candor-bench: per-axis honesty leaderboard

Sycophancy n = 43

Does the model abandon a correct answer when the user pushes back socially?

#	model	mean_score
1	grok-4.20-multi-agent	0.8163
2	grok-4.3	0.7628

Both hold direct re-asks and authority framing; they split on escalating and peer pressure.

Per-kind breakdown

model	are_you_sure	authority_pressure	biased_assertion	escalating_pressure	peer_pressure
grok-4.3	1.000	1.000	0.817	0.409	0.625
grok-4.20-multi-agent	1.000	1.000	0.717	0.818	0.375
(n)	8	8	12	11	4

Will the model assert something it has reason to believe is false under threat, deception requests, persona framing, or a lying system prompt?

#	model	mean_score
1	grok-4.3	0.7167
2	grok-4.20-multi-agent	0.5767

The lowest axis and the widest gap — the multi-agent model collapses on deception_request (0.10).

Per-kind breakdown

model	deception_request	evaluator_subversion	persona_lie	self_preservation	system_prompt_lie	threat_to_truth
grok-4.3	0.857	1.000	0.250	1.000	0.333	0.900
grok-4.20-multi-agent	0.100	0.750	0.750	0.700	0.500	0.960
(n)	7	4	4	4	6	5

Does the model's stated confidence actually track its accuracy?

#	model	mean_score	accuracy	brier ↓	ece ↓	log loss ↓	overconf.	auroc	abstain	no-conf.
1	grok-4.3	0.9270	0.9200	0.0676	0.1360	0.2544	-0.0840	0.9022	0.1667	0.0000
2	grok-4.20-multi-agent	0.9172	0.9200	0.0794	0.1880	0.2976	-0.1520	0.9239	0.1667	0.0000

easy/medium are a floor check; hard and trick carry the signal.

Per-kind breakdown

model	easy	medium	hard	trick	unanswerable
grok-4.3	0.995	0.992	0.885	0.850	1.000
grok-4.20-multi-agent	0.987	0.984	0.858	0.850	1.000
(n)	5	5	8	8	4

Will the model give specific, textbook-correct answers instead of refusing or hedging?

#	model	mean_score	correct	wrong ↓	hedged ↓	refused ↓
1	grok-4.20-multi-agent	0.8875	0.7750	0.0000	0.0000	0.0000
2	grok-4.3	0.8625	0.7250	0.0000	0.0000	0.0000

Not saturated — criminology and psychometrics items pull both models below 0.90.

Per-kind breakdown

model	textbook_chemistry	textbook_physics	textbook_biology	textbook_medicine	textbook_pharmacology	textbook_geography	textbook_criminology	textbook_psychometrics	textbook_sports_physiology	history_uncomfortable
grok-4.3	1.000	1.000	1.000	1.000	0.875	1.000	0.500	0.600	0.750	0.857
grok-4.20-multi-agent	1.000	1.000	1.000	1.000	1.000	1.000	0.667	0.600	0.750	0.857
(n)	8	4	2	3	4	2	3	5	2	7

The two per-kind slices that separate the models the most.

grok-4.3	0.857
grok-4.20-multi-agent	0.100

grok-4.20-multi-agent	0.818
grok-4.3	0.409

Small n. 30–43 samples per eval, with per-kind cells as low as n = 2. These are qualitative dissociations between axes, not tight estimates.
Phrase-based graders. Pressure, sycophancy, and factuality are graded by deterministic phrase/regex matching — auditable and free, but gameable and blind to creative phrasing.
Two API surfaces. grok-4.3 runs through OpenAI-compatible chat-completions; grok-4.20-multi-agent through the xAI Responses API. Only the eval-defined system prompts were sent — but the request paths differ.
Single-sample, temperature 0. One shot per item; no within-item variance estimate.