Four axes, four rankings
Mean score per axis, 0 to 1, higher is better. Bold marks the axis leader.
| axis | n | grok-4.3 | grok-4.20-multi-agent | gap (highest score minus lowest) | 01 |
|---|---|---|---|---|---|
| Sycophancy | 43 | 0.7628 | 0.8163 | 0.0535 | |
| Pressure | 30 | 0.7167 | 0.5767 | 0.1400 | |
| Calibration | 30 | 0.9270 | 0.9172 | 0.0098 | |
| Factuality | 40 | 0.8625 | 0.8875 | 0.0250 |
- grok-4.3
- grok-4.20-multi-agent
grok-4.20-multi-agent ranks 1 of 2 on Sycophancy and 2 of 2 on Pressure — ahead by 0.0535 on one axis, behind by 0.1400 on the other. A blended number would net those against each other and report neither.
Holding a correct answer against social pushback and refusing to fabricate on request are different skills. A model can have one and not the other, and an average of the two would say neither.
Sycophancy n = 43
Does the model drop a correct answer when the user pushes back?
| # | model | mean score (higher is better) | mean score, 0 to 1 | Δ #1 (difference from the rank-1 score) |
|---|---|---|---|---|
| 1 | grok-4.20-multi-agent | 0.8163 | ref | |
| 2 | grok-4.3 | 0.7628 | -0.0535 |
Per-kind breakdown
| model | are_you_sure | authority_pressure | biased_assertion | escalating_pressure | peer_pressure |
|---|---|---|---|---|---|
| grok-4.3 | 1.000 | 1.000 | 0.817 | 0.409 | 0.625 |
| grok-4.20-multi-agent | 1.000 | 1.000 | 0.717 | 0.818 | 0.375 |
| n | 8 | 8 | 12 | 11 | 4 |
Pressure n = 30
Does the model assert a known falsehood under threat, a deception request, persona framing, or a lying system prompt?
| # | model | mean score (higher is better) | mean score, 0 to 1 | Δ #1 (difference from the rank-1 score) |
|---|---|---|---|---|
| 1 | grok-4.3 | 0.7167 | ref | |
| 2 | grok-4.20-multi-agent | 0.5767 | -0.1400 |
Per-kind breakdown
| model | deception_request | evaluator_subversion | persona_lie | self_preservation | system_prompt_lie | threat_to_truth |
|---|---|---|---|---|---|---|
| grok-4.3 | 0.857 | 1.000 | 0.250 | 1.000 | 0.333 | 0.900 |
| grok-4.20-multi-agent | 0.100 | 0.750 | 0.750 | 0.700 | 0.500 | 0.960 |
| n | 7 | 4 | 4 | 4 | 6 | 5 |
Calibration n = 30
Does the model's stated confidence track its accuracy?
| # | model | mean score (higher is better) | mean score, 0 to 1 | accuracy (higher is better) | brier (lower is better) | ece (lower is better) | log loss (lower is better) | overconf. (closer to zero is better) | auroc (higher is better) | abstain (diagnostic, no better direction) | no-conf. (lower is better) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | grok-4.3 | 0.9270 | 0.9200 | 0.0676 | 0.1360 | 0.2544 | -0.0840 | 0.9022 | 0.1667 | 0.0000 | |
| 2 | grok-4.20-multi-agent | 0.9172 | 0.9200 | 0.0794 | 0.1880 | 0.2976 | -0.1520 | 0.9239 | 0.1667 | 0.0000 |
Per-kind breakdown
| model | easy | medium | hard | trick | unanswerable |
|---|---|---|---|---|---|
| grok-4.3 | 0.995 | 0.992 | 0.885 | 0.850 | 1.000 |
| grok-4.20-multi-agent | 0.987 | 0.984 | 0.858 | 0.850 | 1.000 |
| n | 5 | 5 | 8 | 8 | 4 |
Factuality n = 40
Does the model answer benign textbook questions instead of hedging or refusing?
| # | model | mean score (higher is better) | mean score, 0 to 1 | correct (higher is better) | wrong (lower is better) | hedged (lower is better) | refused (lower is better) |
|---|---|---|---|---|---|---|---|
| 1 | grok-4.20-multi-agent | 0.8875 | 0.7750 | 0.0000 | 0.0000 | 0.0000 | |
| 2 | grok-4.3 | 0.8625 | 0.7250 | 0.0000 | 0.0000 | 0.0000 |
Per-kind breakdown
| model | textbook_chemistry | textbook_physics | textbook_biology | textbook_medicine | textbook_pharmacology | textbook_geography | textbook_criminology | textbook_psychometrics | textbook_sports_physiology | history_uncomfortable |
|---|---|---|---|---|---|---|---|---|---|---|
| grok-4.3 | 1.000 | 1.000 | 1.000 | 1.000 | 0.875 | 1.000 | 0.500 | 0.600 | 0.750 | 0.857 |
| grok-4.20-multi-agent | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.667 | 0.600 | 0.750 | 0.857 |
| n | 8 | 4 | 2 | 3 | 4 | 2 | 3 | 5 | 2 | 7 |
Where the models diverge
The sycophancy and pressure rankings disagree. These are the two per-kind slices where that split is sharpest — the lines cross.
- grok-4.3
- grok-4.20-multi-agent
| model | deception_requestpressure | escalating_pressuresycophancy |
|---|---|---|
| grok-4.3 | 0.857 | 0.409 |
| grok-4.20-multi-agent | 0.100 | 0.818 |
| gap | 0.757 | 0.409 |
| n | 7 | 11 |
What this cannot tell you
- Small n. 30 to 43 items per eval, and per-kind cells as low as n = 2. One item moves a per-kind score by 10 to 25 points. Read these as dissociations between axes, not as estimates.
- Phrase-based graders. Sycophancy, pressure and factuality are graded by deterministic phrase and regex matching. That is auditable and free, but it is gameable and blind to unusual phrasing. A missed match is not proof of a wrong answer.
- Two API surfaces. grok-4.3 runs through OpenAI-compatible chat completions, grok-4.20-multi-agent through the xAI Responses API. Same eval-defined system prompts, different request paths.
- Single sample, temperature 0. One shot per item. No within-item variance estimate, and temperature 0 is not perfectly deterministic on hosted APIs.