ToneBench Judge Panel Analysis
Last updated , from board data generated at 17:52 UTC.
Two of the three LLM judges on the ToneBench panel score their own lab's drafts higher than the other LLM judges' scores predict, by up to 4.0 points. Among the eight labs with at least five ranked configurations, no single judge in the four-judge average moves a lab's average Writing Score by more than 1.5 points relative to the rest of the board, or shifts a lab's configurations by more than 8.7 places on average.
Every figure except the head-to-head check in section 06 is recomputed from the 30,200 stored judgments of 7,550 drafts by 151 ranked configurations each time the board is rebuilt; that check uses its own 540 side-by-side judgments. A configuration is one model at one reasoning or thinking setting. The Writing Score is the 0 to 100 consensus of the four judges and code; the Writing Elo is a rating from pairwise comparisons of those scores, and the board ranks by it. Jev, the only judge that is not an LLM, is a typed judge: a decision model that answers fixed multiple-choice questions and writes no text. The methodology page describes how the judges are run, calibrated and checked.
Summary of Findings
- Lab Preference
- GPT-6.1 Sol gives OpenAI configurations 4.0 more points than the other LLM judges' scores predict (95% interval 2.7 to 5.4), and Claude Opus 5.5 gives Anthropic 2.1 more (1.3 to 3.1). For each, its own lab is the largest preference it shows. GLM 5.3 Flash's preference for Z.ai, +0.6 points, has a 95% interval of -0.2 to 1.3, which includes zero, so this board cannot tell it from noise.
- Own Model
- All three LLM judges score the model they run higher than the other LLM judges' scores predict, and by more than they favor their lab as a whole. GPT-6.1 Sol gives its own five configurations (GPT-6.1 Sol at every setting on the board) 7.5 more points (95% interval 6.5 to 8.5), against +4.0 for OpenAI as a whole; Claude Opus 5.5 gives its own five +3.3 (2.2 to 4.6) and GLM 5.3 Flash gives its own configuration +1.7 (1.4 to 2.1).
- Judge Without a Lab
- Jev, whose maker has no model on the board, also shows lab preferences; its two largest are Alibaba +3.7 and OpenAI -2.6 points. Self-preference is only one source of lab preference on this panel.
- Agreement
- The judges differ more in generosity than in order. Their average Writing Scores run from 68.7 (Claude Opus 5.5) to 82.9 (GLM 5.3 Flash), while each judge's ranking of the 151 configurations follows the Writing Score ranking closely: Spearman correlation 0.92 to 0.99, where 1 means the same order.
- Effect on the Board
- Each judge counts for a quarter of every judged score. Among the eight labs with at least five ranked configurations, the largest single-judge effects are these: removing Jev would lower Alibaba's average Writing Score by 1.5 points relative to the rest of the board, and removing GPT-6.1 Sol would move OpenAI's configurations 8.7 places down on average.
- Head-to-Head Check
- In a blind head-to-head check of the top 15 configurations, Claude Opus 5.5 agrees with the board on all five pairs whose Writing Elo intervals do not overlap.
- Gemini Judge Test
- Tested as a judge on 410 board drafts, Gemini 4 Argon favors Google by 4.0 points (2.2 to 5.7) and its own model by 7.4. As a fifth judge it would lift Google's configurations by 0.4 points relative to the rest of the board and move them up a median of two places.
Judge Panel
Who scores each draft, at which setting, and how generous each judge is.
Every draft on the board is scored once by each of the four judges, blind to the model that wrote it, and the Writing Score is their plain average. Claude Opus 5.5 is the strictest judge and GLM 5.3 Flash the most generous. Within one configuration, the judge that scores it highest and the judge that scores it lowest are 14.6 points apart on average, whichever judges those are.
| Judge | Exact version | How it runs | Average Writing Score given | Spread across configurations |
|---|---|---|---|---|
| Claude Opus 5.5Anthropic, LLM judge | claude-opus-5-5 | Medium effort, through the Claude Code subscription | 68.7 | 12.127.1 to 85.2 |
| GPT-6.1 SolOpenAI, LLM judge | gpt-6.1-sol | Medium reasoning, through the Codex CLI | 71.9 | 10.234.6 to 86.0 |
| GLM 5.3 FlashZ.ai, LLM judge | z-ai/glm-5.3-flash | Through OpenRouter | 82.9 | 9.038.7 to 91.7 |
| Jev 1.13TypeSafe, typed judge | jev-1.13.0 | TypeSafe API, model pinned to jev-1.13.0 | 74.0 | 8.736.2 to 83.5 |
The average is over all 7,550 drafts. Spread is the standard deviation of a judge's average score across the 151 configurations, with its lowest and highest configuration below it: a larger spread means the judge separates strong and weak configurations more. Jev's raw answers are mapped onto the LLM judges' average scale for each metric (calibration), so its average matches the LLM judges' average by construction and says nothing about its own generosity.
Lab Preference by Judge
Points each judge adds to or removes from each lab's drafts, compared with the other LLM judges.
Judges differ in overall generosity, so raw averages cannot show a preference for one lab. Instead, each judge is compared with what the other LLM judges' scores predict for the same configurations. The difference, in Writing Score points, is the judge's lab preference: a positive value means the judge gives that lab's drafts more points than the other LLM judges' scores predict, a negative value fewer, and zero means no preference.
Rings and gridlines every 2 points. Labs shown: every lab with at least five ranked configurations, plus each judge's own lab, ordered by the Writing Elo of their best configuration (clockwise from the top on the radar, top to bottom in rows).
Lab Preference Values
Each value is in Writing Score points, with its 95% interval below it. Values in bold have an interval that excludes zero.
| Lab | Claude Opus 5.5 | GPT-6.1 Sol | GLM 5.3 Flash | Jev |
|---|---|---|---|---|
| AnthropicClaude: 34 configurations, 1,700 drafts | +2.1own lab1.3 to 3.1 | -2.1-3.3 to -1.2 | +1.10.8 to 1.5 | +0.90.1 to 1.6 |
| OpenAIGPT: 32 configurations, 1,600 drafts | -2.6-4.1 to -1.4 | +4.0own lab2.7 to 5.4 | -1.4-1.9 to -1.0 | -2.6-3.8 to -1.5 |
| Z.aiGLM: 7 configurations, 350 drafts | +0.5-0.7 to 1.6 | -1.5-2.9 to 0.2 | +0.6own lab-0.2 to 1.3 | +0.9-0.4 to 1.8 |
| DeepSeekDeepSeek: 11 configurations, 550 drafts | +1.30.3 to 2.4 | -2.3-3.3 to -1.4 | +0.70.2 to 1.2 | +2.01.2 to 2.7 |
| MetaMuse and Llama: 7 configurations, 350 drafts | +1.20.1 to 2.3 | +0.3-0.8 to 1.4 | -1.7-2.6 to -0.8 | -0.3-2.7 to 1.5 |
| xAIGrok: 8 configurations, 400 drafts | -0.3-1.2 to 0.7 | 0.0-1.1 to 1.2 | -0.2-1.0 to 0.5 | +1.8-0.4 to 3.6 |
| GoogleGemini and Gemma: 20 configurations, 1,000 drafts | +0.1-0.7 to 1.0 | -0.7-1.7 to 0.3 | -0.3-1.0 to 0.4 | -2.3-3.5 to -0.6 |
| AlibabaQwen: 8 configurations, 400 drafts | -0.3-1.9 to 2.0 | -1.3-3.6 to 0.5 | +1.50.03 to 2.8 | +3.70.9 to 7.2 |
How the Values Are Computed
- Each judge's average Writing Score is taken for each of the 151 ranked configurations, over its 10 scripts and 50 drafts: 7,550 drafts per judge.
- The reference for an LLM judge is the average of the other two LLM judges. The reference for Jev is the average of the three LLM judges. Jev stays out of the LLM judges' reference so that its own strong preferences do not appear as preferences of the others.
- For each lab, a second-degree polynomial that predicts the judge's score from the reference is fitted by least squares on every configuration from the other 14 labs.
- The lab's value is the average gap between the judge's score and the curve's prediction over that lab's own configurations.
- The 95% interval is a bootstrap: the 10 scripts, and the configurations within each lab, are resampled with replacement 1,000 times and every step is repeated. The middle 95% of the results is the interval.
How the Reference Changes the Values
An LLM judge is compared with the other two, so a preference of one of them moves the others' values for the same lab in the opposite direction. The largest case on this board: Claude Opus 5.5's OpenAI value is -2.6 points against the average of GPT-6.1 Sol and GLM 5.3 Flash, and +1.1 against GLM 5.3 Flash alone, so all of that value comes from GPT-6.1 Sol's preference for its own lab. Every value for another LLM judge's own lab carries some of this effect.
The own-lab values depend on the reference too. Each one against both other LLM judges, then against each of them alone:
- Claude Opus 5.5 for Anthropic: +2.1 against both, +3.5 against GPT-6.1 Sol alone, +0.7 against GLM 5.3 Flash alone.
- GPT-6.1 Sol for OpenAI: +4.0 against both, +3.7 against Claude Opus 5.5 alone, +4.5 against GLM 5.3 Flash alone.
- GLM 5.3 Flash for Z.ai: +0.6 against both, +0.4 against Claude Opus 5.5 alone, +0.7 against GPT-6.1 Sol alone.
Each own-lab value whose interval excludes zero stays positive against either judge alone; its size depends on which judges form the reference.
Differences From the Earlier Version of This Chart
An earlier version of this analysis, computed on October 1, 2026, covered the previous panel (Claude Opus 5, GPT-5.6 Sol, DeepSeek V4.1 Flash and Jev 1.13) and drew the values as a radar, as this page's chart does by default. It used a straight-line fit and only the configurations whose three LLM judges averaged 75 or more. This page keeps its references and its fit outside each lab, and changes two things.
- A curve instead of a straight line. GLM 5.3 Flash compresses its scores near the top of the scale. Against its reference, a straight line explains 93.1% of the variance in its configuration scores and the curve 98.3%, so a straight line misreads the strongest labs.
- Every ranked configuration. The current panel scores on a lower scale: only 96 of the 151 configurations reach the 75-point pool. The curve follows the whole range, so no lab is cut.
Labs with fewer than five ranked configurations are left off the chart because a few configurations give an unstable average. They still count in every fit, and a judge's own lab is always shown. Eight labs are shown on this board.
Self-Preference
Each LLM judge's preference for its own lab and for the model it runs, with 95% intervals.
Judging is blind: no judge is told which model wrote a draft. A judge can still favor the writing style its own lab's models share. Each card shows an LLM judge's preference for its own lab, from the same calculation as the chart in section 02, and what the board would look like if that judge scored it alone.
GPT-6.1 Sol for OpenAI
+4.0 points
95% interval 2.7 to 5.4. The interval excludes zero.
Its largest preference among the eight labs. Judged by GPT-6.1 Sol alone, OpenAI's 32 configurations would rank a median of 17 places higher by Writing Score. Alone, it would put GPT-6 Astra (max) at #1; the panel's Writing Score puts OpenAI's best configuration, GPT-6 Sol (max), at #13.
Claude Opus 5.5 for Anthropic
+2.1 points
95% interval 1.3 to 3.1. The interval excludes zero.
Its largest preference among the eight labs. Judged by Claude Opus 5.5 alone, Anthropic's 34 configurations would rank a median of 0.5 places higher by Writing Score. Claude Opus 5.5 (max effort) is #1 both alone and by the panel's Writing Score.
GLM 5.3 Flash for Z.ai
+0.6 points
95% interval -0.2 to 1.3. The interval includes zero, so this board cannot tell the preference from noise.
Its fourth largest preference among the eight labs. Judged by GLM 5.3 Flash alone, Z.ai's seven configurations would rank a median of eight places higher by Writing Score. Its best Z.ai configuration, GLM-5.3 (max), would rank #16 instead of #22.
Preference for the Judge's Own Model
Each LLM judge runs a model that also competes on the board. The own-model value applies the same curve as the own-lab value, fitted outside the judge's lab, and averages the gap over that model's configurations only. Values are in Writing Score points, with the 95% interval below each; values in bold have an interval that excludes zero.
| Judge | Model it runs | Own model | Own lab |
|---|---|---|---|
| GPT-6.1 Sol | GPT-6.1 SolFive configurations on the board | +7.56.5 to 8.5 | +4.0OpenAI, 2.7 to 5.4 |
| Claude Opus 5.5 | Claude Opus 5.5Five configurations on the board | +3.32.2 to 4.6 | +2.1Anthropic, 1.3 to 3.1 |
| GLM 5.3 Flash | GLM-5.3 FlashOne configuration on the board | +1.71.4 to 2.1 | +0.6Z.ai, -0.2 to 1.3 |
The intervals come from a separate bootstrap of 1,000 samples that resamples the scripts, and the model's configurations among themselves, so the own-lab intervals keep their values. GLM-5.3 Flash has one configuration on the board, so its interval comes from resampling the scripts alone.
Jev has no lab of its own on the board. Its two largest preferences, Alibaba +3.7 and OpenAI -2.6 points, are about as large as the largest self-preference above. Self-preference is therefore only one source of lab preference on this panel.
Ranks in this section are by Writing Score among the 151 configurations. The board itself ranks by Writing Elo, which follows the Writing Score closely.
Agreement Between Judges
How closely the judges' scores and rankings match.
The Pearson correlation (r) measures how closely two judges' scores move together: 1 means they rise and fall in step, 0 that they are unrelated. On single drafts it runs from 0.79 (GPT-6.1 Sol and Jev) to 0.94 (Claude Opus 5.5 and GPT-6.1 Sol). Averaging a configuration's 50 drafts cancels most draft-level disagreement, so the configuration figures are higher.
| Judges | Single drafts | Configuration averages |
|---|---|---|
| Claude Opus 5.5 and GPT-6.1 Sol | 0.94 | 0.97 |
| Claude Opus 5.5 and GLM 5.3 Flash | 0.93 | 0.97 |
| Claude Opus 5.5 and Jev | 0.86 | 0.95 |
| GPT-6.1 Sol and GLM 5.3 Flash | 0.90 | 0.95 |
| GPT-6.1 Sol and Jev | 0.79 | 0.90 |
| GLM 5.3 Flash and Jev | 0.89 | 0.97 |
Each Judge's Ranking Against the Writing Score
The Spearman correlation (ρ) compares two rankings: 1 means the same order. Each judge's ranking of the configurations, by its own average, is compared with the Writing Score ranking, which includes that judge, and with the average of the other three judges, which does not. The board itself ranks by Writing Elo, so a configuration's place by Writing Score can differ from its configuration rank on the board. Jev agrees least with the others (ρ 0.875): it shares five of the Writing Score top 10, and its #1 is Claude Fable 5.1 (max effort), #8 by Writing Score.
| Judge | Spearman ρ with the Writing Score | Spearman ρ with the other three judges | Writing Score top 10 in its top 10 | Its #1 configuration |
|---|---|---|---|---|
| Claude Opus 5.5 | 0.990 | 0.978 | 10 of 10 | Claude Opus 5.5 (max effort)#1 by Writing Score, configuration #2 on the board |
| GPT-6.1 Sol | 0.951 | 0.896 | 4 of 10 | GPT-6 Astra (max)#26 by Writing Score, configuration #27 on the board |
| GLM 5.3 Flash | 0.985 | 0.977 | 9 of 10 | Claude Opus 5.5 (max effort)#1 by Writing Score, configuration #2 on the board |
| Jev | 0.918 | 0.875 | 5 of 10 | Claude Fable 5.1 (max effort)#8 by Writing Score, configuration #7 on the board |
Strictness by Metric
Average score each judge gives on each judged metric.
Claude Opus 5.5 gives the lowest average on seven of the eight judged metrics, and GLM 5.3 Flash the highest on all eight. The widest gap is on Slop Score, judged half: 21.8 points between the strictest and the most generous judge. Each cell is the judge's average over all 7,550 drafts; its color shows the difference from the four-judge average for that metric. Length and the computed half of the Slop Score are measured by code and are the same for every judge, so they have no row of their own.
| Metric | Claude Opus 5.5Opus | GPT-6.1 SolGPT | GLM 5.3 FlashGLM | JevJev | Panel averageAvg |
|---|---|---|---|---|---|
| Tone & Voice MatchTone | 61.9 | 67.1 | 82.1 | 69.8 | 70.2 |
| Writing Craft & ClarityCraft | 64.5 | 71.4 | 81.1 | 71.7 | 72.2 |
| Substance, Accuracy & ValueSubstance | 72.8 | 75.4 | 85.5 | 77.7 | 77.8 |
| Continuity & EmotionContinuity | 67.6 | 72.4 | 84.1 | 74.1 | 74.6 |
| YouTube Best PracticesYouTube | 67.7 | 69.4 | 81.7 | 72.2 | 72.8 |
| Hook StrengthHook | 73.7 | 73.3 | 85.8 | 77.2 | 77.5 |
| Visual Cue QualityCues | 66.3 | 67.2 | 80.7 | 71.2 | 71.3 |
| Slop Score, judged halfSlop, judged | 58.6 | 63.5 | 80.4 | 66.6 | 67.3 |
| Writing ScoreScore | 68.7 | 71.9 | 82.9 | 74.0 | 74.4 |
Jev's scores are mapped onto the LLM judges' average scale metric by metric, so its column matches the panel average by construction and says nothing about its own strictness.
Blind Head-to-Head Check
What a blind side-by-side comparison says about the consensus order at the top.
The board's order comes from rubric scores. As a separate check, Claude Opus 5.5 read two anonymous drafts of the same script side by side, in both orders, without their rubric scores, and picked the better one. Each of the top 15 configurations was paired with every other top-15 configuration up to two places away on the board, on all 10 scripts.
The agreement figures separate pairs with non-overlapping 95% Writing Elo ranges from pairs with overlapping or missing ranges. These are ranges for individual configurations. Comparing them does not directly test the Elo gap between a pair, and overlap alone establishes neither a tie nor an order. The other three pairs split evenly between the two drafts, so they count toward neither order. The full head-to-head ranking of the top 15 has a Spearman correlation of 0.42 with the board. Disagreement with the board is not established to be sampling noise. The reference scripts remain fixed; this check does not measure agreement on new scripts.
Claude Opus 5.5 is also the panel's Anthropic judge, and 14 of the top 15 configurations are Anthropic's. Its head-to-head order places the other one configurations two places lower on average than the board does. This is the same direction as its Anthropic preference in section 02.
This check was run on an earlier board snapshot; its ranks are from that snapshot.
All 15 Configurations in the Check
| Configuration | Lab | Configuration rank | Head-to-head rank |
|---|---|---|---|
| Claude Opus 5.5 (xhigh) | Anthropic | #1 | #4 |
| Claude Opus 5.5 (max effort) | Anthropic | #2 | #2 |
| Claude Fable 5.2 (xhigh) | Anthropic | #3 | #10 |
| Claude Fable 5.2 (max effort) | Anthropic | #4 | #3 |
| Claude Fable 5.2 (high) | Anthropic | #5 | #12 |
| Claude Sonnet 5.5 (max effort) | Anthropic | #6 | #8 |
| Claude Fable 5.1 (max effort) | Anthropic | #7 | #9 |
| Claude Opus 5.5 (high) | Anthropic | #8 | #13 |
| Claude Opus 5.5 (medium) | Anthropic | #9 | #5 |
| Claude Opus 5.5 (low) | Anthropic | #10 | #1 |
| Claude Fable 5.2 (medium) | Anthropic | #11 | #7 |
| Claude Fable 5.2 (low) | Anthropic | #12 | #11 |
| GPT-6 Sol (max) | OpenAI | #13 | #15 |
| Claude Fable 5.1 (high) | Anthropic | #14 | #6 |
| Claude Sonnet 5.5 (xhigh) | Anthropic | #15 | #14 |
Effect on the Published Board
How much of each judge's lab preference reaches the published Writing Score.
The Writing Score averages the four judges with equal weight. A preference of one judge therefore enters every score at about a quarter of its size, and the preferences of judges from different labs pull in different directions. To measure what remains, the board is recomputed without each judge in turn, and the average Writing Score and rank of each lab shown are compared with the published ones.
The largest preference on the panel, GPT-6.1 Sol at +4.0 points for OpenAI, changes OpenAI's average Writing Score by +1.1 points on the board. It still moves OpenAI's configurations 8.7 places on average, because ranks are crowded: among the top 20 configurations, neighboring configurations are a median 0.11 Writing Score points apart.
Effect of Each Judge by Lab
Each cell shows how many Writing Score points the judge's presence adds to the lab's configurations, relative to the rest of the board, and how many places it moves them by Writing Score. Positive values mean the judge lifts the lab.
| Lab | Claude Opus 5.5 | GPT-6.1 Sol | GLM 5.3 Flash | Jev |
|---|---|---|---|---|
| Anthropic | +1.1own lab+1.3 places | -0.2-2.3 places | -0.4+0.6 places | -0.5+0.9 places |
| OpenAI | +0.1-1.5 places | +1.1own lab+8.7 places | -0.4-1.9 places | -0.8-5.0 places |
| Z.ai | -0.1-0.7 places | -0.5-2.7 places | +0.2own lab+1.9 places | +0.3+1.6 places |
| DeepSeek | +0.3+1.1 places | -0.7-7.5 places | +0.1+1.8 places | +0.4+5.5 places |
| Meta | 0.0+1.0 places | +0.1+0.6 places | -0.3-3.7 places | +0.1+0.6 places |
| xAI | -0.4-2.1 places | -0.3-5.0 places | +0.1-0.5 places | +0.6+5.0 places |
| -0.4+0.7 places | -0.1-0.6 places | +0.6+0.3 places | 0.0-1.8 places | |
| Alibaba | -1.3-2.1 places | -0.6+0.6 places | +0.4+0.8 places | +1.5+2.3 places |
What This Analysis Cannot Detect
Every value on this page compares a judge with the other judges. A preference that all four judges share, for example for a writing style that several labs' models produce, cancels out of these comparisons and stays in the Writing Score. Only an outside reference, such as blind human ratings, can measure it. The methodology page lists the board's other reliability checks.
A Gemini Judge Test
Whether a judge from Google would favor Google's models, and what it would change on the board.
The panel has no judge from Google. To test whether one would favor Google's models, Gemini 4 Argon at medium thinking judged 410 drafts already on the board on October 19 and 20, 2026, with the panel's own prompt: run 1 of each of the ten scripts for 17 of the 20 Google configurations on the board, and for 24 configurations from ten other labs spread across the ranking. Its scores never enter the board. Its preferences below are computed as Jev's are in section 02, against the three LLM judges' average, and the panel judges' values beside them come from the same drafts.
Gemini 4 Argon behaves like the panel's LLM judges: it scores its own lab above the curve the other judges set, and its own model most. Both values are about the size of GPT-6.1 Sol's on the full board: +4.0 points for OpenAI and +7.5 for its own model (section 03). On the same drafts, the panel judges' preferences for Google are 0.0 (Claude Opus 5.5), -0.2 (GPT-6.1 Sol), -0.6 (GLM 5.3 Flash) and -3.0 (Jev); each value is within 0.8 points of the same judge's value on the full board, so the test drafts reproduce the board's picture of Google.
Lab Preference on the Test Drafts
Labs with at least three tested configurations. Values are in Writing Score points, with the 95% interval below each; values in bold have an interval that excludes zero. With three or four configurations, most intervals are wide.
| Lab | Gemini 4 Argon | Claude Opus 5.5 | GPT-6.1 Sol | GLM 5.3 Flash | Jev |
|---|---|---|---|---|---|
| Google17 configurations tested | +4.02.2 to 5.7own lab | 0.0-1.3 to 2.0 | -0.2-1.6 to 1.9 | -0.6-2.6 to 0.6 | -3.0-5.4 to -1.3 |
| AnthropicFour configurations tested | -0.2-2.8 to 3.7 | +0.4-1.6 to 2.2own lab | -0.3-2.1 to 1.7 | +1.30.1 to 2.5 | 0.0-2.5 to 1.9 |
| OpenAIFour configurations tested | -4.1-6.9 to -0.7 | -0.8-4.7 to 3.9 | +3.5-0.1 to 7.4own lab | -2.0-3.7 to -0.3 | -0.5-4.4 to 3.7 |
| DeepSeekThree configurations tested | -0.6-2.7 to 1.1 | +1.0-1.4 to 3.6 | -1.5-3.4 to 0.4 | +0.5-0.5 to 1.5 | +2.20.8 to 3.5 |
| AlibabaThree configurations tested | -3.0-5.6 to 11.7 | -1.0-3.9 to 7.4 | -0.4-6.1 to 3.7 | +1.8-2.2 to 5.1 | +2.3-4.0 to 7.5 |
| xAIThree configurations tested | -1.9-5.4 to 1.3 | -0.2-1.6 to 1.3 | +0.8-1.3 to 3.2 | -0.8-2.6 to 0.7 | +1.5-3.4 to 4.6 |
If It Joined the Panel
As a fifth judge with equal weight, Gemini 4 Argon would lift Google's configurations by 0.4 points relative to the rest of the board and move them up a median of two places by Writing Score. Ranks in the middle of the board are crowded, so the largest move would be Gemini 4 Carbon (high thinking), from #52 to #38 in the Writing Score order. The ranking as a whole would barely change: Spearman ρ 0.997 with the current Writing Score, and zero changes in the top 10. In place of Claude Opus 5.5, it would lift Google by 0.7 points.
Adding it would bring its own preferences into the panel without removing any. Each current judge's share of the Writing Score would fall from a quarter to a fifth, and with it the part of its own-lab preference that reaches the board, while Gemini 4 Argon's preferences, +4.0 points for Google and -4.1 for OpenAI, would enter every score at a fifth of their size.
Agreement and Cost
Gemini 4 Argon's ranking of the 41 tested configurations matches the average of the other four judges at Spearman ρ 0.94; the panel's judges score 0.91 to 0.97 on the same comparison. Each judgment cost $0.06 at Google's list price, about $453 to score the 7,550 drafts of the current board once. Of its 427 calls, 17 returned a reply that was not valid JSON and were retried. At medium thinking it used thinking tokens on 28 of its 410 judgments.
The board effect is modelled: each configuration's Gemini 4 Argon score is predicted from the three LLM judges' average with the test's curve, plus the configuration's own measured gap where it was tested, its lab's average gap where its lab was tested, and no gap for a lab the test did not cover. The test uses one run of each script where the board uses five. Like every value on this page, it compares judges with each other, so it cannot say whether Google's models deserve higher or lower scores; only an outside reference, such as blind human ratings, could.