Towards AITowards AIToneBench

Lab Comparison

Google Models Compared

Last updated , from board data generated at 17:52 UTC.

All 20 ranked Google configurations on ToneBench, side by side: Writing Elo, Writing Score, the nine metrics, and cost and time per script. A configuration is one model at one thinking level.

Highest Writing Elo
Elo 1849Gemini 4 Carbon (high thinking)#27 of 87 models · $0.586 per script
Lowest Cost Within 200 Elo of the Top
$0.218 per scriptGemini 4 Carbon (medium thinking)Configuration #63 · Elo 1774
Lowest Cost
$0.0009 per scriptGemma 3 4B (no thinking)Configuration #151 · Elo -367

A configuration 200 Elo below the top is expected to score about 24% in its script-by-script comparisons with it, a draw counting as half. The Lowest Cost Within 200 Elo of the Top card names the cheapest configuration that stays that close.

Recommended Configurations

Ranks in these notes are configuration ranks: each setting's place among all 151 configurations. The Highest Writing Elo card above gives the model rank.

Picks and Notes From Louis

For the best Google script, I'd use Gemini 4 Carbon (high thinking). It costs $0.586 a script, and at 8.7 minutes it's the slowest and the priciest row in the family, because Carbon thinks a lot at high. If you call it yourself, read the Carbon thinking page first: most high drafts here needed a second call to finish.

Most days, though, I'd run Gemini 4 Carbon (medium thinking). It's #63, 75 Elo behind on the point estimates, for $0.218 and 3.1 minutes a script, and it's the cheapest Google configuration within 200 Elo of the top. It even costs a little less than Gemini 4 Argon (high thinking) and ranks 210 Elo above it, with ranges that don't overlap. Set the level in your request either way. Leaving it out matched medium here, but I'd rather name the level I'm paying for.

For volume, I'd take Gemini 4 Carbon (low thinking). It costs $0.061 a script and comes back in 0.8 minutes, about what Argon costs with no flag, and it lands 216 Elo above that run, with ranges that don't overlap. Plan to trim, though, because its scripts run long. If you stay on Argon, make sure your request sets high: with no flag, Argon recorded zero thinking tokens on every draft here and lands 190 Elo lower, with ranges that don't touch. And if you're on Gemini 2.5 Pro (dynamic thinking) today, I'd switch: Gemini 4 Carbon (low thinking) costs about the same and sits 476 Elo higher.

Going cheaper doesn't pay off. Gemini 3.8 Flash (medium thinking) costs $0.036 a script and lands 587 Elo below Carbon at low. The Flash-Lite models are cheaper still, and they score far enough below that I'd rather pay the difference. Gemma 3 4B (no thinking) barely costs anything at $0.0009, and it finishes last in the family at #151. It's a small open model, so that's no surprise.

Two I'd skip: Gemini 3.1 Pro (high thinking) and Gemini 3.5 Flash (high thinking), the priciest rows outside Gemini 4. They cost 2.4 and 2.2 times what Gemini 4 Carbon (low thinking) does, yet they sit at #110 and #116, far below it.

Google still won't give you the top script on this board, but Gemini 4 Carbon (high thinking) gets it closer than any Gemini before it. It lands at #54, 285 Elo above Gemini 4 Argon (high thinking), the best Google row until now, and the two ranges don't overlap. On Tone & Voice Match, the metric that weighs the most, all three Carbon rows beat every other Google row. Length is the one score where several older Geminis still beat Argon, but at high, Carbon lands closer to the target than any other Google row. Still, no Google configuration makes the top fifty yet, and voice is the weak spot: every configuration ranked above Carbon scores higher on Tone & Voice Match, while its length and visual cue scores are among the best on the board. Behind Carbon and Argon, the ladder is long. In all, the family has 20 configurations across 12 models, all closed except Gemma, most of them cheap and quick, with 2216 Elo from the top rung to the bottom one. And funny enough, the older Gemini 2.5 Pro (dynamic thinking) still ranks above every Gemini 3 row, though its range overlaps the best of them, so check the table before you trust a version number.

All 20 Google Configurations, Grouped by Model

Every row wrote the same 10 scripts for the same four blind judges, so these Writing Elo ratings sit on the leaderboard's scale. The rank column is the configuration rank, each setting's place among all 151 configurations on the board. Rows are grouped by model: a highlighted row is a model's best configuration, and the rows under it are the same model at other thinking levels. The ± after each Elo is half the width of its 95% range, and scores run from 0 to 100. Cost is one full script at the API list price of the route used. Each name opens its scorecard, and the head-to-head comparisons cover models from different labs.

Score colors:82 and up65 to 81.9under 65

With the metric scores shown, the table scrolls sideways.

Configuration RankConfigurationWriting EloWriting ScoreCost / ScriptTime / ScriptTone & Voice MatchWriting Craft & ClaritySubstance, Accuracy & ValueHook StrengthYouTube Best PracticesContinuity & EmotionSlop ScoreLength AdherenceVisual Cue Quality
#54Gemini 4 Carbon (high thinking)1849 ±3380.1$0.5868.7 min73.674.883.578.880.079.283.697.584.0
#63Gemini 4 Carbon (medium thinking)1774 ±4079.2$0.2183.1 min73.574.883.179.479.779.381.787.683.1
#76Gemini 4 Carbon (low thinking)1590 ±4077.2$0.0610.8 min73.474.282.379.680.579.280.266.181.3
#81Gemini 4 Argon (high thinking)1564 ±4876.2$0.2273.5 min70.473.381.078.378.178.280.971.482.3
#101Gemini 4 Argon (no thinking)1374 ±4074.7$0.0640.9 min71.473.680.279.679.378.579.047.980.9
#108Gemini 2.5 Pro (dynamic thinking)1114 ±3471.0$0.0670.8 min64.670.771.275.169.173.474.681.862.1
#110Gemini 3.1 Pro (high thinking)1102 ±4271.0$0.1481.2 min63.668.173.971.870.872.679.479.469.8
#113Gemini 3.1 Pro (low thinking)1052 ±4270.1$0.1050.8 min62.467.874.272.969.571.978.974.868.0
#111Gemini 3.8 Flash (high thinking)1077 ±4370.2$0.0872.2 min61.968.273.674.372.873.275.567.873.9
#121Gemini 3.8 Flash (medium thinking)1003 ±4669.1$0.0360.9 min60.868.071.774.171.973.574.162.074.7
#112Gemini 3.7 Flash (high thinking)1074 ±5470.1$0.0862.2 min60.968.273.073.673.573.375.668.674.2
#118Gemini 3.7 Flash (medium thinking)1014 ±4869.4$0.0380.9 min60.967.772.974.272.473.574.562.974.1
#116Gemini 3.5 Flash (high thinking)1032 ±4569.5$0.1361.0 min59.166.070.873.070.269.974.987.072.8
#117Gemini 3.5 Flash (medium thinking)1029 ±4369.5$0.1110.8 min58.466.570.773.770.270.474.486.572.4
#119Gemini 3.6 Flash (medium thinking)1010 ±4569.3$0.0520.9 min58.866.371.374.270.069.674.882.871.8
#120Gemini 3.6 Flash (high thinking)1008 ±4569.2$0.0601.1 min58.365.871.472.470.069.873.485.971.8
#124Gemini 2.5 Flash (dynamic thinking)837 ±10565.0$0.0160.4 min59.562.969.969.963.065.265.368.565.1
#129Gemini 3.5 Flash-Lite (minimal thinking)752 ±4365.0$0.00920.1 min56.467.067.672.059.067.976.165.764.5
#139Gemini 2.5 Flash-Lite (no thinking)518 ±3960.7$0.00230.2 min54.462.366.364.051.763.364.764.860.7
#151Gemma 3 4B (no thinking) · open weights-367 ±3134.3$0.00090.9 min31.933.031.131.334.833.148.448.626.0

Effort Ladders for Google Models

Each of these models ran at more than one thinking level. Each page shows how the score, cost and time change from one level to the next.

Gemini 4 Carbon Against Other Google Models

Each comparison lists every thinking level of both models, with Writing Elo, cost and time side by side. The line under each pair gives the Writing Elo gap between the two best configurations. A pair outside the written match-ups opens in the comparison tool.

← All comparisons