Towards AITowards AIToneBench

Lab Comparison

DeepSeek Models Compared

Last updated , from board data generated at 17:52 UTC.

All 11 ranked DeepSeek configurations on ToneBench, side by side: Writing Elo, Writing Score, the nine metrics, and cost and time per script. A configuration is one model at one thinking level.

Highest Writing Elo
Elo 2024DeepSeek V4.1 Flash 1017 (max)#15 of 87 models · $0.014 per script
Lowest Cost Within 200 Elo of the Top
$0.0095 per scriptDeepSeek V4.1 Flash 1017 (high)Configuration #34 · Elo 1974
Lowest Cost
$0.0037 per scriptDeepSeek V4.1 Flash (no thinking)Configuration #82 · Elo 1562

A configuration 200 Elo below the top is expected to score about 24% in its script-by-script comparisons with it, a draw counting as half. The Lowest Cost Within 200 Elo of the Top card names the cheapest configuration that stays that close.

Recommended Configurations

Ranks in these notes are configuration ranks: each setting's place among all 151 configurations. The Highest Writing Elo card above gives the model rank.

Picks and Notes From Louis

I'd use DeepSeek V4.1 Flash 1017 (max). It's #30 on the whole board at $0.014 a script, which is cheap enough that most people can stop there. If you're generating at real volume, DeepSeek V4.1 Flash 1017 (high) is the cheapest configuration within 200 Elo of the top: 50 Elo behind for $0.0095 and 0.9 minutes per script. The two ranges overlap; overlap alone doesn't establish whether max writes better. Most of its edge is length: at max, the scripts land closer to their target.

Both picks come from the newer Flash snapshot, and if you're on the earlier one, I'd switch. At high, the newer snapshot lands 92 Elo above DeepSeek V4.1 Flash (high), and at max it leads DeepSeek V4.1 Flash (max) by 68 Elo. The high ranges don't overlap. The max ranges do, and overlap alone doesn't say which snapshot writes better there. What settles it for me: the newer one costs less at both levels, and at max it comes back in 1.3 minutes instead of 2.4.

I wouldn't go below high to save more. The newer one only ran at high and max, but the earlier one ran the whole ladder, and each step down drops it further: from #36 at max to #48 at high, #60 at low and #82 with thinking off.

The surprise is Pro. It's the pricier tier, and it doesn't pay off for scripts: its best configuration, DeepSeek V4 Pro 0813 (max), costs 5.6 times as much as DeepSeek V4.1 Flash 1017 (max) and only reaches #52. I'd also skip DeepSeek V4 Flash 0731 (high). It costs a little more than DeepSeek V4.1 Flash 1017 (max), ranks lower, and takes 12.3 minutes per script.

If you write a lot of scripts and the bill matters, start with DeepSeek. All 11 configurations are open weights, and even the most expensive one, DeepSeek V4 Pro 0813 (max), costs $0.077 a script. The trap is the spread: 758 Elo separates the best configuration from the worst, and the rows at the bottom all come from the pricier Pro tier. So choosing the right DeepSeek matters a lot more than choosing DeepSeek.

All 11 DeepSeek Configurations, Grouped by Model

Every row wrote the same 10 scripts for the same four blind judges, so these Writing Elo ratings sit on the leaderboard's scale. The rank column is the configuration rank, each setting's place among all 151 configurations on the board. Rows are grouped by model: a highlighted row is a model's best configuration, and the rows under it are the same model at other thinking levels. The ± after each Elo is half the width of its 95% range, and scores run from 0 to 100. Cost is one full script at the API list price of the route used. Each name opens its scorecard, and the head-to-head comparisons cover models from different labs.

Score colors:82 and up65 to 81.9under 65

With the metric scores shown, the table scrolls sideways.

Configuration RankConfigurationWriting EloWriting ScoreCost / ScriptTime / ScriptTone & Voice MatchWriting Craft & ClaritySubstance, Accuracy & ValueHook StrengthYouTube Best PracticesContinuity & EmotionSlop ScoreLength AdherenceVisual Cue Quality
#30DeepSeek V4.1 Flash 1017 (max) · open weights2024 ±4381.6$0.0141.3 min79.279.582.381.180.181.084.194.875.3
#34DeepSeek V4.1 Flash 1017 (high) · open weights1974 ±4681.1$0.00950.9 min79.679.582.281.280.080.984.288.873.4
#36DeepSeek V4.1 Flash (max) · open weights1956 ±4081.0$0.0172.4 min78.178.082.580.179.880.183.295.976.0
#48DeepSeek V4.1 Flash (high) · open weights1882 ±3980.3$0.0121.1 min77.878.280.280.079.680.183.393.174.6
#60DeepSeek V4.1 Flash (low) · open weights1784 ±5779.3$0.00770.7 min77.578.080.080.277.579.582.488.569.8
#82DeepSeek V4.1 Flash (no thinking) · open weights1562 ±4576.8$0.00370.3 min76.075.778.479.073.677.880.880.865.8
#52DeepSeek V4 Pro 0813 (max) · open weights1866 ±4380.1$0.0778.9 min76.376.581.780.978.980.281.493.477.6
#105DeepSeek V4 Pro 0813 (high) · open weights1278 ±7172.3$0.0311.3 min68.669.875.977.969.672.577.572.770.6
#74DeepSeek V4 Flash 0731 (high) · open weights1611 ±4077.5$0.01612.3 min72.871.480.480.575.876.078.295.873.6
#104DeepSeek V4 Pro (high) · open weights1299 ±4873.6$0.0371.3 min70.974.475.477.971.475.872.975.861.9
#106DeepSeek V4 Pro (xhigh) · open weights1266 ±5373.1$0.0703.2 min68.972.773.576.171.274.972.383.464.2

Effort Ladders for DeepSeek Models

Each of these models ran at more than one thinking level. Each page shows how the score, cost and time change from one level to the next.

DeepSeek V4.1 Flash 1017 Against Other DeepSeek Models

Each comparison lists every thinking level of both models, with Writing Elo, cost and time side by side. The line under each pair gives the Writing Elo gap between the two best configurations. A pair outside the written match-ups opens in the comparison tool.

← All comparisons