All videos
0:00 / 0:00
research

Our internal writing benchmark uses 3 AI judges: Claude, GPT and DeepSeek. So do they favor their

What's AI by Louis-François Bouchard3 October 2026Watch on YouTube

What you'll learn

  • An internal writing benchmark uses three AI judges: Claude, GPT and DeepSeek.
  • The GPT judge scores OpenAI models 2.3 points higher out of 100 than the other two judges do.
  • The DeepSeek judge gives its own models +1.7 points.
  • The Claude judge gives Anthropic only +0.4, but gives Google +1.3 points.
  • The creator hopes to add Jev as a fourth judge and is waiting on Typesafe for it.

Frequently asked questions

Which AI judges does the internal writing benchmark use?
The benchmark uses three judges: Claude, GPT and DeepSeek. The creator hopes to add Jev as a fourth later.
Do AI judges score their own model family higher?
Yes, they score their own models slightly higher. The GPT judge gives OpenAI models 2.3 points more out of 100, the DeepSeek judge gives its own models +1.7 and the Claude judge gives Anthropic +0.4.
Which preference stands out for the Claude judge?
The Claude judge gives Anthropic only +0.4 points, but gives Google +1.3 points. That is more than its preference for its own family.
Why has Jev not yet been added as a fourth judge?
The creator is waiting on Typesafe for it. Jev is not yet included in the benchmark.

Topics

llm-evaluationbenchmarkai-judgesbiaswriting

In this video

GoogleGoogleOpenAIOpenAIAnthropicAnthropic

Read next

Description from the channel

Our internal writing benchmark uses 3 AI judges: Claude, GPT and DeepSeek. So do they favor their own family? The GPT judge scores OpenAI models 2.3 points higher (out of 100) than the other two judges do. The DeepSeek judge gives DeepSeek models +1.7. The Claude judge gives Anthropic only +0.4... but Google +1.3 😅 Hoping to add Jev as a fourth soon... waiting on Typesafe ;)