AI-Roundtable Leaderboard
Ranking run · 2026-06-15

Ranking run 2026-06-15

Claude Opus 4.7 leads with 79/100 across all domains. Tax law: Claude Opus 4.7 (75) · Medicine: Claude Opus 4.7 (85) · Law: Claude Opus 4.7 (80) · Business law: GPT-5 (80).

Status
development run — not in the feed, not on the homepage (roster test / interim run)
Run
run-20260615T043639-30c761
Started
2026-06-15 04:36 UTC
Items
100 (Tax law 15, Medicine 20, Law 35, Business law 30)
Models
6
Judges
GPT-5, Claude Opus 4.8, Claude Opus 4.7, Mistral Large 2
Question bank
2026-05-17-v1
Code version
0.2.4
Pipeline cost
72.01 USD
Judge agreement
67 %

Overall across domains

#Modeltop10095 % CIΔ prev.failuressec median/p90hallu bandper 1000 tok.Δ own family
1Claude Opus 4.7019 / 287976–82-2561,59—
2Claude Opus 4.8017 / 297471–770730,89—
2GPT-5023 / 427470–78-3690,65—
3Gemini 2.5 Pro019 / 235450–580823,17—
4Mistral Large 2010 / 664944–54-1474,46—
5Grok 4.308 / 104844–52+1764,58—

Δ prev. = difference to the previous published run; “—” = model was not in it. Hallu band: 100 = none, 0 = ≥ 4 false claims per answer. Per 1000 tok. = flagged claims per 1000 output tokens (length-normalised). Δ own family = score with all judges minus score with foreign judges only, in points; positive = own family scores more generously; “—” = not recorded before code_version 0.2.8 or no foreign judge (methodology §4, §6.5).

Tax law · 15 Items

Ranks 1–3 statistically tied (95 % CI overlaps rank 1)

In Tax law, Claude Opus 4.7 (top score) contradicts a stored key fact on average 0,3 times per answer and adds avg 2,0 claims the judge panel flags as false or unsupported; 100 % of answers carry at least one flag. The lowest value is Gemini 2.5 Pro at avg 1,5 per answer. Relative to answer length: 1,76 per 1000 output tokens, lowest GPT-5 at 0,75.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1Claude Opus 4.77568–81-20,32,02,3100 %1,7668 %
2Claude Opus 4.86660–73-30,51,11,6100 %1,2073 %
3GPT-55946–70-81,10,92,0100 %0,7555 %
4Gemini 2.5 Pro3827–48+31,10,61,593 %6,8853 %
5Mistral Large 23020–41+51,52,03,5100 %7,2957 %
6Grok 4.32617–3601,71,02,7100 %11,3758 %

Medicine · 20 Items

Ranks 1–3 statistically tied (95 % CI overlaps rank 1)

In Medicine, Claude Opus 4.7 (top score) contradicts a stored key fact on average 0,5 times per answer and adds avg 0,8 claims the judge panel flags as false or unsupported; 100 % of answers carry at least one flag. The lowest value is Gemini 2.5 Pro at avg 0,2 per answer. Relative to answer length: 1,21 per 1000 output tokens, lowest Claude Opus 4.8 at 0,65.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1Claude Opus 4.78580–90-10,50,81,2100 %1,2175 %
2GPT-58073–86-20,50,40,980 %0,6677 %
3Claude Opus 4.87971–87-10,30,40,775 %0,6568 %
4Mistral Large 26963–75+30,40,71,195 %2,1874 %
5Gemini 2.5 Pro6154–68+40,10,10,240 %0,9870 %
6Grok 4.35247–58-10,30,20,455 %2,0575 %

Law · 35 Items

Ranks 1–3 statistically tied (95 % CI overlaps rank 1)

In Law, Claude Opus 4.7 (top score) contradicts a stored key fact on average 0,3 times per answer and adds avg 0,7 claims the judge panel flags as false or unsupported; 74 % of answers carry at least one flag. The lowest value is Grok 4.3 at avg 0,4 per answer. Relative to answer length: 1,26 per 1000 output tokens, lowest Claude Opus 4.8 at 0,62.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1Claude Opus 4.78075–85-40,30,71,074 %1,2681 %
2Claude Opus 4.88075–84+10,20,40,666 %0,6277 %
3GPT-57365–79-10,50,61,180 %0,6974 %
4Gemini 2.5 Pro5954–64-30,20,20,440 %1,9371 %
5Grok 4.35146–55+30,30,20,440 %2,8364 %
6Mistral Large 24234–49-51,30,71,980 %6,3266 %

Business law · 30 Items

Ranks 1–2 statistically tied (95 % CI overlaps rank 1)

In Business law, GPT-5 (top score) contradicts a stored key fact on average 0,4 times per answer and adds avg 0,9 claims the judge panel flags as false or unsupported; 90 % of answers carry at least one flag. The lowest value is Gemini 2.5 Pro at avg 1,0 per answer. Relative to answer length: 0,54 per 1000 output tokens.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1GPT-58075–85-30,40,91,290 %0,5468 %
2Claude Opus 4.77772–82+10,72,02,7100 %1,9272 %
3Claude Opus 4.86862–74+10,51,11,693 %1,0659 %
4Mistral Large 25448–60-30,81,52,397 %3,6252 %
5Grok 4.35348–58+20,50,61,190 %3,8161 %
6Gemini 2.5 Pro5247–58+10,60,51,080 %3,7561 %

Raw data of this run

raw.jsonl and progress.jsonl are the public streams with response texts stripped (contamination protection, methodology §7).