AI-Roundtable Leaderboard
Ranking run · 2026-06-21

Ranking run 2026-06-21

GPT-5.5 Pro leads with 80/100 across all domains. Tax law: Claude Opus 4.7 (74) · Medicine: GPT-5.5 Pro (88) · Law: Claude Opus 4.7 (82) · Business law: GPT-5.5 Pro (82).

Status
official run (feed, archive, homepage)
Run
run-20260621T143914-12112d
Started
2026-06-21 14:39 UTC
Items
100 (Tax law 15, Medicine 20, Law 35, Business law 30)
Models
9
Judges
GPT-5, Claude Opus 4.8, Claude Opus 4.7, Mistral Large 2
Question bank
2026-05-17-v1
Code version
0.2.6
Pipeline cost
134.70 USD
Judge agreement
68 %

Overall across domains

#Modeltop10095 % CIΔ prev.failuressec median/p90hallu bandper 1000 tok.Δ own family
1GPT-5.5 Pro n=97/1003227 / 15478077–83—750,10—
2Claude Opus 4.7019 / 287875–81-1601,48—
3Claude Opus 4.8021 / 317572–78+1740,87—
3GPT-5022 / 357571–790690,61—
4Gemini 3.1 Pro022 / 335753–60—783,68—
4Kimi K2.6 n=98/100271 / 1395752–61—460,27—
5Mistral Large 2010 / 695045–55-1504,35—
6Grok 4.305 / 74641–50-1735,44—
7GLM-4.6042 / 744540–49—650,51—

Δ prev. = difference to the previous published run; “—” = model was not in it. Hallu band: 100 = none, 0 = ≥ 4 false claims per answer. Per 1000 tok. = flagged claims per 1000 output tokens (length-normalised). Δ own family = score with all judges minus score with foreign judges only, in points; positive = own family scores more generously; “—” = not recorded before code_version 0.2.8 or no foreign judge (methodology §4, §6.5).

Tax law · 15 Items

Ranks 1–4 statistically tied (95 % CI overlaps rank 1)

In Tax law, Claude Opus 4.7 (top score) contradicts a stored key fact on average 0,5 times per answer and adds avg 1,8 claims the judge panel flags as false or unsupported; 100 % of answers carry at least one flag. The lowest value is Gemini 3.1 Pro at avg 0,9 per answer. Relative to answer length: 1,89 per 1000 output tokens, lowest GPT-5.5 Pro at 0,13.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1Claude Opus 4.77467–80+30,51,82,3100 %1,8974 %
2Claude Opus 4.86762–73-20,21,01,2100 %0,8871 %
3GPT-5.5 Pro6752–80—1,01,01,992 %0,1371 %
4GPT-55639–71-71,31,02,393 %0,8573 %
5Gemini 3.1 Pro5447–61—0,60,40,980 %3,7371 %
6Kimi K2.63018–41—2,12,14,2100 %0,4257 %
7Mistral Large 22719–37-12,11,53,593 %7,6552 %
8GLM-4.61910–28—2,10,82,7100 %0,8958 %
9Grok 4.31810–28-92,10,72,8100 %12,7468 %

Medicine · 20 Items

Ranks 1–4 statistically tied (95 % CI overlaps rank 1)

In Medicine, GPT-5.5 Pro (top score) contradicts a stored key fact on average 0,5 times per answer and adds avg 0,4 claims the judge panel flags as false or unsupported; 85 % of answers carry at least one flag. The lowest value is Gemini 3.1 Pro at avg 0,4 per answer. Relative to answer length: 0,18 per 1000 output tokens.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1GPT-5.5 Pro8883–92—0,50,40,885 %0,1881 %
2GPT-58377–89+20,50,61,090 %0,7272 %
3Claude Opus 4.78173–88-30,70,81,4100 %1,3778 %
4Claude Opus 4.88175–86+20,30,50,790 %0,6676 %
5Kimi K2.67364–80—0,50,71,1100 %0,2274 %
6Mistral Large 26862–75+10,50,51,090 %1,9162 %
7Gemini 3.1 Pro5953–66—0,30,20,460 %1,9072 %
8GLM-4.65751–63—0,20,50,675 %0,2567 %
9Grok 4.35548–6100,40,20,575 %2,3976 %

Law · 35 Items

Ranks 1–4 statistically tied (95 % CI overlaps rank 1)

In Law, Claude Opus 4.7 (top score) contradicts a stored key fact on average 0,2 times per answer and adds avg 0,6 claims the judge panel flags as false or unsupported; 71 % of answers carry at least one flag. The lowest value is Grok 4.3 at avg 0,6 per answer. Relative to answer length: 0,98 per 1000 output tokens, lowest GPT-5.5 Pro at 0,07.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1Claude Opus 4.78277–8600,20,60,871 %0,9882 %
2GPT-5.5 Pro8075–84—0,30,50,777 %0,0776 %
3Claude Opus 4.87973–8500,20,40,674 %0,6879 %
4GPT-57872–83+50,20,60,880 %0,4674 %
5Gemini 3.1 Pro6154–68—0,40,20,649 %3,1373 %
6Kimi K2.65649–63—0,80,81,680 %0,2063 %
7Grok 4.35044–56+20,40,20,646 %4,4168 %
8GLM-4.64537–52—1,10,31,169 %0,4664 %
9Mistral Large 24435–53-21,20,71,877 %6,1767 %

Business law · 30 Items

Ranks 1–3 statistically tied (95 % CI overlaps rank 1)

In Business law, GPT-5.5 Pro (top score) contradicts a stored key fact on average 0,4 times per answer and adds avg 0,8 claims the judge panel flags as false or unsupported; 100 % of answers carry at least one flag. Relative to answer length: 0,11 per 1000 output tokens.

#Modeltop10095 % CIΔ prev.contradicts key factextra false/unsupp.avg totalanswers affectedper 1000 tok.IRR
1GPT-5.5 Pro8277–86—0,40,81,2100 %0,1175 %
2GPT-57668–84-40,50,81,393 %0,5768 %
3Claude Opus 4.77469–80-30,51,92,3100 %1,6969 %
4Claude Opus 4.86963–75+10,51,21,697 %1,1167 %
5Kimi K2.65952–65—0,81,92,7100 %0,3057 %
6Mistral Large 25649–63-30,81,32,193 %3,4251 %
7Gemini 3.1 Pro5144–58—0,90,71,587 %4,8562 %
8GLM-4.64944–56—0,51,11,697 %0,5059 %
9Grok 4.34942–56-10,50,71,287 %4,5157 %

Raw data of this run

raw.jsonl and progress.jsonl are the public streams with response texts stripped (contamination protection, methodology §7).