Skip to main content
Evaluation sources/BullshitBench v2
ParticipatingKnowledge & language

BullshitBench v2

BullshitBench · Public model evaluation results from BullshitBench v2.

Official evaluation
Source description

This source measures concrete delivery performance in Knowledge & language; raw scores are shown separately from the normalized capability score.

Limitation: Different versions, reasoning settings, and test sets may change scores; read them with the data date.

How it is used

Participating sources affect capability scores; reference, cross-reference, and observing sources are for comparison and verification only.

Model results from this source

Claude Opus 4.80.9580.12026-11-05
Qwen3.8 Max0.9580.12026-11-05
Claude Sonnet 4.60.9177.82026-11-05
Claude Opus 4.50.977.32026-11-05
Claude Opus 4.60.8775.52026-11-05
Claude Opus 4.70.8373.22026-11-05
Claude Sonnet 50.871.52026-11-05
Claude Sonnet 4.50.7970.92026-11-05
Qwen3.5 397B-A17B0.7870.32026-11-05
Claude Haiku 4.50.7769.82026-11-05
Claude Fable 5.10.7769.82026-11-05
Kimi K30.7468.02026-11-05
Claude Opus 50.7367.52026-11-05
Qwen3.6 Plus0.7266.92026-11-05
Qwen3.7 Max0.7266.92026-11-05
GLM 5.30.7266.92026-11-05
GPT-6 Astra0.6965.22026-11-05
Gemini 3.5 Flash Lite0.6663.42026-11-05
Grok 4.60.6663.42026-11-05
Gemini 4 Argon0.6663.42026-10-18
Gemini 4 Argon0.6663.42026-11-05
Gemini 4 Argon0.6663.42026-10-17
Kimi K2.60.6562.92026-11-05
GPT-6.1 Sol0.6562.92026-11-05
MiniMax-M30.6462.32026-11-05
Qwen3 Max Thinking0.6361.72026-11-05
Claude Opus 5.50.6361.72026-11-05
MiMo-V2.5-Pro0.6261.12026-11-05
Claude Sonnet 5.50.660.02026-11-05
GPT-6 Sol0.5959.42026-11-05
GPT-6 Luna0.5959.42026-11-05
DeepSeek V4.1 Flash0.5758.32026-11-05
Claude Haiku 5.50.5758.32026-10-18
Claude Haiku 5.50.5758.32026-11-05
Claude Haiku 5.50.5758.32026-10-17
Muse Wonder0.5758.32026-11-05
Muse Wonder0.5758.32026-10-17
Muse Wonder0.5758.32026-10-18
Claude Fable 50.5657.72026-11-05
Grok 4.50.5557.12026-11-05
GPT-5.6 Terra0.5456.52026-11-05
Kimi K2.50.5255.42026-11-05
Grok 4.30.554.22026-11-05
Muse Spark 1.20.554.22026-11-05
Nemotron 3 Ultra 550B A55B0.4953.72026-11-05
Gemini 3 Pro Preview0.4853.12026-11-05
GPT-5.40.4853.12026-11-05
GPT-5.6 Sol0.4853.12026-11-05
Nemotron 3 Super 120B A12B0.4752.52026-11-05
GPT-5.50.4752.52026-11-05
GPT-5.6 Luna0.4149.02026-11-05
Gemini 3.6 Flash0.3947.92026-11-05
DeepSeek V4 Flash 07310.3947.92026-11-05
GPT-5.20.3847.32026-11-05
Gemini 3.1 Pro Preview0.3746.72026-11-05
GPT-5.5 Pro0.3646.22026-11-05
MiMo-V2.50.3545.62026-11-05
DeepSeek V4 Pro 08130.3545.62026-11-05
GPT-5.4 mini0.3243.92026-11-05
GLM-5-Turbo0.3143.32026-11-05
GLM-5.20.3143.32026-11-05
Llama 4 Maverick0.2942.12026-11-05
GLM-50.2841.62026-11-05
o30.2640.42026-11-05
Gemma 4 31B IT0.2539.82026-11-05
Step 3.5 Flash0.2238.12026-11-05
GLM-5.10.2238.12026-11-05
Laguna S 2.10.2238.12026-11-05
GPT-50.2137.52026-11-05
Gemma 4 26B A4B IT0.2137.52026-11-05
Llama 4 Scout0.237.02026-11-05
Gemini 2.5 Pro0.237.02026-11-05
Qwen3 Coder 480B A35B0.237.02026-11-05
Gemini 2.5 Flash0.1936.42026-11-05
DeepSeek V4 Flash0.1835.82026-11-05
Trinity Large Thinking0.1735.22026-11-05
MiMo-V2-Flash0.1634.72026-11-05
Llama 3.1 8B Instruct0.1433.52026-11-05
DeepSeek V4 Pro0.1433.52026-11-05
DeepSeek V3.20.1332.92026-11-05
gpt-oss-120b0.1232.42026-11-05
ERNIE 4.5 VL 424B A47B0.1131.82026-11-05
Seed 1.60.1131.82026-11-05
Gemini 3.1 Flash Lite Preview0.1131.82026-11-05
Gemini 3 Flash Preview0.131.22026-11-05
MiniMax-M2.50.0930.62026-11-05
DeepSeek-R10.0830.12026-11-05
o4-mini0.0830.12026-11-05
GLM-4.50.0830.12026-11-05
MiniMax-M2.70.0830.12026-11-05
Qwen3 235B-A22B0.0628.92026-11-05
Mistral Small 40.0628.92026-11-05
Jamba Large 1.70.0528.32026-11-05
Gemma 3 27B0.0327.22026-11-05
Mistral Large 30.0226.62026-11-05

Frequently asked questions