Skip to main content
Evaluation sources/AutomationBench-AA
ParticipatingProfessional work

AutomationBench-AA

Artificial Analysis · Public model evaluation results from AutomationBench-AA.

Official evaluation
Source description

This source measures concrete delivery performance in Professional work; raw scores are shown separately from the normalized capability score.

Limitation: Different versions, reasoning settings, and test sets may change scores; read them with the data date.

How it is used

Participating sources affect capability scores; reference, cross-reference, and observing sources are for comparison and verification only.

Model results from this source

Gemini 4 Argon0.7751346976.32026-11-05
MiniMax M3.1 Flash0.725218173.52026-11-05
Claude Sonnet 5.50.7176882173.12026-11-05
Muse Wonder Full0.713307572.92026-10-18
Muse Wonder Full0.713307572.92026-10-17
Muse Wonder Full0.713307572.92026-11-05
Claude Opus 5.50.6953868171.92026-11-05
DeepSeek V4.1 Flash0.6889097871.52026-11-05
DeepSeek V4.1 Flash 10170.6849296571.32026-11-05
GPT-6 Astra0.6849174871.32026-11-05
Muse Wonder0.6809455171.12026-10-18
Muse Wonder0.6809455171.12026-11-05
Muse Wonder0.6809455171.12026-10-17
Ling-3.1-flash-10190.6754660770.82026-11-05
Grok 4.60.6699736870.52026-11-05
DeepSeek V4.1 Flash 10170.6664095470.32026-10-31
GPT-6.1 Sol0.6657372170.32026-11-05
Grok 4.70.6555835169.72026-11-05
MiMo V2.6 Flash0.6405168368.92026-10-18
MiMo V2.6 Flash0.6405168368.92026-11-05
MiMo V2.6 Flash0.6405168368.92026-10-17
GLM 5.30.6220286567.82026-11-05
Ling-3.1-flash0.6174373467.62026-11-05
Ling-3.1-flash0.6174373467.62026-10-18
Ling-3.1-flash0.6174373467.62026-10-17
GPT-6 Sol0.6169423567.52026-11-05
Gemini 3.8 Flash0.6088997467.12026-11-05
GLM-5.3 Flash0.6036862866.82026-10-18
GLM-5.3 Flash0.6036862866.82026-11-05
GLM-5.3 Flash0.6036862866.82026-10-17
GPT-6.1 Terra0.6016762366.72026-11-05
GPT-5.6 Sol0.600811566.72026-11-05
Mistral Large 40.5990136266.62026-10-28
GPT-5.6 Terra0.5964995866.42026-11-05
Claude Fable 5.10.5937591766.32026-11-05
MiMo V2.6 Pro0.5862340965.82026-11-05
Kimi K30.5827297865.72026-11-05
Grok 4.50.579348665.52026-11-05
Muse Spark 1.30.5785343865.42026-11-05
Mistral Large 40.5783010565.42026-11-05
DeepSeek V4 Pro 08130.5671165564.82026-11-05
Claude Opus 50.5657355664.72026-11-05
DeepSeek V4 Pro0.5628968664.62026-11-05
Claude Fable 50.5407130163.32026-11-05
DeepSeek V4 Flash 07310.5396771963.32026-11-05
GPT-6 Luna0.531681462.82026-11-05
Gemini 3.6 Flash0.5301669562.72026-11-05
Step 5 Preview0.5095659461.62026-11-05
GPT-5.6 Luna0.5020861861.22026-11-05
Qwen3.8 Max0.4923210460.62026-11-05
GPT-5.50.4730056659.62026-11-05
Claude Opus 4.80.4559343758.62026-11-05
Muse Spark 1.20.4061337855.92026-11-05
Muse Spark 1.10.3884891654.92026-11-05
Claude Sonnet 50.3651267653.62026-11-05
Claude Haiku 5.50.359538953.32026-11-05
Gemini 3.1 Pro Preview0.3542771853.02026-11-05
GLM-5.20.2839897749.12026-11-05
GPT-5.4 mini0.2620488747.92026-11-05
Gemini 3.5 Flash Lite0.2501127147.22026-11-05
Kimi K2.7 Code0.2445430546.92026-11-05
Qwen3.7 Max0.229768946.12026-11-05
MiniMax-M30.2125125845.22026-11-05
GLM-5.10.2030835544.62026-11-05
Claude Sonnet 4.60.2009807444.52026-11-05
MiMo-V2.50.1842951243.62026-11-05
Hy30.1840912843.62026-11-05
Nex-N2-Pro0.1762911743.12026-11-05
Qwen3.7 Plus0.1739122143.02026-11-05
Claude Sonnet 4.50.1407574341.22026-11-05
MiMo-V2.5-Pro0.1370361741.02026-11-05
Kimi K2.60.130207640.62026-11-05
Qwen3.6 27B0.1104522439.52026-11-05
Qwen3.5 397B-A17B0.068866737.22026-11-05
Gemini 3.1 Flash Lite0.0679618237.12026-11-05
GPT-5 mini0.0648746637.02026-11-05
Mistral Medium 3.50.0629964936.92026-11-05
Qwen3.5 122B-A10B0.0548111636.42026-11-05
Qwen3.6 35B-A3B0.0520856636.32026-11-05
Inkling0.049848936.12026-11-05
Inkling Small0.0481422836.02026-11-05
LongCat-2.00.0453391335.92026-11-05
MiniMax-M2.70.044352935.82026-11-05
Nemotron 3 Super 120B A12B0.0377903135.52026-11-05
Claude Haiku 4.50.0324863135.22026-11-05
Ling-3.0-flash (free)0.0321964935.22026-11-05
Nemotron 3 Ultra 550B A55B0.029863535.02026-11-05
DeepSeek V3.1 Terminus0.0270627334.92026-11-05
Gemini 2.5 Pro0.0217494634.62026-11-05
Command A Plus0.0181045434.42026-11-05
Mercury 20.0167977734.32026-11-05
Mistral Large 30.014974734.22026-11-05
North Mini Code0.0149128634.22026-11-05
Grok 4.30.0137596534.12026-11-05
Trinity Large Thinking0.0130692334.12026-11-05
Qwen3 Coder Next0.0105935534.02026-11-05
Qwen3.5 9B0.0104738734.02026-11-05
Nemotron 3 Nano 30B A3B0.0097146933.92026-11-05
Qwen3 32B0.0097112933.92026-11-05
Ring-2.6-1T0.0071786633.82026-11-05
Gemma 4 12B Base0.0068557233.82026-11-05
Ministral 3 3B0.0066586633.72026-11-05
Ministral 3 14B0.0055121333.72026-11-05
o3-mini0.0053018333.72026-10-11
o3 Mini High0.0053018333.72026-11-05
Ministral 3 8B0.0051674933.72026-11-05
Solar Pro 30.0049734233.72026-11-05
Qwen3 8B0.0033260733.62026-11-05
DeepSeek V3 03240.0027298633.52026-11-05
DeepSeek-R10.0024185333.52026-11-05
Llama 4 Maverick0.0021219233.52026-11-05
gpt-oss-120b0.001990733.52026-11-05
Llama 4 Scout0.0019111733.52026-11-05
Qwen3 14B0.0019111733.52026-11-05
gpt-oss-20b0.0019111733.52026-11-05

Frequently asked questions