Skip to main content
Evaluation sources/SWE-Bench Pro
ObservingCoding

SWE-Bench Pro

Scale AI · Public model evaluation results from SWE-Bench Pro.

Official evaluation
Source description

This source measures concrete delivery performance in Coding; raw scores are shown separately from the normalized capability score.

Limitation: Different versions, reasoning settings, and test sets may change scores; read them with the data date.

How it is used

Participating sources affect capability scores; reference, cross-reference, and observing sources are for comparison and verification only.

Model results from this source

Claude Fable 580.380.32026-06-09
Claude Opus 4.869.269.22026-05-28
Grok 4.564.764.72026-07-08
GPT-5.6 Sol64.664.62026-07-09
Claude Opus 4.764.364.32026-05-28
GPT-5.6 Terra63.463.42026-07-09
Claude Sonnet 563.263.22026-06-30
GPT-5.6 Luna62.762.72026-07-09
Ornith 1.0 397B62.262.22026-08-05
GLM-5.262.162.12026-06-16
Muse Spark 1.161.561.52026-07-09
Qwen3.7 Max60.660.62026-05-19
LongCat-2.059.559.52026-06-30
GPT-5.459.159.12026-08-05
MiniMax-M35959.02026-06-01
GPT-5.558.658.62026-05-28
MiMo-V2.5-Pro57.257.22026-04-22
Step 3.7 Flash56.356.32026-05-29
MiniMax-M2.756.256.22026-06-01
Gemini 3.1 Pro Preview54.254.22026-05-28
Claude Opus 4.651.951.92026-08-05
MAI-Code-1-Flash51.251.22026-06-02
Ornith 1.0 35B50.450.42026-08-05
Laguna XS 2.147.647.62026-07-02
Gemini 3.1 Pro Preview46.146.12026-08-05
Claude Opus 4.545.8945.92026-08-05
Claude Opus 4.744.944.92026-08-05
Claude Sonnet 4.543.643.62026-08-05
Gemini 3 Pro Preview43.343.32026-08-05
Ornith 1.0 9B42.942.92026-08-05
GPT-541.7841.82026-08-05
North Mini Code40.240.22026-06-09
Claude Haiku 4.539.4539.52026-08-05
Qwen3-Coder 480B-A35B Instruct38.738.72026-08-05
MiniMax-M2.136.8136.82026-08-05
Claude Opus 4.736.436.42026-08-05
Gemini 3 Flash Preview34.6334.62026-08-05
Claude Opus 4.734.434.42026-08-05
GPT-5.530.930.92026-08-05
GPT-5.229.9429.92026-08-05
Kimi K2.627.327.32026-08-05
GPT-5.526.226.22026-08-05
GPT-5.524.924.92026-08-05
Qwen3 235B-A22B21.4121.42026-08-05
GLM-5.119.819.82026-08-05
GPT-5.418.918.92026-08-05
GPT-5.418.418.42026-08-05
DeepSeek V4 Pro1818.02026-08-05
Gemini 3.1 Pro Preview15.115.12026-08-05
Claude Sonnet 4.614.914.92026-08-05
Claude Opus 4.611.811.82026-08-05
GLM-4.69.679.72026-08-05
Llama 4 Maverick 17B Instruct5.245.22026-08-05

Frequently asked questions