SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Explore score history ↗

SWE-bench: Can Language Models Resolve Real-World GitHub Issues? is an AI benchmark in the Benchmark Radar catalog.

Released: 2023-10-11
Openness: unknown
Importer: Benchmark Saturation

Source provenance

Original evidence

SWE-bench: Can Language Models Resolve Real-World GitHub Issues? results and reported scores

Reported benchmark scores · Benchmark Saturation

Reported scores: 5

Model	Organization	Reported value	Reported	Evidence
Claude 4 Opus	—	67.6	—	evidence
Claude 4 Sonnet	—	64.93	—	evidence
Claude 4.5 Sonnet	—	70.6	—	evidence
GPT-5	—	65.0	—	evidence
GPT-5 mini	—	59.8	—	evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.

Which sources cite SWE-bench: Can Language Models Resolve Real-World GitHub Issues??