Devin + SWE-2 Reasoning Effort Benchmark

What is Devin + SWE-2 Reasoning Effort Benchmark?

A paired software-repair evaluation comparing Devin's SWE-2 Medium and Max modes on 30 unique bugs across three difficulty tiers, with hidden tests and a clean…

Released

2026-09-16

Evaluates

Agents, General AI, Language & Knowledge

Openness

unknown

Importer

Claire Radar

Review status

ai-reviewed

Reported scores

0

Source provenance

• Original evidence https://github.com/MAJORminorStudio/devin-agent-benchmark

Devin + SWE-2 Reasoning Effort Benchmark code

• Code https://github.com/MAJORminorStudio/devin-agent-benchmark
• Website https://majorminor.xyz/blog/value-of-more-reasoning-changed-with-difficulty

Which sources cite Devin + SWE-2 Reasoning Effort Benchmark?

• Devin + SWE-2 Reasoning Effort Benchmark

Related Agents, General AI, Language & Knowledge benchmarks

• $\tau^\tau$-Bench: An Environment for End-To-End, Realistic Agent Construction
• 2d-fet-bench
• 3DCodeBench
• 5-Dialects-BN
• A Large-Scale Dataset and Benchmark
• A Real-World CAD Segmentation Benchmark
• A3S-Bench
• AARRI-Bench