UnderSpecBench

What is UnderSpecBench?

UnderSpecBench evaluates coding agents on DevOps tasks under varying instruction underspecification, measuring action-boundary violations such as wrong-target…

Released

2026-07-02

Evaluates

Code & Software, General AI, cs.SE

Openness

unknown

Importer

Claire Radar

Review status

unreviewed

Reported scores

0

Source provenance

• Original evidence https://arxiv.org/abs/2607.02294

UnderSpecBench paper

• Paper https://arxiv.org/abs/2607.02294

Which sources cite UnderSpecBench?

• UnderSpecBench

Related Code & Software, General AI, cs.SE benchmarks

• $\tau^\tau$-Bench: An Environment for End-To-End, Realistic Agent Construction
• 3DCodeBench
• 5-Dialects-BN
• A3S-Bench
• AARRI-Bench
• ActBench
• Active-SWE
• ActiveSciBench