Benchmark Radar™
Star —

ORCA-bench

Explore score history ↗

ORCA-bench evaluates language model agents on root cause analysis in a production-fidelity oncall setting, using a live OpenTelemetry-instrumented microservice system with 1,079 tasks varying in report specificity, time-to-detection, and fault co-occurrence. Agents access metrics, logs, traces, and source code via real telemetry interfaces.

Released
2026-07-30
Institutions
Cornell Tech · Columbia University
Evaluates
Code & Software, Software & AI Compute, cs.CL
Openness
unknown
Importer
Claire Radar
Review status
unreviewed

ORCA-bench paper and code

No reported scores are on record for this benchmark yet.

Which sources cite ORCA-bench?

Related Code & Software, Software & AI Compute, cs.CL benchmarks