Doc2CI Benchmark

What is Doc2CI?

DOC2CI evaluates LLM-generated CI/CD configuration YAML against reference configurations from four CI services, measuring exact match and schema validity.

Released

2026-08-02

Evaluates

Code & Software, Software & AI Compute, cs.SE

Openness

unknown

Importer

Claire Radar

Review status

unreviewed

Reported scores

0

Source provenance

• Original evidence https://arxiv.org/abs/2608.01451

Doc2CI paper

• Paper https://arxiv.org/abs/2608.01451

Which sources cite Doc2CI?

• Doc2CI

Related Code & Software, Software & AI Compute, cs.SE benchmarks

• Active-SWE
• AFT-Bench
• Agent App Benchmark
• Agent Retrieval Bench
• Agent Systems Benchmark (ASB)
• Agentic Evaluation in 2D Mazes
• Agentic Security Benchmark
• AgentInfraBench