Benchmark Radar™
Star —

NL2Scratch Benchmark

Explore score history ↗

NL2Scratch is an executable benchmark for natural-language-to-Scratch generation, consisting of 311,648 parser-valid NL-program pairs extracted from real Scratch projects. It includes a semantically validated pool of 23,594 examples and an 800-example diagnostic set. Evaluation uses Semantic Alignment Consistency (SAC), an interpretable slot-level metric for measuring semantic agreement.

Released
2026-06-20
Institutions
ETH Zurich
Evaluates
Code & Software, Code generation, General AI, cs.CL
Openness
unknown
Importer
Claire Radar
Review status
unreviewed

NL2Scratch paper

No reported scores are on record for this benchmark yet.

Which sources cite NL2Scratch?

Related Code & Software, Code generation, General AI, cs.CL benchmarks