Benchmark
NL2Repo
Evaluates generating a repository from a natural language description. Temperature=1.0, top_p=1.0, max_new_tokens=64k under 1M context.
- Released
- 2025-06-15
- Categories
- coding_agent
- Openness
- unknown
- Source
- model_reports
- Reported scores
- 2
Reported scores
model_reports
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| GLM-5.3-Flash | Z.ai | 56.3 | 2026-08-27 | evidence |
| Hy4 preview | Tencent | 58.9 | 2026-08-28 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.