SA-Bench
Explore score history ↗SA-Bench (SemanticAlign-Bench) evaluates semantic alignment in LLM-based paper reproduction across 30 papers from top conferences. It decomposes paper specifications into atomic verifiable claims (SAUs) and evaluates repositories along four diagnostic dimensions (numerical, methodological, protocol, ordering drift). Includes 1,491 SAUs across five ML domains and evaluates 12 generator configurations.
- Released
- 2026-08-25
- Evaluates
- General AI, Language & Knowledge, Safety
- Openness
- unknown
- Importer
- Claire Radar
- Review status
- ai-reviewed
SA-Bench paper
No reported scores are on record for this benchmark yet.