MTM-Bench
Explore score history ↗MTM-Bench is a controlled benchmark for language-conditioned task execution in multilingual settings, enumerating all 27 instruction-content-response language triplets across English, Spanish, and Chinese. It contains 2,430 instances per model across semantic reversal, final-state extraction, and language purity tasks, with decomposed metrics for semantic correctness, language adherence, constraint satisfaction, and joint success.
- Released
- 2026-05-26
- Institutions
- University of Pennsylvania · Cornell University · University of Illinois Urbana-Champaign
- Evaluates
- General AI, Language & Knowledge, cs.CL
- Openness
- unknown
- Importer
- Claire Radar
- Review status
- unreviewed
MTM-Bench paper
No reported scores are on record for this benchmark yet.