CMMLU Benchmark
Explore score history ↗CMMLU is a comprehensive Chinese evaluation benchmark specifically designed to assess the knowledge and reasoning abilities of language models in the context of the Chinese language. CMMLU covers 67 topics ranging from basic subjects to advanced professional levels. It includes tasks that require calculations and reasoning in natural sciences, as well as tasks involving knowledge from humanities, social sciences, and practical aspects like Chinese driving rules. Moreover, many tasks within CMMLU have answers specific to China, which might not be universally applicable in other regions or languages. As a result, CMMLU serves as a fully localized Chinese evaluation benchmark.
- Released
- 2023-06-15
- Evaluates
- 学科, Examination, 大语言模型, LLM, 知识储备, Knowledge
- Openness
- unknown
- Importer
- OpenCompass Hub
Source provenance
CMMLU paper and code
No reported scores are on record for this benchmark yet.