Benchmark Radar™
Star —

CMMLU Benchmark

Explore score history ↗

CMMLU is a comprehensive Chinese evaluation benchmark specifically designed to assess the knowledge and reasoning abilities of language models in the context of the Chinese language. CMMLU covers 67 topics ranging from basic subjects to advanced professional levels. It includes tasks that require calculations and reasoning in natural sciences, as well as tasks involving knowledge from humanities, social sciences, and practical aspects like Chinese driving rules. Moreover, many tasks within CMMLU have answers specific to China, which might not be universally applicable in other regions or languages. As a result, CMMLU serves as a fully localized Chinese evaluation benchmark.

Released
2023-06-15
Evaluates
学科, Examination, 大语言模型, LLM, 知识储备, Knowledge
Openness
unknown
Importer
OpenCompass Hub

Source provenance

CMMLU paper and code

No reported scores are on record for this benchmark yet.

Which sources cite CMMLU?

Related 学科, Examination, 大语言模型, LLM, 知识储备, Knowledge benchmarks