Benchmark

MMMU-Pro Benchmark

What is MMMU-Pro?

Harder vision-required subset of MMMU. Input image ordering changes the result, so the protocol has to travel with the number.

Released: 2024-09-04
Evaluates: multimodal
Openness: unknown
Importer: Model reports
Reported scores: 5

Source provenance

Original evidence https://arxiv.org/abs/2409.02813

MMMU-Pro paper

Paper https://arxiv.org/abs/2409.02813

MMMU-Pro results and reported scores

Model reports

Model	Organization	Reported value	Reported	Evidence
Gemini 3.1 Pro	Google	80.5	2026-02-19	evidence
Gemma 4 (31B)	Google	76.9	2026-03-11	evidence
Kimi K3	Moonshot AI	83.4	2026-06-13	evidence
Kimi K3	Moonshot AI	81.6	2026-06-13	evidence
Qwen3.5-397B-A17B	Qwen	79.0	2026-02-16	evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.

Which sources cite MMMU-Pro?

Kimi K3 Moonshot AI, 2026-06-13
Gemma 4 (31B) Google, 2026-03-11
Gemini 3.1 Pro Google, 2026-02-19
Qwen3.5-397B-A17B Qwen, 2026-02-16