Benchmark Radar™
Star —

MMStar Benchmark

Explore score history ↗

MMStar is an elite vision-indispensable multimodal benchmark comprising 1,500 challenge samples meticulously selected by humans to evaluate 6 core capabilities and 18 detailed axes. The benchmark addresses issues of visual content unnecessity and unintentional data leakage in existing multimodal evaluations.

Publisher
Shanghai AI Laboratory (published the hub card)
Released
2024-04-09
Modality
multimodal
Evaluates
multimodal, reasoning, general, vision
Openness
restricted
Importer
LLM Stats

Source provenance

MMStar paper, code and dataset

MMStar results and reported scores

Reported benchmark scores · LLM Stats

Reported scores: 24

ModelOrganizationReported valueReportedEvidence
DeepSeek VL2DeepSeek0.6132024-12-13evidence
DeepSeek VL2 SmallDeepSeek0.572024-12-13evidence
DeepSeek VL2 TinyDeepSeek0.4592024-12-13evidence
LFM2.5-VL-3BLiquid AI0.6332026-08-12evidence
North Micro Vision InstructCohere0.5182026-08-12evidence
Qwen2.5 VL 32B InstructQwen0.6952025-02-28evidence
Qwen2.5 VL 72B InstructQwen0.7082025-01-26evidence
Qwen2.5 VL 7B InstructQwen0.6392025-01-26evidence
Qwen2.5-Omni-7BQwen0.642025-03-27evidence
Qwen3 VL 235B A22B InstructQwen0.7842025-09-22evidence
Qwen3 VL 235B A22B ThinkingQwen0.7872025-09-22evidence
Qwen3 VL 30B A3B InstructQwen0.7212025-09-22evidence
Qwen3 VL 30B A3B ThinkingQwen0.7552025-09-22evidence
Qwen3 VL 32B InstructQwen0.7772025-09-22evidence
Qwen3 VL 32B ThinkingQwen0.7942025-09-22evidence
Qwen3 VL 4B InstructQwen0.6982025-09-22evidence
Qwen3 VL 4B ThinkingQwen0.7322025-09-22evidence
Qwen3 VL 8B InstructQwen0.7092025-09-22evidence
Qwen3 VL 8B ThinkingQwen0.7532025-09-22evidence
Qwen3.5-122B-A10BQwen0.8292026-02-24evidence
Qwen3.5-27BQwen0.812026-02-24evidence
Qwen3.5-35B-A3BQwen0.8192026-02-24evidence
Qwen3.6 PlusQwen0.8332026-04-02evidence
Qwen3.6-27BQwen0.8142026-04-21evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.

Which sources cite MMStar?

Related benchmarks