Benchmark

MMMU-Pro

Visual reasoning

Categories
multimodal, reasoning
Openness
unknown
Source
artificial_analysis
Reported scores
245

Reported scores

artificial_analysis

ModelOrganizationReported valueReportedEvidence
Apriel-v1.5-15B-ThinkerServiceNow0.5705202312138732025-09-30evidence
Claude 3 HaikuAnthropic0.3080924855491332024-03-04evidence
Claude 3.5 HaikuAnthropic0.456069364161852024-10-22evidence
Claude 3.7 Sonnet (Non-reasoning)Anthropic0.6005780346820812025-02-24evidence
Claude 4 Sonnet (Non-reasoning)Anthropic0.6236994219653182025-05-22evidence
Claude 4 Sonnet (Reasoning)Anthropic0.6179190751445092025-05-22evidence
Claude 4.1 Opus (Reasoning)Anthropic0.6791907514450872025-08-05evidence
Claude 4.5 Haiku (Non-reasoning)Anthropic0.5514450867052022025-10-15evidence
Claude 4.5 Haiku (Reasoning)Anthropic0.5855491329479772025-10-15evidence
Claude 4.5 Sonnet (Non-reasoning)Anthropic0.6520231213872832025-09-29evidence
Claude 4.5 Sonnet (Reasoning)Anthropic0.687283236994222025-09-29evidence
Claude Opus 4.5 (Non-reasoning)Anthropic0.7121387283236992025-11-24evidence
Claude Opus 4.5 (Reasoning)Anthropic0.7404624277456652025-11-24evidence
Claude Opus 4.6 (Adaptive Reasoning, Max Effort)Anthropic0.7543352601156072026-02-05evidence
Claude Opus 4.6 (Non-reasoning, High Effort)Anthropic0.7254335260115612026-02-05evidence
Claude Opus 4.7 (Adaptive Reasoning, Max Effort)Anthropic0.7884393063583822026-04-16evidence
Claude Opus 4.7 (Non-reasoning, High Effort)Anthropic0.7635838150289022026-04-16evidence
Claude Opus 5 (Adaptive Reasoning, High Effort)Anthropic0.8242774566473992026-07-24evidence
Claude Opus 5 (Adaptive Reasoning, Low Effort)Anthropic0.7982658959537572026-07-24evidence
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic0.8473988439306362026-07-24evidence
Claude Opus 5 (Adaptive Reasoning, Medium Effort)Anthropic0.8161849710982662026-07-24evidence
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort)Anthropic0.8398843930635842026-07-24evidence
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)Anthropic0.7329479768786132026-02-17evidence
Claude Sonnet 4.6 (Non-reasoning, High Effort)Anthropic0.7057803468208092026-02-17evidence
Claude Sonnet 4.6 (Non-reasoning, Low Effort)Anthropic0.6919075144508672026-02-17evidence
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic0.7728323699421972026-06-30evidence
Claude Sonnet 5 (Non-reasoning, High Effort)Anthropic0.719075144508672026-06-30evidence
Command A+Cohere0.6323699421965322026-05-20evidence
DeepSeek V4 Flash Vision (Reasoning, Max Effort)DeepSeek0.7479768786127172026-08-21evidence
Devstral Small 2Mistral0.4462427745664742025-12-09evidence
DiffusionGemma 26B A4BGoogle0.6653179190751442026-06-10evidence
Doubao Seed CodeByteDance Seed0.680924855491332025-11-11evidence
ERNIE 5.0 Thinking PreviewBaidu0.6456647398843932025-11-13evidence
EXAONE 4.5 33BLG AI Research0.6734104046242772026-04-09evidence
Gemini 1.5 Flash (Sep '24)Google0.4838150289017342024-09-24evidence
Gemini 1.5 Flash-8BGoogle0.3653179190751442024-10-03evidence
Gemini 1.5 Pro (Sep '24)Google0.550289017341042024-09-24evidence
Gemini 2.5 Flash (Non-reasoning)Google0.6549132947976882025-05-20evidence
Gemini 2.5 Flash (Reasoning)Google0.6907514450867052025-05-20evidence
Gemini 2.5 Flash Preview (Non-reasoning)Google0.6196531791907512025-04-17evidence
Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning)Google0.7023121387283242025-09-25evidence
Gemini 2.5 Flash Preview (Sep '25) (Reasoning)Google0.731213872832372025-09-25evidence
Gemini 2.5 Flash-Lite (Non-reasoning)Google0.5398843930635842025-06-17evidence
Gemini 2.5 Flash-Lite (Reasoning)Google0.5820809248554912025-06-17evidence
Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning)Google0.6341040462427752025-09-25evidence
Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning)Google0.649710982658962025-09-08evidence
Gemini 2.5 ProGoogle0.7491329479768792025-06-05evidence
Gemini 3 Flash Preview (Non-reasoning)Google0.7855491329479772025-12-17evidence
Gemini 3 Flash Preview (Reasoning)Google0.7994219653179192025-12-17evidence
Gemini 3 Pro Preview (high)Google0.8017341040462432025-11-18evidence
Gemini 3.1 Flash-LiteGoogle0.7554913294797692026-03-03evidence
Gemini 3.1 Pro PreviewGoogle0.8242774566473992026-02-19evidence
Gemini 3.5 Flash (high)Google0.8427745664739882026-05-19evidence
Gemini 3.5 Flash (medium)Google0.8387283236994222026-05-19evidence
Gemini 3.5 Flash (minimal)Google0.8011560693641622026-05-19evidence
Gemini 3.5 Flash-LiteGoogle0.7901734104046242026-07-21evidence
Gemini 3.6 Flash (high)Google0.8323699421965322026-07-21evidence
Gemini 3.7 Flash (high)Google0.8549132947976882026-08-13evidence
Gemini 3.7 Flash (low)Google0.8485549132947982026-08-13evidence
Gemini 3.7 Flash (medium)Google0.8473988439306362026-08-13evidence
Gemma 3 12B InstructGoogle0.375144508670522025-03-12evidence
Gemma 3 27B InstructGoogle0.4803468208092492025-03-12evidence
Gemma 3 4B InstructGoogle0.2994219653179192025-03-12evidence
Gemma 3n E4B InstructGoogle0.262427745664742025-06-26evidence
Gemma 4 12B (Non-reasoning)Google0.6196531791907512026-06-03evidence
Gemma 4 12B (Reasoning)Google0.6965317919075142026-06-03evidence
Gemma 4 26B A4B (Non-reasoning)Google0.6670520231213872026-04-02evidence
Gemma 4 26B A4B (Reasoning)Google0.6924855491329482026-04-02evidence
Gemma 4 31B (Non-reasoning)Google0.7034682080924862026-04-02evidence
Gemma 4 31B (Reasoning)Google0.7341040462427752026-04-02evidence
Gemma 4 E2B (Non-reasoning)Google0.4179190751445092026-04-02evidence
Gemma 4 E2B (Reasoning)Google0.4456647398843932026-04-02evidence
Gemma 4 E4B (Non-reasoning)Google0.5121387283236992026-04-03evidence
Gemma 4 E4B (Reasoning)Google0.5138728323699422026-04-03evidence
GLM 5V Turbo (Reasoning)Z.ai0.7277456647398842026-04-01evidence
GLM-4.5V (Non-reasoning)Z.ai0.4277456647398842025-08-11evidence
GLM-4.5V (Reasoning)Z.ai0.5046242774566472025-08-11evidence
GLM-4.6V (Non-reasoning)Z.ai0.4219653179190752025-12-08evidence
GLM-4.6V (Reasoning)Z.ai0.4855491329479772025-12-08evidence
GPT-4.1OpenAI0.6121387283236992025-04-14evidence
GPT-4.1 miniOpenAI0.587283236994222025-04-14evidence
GPT-4.1 nanoOpenAI0.4011560693641622025-04-14evidence
GPT-4o (Aug '24)OpenAI0.5630057803468212024-08-06evidence
GPT-4o miniOpenAI0.4150289017341042024-07-18evidence
GPT-5 (high)OpenAI0.7421965317919072025-08-07evidence
GPT-5 (low)OpenAI0.737572254335262025-08-07evidence
GPT-5 (medium)OpenAI0.7433526011560692025-08-07evidence
GPT-5 (minimal)OpenAI0.6208092485549132025-08-07evidence
GPT-5 Codex (high)OpenAI0.7381502890173412025-09-23evidence
GPT-5 mini (high)OpenAI0.7005780346820812025-08-07evidence
GPT-5 mini (medium)OpenAI0.6884393063583812025-08-07evidence
GPT-5 mini (minimal)OpenAI0.5838150289017342025-08-07evidence
GPT-5 nano (high)OpenAI0.6098265895953762025-08-07evidence
GPT-5 nano (medium)OpenAI0.5820809248554912025-08-07evidence
GPT-5 nano (minimal)OpenAI0.3179190751445092025-08-07evidence
GPT-5.1 (high)OpenAI0.7549132947976882025-11-13evidence
GPT-5.1 (Non-reasoning)OpenAI0.6236994219653182025-11-13evidence
GPT-5.1 Codex (high)OpenAI0.724855491329482025-11-13evidence
GPT-5.1 Codex mini (high)OpenAI0.6901734104046242025-11-13evidence
GPT-5.2 (medium)OpenAI0.7456647398843932025-12-11evidence
GPT-5.2 (Non-reasoning)OpenAI0.6583815028901732025-12-11evidence
GPT-5.2 Codex (xhigh)OpenAI0.7630057803468212025-12-11evidence
GPT-5.3 Codex (xhigh)OpenAI0.7849710982658962026-02-05evidence
GPT-5.4 (low)OpenAI0.7797687861271682026-03-05evidence
GPT-5.4 (Non-reasoning)OpenAI0.7057803468208092026-03-05evidence
GPT-5.4 (xhigh)OpenAI0.7843930635838152026-03-05evidence
GPT-5.4 mini (medium)OpenAI0.7115606936416182026-03-17evidence
GPT-5.4 mini (Non-Reasoning)OpenAI0.6046242774566472026-03-17evidence
GPT-5.4 mini (xhigh)OpenAI0.7329479768786132026-03-17evidence
GPT-5.4 nano (medium)OpenAI0.5953757225433532026-03-17evidence
GPT-5.4 nano (Non-Reasoning)OpenAI0.437572254335262026-03-17evidence
GPT-5.4 nano (xhigh)OpenAI0.6537572254335262026-03-17evidence
GPT-5.5 (high)OpenAI0.8109826589595382026-04-23evidence
GPT-5.5 (low)OpenAI0.7901734104046242026-04-23evidence
GPT-5.5 (medium)OpenAI0.8115606936416192026-04-23evidence
GPT-5.5 (Non-reasoning)OpenAI0.7138728323699422026-04-23evidence
GPT-5.5 (xhigh)OpenAI0.7988439306358382026-04-23evidence
GPT-5.6 Luna (high)OpenAI0.7757225433526012026-07-09evidence
GPT-5.6 Luna (low)OpenAI0.7398843930635842026-07-09evidence
GPT-5.6 Luna (max)OpenAI0.7855491329479772026-07-09evidence
GPT-5.6 Luna (medium)OpenAI0.7583815028901732026-07-09evidence
GPT-5.6 Luna (Non-reasoning)OpenAI0.6034682080924862026-07-09evidence
GPT-5.6 Luna (xhigh)OpenAI0.7855491329479772026-07-09evidence
GPT-5.6 Sol (high)OpenAI0.818497109826592026-07-09evidence
GPT-5.6 Sol (low)OpenAI0.8098265895953762026-07-09evidence
GPT-5.6 Sol (max)OpenAI0.8341040462427752026-07-09evidence
GPT-5.6 Sol (medium)OpenAI0.8138728323699422026-07-09evidence
GPT-5.6 Sol (Non-reasoning)OpenAI0.719075144508672026-07-09evidence
GPT-5.6 Sol (xhigh)OpenAI0.8265895953757232026-07-09evidence
GPT-5.6 Terra (high)OpenAI0.7907514450867052026-07-09evidence
GPT-5.6 Terra (low)OpenAI0.7612716763005782026-07-09evidence
GPT-5.6 Terra (max)OpenAI0.8069364161849712026-07-09evidence
GPT-5.6 Terra (medium)OpenAI0.7676300578034682026-07-09evidence
GPT-5.6 Terra (Non-reasoning)OpenAI0.6670520231213872026-07-09evidence
GPT-5.6 Terra (xhigh)OpenAI0.7947976878612722026-07-09evidence
Grok 4xAI0.6884393063583812025-07-10evidence
Grok 4 Fast (Non-reasoning)xAI0.4809248554913292025-09-19evidence
Grok 4 Fast (Reasoning)xAI0.6179190751445092025-09-19evidence
Grok 4.1 Fast (Non-reasoning)xAI0.4843930635838152025-11-19evidence
Grok 4.1 Fast (Reasoning)xAI0.6329479768786132025-11-19evidence
Grok 4.20 0309 (Non-reasoning)xAI0.6404624277456652026-03-10evidence
Grok 4.20 0309 (Reasoning)xAI0.7317919075144512026-03-10evidence
Grok 4.20 0309 v2 (Non-reasoning)xAI0.6491329479768792026-04-07evidence
Grok 4.20 0309 v2 (Reasoning)xAI0.7456647398843932026-04-07evidence
Grok 4.3 (high)xAI0.780924855491332026-04-30evidence
Grok 4.3 (low)xAI0.7277456647398842026-04-30evidence
Grok 4.3 (medium)xAI0.7583815028901732026-04-30evidence
Grok 4.3 (Non-reasoning)xAI0.6479768786127172026-04-30evidence
Grok 4.5 (high)xAI0.8040462427745662026-07-08evidence
Grok Build 0.1 0616xAI0.7653179190751442026-06-16evidence
Inkling (xhigh)Thinking Machines0.7346820809248562026-07-15evidence
Inkling SmallThinking Machines0.7404624277456652026-07-30evidence
JT-4.1 Flash 236B A21BChina Mobile0.6410404624277462026-07-09evidence
Kimi K2.5 (Non-reasoning)Moonshot AI0.7306358381502892026-01-27evidence
Kimi K2.5 (Reasoning)Moonshot AI0.7537572254335262026-01-27evidence
Kimi K2.6Moonshot AI0.793641618497112026-04-20evidence
Kimi K3 (low)Moonshot AI0.7849710982658962026-07-16evidence
Kimi K3 (max)Moonshot AI0.8052023121387282026-07-16evidence
LFM2.5-VL-1.6BLiquid AI0.2653179190751452026-01-05evidence
Llama 3.2 Instruct 11B (Vision)Meta0.2930635838150292024-09-25evidence
Llama 3.2 Instruct 90B (Vision)Meta0.3947976878612722024-09-25evidence
Llama 4 MaverickMeta0.6213872832369942025-04-05evidence
Llama 4 ScoutMeta0.5294797687861272025-04-05evidence
Magistral Medium 1.2Mistral0.5965317919075142025-09-18evidence
Magistral Small 1.2Mistral0.5549132947976882025-09-17evidence
MiMo-V2-OmniXiaomi0.6988439306358382026-03-19evidence
MiMo-V2-Omni-0327Xiaomi0.7393063583815032026-03-27evidence
MiMo-V2.5Xiaomi0.7543352601156072026-04-22evidence
MiniCPM-V 4.6 1.3BOpenBMB0.3791907514450872026-05-11evidence
MiniMax-M3MiniMax0.7855491329479772026-06-01evidence
Ministral 3 14BMistral0.4982658959537572025-12-02evidence
Ministral 3 3BMistral0.3809248554913292025-12-02evidence
Ministral 3 8BMistral0.4595375722543352025-12-02evidence
Mistral Large 3Mistral0.5566473988439312025-12-02evidence
Mistral Medium 3Mistral0.5300578034682082025-05-07evidence
Mistral Medium 3.1Mistral0.5416184971098272025-08-12evidence
Mistral Medium 3.5Mistral0.6485549132947982026-04-29evidence
Mistral Small 3.2Mistral0.4797687861271682025-06-20evidence
Mistral Small 4 (Non-reasoning)Mistral0.4647398843930642026-03-16evidence
Mistral Small 4 (Reasoning)Mistral0.5682080924855492026-03-16evidence
Molmo 7B-DAllen Institute for AI0.2450867052023122024-09-25evidence
Molmo2-8BAllen Institute for AI0.3745664739884392025-12-11evidence
Muse Glimmer (high)Meta0.7433526011560692026-08-10evidence
Muse SparkMeta0.8052023121387282026-04-08evidence
Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIA0.5317919075144512026-04-29evidence
Nova 2.0 Lite (high)Amazon0.637572254335262025-10-29evidence
Nova 2.0 Lite (low)Amazon0.5797687861271682025-10-29evidence
Nova 2.0 Lite (medium)Amazon0.6254335260115612025-10-29evidence
Nova 2.0 Lite (Non-reasoning)Amazon0.4901734104046242025-10-29evidence
Nova 2.0 Omni (low)Amazon0.5982658959537572025-11-26evidence
Nova 2.0 Omni (medium)Amazon0.6190751445086712025-11-26evidence
Nova 2.0 Omni (Non-reasoning)Amazon0.4988439306358382025-11-26evidence
Nova 2.0 Pro Preview (low)Amazon0.6271676300578042025-11-27evidence
Nova 2.0 Pro Preview (medium)Amazon0.6450867052023122025-11-27evidence
Nova LiteAmazon0.3780346820809252024-12-03evidence
Nova ProAmazon0.4433526011560692024-12-03evidence
NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)NVIDIA0.4450867052023122025-10-28evidence
NVIDIA Nemotron Nano 12B v2 VL (Reasoning)NVIDIA0.5289017341040462025-10-28evidence
o3OpenAI0.7005780346820812025-04-16evidence
o4-mini (high)OpenAI0.6924855491329482025-04-16evidence
Phi-4 Multimodal InstructMicrosoft0.1450867052023122025-02-26evidence
Pixtral LargeMistral0.5057803468208092024-11-18evidence
Qwen3 Omni 30B A3B (Reasoning)Qwen0.6023121387283242025-09-22evidence
Qwen3 Omni 30B A3B InstructQwen0.5549132947976882025-09-22evidence
Qwen3 VL 235B A22B (Reasoning)Qwen0.687283236994222025-09-23evidence
Qwen3 VL 235B A22B InstructQwen0.6757225433526012025-09-23evidence
Qwen3 VL 30B A3B (Reasoning)Qwen0.618497109826592025-10-03evidence
Qwen3 VL 30B A3B InstructQwen0.6213872832369942025-10-03evidence
Qwen3 VL 32B (Reasoning)Qwen0.6341040462427752025-10-21evidence
Qwen3 VL 32B InstructQwen0.6427745664739882025-10-21evidence
Qwen3 VL 4B (Reasoning)Qwen0.5202312138728322025-10-14evidence
Qwen3 VL 4B InstructQwen0.4387283236994222025-10-14evidence
Qwen3 VL 8B (Reasoning)Qwen0.5664739884393062025-10-14evidence
Qwen3 VL 8B InstructQwen0.4734104046242772025-10-14evidence
Qwen3.5 0.8B (Non-reasoning)Qwen0.2566473988439312026-03-02evidence
Qwen3.5 0.8B (Reasoning)Qwen0.2583815028901732026-03-02evidence
Qwen3.5 122B A10B (Non-reasoning)Qwen0.7028901734104052026-02-24evidence
Qwen3.5 122B A10B (Reasoning)Qwen0.749710982658962026-02-24evidence
Qwen3.5 27B (Non-reasoning)Qwen0.72026-02-24evidence
Qwen3.5 27B (Reasoning)Qwen0.750289017341042026-02-24evidence
Qwen3.5 2B (Non-reasoning)Qwen0.4265895953757232026-03-02evidence
Qwen3.5 2B (Reasoning)Qwen0.4300578034682082026-03-02evidence
Qwen3.5 35B A3B (Non-reasoning)Qwen0.6919075144508672026-02-24evidence
Qwen3.5 35B A3B (Reasoning)Qwen0.7265895953757232026-02-24evidence
Qwen3.5 397B A17B (Non-reasoning)Qwen0.5265895953757232026-02-16evidence
Qwen3.5 397B A17B (Reasoning)Qwen0.7728323699421972026-02-16evidence
Qwen3.5 4B (Non-reasoning)Qwen0.6208092485549132026-03-02evidence
Qwen3.5 4B (Reasoning)Qwen0.6537572254335262026-03-02evidence
Qwen3.5 9B (Non-reasoning)Qwen0.6676300578034682026-03-02evidence
Qwen3.5 9B (Reasoning)Qwen0.6924855491329482026-03-02evidence
Qwen3.5 Omni FlashQwen0.6473988439306362026-03-30evidence
Qwen3.5 Omni PlusQwen0.7052023121387282026-03-30evidence
Qwen3.6 27B (Non-reasoning)Qwen0.7173410404624282026-04-22evidence
Qwen3.6 27B (Reasoning)Qwen0.7462427745664742026-04-22evidence
Qwen3.6 35B A3B (Non-reasoning)Qwen0.7104046242774572026-04-16evidence
Qwen3.6 35B A3B (Reasoning)Qwen0.750289017341042026-04-16evidence
Qwen3.6 PlusQwen0.7797687861271682026-04-02evidence
Qwen3.7 PlusQwen0.8046242774566472026-06-01evidence
Qwen3.8 27B (low)Qwen0.737572254335262026-08-14evidence
Qwen3.8 27B (medium)Qwen0.7416184971098272026-08-14evidence
Qwen3.8 27B (Non-reasoning)Qwen0.6994219653179192026-08-14evidence
Qwen3.8 27B (xhigh)Qwen0.7630057803468212026-08-14evidence
Qwen3.8 MaxQwen0.8231213872832372026-08-03evidence
Step 3.7 FlashStepFun0.7531791907514452026-05-29evidence
Step3 VL 10BStepFun0.6398843930635842026-01-20evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.