Benchmark
Video-MME Benchmark
What is Video-MME?
Video-MME is the first-ever comprehensive evaluation benchmark of Multi-modal Large Language Models (MLLMs) in video analysis. It features 900 videos totaling…
- Released
- 2024-05-31
- Modality
- multimodal
- Evaluates
- multimodal, reasoning, vision
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 17
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/video-mme?top_n=500
Video-MME results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Gemini 1.5 Flash | 0.761 | 2024-05-01 | evidence | |
| Gemini 1.5 Flash 8B | 0.662 | 2024-03-15 | evidence | |
| Gemini 1.5 Pro | 0.786 | 2024-05-01 | evidence | |
| Gemini 2.5 Pro | 0.848 | 2025-05-20 | evidence | |
| Kimi K2.5 | Moonshot AI | 0.874 | 2026-01-27 | evidence |
| MiMo-V2.5 | Xiaomi | 0.877 | 2026-04-22 | evidence |
| MiniMax M3 | MiniMax | 0.854 | 2026-06-01 | evidence |
| Nova 2 Omni | Amazon | 0.779 | 2025-12-02 | evidence |
| Phi-4-multimodal-instruct | Microsoft | 0.55 | 2025-02-01 | evidence |
| Qwen3 VL 30B A3B Instruct | Qwen | 0.745 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Thinking | Qwen | 0.733 | 2025-09-22 | evidence |
| Qwen3 VL 8B Instruct | Qwen | 0.714 | 2025-09-22 | evidence |
| Qwen3 VL 8B Thinking | Qwen | 0.718 | 2025-09-22 | evidence |
| Qwen3.6 Plus | Qwen | 0.842 | 2026-04-02 | evidence |
| Qwen3.7-Plus | Qwen | 0.88 | 2026-05-31 | evidence |
| Seed 2.1 Pro | ByteDance | 0.892 | 2026-06-24 | evidence |
| Seed 2.1 Turbo | ByteDance | 0.89 | 2026-06-24 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.