Benchmark
Video-MME
Video-MME is the first-ever comprehensive evaluation benchmark of Multi-modal Large Language Models (MLLMs) in video analysis. It features 900 videos…
- Modality
- multimodal
- Categories
- multimodal, reasoning, vision
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 17
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Gemini 1.5 Flash | 0.761 | 2024-05-01 | evidence | |
| Gemini 1.5 Flash 8B | 0.662 | 2024-03-15 | evidence | |
| Gemini 1.5 Pro | 0.786 | 2024-05-01 | evidence | |
| Gemini 2.5 Pro | 0.848 | 2025-05-20 | evidence | |
| Kimi K2.5 | Moonshot AI | 0.874 | 2026-01-27 | evidence |
| MiMo-V2.5 | Xiaomi | 0.877 | 2026-04-22 | evidence |
| MiniMax M3 | MiniMax | 0.854 | 2026-06-01 | evidence |
| Nova 2 Omni | Amazon | 0.779 | 2025-12-02 | evidence |
| Phi-4-multimodal-instruct | Microsoft | 0.55 | 2025-02-01 | evidence |
| Qwen3 VL 30B A3B Instruct | Qwen | 0.745 | 2025-09-22 | evidence |
| Qwen3 VL 30B A3B Thinking | Qwen | 0.733 | 2025-09-22 | evidence |
| Qwen3 VL 8B Instruct | Qwen | 0.714 | 2025-09-22 | evidence |
| Qwen3 VL 8B Thinking | Qwen | 0.718 | 2025-09-22 | evidence |
| Qwen3.6 Plus | Qwen | 0.842 | 2026-04-02 | evidence |
| Qwen3.7-Plus | Qwen | 0.88 | 2026-05-31 | evidence |
| Seed 2.1 Pro | ByteDance | 0.892 | 2026-06-24 | evidence |
| Seed 2.1 Turbo | ByteDance | 0.89 | 2026-06-24 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.