Benchmark
Multi-Challenge
MultiChallenge is a realistic multi-turn conversation evaluation benchmark that challenges frontier LLMs across four key categories: instruction retention…
- Modality
- text
- Categories
- reasoning, communication
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 29
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| GPT-4.1 | OpenAI | 0.383 | 2025-04-14 | evidence |
| GPT-4.1 mini | OpenAI | 0.358 | 2025-04-14 | evidence |
| GPT-4.1 nano | OpenAI | 0.15 | 2025-04-14 | evidence |
| GPT-4.5 | OpenAI | 0.438 | 2025-02-27 | evidence |
| GPT-4o | OpenAI | 0.403 | 2024-08-06 | evidence |
| GPT-5 | OpenAI | 0.696 | 2025-08-07 | evidence |
| Kimi K2 Instruct | Moonshot AI | 0.541 | 2025-07-11 | evidence |
| Kimi K2-Instruct-0905 | Moonshot AI | 0.541 | 2025-09-05 | evidence |
| MAI-Thinking-1 | Microsoft | 0.53 | 2026-06-02 | evidence |
| MiniMax M1 40K | MiniMax | 0.447 | 2025-06-16 | evidence |
| MiniMax M1 80K | MiniMax | 0.447 | 2025-06-16 | evidence |
| Nemotron 3 Nano (30B A3B) | NVIDIA | 0.385 | 2025-12-15 | evidence |
| Nemotron 3 Super (120B A12B) | NVIDIA | 0.5523 | 2026-03-11 | evidence |
| Nemotron 3 Ultra (550B A55B) | NVIDIA | 0.638 | 2026-06-04 | evidence |
| Nova 2 Lite | Amazon | 0.766 | 2025-12-02 | evidence |
| Nova 2 Omni | Amazon | 0.755 | 2025-12-02 | evidence |
| Nova 2 Pro | Amazon | 0.777 | 2025-12-02 | evidence |
| o3 | OpenAI | 0.604 | 2025-04-16 | evidence |
| o3-mini | OpenAI | 0.399 | 2025-01-30 | evidence |
| o4-mini | OpenAI | 0.43 | 2025-04-16 | evidence |
| Qwen3.5-0.8B | Qwen | 0.189 | 2026-03-02 | evidence |
| Qwen3.5-122B-A10B | Qwen | 0.615 | 2026-02-24 | evidence |
| Qwen3.5-27B | Qwen | 0.608 | 2026-02-24 | evidence |
| Qwen3.5-2B | Qwen | 0.337 | 2026-03-02 | evidence |
| Qwen3.5-35B-A3B | Qwen | 0.6 | 2026-02-24 | evidence |
| Qwen3.5-397B-A17B | Qwen | 0.676 | 2026-02-16 | evidence |
| Qwen3.5-4B | Qwen | 0.49 | 2026-03-02 | evidence |
| Qwen3.5-9B | Qwen | 0.545 | 2026-03-02 | evidence |
| Step3-VL-10B | StepFun | 0.626 | 2026-01-15 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.