Benchmark
LiveCodeBench
What is LiveCodeBench?
LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from…
- Released
- 2024-03-12
- Modality
- text
- Evaluates
- reasoning, general, code
- Openness
- unknown
- Importer
- LLM Stats
- Reported scores
- 75
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/livecodebench?top_n=500
LiveCodeBench paper and code
LiveCodeBench results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| DeepSeek R1 Distill Llama 70B | DeepSeek | 0.575 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Llama 8B | DeepSeek | 0.396 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 1.5B | DeepSeek | 0.169 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 14B | DeepSeek | 0.531 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 32B | DeepSeek | 0.572 | 2025-01-20 | evidence |
| DeepSeek R1 Distill Qwen 7B | DeepSeek | 0.376 | 2025-01-20 | evidence |
| DeepSeek R1 Zero | DeepSeek | 0.5 | 2025-01-20 | evidence |
| DeepSeek-R1-0528 | DeepSeek | 0.733 | 2025-05-28 | evidence |
| DeepSeek-V3 | DeepSeek | 0.376 | 2024-12-25 | evidence |
| DeepSeek-V3 0324 | DeepSeek | 0.492 | 2025-03-25 | evidence |
| DeepSeek-V3.1 | DeepSeek | 0.564 | 2025-01-10 | evidence |
| DeepSeek-V3.2 | DeepSeek | 0.833 | 2025-12-01 | evidence |
| DeepSeek-V3.2 (Thinking) | DeepSeek | 0.833 | 2025-12-01 | evidence |
| DeepSeek-V3.2-Exp | DeepSeek | 0.741 | 2025-09-29 | evidence |
| DeepSeek-V4-Flash-0423 | DeepSeek | 0.884 | 2026-04-23 | evidence |
| DeepSeek-V4-Flash-Max | DeepSeek | 0.916 | 2026-04-23 | evidence |
| DeepSeek-V4-Pro-Max | DeepSeek | 0.935 | 2026-04-23 | evidence |
| Gemini 2.0 Flash | 0.351 | 2024-12-01 | evidence | |
| Gemini 2.5 Flash-Lite | 0.337 | 2025-06-17 | evidence | |
| Gemini 2.5 Pro Preview 06-05 | 0.69 | 2025-06-05 | evidence | |
| Gemini Diffusion | 0.309 | 2025-05-20 | evidence | |
| Gemma 3 12B | 0.246 | 2025-03-12 | evidence | |
| Gemma 3 1B | 0.019 | 2025-03-12 | evidence | |
| Gemma 3 27B | 0.297 | 2025-03-12 | evidence | |
| Gemma 3 4B | 0.126 | 2025-03-12 | evidence | |
| Gemma 3n E2B Instructed | 0.132 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.132 | 2025-05-20 | evidence | |
| Gemma 3n E4B Instructed | 0.132 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.132 | 2025-05-20 | evidence | |
| GLM-4.5 | Z.ai | 0.729 | 2025-07-28 | evidence |
| GLM-4.5-Air | Z.ai | 0.707 | 2025-07-28 | evidence |
| Grok 4 Fast | xAI | 0.8 | 2025-08-28 | evidence |
| Grok-3 | xAI | 0.794 | 2025-02-17 | evidence |
| Grok-3 Mini | xAI | 0.804 | 2025-02-17 | evidence |
| Grok-4 | xAI | 0.79 | 2025-07-09 | evidence |
| Grok-4 Heavy | xAI | 0.794 | 2025-07-09 | evidence |
| Kimi K2-Instruct-0905 | Moonshot AI | 0.537 | 2025-09-05 | evidence |
| Llama 3.1 Nemotron Ultra 253B v1 | NVIDIA | 0.6631 | 2025-04-07 | evidence |
| Llama 4 Maverick | Meta | 0.434 | 2025-04-05 | evidence |
| Llama 4 Scout | Meta | 0.328 | 2025-04-05 | evidence |
| LongCat-Flash-Chat | Meituan | 0.4802 | 2025-08-29 | evidence |
| LongCat-Flash-Thinking | Meituan | 0.794 | 2025-09-22 | evidence |
| LongCat-Flash-Thinking-2601 | Meituan | 0.828 | 2026-01-14 | evidence |
| Magistral Medium | Mistral | 0.503 | 2025-06-10 | evidence |
| Magistral Small 2506 | Mistral | 0.513 | 2025-06-10 | evidence |
| Mercury 2 | Inception | 0.67 | 2026-02-24 | evidence |
| Min istral 3 (3B Reasoning 2512) | Mistral | 0.548 | 2025-12-04 | evidence |
| MiniMax M1 40K | MiniMax | 0.623 | 2025-06-16 | evidence |
| MiniMax M1 80K | MiniMax | 0.65 | 2025-06-16 | evidence |
| MiniMax M2 | MiniMax | 0.83 | 2025-10-27 | evidence |
| MiniMax M2.1 | MiniMax | 0.78 | 2025-12-23 | evidence |
| Ministral 3 (14B Reasoning 2512) | Mistral | 0.646 | 2025-12-04 | evidence |
| Ministral 3 (8B Reasoning 2512) | Mistral | 0.616 | 2025-12-04 | evidence |
| Mistral Large 3 (675B Base) | Mistral | 0.344 | 2025-12-04 | evidence |
| Mistral Large 3 (675B Instruct 2512 Eagle) | Mistral | 0.344 | 2025-12-04 | evidence |
| Mistral Large 3 (675B Instruct 2512 NVFP4) | Mistral | 0.344 | 2025-12-04 | evidence |
| Mistral Large 3 (675B Instruct 2512) | Mistral | 0.344 | 2025-12-04 | evidence |
| Mistral Small 4 | Mistral | 0.636 | 2026-03-16 | evidence |
| Nemotron 3 Super (120B A12B) | NVIDIA | 0.8119 | 2026-03-11 | evidence |
| Nemotron Nano 9B v2 | NVIDIA | 0.711 | 2025-08-18 | evidence |
| Nova 2 Lite | Amazon | 0.71 | 2025-12-02 | evidence |
| Nova 2 Pro | Amazon | 0.746 | 2025-12-02 | evidence |
| Phi 4 Reasoning | Microsoft | 0.538 | 2025-04-30 | evidence |
| Phi 4 Reasoning Plus | Microsoft | 0.531 | 2025-04-30 | evidence |
| Qwen2 7B Instruct | Qwen | 0.266 | 2024-07-23 | evidence |
| Qwen2.5 72B Instruct | Qwen | 0.555 | 2024-09-19 | evidence |
| Qwen2.5 7B Instruct | Qwen | 0.287 | 2024-09-19 | evidence |
| Qwen2.5-Coder 32B Instruct | Qwen | 0.314 | 2024-09-19 | evidence |
| Qwen2.5-Coder 7B Instruct | Qwen | 0.182 | 2024-09-19 | evidence |
| Qwen3 235B A22B | Qwen | 0.707 | 2025-04-29 | evidence |
| Qwen3 30B A3B | Qwen | 0.626 | 2025-04-29 | evidence |
| Qwen3 32B | Qwen | 0.657 | 2025-04-29 | evidence |
| QwQ-32B | Qwen | 0.634 | 2025-03-05 | evidence |
| QwQ-32B-Preview | Qwen | 0.5 | 2024-11-28 | evidence |
| Solar Pro 4 | Upstage | 0.878 | 2026-08-06 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.