Benchmark
Aider-Polyglot
A coding benchmark that evaluates LLMs on 225 challenging Exercism programming exercises across C++, Go, Java, JavaScript, Python, and Rust. Models…
- Modality
- text
- Categories
- general, code
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 22
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| DeepSeek-R1-0528 | DeepSeek | 0.716 | 2025-05-28 | evidence |
| DeepSeek-V3 | DeepSeek | 0.496 | 2024-12-25 | evidence |
| DeepSeek-V3.1 | DeepSeek | 0.684 | 2025-01-10 | evidence |
| DeepSeek-V3.2-Exp | DeepSeek | 0.745 | 2025-09-29 | evidence |
| Gemini 2.5 Flash | 0.619 | 2025-05-20 | evidence | |
| Gemini 2.5 Flash-Lite | 0.267 | 2025-06-17 | evidence | |
| Gemini 2.5 Pro | 0.765 | 2025-05-20 | evidence | |
| Gemini 2.5 Pro Preview 06-05 | 0.822 | 2025-06-05 | evidence | |
| GPT-4.1 | OpenAI | 0.516 | 2025-04-14 | evidence |
| GPT-4.1 mini | OpenAI | 0.347 | 2025-04-14 | evidence |
| GPT-4.1 nano | OpenAI | 0.098 | 2025-04-14 | evidence |
| GPT-4o | OpenAI | 0.307 | 2024-08-06 | evidence |
| GPT-5 | OpenAI | 0.88 | 2025-08-07 | evidence |
| Kimi K2 Instruct | Moonshot AI | 0.6 | 2025-07-11 | evidence |
| Kimi K2-Instruct-0905 | Moonshot AI | 0.6 | 2025-09-05 | evidence |
| Magistral Medium | Mistral | 0.471 | 2025-06-10 | evidence |
| o3 | OpenAI | 0.813 | 2025-04-16 | evidence |
| o3-mini | OpenAI | 0.667 | 2025-01-30 | evidence |
| o4-mini | OpenAI | 0.689 | 2025-04-16 | evidence |
| Qwen3-235B-A22B-Instruct-2507 | Qwen | 0.573 | 2025-07-22 | evidence |
| Qwen3-Coder 480B A35B Instruct | Qwen | 0.618 | 2025-01-31 | evidence |
| Qwen3-Next-80B-A3B-Instruct | Qwen | 0.498 | 2025-09-10 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.