Benchmark
SciCode
SciCode is a research coding benchmark curated by scientists that challenges language models to code solutions for scientific problems. It contains 338…
- Modality
- text
- Categories
- math, physics, reasoning, biology, chemistry, code
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 21
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Command A+ | Cohere | 0.38 | 2026-05-20 | evidence |
| Gemini 3.1 Pro | 0.59 | 2026-02-19 | evidence | |
| GLM-4.5 | Z.ai | 0.417 | 2025-07-28 | evidence |
| GLM-4.5-Air | Z.ai | 0.373 | 2025-07-28 | evidence |
| Inkling-Small | Thinking Machines Lab | 0.487 | 2026-07-30 | evidence |
| Kimi K2-Thinking-0905 | Moonshot AI | 0.448 | 2025-09-05 | evidence |
| Kimi K2.5 | Moonshot AI | 0.487 | 2026-01-27 | evidence |
| Kimi K2.6 | Moonshot AI | 0.522 | 2026-04-20 | evidence |
| Mercury 2 | Inception | 0.38 | 2026-02-24 | evidence |
| MiniMax M2 | MiniMax | 0.36 | 2025-10-27 | evidence |
| MiniMax M2.1 | MiniMax | 0.39 | 2025-12-23 | evidence |
| Muse Glimmer-30B | Meta | 0.436 | 2026-08-10 | evidence |
| Nemotron 3 Nano (30B A3B) | NVIDIA | 0.333 | 2025-12-15 | evidence |
| Nemotron 3 Super (120B A12B) | NVIDIA | 0.4205 | 2026-03-11 | evidence |
| Nemotron 3 Ultra (550B A55B) | NVIDIA | 0.446 | 2026-06-04 | evidence |
| Nemotron 3.5 Lightning (30B A3B) | NVIDIA | 0.326 | 2026-08-11 | evidence |
| North Mini Code 1.0 | Cohere | 0.382 | 2026-06-09 | evidence |
| Qwen3.7 Max | Qwen | 0.535 | 2026-05-19 | evidence |
| Qwen3.7-Plus | Qwen | 0.513 | 2026-05-31 | evidence |
| Seed 2.1 Pro | ByteDance | 0.598 | 2026-06-24 | evidence |
| Seed 2.1 Turbo | ByteDance | 0.578 | 2026-06-24 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.