SciCode Benchmark
Explore score history ↗SciCode is a research coding benchmark curated by scientists that challenges language models to code solutions for scientific problems. It contains 338 subproblems decomposed from 80 challenging main problems across 16 natural science sub-fields including mathematics, physics, chemistry, biology, and materials science. Problems require knowledge recall, reasoning, and code synthesis skills.
- Released
- 2024-07-18
- Modality
- text
- Evaluates
- math, physics, reasoning, biology, chemistry, code
- Openness
- unknown
- Importer
- LLM Stats
Source provenance
SciCode paper and code
SciCode results and reported scores
Reported benchmark scores · LLM Stats
Reported scores: 21
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Command A+ | Cohere | 0.38 | 2026-05-20 | evidence |
| Gemini 3.1 Pro | 0.59 | 2026-02-19 | evidence | |
| GLM-4.5 | Z.ai | 0.417 | 2025-07-28 | evidence |
| GLM-4.5-Air | Z.ai | 0.373 | 2025-07-28 | evidence |
| Inkling-Small | Thinking Machines Lab | 0.487 | 2026-07-30 | evidence |
| Kimi K2-Thinking-0905 | Moonshot AI | 0.448 | 2025-09-05 | evidence |
| Kimi K2.5 | Moonshot AI | 0.487 | 2026-01-27 | evidence |
| Kimi K2.6 | Moonshot AI | 0.522 | 2026-04-20 | evidence |
| Mercury 2 | Inception | 0.38 | 2026-02-24 | evidence |
| MiniMax M2 | MiniMax | 0.36 | 2025-10-27 | evidence |
| MiniMax M2.1 | MiniMax | 0.39 | 2025-12-23 | evidence |
| Muse Glimmer-30B | Meta | 0.436 | 2026-08-10 | evidence |
| Nemotron 3 Nano (30B A3B) | NVIDIA | 0.333 | 2025-12-15 | evidence |
| Nemotron 3 Super (120B A12B) | NVIDIA | 0.4205 | 2026-03-11 | evidence |
| Nemotron 3 Ultra (550B A55B) | NVIDIA | 0.446 | 2026-06-04 | evidence |
| Nemotron 3.5 Lightning (30B A3B) | NVIDIA | 0.326 | 2026-08-11 | evidence |
| North Mini Code 1.0 | Cohere | 0.382 | 2026-06-09 | evidence |
| Qwen3.7 Max | Qwen | 0.535 | 2026-05-19 | evidence |
| Qwen3.7-Plus | Qwen | 0.513 | 2026-05-31 | evidence |
| Seed 2.1 Pro | ByteDance | 0.598 | 2026-06-24 | evidence |
| Seed 2.1 Turbo | ByteDance | 0.578 | 2026-06-24 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.