Benchmark

HumanEval

What is HumanEval?

Saturated. Retained because open-weight cards still report it. Benchmark Radar tracks 2 reported scores for HumanEval, cited by 9 source documents, with links…

Released: 2021-07-07
Evaluates: coding
Openness: unknown
Importer: Model reports
Reported scores: 2

Source provenance

Original evidence https://github.com/openai/human-eval

HumanEval code

Code https://github.com/openai/human-eval

HumanEval results and reported scores

Model reports

Model	Organization	Reported value	Reported	Evidence
Gemini 1.5 Pro	Google	84.1	2024-03-08	evidence
Qwen2.5-Coder-32B-Instruct	Qwen	92.7	2024-09-18	evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.

Which sources cite HumanEval?

DeepSeek-V4-Pro DeepSeek, 2026-04-22
Olmo 3 (7B and 32B) Ai2, 2025-11-20
Qwen3 Qwen, 2025-05-14
Mistral Medium 3 Mistral, 2025-05-07
DeepSeek-V3 DeepSeek, 2024-12-27
Qwen2.5-Coder Qwen, 2024-09-18
Mistral Large 2 Mistral, 2024-07-24
Llama 3.1 405B Meta, 2024-07-23
Gemini 1.5 Google, 2024-03-08