Benchmark
BIG-Bench Hard
BIG-Bench Hard (BBH) is a subset of 23 challenging BIG-Bench tasks selected because prior language model evaluations did not outperform average…
- Modality
- text
- Categories
- math, reasoning, language
- Openness
- unknown
- Source
- llm_stats
- Reported scores
- 21
Reported scores
llm_stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 3 Haiku | Anthropic | 0.737 | 2024-03-13 | evidence |
| Claude 3 Opus | Anthropic | 0.868 | 2024-02-29 | evidence |
| Claude 3 Sonnet | Anthropic | 0.829 | 2024-02-29 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.931 | 2024-06-21 | evidence |
| Claude 3.5 Sonnet | Anthropic | 0.931 | 2024-10-22 | evidence |
| Gemini 1.5 Flash | 0.855 | 2024-05-01 | evidence | |
| Gemini 1.5 Pro | 0.892 | 2024-05-01 | evidence | |
| Gemma 3 12B | 0.857 | 2025-03-12 | evidence | |
| Gemma 3 1B | 0.391 | 2025-03-12 | evidence | |
| Gemma 3 27B | 0.876 | 2025-03-12 | evidence | |
| Gemma 3 4B | 0.722 | 2025-03-12 | evidence | |
| Gemma 3n E2B | 0.443 | 2025-06-26 | evidence | |
| Gemma 3n E2B Instructed LiteRT (Preview) | 0.443 | 2025-05-20 | evidence | |
| Gemma 3n E4B | 0.529 | 2025-06-26 | evidence | |
| Gemma 3n E4B Instructed LiteRT Preview | 0.529 | 2025-05-20 | evidence | |
| Granite 3.3 8B Base | IBM | 0.6913 | 2025-04-16 | evidence |
| Granite 3.3 8B Instruct | IBM | 0.6913 | 2025-04-16 | evidence |
| IBM Granite 4.0 Tiny Preview | IBM | 0.557 | 2025-05-02 | evidence |
| Phi 4 Mini | Microsoft | 0.704 | 2025-02-01 | evidence |
| Phi-3.5-mini-instruct | Microsoft | 0.69 | 2024-08-23 | evidence |
| Phi-3.5-MoE-instruct | Microsoft | 0.791 | 2024-08-23 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.