Benchmark
Harvey LAB-AA
Legal agentic work, criterion pass rate
- Categories
- agentic, legal
- Openness
- unknown
- Source
- artificial_analysis
- Reported scores
- 36
Reported scores
artificial_analysis
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude 4.5 Haiku (Reasoning) | Anthropic | 0.610508033000434 | 2025-10-15 | evidence |
| Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) | Anthropic | 0.935591257779708 | 2026-06-09 | evidence |
| Claude Opus 4.8 (Adaptive Reasoning, Max Effort) | Anthropic | 0.910840932117528 | 2026-05-28 | evidence |
| Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) | Anthropic | 0.859748154580981 | 2026-02-17 | evidence |
| Claude Sonnet 4.6 (Non-reasoning, High Effort) | Anthropic | 0.844405847445361 | 2026-02-17 | evidence |
| Claude Sonnet 5 (Adaptive Reasoning, Max Effort) | Anthropic | 0.900709219858156 | 2026-06-30 | evidence |
| DeepSeek V4 Flash (Reasoning, Max Effort) | DeepSeek | 0.813287016934433 | 2026-04-24 | evidence |
| DeepSeek V4 Pro (Reasoning, Max Effort) | DeepSeek | 0.843971631205674 | 2026-04-24 | evidence |
| Gemini 3.1 Flash-Lite | 0.311477782602403 | 2026-03-03 | evidence | |
| Gemini 3.1 Pro Preview | 0.588507743522941 | 2026-02-19 | evidence | |
| Gemini 3.5 Flash (high) | 0.820813431755681 | 2026-05-19 | evidence | |
| Gemini 3.6 Flash (high) | 0.851498046026921 | 2026-07-21 | evidence | |
| Gemini 3.7 Flash (high) | 0.906643508467217 | 2026-08-13 | evidence | |
| Gemma 4 31B (Reasoning) | 0.47228253003329 | 2026-04-02 | evidence | |
| GLM-5.2 (max) | Z.ai | 0.909683022145028 | 2026-06-16 | evidence |
| GPT-5.4 mini (xhigh) | OpenAI | 0.607468519322623 | 2026-03-17 | evidence |
| GPT-5.4 nano (xhigh) | OpenAI | 0.522362136343899 | 2026-03-17 | evidence |
| GPT-5.5 (xhigh) | OpenAI | 0.862787668258793 | 2026-04-23 | evidence |
| GPT-5.6 Luna (max) | OpenAI | 0.878998407873788 | 2026-07-09 | evidence |
| GPT-5.6 Sol (max) | OpenAI | 0.871761470545665 | 2026-07-09 | evidence |
| GPT-5.6 Terra (max) | OpenAI | 0.851787523520046 | 2026-07-09 | evidence |
| gpt-oss-120b (high) | OpenAI | 0.138949196699957 | 2025-08-05 | evidence |
| Grok 4.3 (high) | xAI | 0.683890577507599 | 2026-04-30 | evidence |
| Grok 4.5 (high) | xAI | 0.924156896801274 | 2026-07-08 | evidence |
| Kimi K2.6 | Moonshot AI | 0.841221595020987 | 2026-04-20 | evidence |
| Kimi K2.7 Code | Moonshot AI | 0.850195397307859 | 2026-06-12 | evidence |
| Kimi K3 (max) | Moonshot AI | 0.946446663771892 | 2026-07-16 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.732522796352584 | 2026-04-22 | evidence |
| MiniMax-M3 | MiniMax | 0.884064264003474 | 2026-06-01 | evidence |
| Mistral Medium 3.5 | Mistral | 0.690982776089159 | 2026-04-29 | evidence |
| Nemotron 3 Ultra 550B A55B (Reasoning) | NVIDIA | 0.81719496309162 | 2026-06-04 | evidence |
| Qwen3.5 397B A17B (Reasoning) | Qwen | 0.723838471558836 | 2026-02-16 | evidence |
| Qwen3.6 27B (Reasoning) | Qwen | 0.822984512954118 | 2026-04-22 | evidence |
| Qwen3.7 Max | Qwen | 0.834274135185989 | 2026-05-19 | evidence |
| Qwen3.7 Plus | Qwen | 0.817918656824432 | 2026-06-01 | evidence |
| Step 3.7 Flash | StepFun | 0.727312201476335 | 2026-05-29 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.