Benchmark
Tau2 Telecom Benchmark
What is Tau2 Telecom?
τ²-Bench telecom domain evaluates conversational agents in a dual-control environment modeled as a Dec-POMDP, where both agent and user use tools in shared…
- Released
- 2025-06-09
- Modality
- text
- Evaluates
- reasoning, communication, tool calling
- Openness
- restricted
- Importer
- LLM Stats
- Reported scores
- 35
Source provenance
- Original evidence https://api.zeroeval.com/leaderboard/benchmarks/tau2-telecom?top_n=500
Tau2 Telecom paper and code
Tau2 Telecom results and reported scores
LLM Stats
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| Claude Haiku 4.5 | Anthropic | 0.83 | 2025-10-15 | evidence |
| Claude Opus 4.5 | Anthropic | 0.982 | 2025-11-24 | evidence |
| Claude Opus 4.6 | Anthropic | 0.993 | 2026-02-05 | evidence |
| Claude Sonnet 4.6 | Anthropic | 0.979 | 2026-02-17 | evidence |
| Command A+ | Cohere | 0.85 | 2026-05-20 | evidence |
| GPT-4o | OpenAI | 0.235 | 2024-08-06 | evidence |
| GPT-5 | OpenAI | 0.967 | 2025-08-07 | evidence |
| GPT-5.1 | OpenAI | 0.956 | 2025-11-13 | evidence |
| GPT-5.1 Instant | OpenAI | 0.956 | 2025-11-12 | evidence |
| GPT-5.1 Thinking | OpenAI | 0.956 | 2025-11-12 | evidence |
| GPT-5.2 | OpenAI | 0.987 | 2025-12-11 | evidence |
| GPT-5.4 | OpenAI | 0.989 | 2026-03-05 | evidence |
| GPT-5.4 mini | OpenAI | 0.934 | 2026-03-17 | evidence |
| GPT-5.4 nano | OpenAI | 0.925 | 2026-03-17 | evidence |
| GPT-5.5 | OpenAI | 0.98 | 2026-04-23 | evidence |
| Kimi K2 Instruct | Moonshot AI | 0.658 | 2025-07-11 | evidence |
| Kimi K2-Instruct-0905 | Moonshot AI | 0.658 | 2025-09-05 | evidence |
| LongCat-Flash-Chat | Meituan | 0.7368 | 2025-08-29 | evidence |
| LongCat-Flash-Lite | Meituan | 0.728 | 2026-02-05 | evidence |
| LongCat-Flash-Thinking | Meituan | 0.831 | 2025-09-22 | evidence |
| LongCat-Flash-Thinking-2601 | Meituan | 0.993 | 2026-01-14 | evidence |
| MAI-Code-1-Flash | Microsoft | 0.717 | 2026-06-02 | evidence |
| MiMo-V2-Pro | Xiaomi | 0.968 | 2026-03-18 | evidence |
| MiniMax M2 | MiniMax | 0.87 | 2025-10-27 | evidence |
| MiniMax M2.1 | MiniMax | 0.87 | 2025-12-23 | evidence |
| Muse Spark | Meta | 0.915 | 2026-04-08 | evidence |
| Nemotron 3 Nano (30B A3B) | NVIDIA | 0.422 | 2025-12-15 | evidence |
| Nemotron 3 Super (120B A12B) | NVIDIA | 0.6436 | 2026-03-11 | evidence |
| Nova 2 Lite | Amazon | 0.76 | 2025-12-02 | evidence |
| Nova 2 Omni | Amazon | 0.8 | 2025-12-02 | evidence |
| Nova 2 Pro | Amazon | 0.927 | 2025-12-02 | evidence |
| o3 | OpenAI | 0.582 | 2025-04-16 | evidence |
| Qwen3-235B-A22B-Thinking-2507 | Qwen | 0.456 | 2025-07-25 | evidence |
| Qwen3-Next-80B-A3B-Instruct | Qwen | 0.132 | 2025-09-10 | evidence |
| Qwen3-Next-80B-A3B-Thinking | Qwen | 0.439 | 2025-09-10 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.