Benchmark

tau2-bench

What is tau2-bench?

Per-domain scores (Retail, Telecom, Airline) diverge sharply and several cards apply the Airline domain fixes from the Claude Opus 4.5 system card, so an…

Released: 2025-06-09
Evaluates: tool use
Openness: unknown
Importer: Model reports
Reported scores: 5

Source provenance

Original evidence https://arxiv.org/abs/2506.07982

tau2-bench paper

Paper https://arxiv.org/abs/2506.07982

tau2-bench results and reported scores

Model reports

Model	Organization	Reported value	Reported	Evidence
Gemini 3.1 Pro	Google	90.8	2026-02-19	evidence
Gemini 3.1 Pro	Google	99.3	2026-02-19	evidence
Gemma 4 (31B)	Google	76.9	2026-03-11	evidence
GLM-5	Z.ai	89.7	2026-02-11	evidence
Qwen3.5-397B-A17B	Qwen	86.7	2026-02-16	evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.

Which sources cite tau2-bench?

GLM-5.1 Z.ai, 2026-04-03
Gemma 4 (31B) Google, 2026-03-11
Gemini 3.1 Pro Google, 2026-02-19
Qwen3.5-397B-A17B Qwen, 2026-02-16
GLM-5 Z.ai, 2026-02-11
GLM-5 Z.ai, 2026-02-11