Benchmark
τ³-Banking
Agentic tool use
- Categories
- intelligence-index, agentic, tool-use
- Openness
- unknown
- Source
- artificial_analysis
- Reported scores
- 178
Reported scores
artificial_analysis
| Model | Organization | Reported value | Reported | Evidence |
|---|---|---|---|---|
| A.X-K2 | SK Telecom | 0.160824742268041 | 2026-08-12 | evidence |
| Agnes 2.5 Pro Alpha | Sapiens AI | 0.123711340206186 | 2026-07-24 | evidence |
| Celeris-1 | Celeris | 0.0391752577319588 | 2026-07-24 | evidence |
| Claude 4 Sonnet (Reasoning) | Anthropic | 0.167010309278351 | 2025-05-22 | evidence |
| Claude 4.5 Haiku (Reasoning) | Anthropic | 0.0927835051546392 | 2025-10-15 | evidence |
| Claude 4.5 Sonnet (Reasoning) | Anthropic | 0.245360824742268 | 2025-09-29 | evidence |
| Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) | Anthropic | 0.381443298969072 | 2026-06-09 | evidence |
| Claude Opus 4.7 (Adaptive Reasoning, Max Effort) | Anthropic | 0.34639175257732 | 2026-04-16 | evidence |
| Claude Opus 4.8 (Adaptive Reasoning, Max Effort) | Anthropic | 0.342268041237113 | 2026-05-28 | evidence |
| Claude Opus 5 (Adaptive Reasoning, High Effort) | Anthropic | 0.447422680412371 | 2026-07-24 | evidence |
| Claude Opus 5 (Adaptive Reasoning, Low Effort) | Anthropic | 0.303092783505155 | 2026-07-24 | evidence |
| Claude Opus 5 (Adaptive Reasoning, Max Effort) | Anthropic | 0.420618556701031 | 2026-07-24 | evidence |
| Claude Opus 5 (Adaptive Reasoning, Medium Effort) | Anthropic | 0.385567010309278 | 2026-07-24 | evidence |
| Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) | Anthropic | 0.43298969072165 | 2026-07-24 | evidence |
| Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) | Anthropic | 0.344329896907216 | 2026-02-17 | evidence |
| Claude Sonnet 5 (Adaptive Reasoning, Max Effort) | Anthropic | 0.37319587628866 | 2026-06-30 | evidence |
| Claude Sonnet 5 (Non-reasoning, High Effort) | Anthropic | 0.156701030927835 | 2026-06-30 | evidence |
| Command A+ | Cohere | 0.0597938144329897 | 2026-05-20 | evidence |
| DeepSeek R1 (Jan '25) | DeepSeek | 0.0639175257731959 | 2025-01-20 | evidence |
| DeepSeek V3 (Dec '24) | DeepSeek | 0.0474226804123711 | 2024-12-26 | evidence |
| DeepSeek V3 0324 | DeepSeek | 0.0474226804123711 | 2025-03-25 | evidence |
| DeepSeek V3.1 Terminus (Reasoning) | DeepSeek | 0.210309278350515 | 2025-09-22 | evidence |
| DeepSeek V4 Flash (Reasoning, High Effort) | DeepSeek | 0.261855670103093 | 2026-04-24 | evidence |
| DeepSeek V4 Flash (Reasoning, Max Effort) | DeepSeek | 0.309278350515464 | 2026-04-24 | evidence |
| DeepSeek V4 Flash 0731 (Reasoning, Max Effort) | DeepSeek | 0.393814432989691 | 2026-07-31 | evidence |
| DeepSeek V4 Flash Vision (Reasoning, Max Effort) | DeepSeek | 0.410309278350515 | 2026-08-21 | evidence |
| DeepSeek V4 Pro (Reasoning, High Effort) | DeepSeek | 0.261855670103093 | 2026-04-24 | evidence |
| DeepSeek V4 Pro (Reasoning, Max Effort) | DeepSeek | 0.301030927835052 | 2026-04-24 | evidence |
| DeepSeek V4 Pro 0813 (Reasoning, Max Effort) | DeepSeek | 0.395876288659794 | 2026-08-13 | evidence |
| Devstral 2 | Mistral | 0.105154639175258 | 2025-12-09 | evidence |
| Devstral Small 2 | Mistral | 0.107216494845361 | 2025-12-09 | evidence |
| G9v3-39A5B | AI9Stars | 0.220618556701031 | 2026-08-20 | evidence |
| Gemini 2.5 Pro | 0.0969072164948454 | 2025-06-05 | evidence | |
| Gemini 3 Flash Preview (Reasoning) | 0.208247422680412 | 2025-12-17 | evidence | |
| Gemini 3.1 Flash-Lite | 0.0969072164948454 | 2026-03-03 | evidence | |
| Gemini 3.1 Pro Preview | 0.214432989690722 | 2026-02-19 | evidence | |
| Gemini 3.5 Flash (high) | 0.321649484536082 | 2026-05-19 | evidence | |
| Gemini 3.5 Flash-Lite | 0.175257731958763 | 2026-07-21 | evidence | |
| Gemini 3.6 Flash (high) | 0.298969072164948 | 2026-07-21 | evidence | |
| Gemini 3.7 Flash (high) | 0.327835051546392 | 2026-08-13 | evidence | |
| Gemini 3.7 Flash (low) | 0.294845360824742 | 2026-08-13 | evidence | |
| Gemini 3.7 Flash (medium) | 0.354639175257732 | 2026-08-13 | evidence | |
| Gemma 3 12B Instruct | 0.00824742268041237 | 2025-03-12 | evidence | |
| Gemma 3 27B Instruct | 0.00824742268041237 | 2025-03-12 | evidence | |
| Gemma 3 4B Instruct | 0.00412371134020619 | 2025-03-12 | evidence | |
| Gemma 3n E4B Instruct | 0.00206185567010309 | 2025-06-26 | evidence | |
| Gemma 4 26B A4B (Reasoning) | 0.119587628865979 | 2026-04-02 | evidence | |
| Gemma 4 31B (Non-reasoning) | 0.088659793814433 | 2026-04-02 | evidence | |
| Gemma 4 31B (Reasoning) | 0.148453608247423 | 2026-04-02 | evidence | |
| GLM-4.6 (Reasoning) | Z.ai | 0.134020618556701 | 2025-09-30 | evidence |
| GLM-4.7 (Reasoning) | Z.ai | 0.121649484536082 | 2025-12-22 | evidence |
| GLM-5.1 (Reasoning) | Z.ai | 0.136082474226804 | 2026-04-07 | evidence |
| GLM-5.2 (max) | Z.ai | 0.34639175257732 | 2026-06-16 | evidence |
| GLM-5.2 (Non-reasoning) | Z.ai | 0.167010309278351 | 2026-06-16 | evidence |
| GLM-5.3 (max) | Z.ai | 0.503092783505155 | 2026-08-18 | evidence |
| GPT-4.1 mini | OpenAI | 0.0536082474226804 | 2025-04-14 | evidence |
| GPT-4.1 nano | OpenAI | 0.0350515463917526 | 2025-04-14 | evidence |
| GPT-4o mini | OpenAI | 0.0288659793814433 | 2024-07-18 | evidence |
| GPT-5 (high) | OpenAI | 0.220618556701031 | 2025-08-07 | evidence |
| GPT-5 mini (high) | OpenAI | 0.154639175257732 | 2025-08-07 | evidence |
| GPT-5.1 (high) | OpenAI | 0.158762886597938 | 2025-11-13 | evidence |
| GPT-5.2 (Non-reasoning) | OpenAI | 0.111340206185567 | 2025-12-11 | evidence |
| GPT-5.4 (xhigh) | OpenAI | 0.395876288659794 | 2026-03-05 | evidence |
| GPT-5.4 mini (xhigh) | OpenAI | 0.255670103092783 | 2026-03-17 | evidence |
| GPT-5.4 nano (xhigh) | OpenAI | 0.274226804123711 | 2026-03-17 | evidence |
| GPT-5.5 (high) | OpenAI | 0.36701030927835 | 2026-04-23 | evidence |
| GPT-5.5 (low) | OpenAI | 0.249484536082474 | 2026-04-23 | evidence |
| GPT-5.5 (medium) | OpenAI | 0.298969072164948 | 2026-04-23 | evidence |
| GPT-5.5 (Non-reasoning) | OpenAI | 0.148453608247423 | 2026-04-23 | evidence |
| GPT-5.5 (xhigh) | OpenAI | 0.389690721649485 | 2026-04-23 | evidence |
| GPT-5.5 Instant (June 2026) | OpenAI | 0.117525773195876 | 2026-06-25 | evidence |
| GPT-5.6 Luna (high) | OpenAI | 0.251546391752577 | 2026-07-09 | evidence |
| GPT-5.6 Luna (low) | OpenAI | 0.127835051546392 | 2026-07-09 | evidence |
| GPT-5.6 Luna (max) | OpenAI | 0.311340206185567 | 2026-07-09 | evidence |
| GPT-5.6 Luna (medium) | OpenAI | 0.177319587628866 | 2026-07-09 | evidence |
| GPT-5.6 Luna (Non-reasoning) | OpenAI | 0.0969072164948454 | 2026-07-09 | evidence |
| GPT-5.6 Luna (xhigh) | OpenAI | 0.28659793814433 | 2026-07-09 | evidence |
| GPT-5.6 Sol (high) | OpenAI | 0.36701030927835 | 2026-07-09 | evidence |
| GPT-5.6 Sol (low) | OpenAI | 0.290721649484536 | 2026-07-09 | evidence |
| GPT-5.6 Sol (max) | OpenAI | 0.443298969072165 | 2026-07-09 | evidence |
| GPT-5.6 Sol (medium) | OpenAI | 0.364948453608247 | 2026-07-09 | evidence |
| GPT-5.6 Sol (Non-reasoning) | OpenAI | 0.195876288659794 | 2026-07-09 | evidence |
| GPT-5.6 Sol (xhigh) | OpenAI | 0.381443298969072 | 2026-07-09 | evidence |
| GPT-5.6 Terra (high) | OpenAI | 0.28659793814433 | 2026-07-09 | evidence |
| GPT-5.6 Terra (low) | OpenAI | 0.187628865979381 | 2026-07-09 | evidence |
| GPT-5.6 Terra (max) | OpenAI | 0.402061855670103 | 2026-07-09 | evidence |
| GPT-5.6 Terra (medium) | OpenAI | 0.255670103092783 | 2026-07-09 | evidence |
| GPT-5.6 Terra (Non-reasoning) | OpenAI | 0.156701030927835 | 2026-07-09 | evidence |
| GPT-5.6 Terra (xhigh) | OpenAI | 0.296907216494845 | 2026-07-09 | evidence |
| gpt-oss-120b (high) | OpenAI | 0.127835051546392 | 2025-08-05 | evidence |
| gpt-oss-120b (low) | OpenAI | 0.0288659793814433 | 2025-08-05 | evidence |
| gpt-oss-20b (high) | OpenAI | 0.0701030927835052 | 2025-08-05 | evidence |
| Grok 4.3 (high) | xAI | 0.123711340206186 | 2026-04-30 | evidence |
| Grok 4.3 (Non-reasoning) | xAI | 0.0804123711340206 | 2026-04-30 | evidence |
| Grok 4.5 (high) | xAI | 0.420618556701031 | 2026-07-08 | evidence |
| Grok 4.6 (high) | xAI | 0.507216494845361 | 2026-08-12 | evidence |
| Grok 4.6 (low) | xAI | 0.381443298969072 | 2026-08-12 | evidence |
| Grok 4.6 (medium) | xAI | 0.443298969072165 | 2026-08-12 | evidence |
| Grok 4.6 (xhigh) | xAI | 0.43298969072165 | 2026-08-12 | evidence |
| Grok Build 0.1 0616 | xAI | 0.134020618556701 | 2026-06-16 | evidence |
| Hy3 | Tencent | 0.228865979381443 | 2026-07-06 | evidence |
| HyperNova 60B 2605 | Multiverse Computing | 0.0556701030927835 | 2026-05-26 | evidence |
| Inkling (xhigh) | Thinking Machines | 0.290721649484536 | 2026-07-15 | evidence |
| Inkling Small | Thinking Machines | 0.187628865979381 | 2026-07-30 | evidence |
| K-EXAONE 2.0 0803 | LG AI Research | 0.115463917525773 | 2026-08-12 | evidence |
| KAT Coder Pro V2 | KwaiKAT | 0.045360824742268 | 2026-03-27 | evidence |
| Kimi K2.5 (Reasoning) | Moonshot AI | 0.142268041237113 | 2026-01-27 | evidence |
| Kimi K2.6 | Moonshot AI | 0.232989690721649 | 2026-04-20 | evidence |
| Kimi K2.7 Code | Moonshot AI | 0.202061855670103 | 2026-06-12 | evidence |
| Kimi K3 (low) | Moonshot AI | 0.416494845360825 | 2026-07-16 | evidence |
| Kimi K3 (max) | Moonshot AI | 0.45979381443299 | 2026-07-16 | evidence |
| LFM2.5-2.6B | Liquid AI | 0.0721649484536082 | 2026-08-04 | evidence |
| Ling 2.6 Flash | InclusionAI | 0.0288659793814433 | 2026-04-21 | evidence |
| Ling 3.0 Flash | InclusionAI | 0.272164948453608 | 2026-08-04 | evidence |
| Ling 3.0 Tiny | InclusionAI | 0.208247422680412 | 2026-08-06 | evidence |
| Llama 4 Maverick | Meta | 0.0371134020618557 | 2025-04-05 | evidence |
| Llama 4 Scout | Meta | 0.0329896907216495 | 2025-04-05 | evidence |
| LongCat 2.0 | LongCat | 0.131958762886598 | 2026-06-29 | evidence |
| Magistral Medium 1.2 | Mistral | 0.0556701030927835 | 2025-09-18 | evidence |
| Magistral Small 1.2 | Mistral | 0.045360824742268 | 2025-09-17 | evidence |
| Mercury 2 | Inception | 0.0948453608247423 | 2026-02-20 | evidence |
| MiMo-V2.5 | Xiaomi | 0.0865979381443299 | 2026-04-22 | evidence |
| MiMo-V2.5-Pro | Xiaomi | 0.0989690721649484 | 2026-04-22 | evidence |
| MiniMax-M2.7 | MiniMax | 0.0989690721649484 | 2026-03-18 | evidence |
| MiniMax-M3 | MiniMax | 0.152577319587629 | 2026-06-01 | evidence |
| Ministral 3 14B | Mistral | 0.065979381443299 | 2025-12-02 | evidence |
| Ministral 3 3B | Mistral | 0.0474226804123711 | 2025-12-02 | evidence |
| Ministral 3 8B | Mistral | 0.0371134020618557 | 2025-12-02 | evidence |
| Mistral Large 3 | Mistral | 0.0577319587628866 | 2025-12-02 | evidence |
| Mistral Medium 3.1 | Mistral | 0.0824742268041237 | 2025-08-12 | evidence |
| Mistral Medium 3.5 | Mistral | 0.150515463917526 | 2026-04-29 | evidence |
| Mistral Small 3.1 | Mistral | 0.0742268041237113 | 2025-03-17 | evidence |
| Mistral Small 3.2 | Mistral | 0.0618556701030928 | 2025-06-20 | evidence |
| Mistral Small 4 (Reasoning) | Mistral | 0.0494845360824742 | 2026-03-16 | evidence |
| Motif 3 | Motif Technologies | 0.352577319587629 | 2026-08-12 | evidence |
| Muse Glimmer (high) | Meta | 0.235051546391753 | 2026-08-10 | evidence |
| Muse Spark 1.1 (xhigh) | Meta | 0.317525773195876 | 2026-07-09 | evidence |
| Muse Spark 1.2 (xhigh) | Meta | 0.348453608247423 | 2026-08-05 | evidence |
| Nemotron 3 Super 120B A12B (Reasoning) | NVIDIA | 0.103092783505155 | 2026-03-11 | evidence |
| Nemotron 3 Ultra 550B A55B (Reasoning) | NVIDIA | 0.142268041237113 | 2026-06-04 | evidence |
| Nemotron 3.5 Lightning | NVIDIA | 0.088659793814433 | 2026-08-11 | evidence |
| Nex-N2-Pro | Nex AGI | 0.175257731958763 | 2026-06-02 | evidence |
| North Mini Code | Cohere | 0.0639175257731959 | 2026-06-09 | evidence |
| NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) | NVIDIA | 0.0597938144329897 | 2025-12-15 | evidence |
| o3-mini (high) | OpenAI | 0.0515463917525773 | 2025-01-31 | evidence |
| Qwen3 14B (Reasoning) | Qwen | 0.0556701030927835 | 2025-04-28 | evidence |
| Qwen3 235B A22B 2507 (Reasoning) | Qwen | 0.0783505154639175 | 2025-07-25 | evidence |
| Qwen3 30B A3B 2507 (Reasoning) | Qwen | 0.0536082474226804 | 2025-07-30 | evidence |
| Qwen3 32B (Reasoning) | Qwen | 0.0536082474226804 | 2025-04-28 | evidence |
| Qwen3 8B (Reasoning) | Qwen | 0.0474226804123711 | 2025-04-28 | evidence |
| Qwen3 Coder Next | Qwen | 0.0536082474226804 | 2026-02-03 | evidence |
| Qwen3 Next 80B A3B (Reasoning) | Qwen | 0.0618556701030928 | 2025-09-11 | evidence |
| Qwen3.5 122B A10B (Non-reasoning) | Qwen | 0.103092783505155 | 2026-02-24 | evidence |
| Qwen3.5 122B A10B (Reasoning) | Qwen | 0.152577319587629 | 2026-02-24 | evidence |
| Qwen3.5 35B A3B (Non-reasoning) | Qwen | 0.0494845360824742 | 2026-02-24 | evidence |
| Qwen3.5 397B A17B (Reasoning) | Qwen | 0.134020618556701 | 2026-02-16 | evidence |
| Qwen3.5 4B (Non-reasoning) | Qwen | 0.0432989690721649 | 2026-03-02 | evidence |
| Qwen3.5 4B (Reasoning) | Qwen | 0.0680412371134021 | 2026-03-02 | evidence |
| Qwen3.5 9B (Reasoning) | Qwen | 0.0701030927835052 | 2026-03-02 | evidence |
| Qwen3.6 27B (Non-reasoning) | Qwen | 0.0927835051546392 | 2026-04-22 | evidence |
| Qwen3.6 27B (Reasoning) | Qwen | 0.167010309278351 | 2026-04-22 | evidence |
| Qwen3.6 35B A3B (Non-reasoning) | Qwen | 0.0536082474226804 | 2026-04-16 | evidence |
| Qwen3.6 35B A3B (Reasoning) | Qwen | 0.0927835051546392 | 2026-04-16 | evidence |
| Qwen3.6 Plus | Qwen | 0.208247422680412 | 2026-04-02 | evidence |
| Qwen3.7 Max | Qwen | 0.117525773195876 | 2026-05-19 | evidence |
| Qwen3.7 Plus | Qwen | 0.175257731958763 | 2026-06-01 | evidence |
| Qwen3.8 2.4T A95B | Qwen | 0.490721649484536 | 2026-08-12 | evidence |
| Qwen3.8 27B (low) | Qwen | 0.321649484536082 | 2026-08-14 | evidence |
| Qwen3.8 27B (medium) | Qwen | 0.474226804123711 | 2026-08-14 | evidence |
| Qwen3.8 27B (Non-reasoning) | Qwen | 0.2 | 2026-08-14 | evidence |
| Qwen3.8 27B (xhigh) | Qwen | 0.480412371134021 | 2026-08-14 | evidence |
| Qwen3.8 Max | Qwen | 0.51340206185567 | 2026-08-03 | evidence |
| Ring-2.6-1T | InclusionAI | 0.179381443298969 | 2026-05-08 | evidence |
| Solar Open2 250B | Upstage | 0.216494845360825 | 2026-08-12 | evidence |
| Solar Pro 3 | Upstage | 0.0865979381443299 | 2026-04-06 | evidence |
| Solar Pro 4 | Upstage | 0.232989690721649 | 2026-08-06 | evidence |
| Step 3.7 Flash | StepFun | 0.119587628865979 | 2026-05-29 | evidence |
| Trinity Large Thinking | Arcee AI | 0.0577319587628866 | 2026-04-01 | evidence |
Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.