Benchmark

τ²-Bench Telecom

Agentic tool use

Categories
agentic, tool-use
Openness
unknown
Source
artificial_analysis
Reported scores
440

Reported scores

artificial_analysis

ModelOrganizationReported valueReportedEvidence
Apertus 70B InstructSwiss AI Initiative0.1286549707602342025-09-02evidence
Apertus 8B InstructSwiss AI Initiative0.1140350877192982025-09-02evidence
Apriel-v1.5-15B-ThinkerServiceNow0.6842105263157892025-09-30evidence
Apriel-v1.6-15B-ThinkerServiceNow0.6929824561403512025-11-25evidence
Claude 3 HaikuAnthropic0.2105263157894742024-03-04evidence
Claude 3.5 HaikuAnthropic0.2456140350877192024-10-22evidence
Claude 3.7 Sonnet (Non-reasoning)Anthropic0.52025-02-24evidence
Claude 3.7 Sonnet (Reasoning)Anthropic0.5467836257309942025-02-24evidence
Claude 4 Opus (Reasoning)Anthropic0.7339181286549712025-05-22evidence
Claude 4 Sonnet (Non-reasoning)Anthropic0.5233918128654972025-05-22evidence
Claude 4 Sonnet (Reasoning)Anthropic0.6461988304093572025-05-22evidence
Claude 4.1 Opus (Reasoning)Anthropic0.7136609336609342025-08-05evidence
Claude 4.5 Haiku (Non-reasoning)Anthropic0.3245614035087722025-10-15evidence
Claude 4.5 Haiku (Reasoning)Anthropic0.5467836257309942025-10-15evidence
Claude 4.5 Sonnet (Non-reasoning)Anthropic0.7046783625730992025-09-29evidence
Claude 4.5 Sonnet (Reasoning)Anthropic0.7807017543859652025-09-29evidence
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic0.9853801169590642026-06-09evidence
Claude Opus 4.5 (Non-reasoning)Anthropic0.8625730994152052025-11-24evidence
Claude Opus 4.5 (Reasoning)Anthropic0.8947368421052632025-11-24evidence
Claude Opus 4.6 (Adaptive Reasoning, Max Effort)Anthropic0.9210526315789472026-02-05evidence
Claude Opus 4.6 (Non-reasoning, High Effort)Anthropic0.8479532163742692026-02-05evidence
Claude Opus 4.7 (Adaptive Reasoning, Max Effort)Anthropic0.8859649122807022026-04-16evidence
Claude Opus 4.7 (Non-reasoning, High Effort)Anthropic0.7397660818713452026-04-16evidence
Claude Opus 4.8 (Adaptive Reasoning, Max Effort)Anthropic0.9444444444444452026-05-28evidence
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort)Anthropic0.7573099415204682026-02-17evidence
Claude Sonnet 4.6 (Non-reasoning, High Effort)Anthropic0.7953216374269012026-02-17evidence
Claude Sonnet 4.6 (Non-reasoning, Low Effort)Anthropic0.7894736842105262026-02-17evidence
Command ACohere0.1520467836257312025-03-13evidence
Command A+Cohere0.8070175438596492026-05-20evidence
DeepSeek R1 (Jan '25)DeepSeek0.1140350877192982025-01-20evidence
DeepSeek R1 0528 (May '25)DeepSeek0.3654970760233922025-05-28evidence
DeepSeek R1 0528 Qwen3 8BDeepSeek02025-05-29evidence
DeepSeek R1 Distill Llama 70BDeepSeek0.2192982456140352025-01-20evidence
DeepSeek V3 (Dec '24)DeepSeek0.2280701754385962024-12-26evidence
DeepSeek V3 0324DeepSeek0.4707602339181292025-03-25evidence
DeepSeek V3.1 (Non-reasoning)DeepSeek0.3479532163742692025-08-21evidence
DeepSeek V3.1 (Reasoning)DeepSeek0.3742690058479532025-08-21evidence
DeepSeek V3.1 Terminus (Non-reasoning)DeepSeek0.3713450292397662025-09-22evidence
DeepSeek V3.1 Terminus (Reasoning)DeepSeek0.3713450292397662025-09-22evidence
DeepSeek V3.2 (Non-reasoning)DeepSeek0.7894736842105262025-12-01evidence
DeepSeek V3.2 (Reasoning)DeepSeek0.9064327485380122025-12-01evidence
DeepSeek V3.2 Exp (Non-reasoning)DeepSeek0.3391812865497082025-09-29evidence
DeepSeek V3.2 Exp (Reasoning)DeepSeek0.3391812865497082025-09-29evidence
DeepSeek V3.2 SpecialeDeepSeek02025-12-01evidence
DeepSeek V4 Flash (Non-reasoning)DeepSeek0.9444444444444452026-04-24evidence
DeepSeek V4 Flash (Reasoning, High Effort)DeepSeek0.9561403508771932026-04-24evidence
DeepSeek V4 Flash (Reasoning, Max Effort)DeepSeek0.9502923976608192026-04-24evidence
DeepSeek V4 Pro (Non-reasoning)DeepSeek0.9122807017543862026-04-24evidence
DeepSeek V4 Pro (Reasoning, High Effort)DeepSeek0.9415204678362572026-04-24evidence
DeepSeek V4 Pro (Reasoning, Max Effort)DeepSeek0.9619883040935672026-04-24evidence
Devstral 2Mistral0.2485380116959062025-12-09evidence
Devstral MediumMistral0.1988304093567252025-07-10evidence
Devstral Small (Jul '25)Mistral0.2836257309941522025-07-10evidence
Devstral Small (May '25)Mistral0.3801169590643272025-05-21evidence
Devstral Small 2Mistral0.2339181286549712025-12-09evidence
Doubao Seed CodeByteDance Seed0.581871345029242025-11-11evidence
ERNIE 4.5 300B A47BBaidu02025-06-30evidence
ERNIE 5.0 Thinking PreviewBaidu0.8391812865497072025-11-13evidence
Exaone 4.0 1.2B (Non-reasoning)LG AI Research0.2046783625730992025-07-15evidence
Exaone 4.0 1.2B (Reasoning)LG AI Research0.163742690058482025-07-15evidence
EXAONE 4.0 32B (Non-reasoning)LG AI Research0.04093567251461992025-07-15evidence
EXAONE 4.0 32B (Reasoning)LG AI Research0.1725146198830412025-07-15evidence
EXAONE 4.5 33BLG AI Research0.7807017543859652026-04-09evidence
Falcon-H1R-7BTII UAE0.2777777777777782026-01-04evidence
Gemini 2.0 Flash (Feb '25)Google0.2953216374269012025-02-05evidence
Gemini 2.5 Flash (Non-reasoning)Google0.1491228070175442025-05-20evidence
Gemini 2.5 Flash (Reasoning)Google0.3157894736842112025-05-20evidence
Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning)Google0.2836257309941522025-09-25evidence
Gemini 2.5 Flash Preview (Sep '25) (Reasoning)Google0.4561403508771932025-09-25evidence
Gemini 2.5 Flash-Lite (Non-reasoning)Google0.1900584795321642025-06-17evidence
Gemini 2.5 Flash-Lite (Reasoning)Google0.1842105263157892025-06-17evidence
Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning)Google0.3040935672514622025-09-25evidence
Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning)Google0.3070175438596492025-09-08evidence
Gemini 2.5 ProGoogle0.540935672514622025-06-05evidence
Gemini 3 Flash Preview (Non-reasoning)Google0.4327485380116962025-12-17evidence
Gemini 3 Flash Preview (Reasoning)Google0.8040935672514622025-12-17evidence
Gemini 3 Pro Preview (high)Google0.8713450292397662025-11-18evidence
Gemini 3 Pro Preview (low)Google0.6812865497076022025-11-18evidence
Gemini 3.1 Flash-LiteGoogle0.3128654970760232026-03-03evidence
Gemini 3.1 Pro PreviewGoogle0.9561403508771932026-02-19evidence
Gemini 3.5 Flash (high)Google0.9532163742690062026-05-19evidence
Gemini 3.5 Flash (medium)Google0.9561403508771932026-05-19evidence
Gemini 3.5 Flash (minimal)Google0.5877192982456142026-05-19evidence
Gemma 3 12B InstructGoogle0.1081871345029242025-03-12evidence
Gemma 3 1B InstructGoogle0.1052631578947372025-03-13evidence
Gemma 3 270MGoogle0.09064327485380122025-08-14evidence
Gemma 3 27B InstructGoogle0.1052631578947372025-03-12evidence
Gemma 3 4B InstructGoogle0.04970760233918132025-03-12evidence
Gemma 3n E2B InstructGoogle02025-06-26evidence
Gemma 3n E4B InstructGoogle0.04970760233918132025-06-26evidence
Gemma 4 12B (Non-reasoning)Google0.3187134502923982026-06-03evidence
Gemma 4 12B (Reasoning)Google0.3625730994152052026-06-03evidence
Gemma 4 26B A4B (Non-reasoning)Google0.4035087719298252026-04-02evidence
Gemma 4 26B A4B (Reasoning)Google0.4356725146198832026-04-02evidence
Gemma 4 31B (Non-reasoning)Google0.6549707602339182026-04-02evidence
Gemma 4 31B (Reasoning)Google0.5994152046783632026-04-02evidence
Gemma 4 E2B (Non-reasoning)Google0.2222222222222222026-04-02evidence
Gemma 4 E2B (Reasoning)Google0.2076023391812872026-04-02evidence
Gemma 4 E4B (Non-reasoning)Google0.2602339181286552026-04-03evidence
Gemma 4 E4B (Reasoning)Google0.2076023391812872026-04-03evidence
GLM 5V Turbo (Reasoning)Z.ai0.9853801169590642026-04-01evidence
GLM-4.5 (Reasoning)Z.ai0.4298245614035092025-07-28evidence
GLM-4.5-AirZ.ai0.4649122807017542025-07-28evidence
GLM-4.5V (Non-reasoning)Z.ai0.1959064327485382025-08-11evidence
GLM-4.5V (Reasoning)Z.ai0.2251461988304092025-08-11evidence
GLM-4.6 (Non-reasoning)Z.ai0.7690058479532162025-09-30evidence
GLM-4.6 (Reasoning)Z.ai0.7046783625730992025-09-30evidence
GLM-4.6V (Non-reasoning)Z.ai0.3070175438596492025-12-08evidence
GLM-4.6V (Reasoning)Z.ai0.3157894736842112025-12-08evidence
GLM-4.7 (Non-reasoning)Z.ai0.9415204678362572025-12-22evidence
GLM-4.7 (Reasoning)Z.ai0.959064327485382025-12-22evidence
GLM-4.7-Flash (Non-reasoning)Z.ai0.918128654970762026-01-19evidence
GLM-4.7-Flash (Reasoning)Z.ai0.9883040935672512026-01-19evidence
GLM-5 (Non-reasoning)Z.ai0.9736842105263162026-02-11evidence
GLM-5 (Reasoning)Z.ai0.9824561403508772026-02-11evidence
GLM-5-TurboZ.ai0.9853801169590642026-03-15evidence
GLM-5.1 (Non-reasoning)Z.ai0.9707602339181292026-04-07evidence
GLM-5.1 (Reasoning)Z.ai0.9766081871345032026-04-07evidence
GLM-5.2 (max)Z.ai0.9912280701754392026-06-16evidence
GPT-4.1OpenAI0.4707602339181292025-04-14evidence
GPT-4.1 miniOpenAI0.5292397660818712025-04-14evidence
GPT-4.1 nanoOpenAI0.1725146198830412025-04-14evidence
GPT-4o (Aug '24)OpenAI0.2894736842105262024-08-06evidence
GPT-4o (Nov '24)OpenAI0.2514619883040942024-11-20evidence
GPT-5 (ChatGPT)OpenAI02025-08-07evidence
GPT-5 (high)OpenAI0.8479532163742692025-08-07evidence
GPT-5 (low)OpenAI0.8421052631578952025-08-07evidence
GPT-5 (medium)OpenAI0.8654970760233922025-08-07evidence
GPT-5 (minimal)OpenAI0.6695906432748542025-08-07evidence
GPT-5 Codex (high)OpenAI0.8684210526315792025-09-23evidence
GPT-5 mini (high)OpenAI0.6842105263157892025-08-07evidence
GPT-5 mini (medium)OpenAI0.7105263157894742025-08-07evidence
GPT-5 mini (minimal)OpenAI0.3187134502923982025-08-07evidence
GPT-5 nano (high)OpenAI0.3654970760233922025-08-07evidence
GPT-5 nano (medium)OpenAI0.3040935672514622025-08-07evidence
GPT-5 nano (minimal)OpenAI0.2573099415204682025-08-07evidence
GPT-5.1 (high)OpenAI0.8187134502923982025-11-13evidence
GPT-5.1 (Non-reasoning)OpenAI0.4649122807017542025-11-13evidence
GPT-5.1 Codex (high)OpenAI0.8304093567251462025-11-13evidence
GPT-5.1 Codex mini (high)OpenAI0.6286549707602342025-11-13evidence
GPT-5.2 (medium)OpenAI0.7426900584795322025-12-11evidence
GPT-5.2 (Non-reasoning)OpenAI0.4649122807017542025-12-11evidence
GPT-5.2 (xhigh)OpenAI0.8479532163742692025-12-11evidence
GPT-5.2 Codex (xhigh)OpenAI0.9210526315789472025-12-11evidence
GPT-5.3 Codex (xhigh)OpenAI0.8596491228070182026-02-05evidence
GPT-5.4 (low)OpenAI0.7456140350877192026-03-05evidence
GPT-5.4 (Non-reasoning)OpenAI0.3596491228070182026-03-05evidence
GPT-5.4 (xhigh)OpenAI0.8713450292397662026-03-05evidence
GPT-5.4 mini (medium)OpenAI0.3654970760233922026-03-17evidence
GPT-5.4 mini (Non-Reasoning)OpenAI0.2339181286549712026-03-17evidence
GPT-5.4 mini (xhigh)OpenAI0.8333333333333332026-03-17evidence
GPT-5.4 nano (medium)OpenAI0.5263157894736842026-03-17evidence
GPT-5.4 nano (Non-Reasoning)OpenAI0.3479532163742692026-03-17evidence
GPT-5.4 nano (xhigh)OpenAI0.7602339181286552026-03-17evidence
GPT-5.5 (high)OpenAI0.9298245614035092026-04-23evidence
GPT-5.5 (low)OpenAI0.8391812865497082026-04-23evidence
GPT-5.5 (medium)OpenAI0.918128654970762026-04-23evidence
GPT-5.5 (Non-reasoning)OpenAI0.6929824561403512026-04-23evidence
GPT-5.5 (xhigh)OpenAI0.938596491228072026-04-23evidence
GPT-5.5 Instant (May 2026)OpenAI0.4941520467836262026-05-05evidence
GPT-5.6 Sol (high)OpenAI0.8333333333333332026-07-09evidence
GPT-5.6 Sol (low)OpenAI0.7602339181286552026-07-09evidence
GPT-5.6 Sol (max)OpenAI0.8508771929824562026-07-09evidence
GPT-5.6 Sol (medium)OpenAI0.8099415204678362026-07-09evidence
GPT-5.6 Sol (xhigh)OpenAI0.8479532163742692026-07-09evidence
GPT-5.6 Terra (high)OpenAI0.7836257309941522026-07-09evidence
GPT-5.6 Terra (low)OpenAI0.6052631578947372026-07-09evidence
GPT-5.6 Terra (max)OpenAI0.8625730994152052026-07-09evidence
GPT-5.6 Terra (medium)OpenAI0.7280701754385962026-07-09evidence
GPT-5.6 Terra (xhigh)OpenAI0.8040935672514622026-07-09evidence
gpt-oss-120b (high)OpenAI0.6578947368421052025-08-05evidence
gpt-oss-120b (low)OpenAI0.4502923976608192025-08-05evidence
gpt-oss-20b (high)OpenAI0.602339181286552025-08-05evidence
gpt-oss-20b (low)OpenAI0.5029239766081872025-08-05evidence
Granite 3.3 8B (Non-reasoning)IBM0.1052631578947372025-04-16evidence
Granite 4.0 1BIBM0.2280701754385962025-10-28evidence
Granite 4.0 350MIBM0.1315789473684212025-10-28evidence
Granite 4.0 H 1BIBM0.1959064327485382025-10-28evidence
Granite 4.0 H 350MIBM0.1461988304093572025-10-28evidence
Granite 4.0 H SmallIBM0.1725146198830412025-09-22evidence
Granite 4.0 MicroIBM0.1257309941520472025-09-22evidence
Granite 4.1 30BIBM0.4210526315789472026-04-29evidence
Granite 4.1 3BIBM0.1959064327485382026-04-29evidence
Granite 4.1 8BIBM0.2777777777777782026-04-29evidence
Grok 3xAI0.4883040935672522025-02-19evidence
Grok 3 mini Reasoning (high)xAI0.9035087719298242025-02-19evidence
Grok 4xAI0.74853801172025-07-10evidence
Grok 4 Fast (Non-reasoning)xAI0.6374269005847952025-09-19evidence
Grok 4 Fast (Reasoning)xAI0.6578947368421052025-09-19evidence
Grok 4.1 Fast (Non-reasoning)xAI0.6374269005847952025-11-19evidence
Grok 4.1 Fast (Reasoning)xAI0.9327485380116962025-11-19evidence
Grok 4.20 0309 (Non-reasoning)xAI0.6959064327485382026-03-10evidence
Grok 4.20 0309 (Reasoning)xAI0.9649122807017542026-03-10evidence
Grok 4.20 0309 v2 (Non-reasoning)xAI0.5994152046783632026-04-07evidence
Grok 4.20 0309 v2 (Reasoning)xAI0.9298245614035092026-04-07evidence
Grok 4.3 (high)xAI0.9766081871345032026-04-30evidence
Grok 4.3 (low)xAI0.8888888888888892026-04-30evidence
Grok 4.3 (medium)xAI0.9122807017543862026-04-30evidence
Grok 4.3 (Non-reasoning)xAI0.6578947368421052026-04-30evidence
Grok Code Fast 1xAI0.7573099415204682025-08-28evidence
Hermes 4 - Llama-3.1 405B (Non-reasoning)Nous Research0.2660818713450292025-08-27evidence
Hermes 4 - Llama-3.1 405B (Reasoning)Nous Research0.2222222222222222025-08-27evidence
Hermes 4 - Llama-3.1 70B (Non-reasoning)Nous Research0.2163742690058482025-08-27evidence
Hermes 4 - Llama-3.1 70B (Reasoning)Nous Research0.2251461988304092025-08-27evidence
Hy3-preview (Non-reasoning)Tencent0.6754385964912282026-04-23evidence
Hy3-preview (Reasoning)Tencent0.9269005847953222026-04-23evidence
HyperCLOVA X SEED Think (32B)Naver0.8742690058479532025-12-26evidence
HyperNova 60B 2605Multiverse Computing0.6315789473684212026-05-26evidence
INTELLECT-3Prime Intellect0.2660818713450292025-11-27evidence
Jamba 1.7 LargeAI21 Labs0.1345029239766082025-07-07evidence
Jamba 1.7 MiniAI21 Labs0.1257309941520472025-07-07evidence
Jamba Reasoning 3BAI21 Labs0.1578947368421052025-10-08evidence
JT-35B-FlashChina Mobile0.9912280701754392026-05-14evidence
JT-MINIChina Mobile0.9298245614035092026-04-15evidence
K-EXAONE (Non-reasoning)LG AI Research0.5906432748538012025-12-31evidence
K-EXAONE (Reasoning)LG AI Research0.7426900584795322025-12-31evidence
K2 Think V2MBZUAI Institute of Foundation Models0.2543859649122812025-12-15evidence
K2-V2 (high)MBZUAI Institute of Foundation Models0.2777777777777782025-12-05evidence
K2-V2 (low)MBZUAI Institute of Foundation Models0.2076023391812872025-12-05evidence
K2-V2 (medium)MBZUAI Institute of Foundation Models0.2485380116959062025-12-05evidence
KAT Coder Pro V2KwaiKAT0.8947368421052632026-03-27evidence
KAT-Coder-Pro V1KwaiKAT0.8859649122807022025-11-11evidence
Kimi K2Moonshot AI0.6111111111111112025-07-11evidence
Kimi K2 0905Moonshot AI0.7339181286549712025-09-05evidence
Kimi K2 ThinkingMoonshot AI0.9298245614035092025-11-06evidence
Kimi K2.5 (Non-reasoning)Moonshot AI0.8128654970760232026-01-27evidence
Kimi K2.5 (Reasoning)Moonshot AI0.959064327485382026-01-27evidence
Kimi K2.6Moonshot AI0.959064327485382026-04-20evidence
Kimi K2.6 (Non-reasoning)Moonshot AI0.938596491228072026-04-20evidence
Kimi K2.7 CodeMoonshot AI0.9005847953216372026-06-12evidence
Kimi Linear 48B A3B InstructMoonshot AI02025-10-30evidence
LFM2 1.2BLiquid AI0.1257309941520472025-07-10evidence
LFM2 2.6BLiquid AI0.1345029239766082025-09-23evidence
LFM2 24B A2BLiquid AI0.1111111111111112026-02-25evidence
LFM2 8B A1BLiquid AI0.1052631578947372025-10-07evidence
LFM2.5-1.2B-InstructLiquid AI0.1081871345029242026-01-05evidence
LFM2.5-1.2B-ThinkingLiquid AI0.1959064327485382026-01-20evidence
LFM2.5-8B-A1BLiquid AI0.1608187134502922026-05-28evidence
LFM2.5-VL-1.6BLiquid AI0.08479532163742692026-01-05evidence
Ling 2.6 FlashInclusionAI0.8596491228070182026-04-21evidence
Ling-1TInclusionAI0.3274853801169592025-10-08evidence
Ling-2.6-1TInclusionAI0.897660818713452026-04-23evidence
Ling-flash-2.0InclusionAI0.2076023391812872025-09-17evidence
Ling-mini-2.0InclusionAI0.1315789473684212025-09-09evidence
Llama 3 Instruct 70BMeta02024-04-18evidence
Llama 3 Instruct 8BMeta02024-04-18evidence
Llama 3.1 Instruct 405BMeta0.1900584795321642024-07-23evidence
Llama 3.1 Instruct 70BMeta0.1520467836257312024-07-23evidence
Llama 3.1 Instruct 8BMeta0.163742690058482024-07-23evidence
Llama 3.1 Nemotron Instruct 70BNVIDIA0.2309941520467842024-10-15evidence
Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning)NVIDIA0.1169590643274852025-05-20evidence
Llama 3.1 Nemotron Ultra 253B v1 (Reasoning)NVIDIA0.1140350877192982025-04-07evidence
Llama 3.2 Instruct 11B (Vision)Meta0.1461988304093572024-09-25evidence
Llama 3.2 Instruct 1BMeta02024-09-25evidence
Llama 3.2 Instruct 3BMeta0.2105263157894742024-09-25evidence
Llama 3.3 Instruct 70BMeta0.2660818713450292024-12-06evidence
Llama 3.3 Nemotron Super 49B v1 (Reasoning)NVIDIA0.2690058479532162025-03-18evidence
Llama 4 MaverickMeta0.1783625730994152025-04-05evidence
Llama 4 ScoutMeta0.1549707602339182025-04-05evidence
Llama Nemotron Super 49B v1.5 (Non-reasoning)NVIDIA0.2514619883040942025-07-25evidence
Llama Nemotron Super 49B v1.5 (Reasoning)NVIDIA0.2807017543859652025-07-25evidence
LongCat Flash LiteLongCat0.7953216374269012026-01-28evidence
Magistral Medium 1Mistral0.2309941520467842025-06-10evidence
Magistral Medium 1.2Mistral0.520467836257312025-09-18evidence
Magistral Small 1Mistral0.2660818713450292025-06-10evidence
Magistral Small 1.2Mistral0.2777777777777782025-09-17evidence
Mercury 2Inception0.7076023391812872026-02-20evidence
Mi:dm K 2.5 ProKorea Telecom0.8654970760233922025-12-11evidence
Mi:dm K 2.5 Pro PreviewKorea Telecom0.4941520467836262025-12-11evidence
MiMo-V2-Flash (Feb 2026)Xiaomi0.9327485380116962025-12-16evidence
MiMo-V2-Flash (Non-reasoning)Xiaomi0.8391812865497082025-12-16evidence
MiMo-V2-Flash (Reasoning)Xiaomi0.9502923976608192025-12-16evidence
MiMo-V2-OmniXiaomi0.9122807017543862026-03-19evidence
MiMo-V2-Omni-0327Xiaomi0.8801169590643282026-03-27evidence
MiMo-V2-ProXiaomi0.9502923976608192026-03-18evidence
MiMo-V2.5Xiaomi0.9064327485380122026-04-22evidence
MiMo-V2.5-ProXiaomi0.9415204678362572026-04-22evidence
MiMo-V2.5-Pro (Non-reasoning)Xiaomi0.7251461988304092026-04-22evidence
MiniCPM-V 4.6 1.3BOpenBMB0.877192982456142026-05-11evidence
MiniCPM5-1B (Non-reasoning)OpenBMB0.8245614035087722026-05-25evidence
MiniCPM5-1B (Reasoning)OpenBMB0.8099415204678362026-05-25evidence
MiniMax M1 40kMiniMax0.3157894736842112025-06-17evidence
MiniMax M1 80kMiniMax0.3421052631578952025-06-17evidence
MiniMax-M2MiniMax0.8684210526315792025-10-26evidence
MiniMax-M2.1MiniMax0.8538011695906432025-12-23evidence
MiniMax-M2.5MiniMax0.9532163742690062026-02-12evidence
MiniMax-M2.7MiniMax0.8479532163742692026-03-18evidence
MiniMax-M3MiniMax0.8888888888888892026-06-01evidence
Ministral 3 14BMistral0.2719298245614042025-12-02evidence
Ministral 3 3BMistral0.2485380116959062025-12-02evidence
Ministral 3 8BMistral0.2660818713450292025-12-02evidence
Mistral 7B InstructMistral02023-09-27evidence
Mistral Large 2 (Jul '24)Mistral0.3304093567251462024-07-24evidence
Mistral Large 2 (Nov '24)Mistral0.3070175438596492024-11-18evidence
Mistral Large 3Mistral0.2456140350877192025-12-02evidence
Mistral Medium 3Mistral0.2426900584795322025-05-07evidence
Mistral Medium 3.1Mistral0.4064327485380122025-08-12evidence
Mistral Medium 3.5Mistral0.9415204678362572026-04-29evidence
Mistral Small 3Mistral0.1959064327485382025-01-30evidence
Mistral Small 3.1Mistral0.2514619883040942025-03-17evidence
Mistral Small 3.2Mistral0.2953216374269012025-06-20evidence
Mistral Small 4 (Non-reasoning)Mistral0.1842105263157892026-03-16evidence
Mistral Small 4 (Reasoning)Mistral0.4122807017543862026-03-16evidence
Molmo 7B-DAllen Institute for AI02024-09-25evidence
Molmo2-8BAllen Institute for AI02025-12-11evidence
Motif-2-12.7B-ReasoningMotif Technologies0.4649122807017542025-12-04evidence
Muse SparkMeta0.9152046783625732026-04-08evidence
Nanbeige4.1-3BNanbeige0.2163742690058482026-02-11evidence
Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIA0.4532163742690062026-04-29evidence
Nemotron 3 Super 120B A12B (Reasoning)NVIDIA0.6783625730994152026-03-11evidence
Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIA0.8333333333333332026-06-04evidence
Nemotron Cascade 2 30B A3BNVIDIA0.5321637426900592026-03-19evidence
Nex-N2-ProNex AGI0.8157894736842112026-06-02evidence
North Mini CodeCohere0.3742690058479532026-06-09evidence
Nova 2.0 Lite (high)Amazon0.7280701754385962025-10-29evidence
Nova 2.0 Lite (low)Amazon0.7192982456140352025-10-29evidence
Nova 2.0 Lite (medium)Amazon0.7573099415204682025-10-29evidence
Nova 2.0 Lite (Non-reasoning)Amazon0.6198830409356722025-10-29evidence
Nova 2.0 Omni (low)Amazon0.6783625730994152025-11-26evidence
Nova 2.0 Omni (medium)Amazon0.8040935672514622025-11-26evidence
Nova 2.0 Omni (Non-reasoning)Amazon0.4473684210526322025-11-26evidence
Nova 2.0 Pro Preview (low)Amazon0.9064327485380122025-11-27evidence
Nova 2.0 Pro Preview (medium)Amazon0.9269005847953222025-11-27evidence
Nova 2.0 Pro Preview (Non-reasoning)Amazon0.7163742690058482025-11-27evidence
Nova LiteAmazon0.1754385964912282024-12-03evidence
Nova MicroAmazon0.1403508771929822024-12-03evidence
Nova PremierAmazon0.3830409356725152025-04-30evidence
Nova ProAmazon0.1403508771929822024-12-03evidence
NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)NVIDIA0.2543859649122812025-12-15evidence
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)NVIDIA0.4093567251461992025-12-15evidence
NVIDIA Nemotron 3 Nano 4BNVIDIA0.2807017543859652026-03-16evidence
NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)NVIDIA0.1929824561403512025-10-28evidence
NVIDIA Nemotron Nano 12B v2 VL (Reasoning)NVIDIA0.2134502923976612025-10-28evidence
NVIDIA Nemotron Nano 9B V2 (Non-reasoning)NVIDIA0.2339181286549712025-08-18evidence
NVIDIA Nemotron Nano 9B V2 (Reasoning)NVIDIA0.2192982456140352025-08-18evidence
o1OpenAI0.6257309941520472024-12-05evidence
o3OpenAI0.8070175438596492025-04-16evidence
o3-miniOpenAI0.2865497076023392025-01-31evidence
o3-mini (high)OpenAI0.3128654970760232025-01-31evidence
o4-mini (high)OpenAI0.5555555555555562025-04-16evidence
OLMo 2 32BAllen Institute for AI02025-03-13evidence
OLMo 2 7BAllen Institute for AI02024-11-26evidence
Olmo 3 32B ThinkAllen Institute for AI02025-11-20evidence
Olmo 3 7B InstructAllen Institute for AI0.1257309941520472025-11-20evidence
Olmo 3 7B ThinkAllen Institute for AI02025-11-20evidence
Olmo 3.1 32B InstructAllen Institute for AI0.2134502923976612026-01-13evidence
Olmo 3.1 32B ThinkAllen Institute for AI02025-12-12evidence
Phi-3 Mini Instruct 3.8BMicrosoft02024-04-23evidence
Phi-4Microsoft02024-12-12evidence
Phi-4 Mini InstructMicrosoft0.08187134502923982024-02-26evidence
Pixtral LargeMistral0.3654970760233922024-11-18evidence
Qwen2.5 Instruct 72BQwen0.3450292397660822024-09-19evidence
Qwen3 0.6B (Non-reasoning)Qwen0.1461988304093572025-04-28evidence
Qwen3 0.6B (Reasoning)Qwen0.2105263157894742025-04-28evidence
Qwen3 1.7B (Non-reasoning)Qwen0.2163742690058482025-04-28evidence
Qwen3 1.7B (Reasoning)Qwen0.2602339181286552025-04-28evidence
Qwen3 14B (Non-reasoning)Qwen0.3216374269005852025-04-28evidence
Qwen3 14B (Reasoning)Qwen0.3450292397660822025-04-28evidence
Qwen3 235B A22B (Non-reasoning)Qwen0.2719298245614042025-04-28evidence
Qwen3 235B A22B (Reasoning)Qwen0.2397660818713452025-04-28evidence
Qwen3 235B A22B 2507 (Reasoning)Qwen0.5321637426900592025-07-25evidence
Qwen3 235B A22B 2507 InstructQwen0.3333333333333332025-07-21evidence
Qwen3 30B A3B (Non-reasoning)Qwen0.2222222222222222025-04-28evidence
Qwen3 30B A3B (Reasoning)Qwen0.2602339181286552025-04-28evidence
Qwen3 30B A3B 2507 (Reasoning)Qwen0.2807017543859652025-07-30evidence
Qwen3 30B A3B 2507 InstructQwen0.102339181286552025-07-29evidence
Qwen3 32B (Reasoning)Qwen0.2982456140350882025-04-28evidence
Qwen3 4B (Reasoning)Qwen0.1900584795321642025-04-28evidence
Qwen3 4B 2507 (Reasoning)Qwen0.2543859649122812025-08-06evidence
Qwen3 4B 2507 InstructQwen0.2660818713450292025-08-06evidence
Qwen3 8B (Non-reasoning)Qwen0.2485380116959062025-04-28evidence
Qwen3 8B (Reasoning)Qwen0.2777777777777782025-04-28evidence
Qwen3 Coder 30B A3B InstructQwen0.3450292397660822025-07-31evidence
Qwen3 Coder 480B A35B InstructQwen0.4356725146198832025-07-22evidence
Qwen3 Coder NextQwen0.7953216374269012026-02-03evidence
Qwen3 MaxQwen0.7426900584795322025-09-23evidence
Qwen3 Max (Preview)Qwen0.3274853801169592025-09-05evidence
Qwen3 Max ThinkingQwen0.8362573099415212026-01-26evidence
Qwen3 Max Thinking (Preview)Qwen0.836257309941522025-11-03evidence
Qwen3 Next 80B A3B (Reasoning)Qwen0.4152046783625732025-09-11evidence
Qwen3 Next 80B A3B InstructQwen0.2163742690058482025-09-11evidence
Qwen3 Omni 30B A3B (Reasoning)Qwen0.2134502923976612025-09-22evidence
Qwen3 Omni 30B A3B InstructQwen0.163742690058482025-09-22evidence
Qwen3 VL 235B A22B (Reasoning)Qwen0.540935672514622025-09-23evidence
Qwen3 VL 235B A22B InstructQwen0.3508771929824562025-09-23evidence
Qwen3 VL 30B A3B (Reasoning)Qwen0.1988304093567252025-10-03evidence
Qwen3 VL 30B A3B InstructQwen0.1900584795321642025-10-03evidence
Qwen3 VL 32B (Reasoning)Qwen0.4561403508771932025-10-21evidence
Qwen3 VL 32B InstructQwen0.2923976608187132025-10-21evidence
Qwen3 VL 4B (Reasoning)Qwen0.1549707602339182025-10-14evidence
Qwen3 VL 4B InstructQwen0.2339181286549712025-10-14evidence
Qwen3 VL 8B (Reasoning)Qwen0.2251461988304092025-10-14evidence
Qwen3 VL 8B InstructQwen0.2923976608187132025-10-14evidence
Qwen3.5 0.8B (Non-reasoning)Qwen0.6520467836257312026-03-02evidence
Qwen3.5 0.8B (Reasoning)Qwen0.4766081871345032026-03-02evidence
Qwen3.5 122B A10B (Non-reasoning)Qwen0.8450292397660822026-02-24evidence
Qwen3.5 122B A10B (Reasoning)Qwen0.9356725146198832026-02-24evidence
Qwen3.5 27B (Non-reasoning)Qwen0.8713450292397662026-02-24evidence
Qwen3.5 27B (Reasoning)Qwen0.938596491228072026-02-24evidence
Qwen3.5 2B (Non-reasoning)Qwen0.8157894736842112026-03-02evidence
Qwen3.5 2B (Reasoning)Qwen0.6900584795321642026-03-02evidence
Qwen3.5 35B A3B (Non-reasoning)Qwen0.8625730994152052026-02-24evidence
Qwen3.5 35B A3B (Reasoning)Qwen0.8918128654970762026-02-24evidence
Qwen3.5 397B A17B (Non-reasoning)Qwen0.8391812865497082026-02-16evidence
Qwen3.5 397B A17B (Reasoning)Qwen0.9561403508771932026-02-16evidence
Qwen3.5 4B (Non-reasoning)Qwen0.877192982456142026-03-02evidence
Qwen3.5 4B (Reasoning)Qwen0.9210526315789472026-03-02evidence
Qwen3.5 9B (Non-reasoning)Qwen0.8508771929824562026-03-02evidence
Qwen3.5 9B (Reasoning)Qwen0.8684210526315792026-03-02evidence
Qwen3.5 Omni FlashQwen0.8450292397660822026-03-30evidence
Qwen3.5 Omni PlusQwen0.8830409356725152026-03-30evidence
Qwen3.6 27B (Non-reasoning)Qwen0.9356725146198832026-04-22evidence
Qwen3.6 27B (Reasoning)Qwen0.9415204678362572026-04-22evidence
Qwen3.6 35B A3B (Non-reasoning)Qwen0.8508771929824562026-04-16evidence
Qwen3.6 35B A3B (Reasoning)Qwen0.9532163742690062026-04-16evidence
Qwen3.6 Max PreviewQwen0.959064327485382026-04-20evidence
Qwen3.6 PlusQwen0.9766081871345032026-04-02evidence
Qwen3.7 MaxQwen0.9473684210526322026-05-19evidence
Qwen3.7 PlusQwen0.9298245614035092026-06-01evidence
Reka Flash 3Reka AI02025-03-10evidence
Ring-1TInclusionAI0.2631578947368422025-10-13evidence
Ring-2.6-1TInclusionAI0.9239766081871352026-05-08evidence
Ring-flash-2.0InclusionAI02025-09-19evidence
Sarvam 105B (high)Sarvam0.4678362573099412026-03-06evidence
Sarvam 30B (high)Sarvam0.3450292397660822026-03-06evidence
Sarvam M (Reasoning)Sarvam02025-05-23evidence
Seed-OSS-36B-InstructByteDance Seed0.4941520467836262025-08-20evidence
Solar MiniUpstage0.2017543859649122024-01-25evidence
Solar Open 100B (Reasoning)Upstage0.4824561403508772025-12-17evidence
Solar Pro 2 (Non-reasoning)Upstage0.3187134502923982025-07-09evidence
Solar Pro 2 (Reasoning)Upstage0.2807017543859652025-07-09evidence
Solar Pro 3Upstage0.8625730994152052026-04-06evidence
Step 3.5 FlashStepFun0.9444444444444442026-02-02evidence
Step 3.5 Flash 2603StepFun0.8742690058479532026-04-02evidence
Step 3.7 FlashStepFun0.9853801169590642026-05-29evidence
Step3 VL 10BStepFun0.1608187134502922026-01-20evidence
Tiny Aya GlobalCohere02026-02-17evidence
Tri-21B-ThinkTrillion Labs0.8099415204678362026-02-10evidence
Tri-21B-think PreviewTrillion Labs0.9327485380116962026-02-10evidence
Trinity Large ThinkingArcee AI0.9005847953216372026-04-01evidence

Scores are partitioned by the source that reported them and are never merged into a single cross-source ranking, because the sources measure different things and say so.