Agents

τ²-bench

Sierra Lab's next-gen tool-use benchmark — multi-turn customer-service workflows under realistic constraints.

Source

296Models

99.1Top score

35.1Median

State of the art over time

Each point is a model at its release date; the line traces the best score to date.

Ranking

#1	JT-35B-FlashChina Mobile	99.1	99.1
#2	GLM 4.7 FlashZhipu AI	98.8	98.8
#3	GLM 5 TurboZhipu AI	98.5	98.5
#4	GLM 5V TurboZhipu AI	98.5	98.5
#5	GLM-5Zhipu AI	98.2	98.2
#6	Qwen3.6 PlusAlibaba	97.7	97.7
#7	GLM 5.1Zhipu AI	97.7	97.7
#8	Grok 4.3xAI	97.7	97.7
#9	Grok 4.20 0309xAI	96.5	96.5
#10	DeepSeek-V4-ProDeepSeek	96.2	96.2
#11	GLM 4.7Zhipu AI	95.9	95.9
#12	Qwen3.6 MaxAlibaba	95.9	95.9
#13	Kimi K2.6Moonshot AI	95.9	95.9
#14	Kimi K2.5Moonshot AI	95.9	95.9
#15	Qwen3.5 397B A17BAlibaba	95.6	95.6
#16	DeepSeek-V4-FlashDeepSeek	95.6	95.6
#17	Gemini 3.5 FlashGoogle	95.6	95.6
#18	Gemini 3.1 ProGoogle	95.6	95.6
#19	MiniMax M2.5MiniMax	95.3	95.3
#20	Qwen3.6 35B A3BAlibaba	95.3	95.3
#21	MiMo-V2-FlashXiaomi	95	95
#22	MiMo-V2-ProXiaomi	95	95
#23	Qwen3.7 MaxAlibaba	94.7	94.7
#24	Claude Opus 4.8Anthropic	94.4	94.4
#25	Step 3.5 FlashStepFun	94.4	94.4
#26	MiMo-V2.5-ProXiaomi	94.2	94.2
#27	Qwen3.6 27BAlibaba	94.2	94.2
#28	Mistral Medium 3.5Mistral AI	94.2	94.2
#29	Qwen3.5-27BAlibaba	93.9	93.9
#30	GPT-5.5OpenAI	93.9	93.9
#31	Qwen3.5-122B-A10BAlibaba	93.6	93.6
#32	Tri-21B-ThinkTrillion Labs	93.3	93.3
#33	Grok 4.1 FastxAI	93.3	93.3
#34	JT-MINIChina Mobile	93	93
#35	Grok 4.20 0309 v2xAI	93	93
#36	Kimi K2 ThinkingMoonshot AI	93	93
#37	Nova 2.0 ProAmazon	92.7	92.7
#38	Hy3Tencent	92.7	92.7
#39	Ring-2.6-1TInclusionAI	92.4	92.4
#40	Qwen3.5 4BAlibaba	92.1	92.1
#41	GPT-5.2-CodexOpenAI	92.1	92.1
#42	Claude Opus 4.6Anthropic	92.1	92.1
#43	Muse SparkMeta	91.5	91.5
#44	MiMo-V2-OmniXiaomi	91.2	91.2
#45	MiMo-V2.5Xiaomi	90.6	90.6
#46	DeepSeek-V3.2DeepSeek	90.6	90.6
#47	Grok 3 mini ReasoningxAI	90.4	90.4
#48	Trinity Large ThinkingArcee AI	90.1	90.1
#49	Ling-2.6-1TInclusionAI	89.8	89.8
#50	KAT-Coder-Pro V2Kuaishou	89.5	89.5
#51	Claude Opus 4.5Anthropic	89.5	89.5
#52	Qwen3.5-35B-A3BAlibaba	89.2	89.2
#53	KAT-Coder-Pro V1Kuaishou	88.6	88.6
#54	Claude Opus 4.7Anthropic	88.6	88.6
#55	Qwen3.5 Omni PlusAlibaba	88.3	88.3
#56	MiMo-V2-Omni-0327Xiaomi	88	88
#57	MiniCPM-V 4.6 1.3BOpenBMB	87.7	87.7
#58	HyperCLOVA X SEED ThinkNaver	87.4	87.4
#59	Step 3.5 Flash 2603StepFun	87.4	87.4
#60	Gemini 3 ProGoogle	87.1	87.1
#61	GPT-5.4OpenAI	87.1	87.1
#62	GPT-5 CodexOpenAI	86.8	86.8
#63	Qwen3.5-9BAlibaba	86.8	86.8
#64	MiniMax-M2MiniMax	86.8	86.8
#65	Mi:dm K 2.5 ProKorea Telecom	86.5	86.5
#66	GPT-5OpenAI	86.5	86.5
#67	Solar Pro 3Upstage	86.3	86.3
#68	GPT-5.3-CodexOpenAI	86	86
#69	Ling-2.6-flashInclusionAI	86	86
#70	MiniMax M2.1MiniMax	85.4	85.4
#71	MiniMax M2.7MiniMax	84.8	84.8
#72	GPT-5.2OpenAI	84.8	84.8
#73	Qwen3.5 Omni FlashAlibaba	84.5	84.5
#74	ERNIE 5.0 ThinkingBaidu	83.9	83.9
#75	Qwen3 Max ThinkingAlibaba	83.6	83.6
#76	GPT-5.4 miniOpenAI	83.3	83.3
#77	GPT-5.1-CodexOpenAI	83	83
#78	MiniCPM5-1BOpenBMB	82.5	82.5
#79	GPT-5.1OpenAI	81.9	81.9
#80	Qwen3.5 2BAlibaba	81.6	81.6
#81	Command ACohere	80.7	80.7
#82	o3OpenAI	80.7	80.7
#83	Nova 2.0 OmniAmazon	80.4	80.4
#84	Gemini 3 FlashGoogle	80.4	80.4
#85	LongCat Flash LiteLongCat	79.5	79.5
#86	Qwen3 Coder NextAlibaba	79.5	79.5
#87	Claude Sonnet 4.6Anthropic	79.5	79.5
#88	EXAONE 4.5 33BLG AI Research	78.1	78.1
#89	Claude Sonnet 4.5Anthropic	78.1	78.1
#90	GLM-4.6Zhipu AI	76.9	76.9
#91	GPT-5.4 nanoOpenAI	76	76
#92	Grok Code Fast 1xAI	75.7	75.7
#93	Nova 2 LiteAmazon	75.7	75.7
#94	Grok 4xAI	74.9	74.9
#95	K-EXAONELG AI Research	74.3	74.3
#96	Qwen3 MaxAlibaba	74.3	74.3
#97	Kimi K2 0905Moonshot AI	73.4	73.4
#98	Claude Opus 4Anthropic	73.4	73.4
#99	Claude Opus 4.1Anthropic	71.4	71.4
#100	GPT-5 miniOpenAI	71.1	71.1
#101	Mercury 2Inception	70.8	70.8
#102	Apriel-v1.6-15B-ThinkerServiceNow	69.3	69.3
#103	Apriel-v1.5-15B-ThinkerServiceNow	68.4	68.4
#104	NVIDIA Nemotron 3 Super 120B A12BNVIDIA	67.8	67.8
#105	gpt-oss-120bOpenAI	65.8	65.8
#106	Grok 4 FastxAI	65.8	65.8
#107	Gemma 4 31BGoogle	65.5	65.5
#108	Qwen3.5 0.8BAlibaba	65.2	65.2
#109	Claude Sonnet 4Anthropic	64.6	64.6
#110	GPT-5.1-Codex-MiniOpenAI	62.9	62.9
#111	o1OpenAI	62.6	62.6
#112	Kimi K2Moonshot AI	61.1	61.1
#113	gpt-oss-20bOpenAI	60.2	60.2
#114	Doubao Seed CodeByteDance	58.2	58.2
#115	o4-miniOpenAI	55.6	55.6
#116	Claude Haiku 4.5Anthropic	54.7	54.7
#117	Claude 3.7 SonnetAnthropic	54.7	54.7
#118	Qwen3 VL 235B A22BAlibaba	54.1	54.1
#119	Gemini 2.5 ProGoogle	54.1	54.1
#120	Qwen3 235B A22B 2507Alibaba	53.2	53.2
#121	Nemotron Cascade 2 30B A3BNVIDIA	53.2	53.2
#122	GPT-4.1 MiniOpenAI	52.9	52.9
#123	Magistral Medium 1.2Mistral AI	52	52
#124	Seed-OSS-36B-InstructByteDance	49.4	49.4
#125	Grok-3xAI	48.8	48.8
#126	Solar Open 100BUpstage	48.2	48.2
#127	DeepSeek-V3 0324DeepSeek	47.1	47.1
#128	GPT-4.1OpenAI	47.1	47.1
#129	Sarvam 105BSarvam	46.8	46.8
#130	Motif-2-12.7B-ReasoningMotif Technologies	46.5	46.5
#131	GLM 4.5 AirZhipu AI	46.5	46.5
#132	Qwen3 VL 32BAlibaba	45.6	45.6
#133	Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIA	45.3	45.3
#134	Qwen3 Coder 480B A35B InstructAlibaba	43.6	43.6
#135	Gemma 4 26B A4BGoogle	43.6	43.6
#136	GLM-4.5Zhipu AI	43	43
#137	Granite 4.1 30BIBM	42.1	42.1
#138	Qwen3-Next-80B-A3BAlibaba	41.5	41.5
#139	Mistral Small 4Mistral AI	41.2	41.2
#140	NVIDIA Nemotron 3 Nano 30B A3BNVIDIA	40.9	40.9
#141	Mistral Medium 3.1Mistral AI	40.6	40.6
#142	Nova PremierAmazon	38.3	38.3
#143	Devstral SmallMistral AI	38	38
#144	DeepSeek-V3.1DeepSeek	37.4	37.4
#145	DeepSeek V3.1 TerminusDeepSeek	37.1	37.1
#146	DeepSeek-R1-0528DeepSeek	36.5	36.5
#147	Pixtral LargeMistral AI	36.5	36.5
#148	GPT-5 nanoOpenAI	36.5	36.5
#149	Qwen3 VL 235B A22B InstructAlibaba	35.1	35.1
#150	Sarvam 30BSarvam	34.5	34.5
#151	Qwen2.5 72B InstructAlibaba	34.5	34.5
#152	Qwen3 14BAlibaba	34.5	34.5
#153	Qwen3 Coder 30B A3B InstructAlibaba	34.5	34.5
#154	MiniMax M1 80kMiniMax	34.2	34.2
#155	DeepSeek V3.2 ExpDeepSeek	33.9	33.9
#156	Qwen3-235B-A22B-Instruct-2507Alibaba	33.3	33.3
#157	Mistral Large 2Mistral AI	33	33
#158	Ling-1TInclusionAI	32.7	32.7
#159	Solar Pro 2Upstage	31.9	31.9
#160	MiniMax M1 40kMiniMax	31.6	31.6
#161	GLM 4.6VZhipu AI	31.6	31.6
#162	Gemini 2.5 FlashGoogle	31.6	31.6
#163	Gemini 3.1 Flash LiteGoogle	31.3	31.3
#164	o3-miniOpenAI	31.3	31.3
#165	Qwen3 32BAlibaba	29.8	29.8
#166	Mistral Small 3.2Mistral AI	29.5	29.5
#167	Gemini 2.0 FlashGoogle	29.5	29.5
#168	Qwen3 VL 8B InstructAlibaba	29.2	29.2
#169	Qwen3 VL 32B InstructAlibaba	29.2	29.2
#170	GPT-4oOpenAI	28.9	28.9
#171	Qwen3 30B A3B 2507Alibaba	28.1	28.1
#172	NVIDIA Nemotron 3 Nano 4BNVIDIA	28.1	28.1
#173	Llama Nemotron Super 49B v1.5NVIDIA	28.1	28.1
#174	K2-V2MBZUAI Institute of Foundation Models	27.8	27.8
#175	Falcon-H1R-7BTII UAE	27.8	27.8
#176	Magistral Small 1.2Mistral AI	27.8	27.8
#177	Qwen3 8BAlibaba	27.8	27.8
#178	Granite 4.1 8BIBM	27.8	27.8
#179	Qwen3 235B A22BAlibaba	27.2	27.2
#180	Ministral 3 14BMistral AI	27.2	27.2
#181	Llama-3.3 Nemotron Super 49B v1NVIDIA	26.9	26.9
#182	Qwen3 4B 2507 InstructAlibaba	26.6	26.6
#183	Magistral Small 1Mistral AI	26.6	26.6
#184	Hermes 4 - Llama-3.1 405BNous Research	26.6	26.6
#185	INTELLECT-3Prime Intellect	26.6	26.6
#186	Llama 3.3 70B InstructMeta	26.6	26.6
#187	Ministral 3 8BMistral AI	26.6	26.6
#188	Ring-1TInclusionAI	26.3	26.3
#189	Qwen3 1.7BAlibaba	26	26
#190	Gemma 4 E4BGoogle	26	26
#191	Qwen3 30B A3BAlibaba	26	26
#192	Qwen3 4B 2507Alibaba	25.4	25.4
#193	K2 Think V2MBZUAI Institute of Foundation Models	25.4	25.4
#194	Mistral Small 3.1Mistral AI	25.1	25.1
#195	Ministral 3 3BMistral AI	24.9	24.9
#196	Devstral 2Mistral AI	24.9	24.9
#197	Mistral Large 3Mistral AI	24.6	24.6
#198	Claude 3.5 HaikuAnthropic	24.6	24.6
#199	Mistral Medium 3Mistral AI	24.3	24.3
#200	Qwen3 VL 4B InstructAlibaba	23.4	23.4
#201	NVIDIA Nemotron Nano 9B V2NVIDIA	23.4	23.4
#202	Devstral Small 2Mistral AI	23.4	23.4
#203	Magistral Medium 1Mistral AI	23.1	23.1
#204	Llama 3.1 Nemotron 70B InstructNVIDIA	23.1	23.1
#205	Granite 4.0 1BIBM	22.8	22.8
#206	DeepSeek-V3DeepSeek	22.8	22.8
#207	Qwen3 VL 8BAlibaba	22.5	22.5
#208	Hermes 4 - Llama-3.1 70BNous Research	22.5	22.5
#209	GLM 4.5VZhipu AI	22.5	22.5
#210	Gemma 4 E2BGoogle	22.2	22.2
#211	DeepSeek R1 Distill Llama 70BDeepSeek	21.9	21.9
#212	Nanbeige4.1-3BNanbeige	21.6	21.6
#213	Qwen3 Next 80B A3B InstructAlibaba	21.6	21.6
#214	Qwen3 Omni 30B A3BAlibaba	21.3	21.3
#215	Olmo 3.1 32B InstructAllen Institute for AI	21.3	21.3
#216	NVIDIA Nemotron Nano 12B v2 VLNVIDIA	21.3	21.3
#217	Qwen3 0.6BAlibaba	21.1	21.1
#218	Claude 3 HaikuAnthropic	21.1	21.1
#219	Llama 3.2 3B InstructMeta	21.1	21.1
#220	Ling-flash-2.0InclusionAI	20.8	20.8
#221	Exaone 4.0 1.2BLG AI Research	20.5	20.5
#222	Solar MiniUpstage	20.2	20.2
#223	Qwen3 VL 30B A3BAlibaba	19.9	19.9
#224	Devstral MediumMistral AI	19.9	19.9
#225	Granite 4.0 H 1BIBM	19.6	19.6
#226	Granite 4.1 3BIBM	19.6	19.6
#227	LFM2.5-1.2B-ThinkingLiquid AI	19.6	19.6
#228	Mistral Small 3Mistral AI	19.6	19.6
#229	Qwen3 4BAlibaba	19	19
#230	Llama 3.1 405B InstructMeta	19	19
#231	Gemini 2.5 Flash LiteGoogle	19	19
#232	Qwen3 VL 30B A3B InstructAlibaba	19	19
#233	Llama 4 MaverickMeta	17.8	17.8
#234	Nova LiteAmazon	17.5	17.5
#235	EXAONE 4.0 32BLG AI Research	17.3	17.3
#236	Granite 4.0 H SmallIBM	17.3	17.3
#237	GPT-4.1 NanoOpenAI	17.3	17.3
#238	Qwen3 Omni 30B A3B InstructAlibaba	16.4	16.4
#239	Llama 3.1 8B InstructMeta	16.4	16.4
#240	Step3 VL 10BStepFun	16.1	16.1
#241	Jamba Reasoning 3BAI21 Labs	15.8	15.8
#242	Qwen3 VL 4BAlibaba	15.5	15.5
#243	Llama 4 ScoutMeta	15.5	15.5
#244	Llama 3.1 70B InstructMeta	15.2	15.2
#245	Granite 4.0 H 350MIBM	14.6	14.6
#246	Llama 3.2 11B InstructMeta	14.6	14.6
#247	Nova ProAmazon	14	14
#248	Nova MicroAmazon	14	14
#249	LFM2 2.6BLiquid AI	13.5	13.5
#250	Jamba Large 1.7AI21 Labs	13.5	13.5
#251	Ling-mini-2.0InclusionAI	13.2	13.2
#252	Granite 4.0 350MIBM	13.2	13.2
#253	Apertus 70B InstructSwiss AI Initiative	12.9	12.9
#254	LFM2 1.2BLiquid AI	12.6	12.6
#255	Jamba 1.7 MiniAI21 Labs	12.6	12.6
#256	Olmo 3 7B InstructAllen Institute for AI	12.6	12.6
#257	Granite 4.0 MicroIBM	12.6	12.6
#258	Llama 3.1 Nemotron Nano 4B v1.1NVIDIA	11.7	11.7
#259	Apertus 8B InstructSwiss AI Initiative	11.4	11.4
#260	Llama 3.1 Nemotron Ultra 253B v1NVIDIA	11.4	11.4
#261	DeepSeek-R1DeepSeek	11.4	11.4
#262	LFM2-24B-A2BLiquid AI	11.1	11.1
#263	Gemma 3 12B InstructGoogle	10.8	10.8
#264	LFM2.5-1.2B-InstructLiquid AI	10.8	10.8
#265	Granite 3.3 8BIBM	10.5	10.5
#266	Gemma 3 1B InstructGoogle	10.5	10.5
#267	Gemma 3 27B InstructGoogle	10.5	10.5
#268	LFM2 8B A1BLiquid AI	10.5	10.5
#269	Qwen3 30B A3B 2507 InstructAlibaba	10.2	10.2
#270	Gemma 3 270MGoogle	9.1	9.1
#271	LFM2.5-VL-1.6BLiquid AI	8.5	8.5
#272	Phi 4 Mini InstructMicrosoft	8.2	8.2
#273	Gemma 3n E4B InstructGoogle	5	5
#274	Gemma 3 4B InstructGoogle	5	5
#275	Sarvam MSarvam	0	0
#276	OLMo 2 7BAllen Institute for AI	0	0
#277	OLMo 2 32BAllen Institute for AI	0	0
#278	Phi-3 Mini Instruct 3.8BMicrosoft	0	0
#279	DeepSeek R1 0528 Qwen3 8BDeepSeek	0	0
#280	Mistral 7B InstructMistral AI	0	0
#281	Gemma 3n E2B InstructGoogle	0	0
#282	Ring-flash-2.0InclusionAI	0	0
#283	Tiny Aya GlobalCohere	0	0
#284	Olmo 3.1 32B ThinkAllen Institute for AI	0	0
#285	Molmo2-8BAllen Institute for AI	0	0
#286	Olmo 3 7B ThinkAllen Institute for AI	0	0
#287	Molmo 7B-DAllen Institute for AI	0	0
#288	Kimi Linear 48B A3B InstructMoonshot AI	0	0
#289	Reka Flash 3Reka AI	0	0
#290	Llama 3 8B InstructMeta	0	0
#291	Llama 3 70B InstructMeta	0	0
#292	Llama 3.2 1B InstructMeta	0	0
#293	Phi 4Microsoft	0	0
#294	ERNIE 4.5 300B A47BBaidu	0	0
#295	Olmo 3 32B ThinkAllen Institute for AI	0	0
#296	DeepSeek V3.2 SpecialeDeepSeek	0	0

Related Agents benchmarks

TAU-bench Retail22 TAU-bench Airline20 BFCL10 τ²-bench Airline10 τ²-bench Retail10 BrowseComp10