Coding

SciCode

Scientific-coding benchmark — graduate-level coding problems drawn from physics, biology, math, and CS research.

Source

348Models

58.9Top score

29.9Median

State of the art over time

Each point is a model at its release date; the line traces the best score to date.

Ranking

#1	Gemini 3.1 ProGoogle	58.9	58.9
#2	GPT-5.4OpenAI	56.6	56.6
#3	Gemini 3 ProGoogle	56.1	56.1
#4	GPT-5.5OpenAI	56.1	56.1
#5	GPT-5.2-CodexOpenAI	54.6	54.6
#6	Claude Opus 4.7Anthropic	54.5	54.5
#7	Claude Opus 4.8Anthropic	53.5	53.5
#8	Kimi K2.6Moonshot AI	53.5	53.5
#9	GPT-5.3-CodexOpenAI	53.2	53.2
#10	Gemini 3.5 FlashGoogle	53.1	53.1
#11	GPT-5.2OpenAI	52.1	52.1
#12	Claude Opus 4.6Anthropic	51.9	51.9
#13	Muse SparkMeta	51.5	51.5
#14	Gemini 3 FlashGoogle	50.6	50.6
#15	MiMo-V2.5-ProXiaomi	50.2	50.2
#16	DeepSeek-V4-ProDeepSeek	50	50
#17	GPT-5.4 miniOpenAI	49.9	49.9
#18	Claude Opus 4.5Anthropic	49.5	49.5
#19	Kimi K2.5Moonshot AI	49	49
#20	Qwen3.7 MaxAlibaba	48.8	48.8
#21	Grok 4.3xAI	47.3	47.3
#22	MiniMax M2.7MiniMax	47	47
#23	Qwen3.6 MaxAlibaba	46.9	46.9
#24	Claude Sonnet 4.6Anthropic	46.9	46.9
#25	GPT-5.4 nanoOpenAI	46.9	46.9
#26	o4-miniOpenAI	46.5	46.5
#27	GLM-5Zhipu AI	46.2	46.2
#28	Grok 4xAI	45.7	45.7
#29	Grok 4.20 0309 v2xAI	45.6	45.6
#30	GLM 4.7Zhipu AI	45.1	45.1
#31	DeepSeek-V4-FlashDeepSeek	44.9	44.9
#32	Grok 4.20 0309xAI	44.7	44.7
#33	Claude Sonnet 4.5Anthropic	44.7	44.7
#34	Grok 4.1 FastxAI	44.2	44.2
#35	Grok 4 FastxAI	44.2	44.2
#36	DeepSeek V3.2 SpecialeDeepSeek	44	44
#37	GLM 5.1Zhipu AI	43.8	43.8
#38	GLM 5 TurboZhipu AI	43.6	43.6
#39	GLM 5V TurboZhipu AI	43.5	43.5
#40	Gemma 4 31BGoogle	43.4	43.4
#41	Claude Haiku 4.5Anthropic	43.3	43.3
#42	GPT-5.1OpenAI	43.3	43.3
#43	MiMo-V2.5Xiaomi	43.1	43.1
#44	Qwen3 Max ThinkingAlibaba	43.1	43.1
#45	GPT-5OpenAI	42.9	42.9
#46	Gemini 2.5 ProGoogle	42.8	42.8
#47	Nova 2.0 ProAmazon	42.7	42.7
#48	GPT-5.1-Codex-MiniOpenAI	42.6	42.6
#49	MiniMax M2.5MiniMax	42.6	42.6
#50	MiMo-V2-ProXiaomi	42.5	42.5
#51	Qwen3 235B A22B 2507Alibaba	42.4	42.4
#52	Ring-2.6-1TInclusionAI	42.4	42.4
#53	Kimi K2 ThinkingMoonshot AI	42.4	42.4
#54	Qwen3.5 397B A17BAlibaba	42	42
#55	Qwen3.5-122B-A10BAlibaba	42	42
#56	Gemini 3.1 Flash LiteGoogle	41.9	41.9
#57	Hy3Tencent	41.2	41.2
#58	Cogito v2.1Deep Cogito	41	41
#59	GPT-5 miniOpenAI	41	41
#60	o3OpenAI	41	41
#61	GPT-5 CodexOpenAI	40.9	40.9
#62	Claude Opus 4.1Anthropic	40.9	40.9
#63	Claude Opus 4Anthropic	40.9	40.9
#64	Doubao Seed CodeByteDance	40.7	40.7
#65	MiniMax M2.1MiniMax	40.7	40.7
#66	Qwen3.6 PlusAlibaba	40.7	40.7
#67	Grok 3 mini ReasoningxAI	40.6	40.6
#68	DeepSeek V3.1 TerminusDeepSeek	40.6	40.6
#69	Qwen3.5 Omni PlusAlibaba	40.5	40.5
#70	GPT-4.1 MiniOpenAI	40.4	40.4
#71	Step 3.5 FlashStepFun	40.4	40.4
#72	DeepSeek-R1-0528DeepSeek	40.3	40.3
#73	Claude 3.7 SonnetAnthropic	40.3	40.3
#74	GPT-5.1-CodexOpenAI	40.2	40.2
#75	Gemma 4 26B A4BGoogle	40	40
#76	Claude Sonnet 4Anthropic	40	40
#77	Qwen3 VL 235B A22BAlibaba	39.9	39.9
#78	Qwen3 235B A22BAlibaba	39.9	39.9
#79	DeepSeek V3.2 ExpDeepSeek	39.9	39.9
#80	o3-miniOpenAI	39.9	39.9
#81	Qwen3.6 27BAlibaba	39.8	39.8
#82	Mistral Medium 3.5Mistral AI	39.6	39.6
#83	MiMo-V2-Omni-0327Xiaomi	39.5	39.5
#84	Qwen3.5-27BAlibaba	39.5	39.5
#85	MiMo-V2-FlashXiaomi	39.4	39.4
#86	Gemini 2.5 FlashGoogle	39.4	39.4
#87	Magistral Medium 1.2Mistral AI	39.2	39.2
#88	INTELLECT-3Prime Intellect	39.1	39.1
#89	DeepSeek-V3.1DeepSeek	39.1	39.1
#90	gpt-oss-120bOpenAI	38.9	38.9
#91	DeepSeek-V3.2DeepSeek	38.9	38.9
#92	Qwen3-Next-80B-A3BAlibaba	38.8	38.8
#93	Mercury 2Inception	38.7	38.7
#94	Step 3.5 Flash 2603StepFun	38.5	38.5
#95	GLM-4.6Zhipu AI	38.4	38.4
#96	KAT-Coder-Pro V2Kuaishou	38.3	38.3
#97	Qwen3 MaxAlibaba	38.3	38.3
#98	GPT-4.1OpenAI	38.1	38.1
#99	Mistral Small 4Mistral AI	38	38
#100	MiniMax M1 40kMiniMax	37.8	37.8
#101	Command ACohere	37.8	37.8
#102	Qwen3.5-35B-A3BAlibaba	37.7	37.7
#103	DeepSeek R1 Distill Qwen 32BDeepSeek	37.6	37.6
#104	ERNIE 5.0 ThinkingBaidu	37.5	37.5
#105	MiniMax M1 80kMiniMax	37.4	37.4
#106	Apriel-v1.6-15B-ThinkerServiceNow	37.3	37.3
#107	Ling-2.6-1TInclusionAI	37	37
#108	Nova 2 LiteAmazon	36.9	36.9
#109	Grok-3xAI	36.8	36.8
#110	Ring-1TInclusionAI	36.7	36.7
#111	MiMo-V2-OmniXiaomi	36.7	36.7
#112	KAT-Coder-Pro V1Kuaishou	36.6	36.6
#113	GPT-5 nanoOpenAI	36.6	36.6
#114	GPT-4oOpenAI	36.6	36.6
#115	Claude 3.5 SonnetAnthropic	36.6	36.6
#116	Seed-OSS-36B-InstructByteDance	36.5	36.5
#117	Nova 2.0 OmniAmazon	36.2	36.2
#118	Grok Code Fast 1xAI	36.2	36.2
#119	Mistral Large 3Mistral AI	36.2	36.2
#120	Trinity Large ThinkingArcee AI	36.1	36.1
#121	MiniMax-M2MiniMax	36.1	36.1
#122	NVIDIA Nemotron 3 Super 120B A12BNVIDIA	36	36
#123	Qwen3-235B-A22B-Instruct-2507Alibaba	36	36
#124	Qwen3 Coder 480B A35B InstructAlibaba	35.9	35.9
#125	Qwen3 VL 235B A22B InstructAlibaba	35.9	35.9
#126	QwQ-32BAlibaba	35.8	35.8
#127	DeepSeek-V3 0324DeepSeek	35.8	35.8
#128	Qwen3.6 35B A3BAlibaba	35.8	35.8
#129	o1OpenAI	35.8	35.8
#130	DeepSeek-R1DeepSeek	35.7	35.7
#131	K-EXAONELG AI Research	35.6	35.6
#132	Qwen3 32BAlibaba	35.4	35.4
#133	DeepSeek-V3DeepSeek	35.4	35.4
#134	Ling-1TInclusionAI	35.2	35.2
#135	Magistral Small 1.2Mistral AI	35.2	35.2
#136	Apriel-v1.5-15B-ThinkerServiceNow	34.8	34.8
#137	Nemotron Cascade 2 30B A3BNVIDIA	34.8	34.8
#138	Llama Nemotron Super 49B v1.5NVIDIA	34.8	34.8
#139	GLM-4.5Zhipu AI	34.8	34.8
#140	Llama 3.1 Nemotron Ultra 253B v1NVIDIA	34.7	34.7
#141	Hermes 4 - Llama-3.1 405BNous Research	34.6	34.6
#142	Kimi K2Moonshot AI	34.5	34.5
#143	EXAONE 4.0 32BLG AI Research	34.4	34.4
#144	gpt-oss-20bOpenAI	34.4	34.4
#145	Hermes 4 - Llama-3.1 70BNous Research	34.1	34.1
#146	Gemini 2.0 FlashGoogle	34	34
#147	Mistral Medium 3.1Mistral AI	33.8	33.8
#148	Qwen2.5 MaxAlibaba	33.7	33.7
#149	GLM 4.7 FlashZhipu AI	33.7	33.7
#150	Qwen3 30B A3B 2507Alibaba	33.3	33.3
#151	Mi:dm K 2.5 ProKorea Telecom	33.2	33.2
#152	Mistral Medium 3Mistral AI	33.1	33.1
#153	Devstral 2Mistral AI	33.1	33.1
#154	Llama 4 MaverickMeta	33.1	33.1
#155	K2 Think V2MBZUAI Institute of Foundation Models	33	33
#156	Gemini 2.0 Flash ThinkingGoogle	32.9	32.9
#157	o1-miniOpenAI	32.3	32.3
#158	Qwen3 Coder NextAlibaba	32.3	32.3
#159	GPT-4 TurboOpenAI	31.9	31.9
#160	Qwen3 14BAlibaba	31.6	31.6
#161	ERNIE 4.5 300B A47BBaidu	31.5	31.5
#162	Gemini 2.0 ProGoogle	31.3	31.3
#163	DeepSeek R1 Distill Llama 70BDeepSeek	31.3	31.3
#164	Step3 VL 10BStepFun	31.1	31.1
#165	Qwen3 VL 30B A3B InstructAlibaba	30.8	30.8
#166	Kimi K2 0905Moonshot AI	30.7	30.7
#167	Qwen3 Next 80B A3B InstructAlibaba	30.7	30.7
#168	Qwen3 Omni 30B A3BAlibaba	30.6	30.6
#169	GLM 4.5 AirZhipu AI	30.6	30.6
#170	Qwen3 30B A3B 2507 InstructAlibaba	30.4	30.4
#171	GLM 4.6VZhipu AI	30.4	30.4
#172	Llama 3.1 Tulu3 405BAllen Institute for AI	30.2	30.2
#173	Solar Pro 2Upstage	30.2	30.2
#174	Qwen3 VL 32B InstructAlibaba	30.1	30.1
#175	Llama 3.1 405B InstructMeta	29.9	29.9
#176	Magistral Medium 1Mistral AI	29.7	29.7
#177	NVIDIA Nemotron 3 Nano 30B A3BNVIDIA	29.6	29.6
#178	GrokxAI	29.5	29.5
#179	Gemini 1.5 ProGoogle	29.5	29.5
#180	Devstral MediumMistral AI	29.4	29.4
#181	Olmo 3.1 32B ThinkAllen Institute for AI	29.3	29.3
#182	Pixtral LargeMistral AI	29.2	29.2
#183	Mistral Large 2Mistral AI	29.2	29.2
#184	JT-35B-FlashChina Mobile	29.1	29.1
#185	Ling-flash-2.0InclusionAI	28.9	28.9
#186	Qwen3 VL 30B A3BAlibaba	28.8	28.8
#187	Devstral Small 2Mistral AI	28.8	28.8
#188	K2-V2MBZUAI Institute of Foundation Models	28.6	28.6
#189	Olmo 3 32B ThinkAllen Institute for AI	28.6	28.6
#190	Qwen3 VL 32BAlibaba	28.5	28.5
#191	Qwen3 30B A3BAlibaba	28.5	28.5
#192	Grok-2xAI	28.5	28.5
#193	LongCat Flash LiteLongCat	28.4	28.4
#194	HyperCLOVA X SEED ThinkNaver	28.4	28.4
#195	Motif-2-12.7B-ReasoningMotif Technologies	28.2	28.2
#196	Llama-3.3 Nemotron Super 49B v1NVIDIA	28.2	28.2
#197	EXAONE 4.5 33BLG AI Research	28	28
#198	Nova PremierAmazon	27.9	27.9
#199	Nemotron 3 Nano Omni 30B A3B ReasoningNVIDIA	27.8	27.8
#200	Qwen3 Coder 30B A3B InstructAlibaba	27.8	27.8
#201	Qwen3.5-9BAlibaba	27.7	27.7
#202	Claude 3.5 HaikuAnthropic	27.4	27.4
#203	JT-MINIChina Mobile	27.2	27.2
#204	Qwen2.5 Coder 32B InstructAlibaba	27.1	27.1
#205	Ling-2.6-flashInclusionAI	27.1	27.1
#206	Solar Open 100BUpstage	26.9	26.9
#207	Reka Flash 3Reka AI	26.7	26.7
#208	Gemini 1.5 FlashGoogle	26.7	26.7
#209	Llama 3.1 70B InstructMeta	26.7	26.7
#210	Qwen2.5 72B InstructAlibaba	26.7	26.7
#211	Nanbeige4.1-3BNanbeige	26.6	26.6
#212	Mistral Small 3.1Mistral AI	26.5	26.5
#213	Mistral Small 3.2Mistral AI	26.4	26.4
#214	Sarvam 105BSarvam	26.4	26.4
#215	NVIDIA Nemotron Nano 12B v2 VLNVIDIA	26.2	26.2
#216	Llama 3.3 70B InstructMeta	26	26
#217	Phi 4Microsoft	26	26
#218	GPT-4.1 NanoOpenAI	25.9	25.9
#219	Granite 4.1 30BIBM	25.8	25.8
#220	Qwen3 4B 2507Alibaba	25.6	25.6
#221	Qwen3.5 Omni FlashAlibaba	25.5	25.5
#222	Gemini 2.0 Flash LiteGoogle	25	25
#223	Falcon-H1R-7BTII UAE	24.9	24.9
#224	Solar Pro 3Upstage	24.7	24.7
#225	Devstral SmallMistral AI	24.5	24.5
#226	Gemma 4 E4BGoogle	24.4	24.4
#227	Mistral SabaMistral AI	24.1	24.1
#228	Magistral Small 1Mistral AI	24.1	24.1
#229	Llama 3.2 90B InstructMeta	24	24
#230	DeepSeek R1 Distill Qwen 14BDeepSeek	23.9	23.9
#231	Mistral Small 3Mistral AI	23.6	23.6
#232	Ministral 3 14BMistral AI	23.6	23.6
#233	Llama 3.1 Nemotron 70B InstructNVIDIA	23.3	23.3
#234	Claude 3 OpusAnthropic	23.3	23.3
#235	Hermes 3 - Llama-3.1 70BNous Research	23.1	23.1
#236	SonarPerplexity	22.9	22.9
#237	Qwen2.5 32B InstructAlibaba	22.9	22.9
#238	Qwen2 72B InstructAlibaba	22.9	22.9
#239	Gemini 1.5 Flash 8BGoogle	22.9	22.9
#240	Claude 3 SonnetAnthropic	22.9	22.9
#241	GPT-4o-miniOpenAI	22.9	22.9
#242	DeepHermes 3 - Mistral 24BNous Research	22.8	22.8
#243	Sonar ProPerplexity	22.6	22.6
#244	Qwen3 8BAlibaba	22.6	22.6
#245	GLM 4.5VZhipu AI	22.1	22.1
#246	NVIDIA Nemotron Nano 9B V2NVIDIA	22	22
#247	Qwen3 VL 8BAlibaba	21.9	21.9
#248	Granite 4.1 8BIBM	21.8	21.8
#249	Gemma 3 27B InstructGoogle	21.2	21.2
#250	Olmo 3 7B ThinkAllen Institute for AI	21.2	21.2
#251	Granite 4.0 H SmallIBM	20.9	20.9
#252	Gemma 4 E2BGoogle	20.9	20.9
#253	Nova ProAmazon	20.8	20.8
#254	Mistral LargeMistral AI	20.8	20.8
#255	Ministral 3 8BMistral AI	20.8	20.8
#256	DeepSeek R1 0528 Qwen3 8BDeepSeek	20.4	20.4
#257	Kimi Linear 48B A3B InstructMoonshot AI	19.9	19.9
#258	Claude 2Anthropic	19.4	19.4
#259	Gemini 2.5 Flash LiteGoogle	19.3	19.3
#260	Sarvam 30BSarvam	19.2	19.2
#261	Llama 3 70B InstructMeta	18.9	18.9
#262	Mixtral 8x22B InstructMistral AI	18.8	18.8
#263	Jamba Large 1.7AI21 Labs	18.8	18.8
#264	Qwen3 Omni 30B A3B InstructAlibaba	18.6	18.6
#265	Claude 3 HaikuAnthropic	18.6	18.6
#266	Jamba 1.6 LargeAI21 Labs	18.4	18.4
#267	Claude 2.1Anthropic	18.4	18.4
#268	Qwen3.5 4BAlibaba	18.3	18.3
#269	Qwen3 4B 2507 InstructAlibaba	18.1	18.1
#270	Sarvam MSarvam	17.8	17.8
#271	Tri-21B-ThinkTrillion Labs	17.8	17.8
#272	Gemma 3 12B InstructGoogle	17.4	17.4
#273	Qwen3 VL 8B InstructAlibaba	17.4	17.4
#274	Qwen3 VL 4BAlibaba	17.1	17.1
#275	Llama 4 ScoutMeta	17	17
#276	Ring-flash-2.0InclusionAI	16.8	16.8
#277	Qwen3 4BAlibaba	16.7	16.7
#278	Olmo 3.1 32B InstructAllen Institute for AI	16.7	16.7
#279	NVIDIA Nemotron 3 Nano 4BNVIDIA	16.4	16.4
#280	Jamba 1.5 LargeAI21 Labs	16.3	16.3
#281	Mistral SmallMistral AI	15.6	15.6
#282	Qwen2.5 TurboAlibaba	15.3	15.3
#283	Qwen2.5-Coder 7B InstructAlibaba	14.8	14.8
#284	Ministral 3 3BMistral AI	14.4	14.4
#285	DeepSeek Coder V2 Lite InstructDeepSeek	13.9	13.9
#286	Nova LiteAmazon	13.9	13.9
#287	Qwen3 VL 4B InstructAlibaba	13.7	13.7
#288	Ling-mini-2.0InclusionAI	13.5	13.5
#289	Molmo2-8BAllen Institute for AI	13.3	13.3
#290	Llama 3.1 8B InstructMeta	13.2	13.2
#291	Granite 4.1 3BIBM	11.9	11.9
#292	DeepSeek R1 Distill Llama 8BDeepSeek	11.9	11.9
#293	Llama 3 8B InstructMeta	11.9	11.9
#294	Granite 4.0 MicroIBM	11.9	11.9
#295	DBRX InstructDatabricks	11.8	11.8
#296	Mistral MediumMistral AI	11.8	11.8
#297	Llama 2 Chat 13BMeta	11.8	11.8
#298	Command R+Cohere	11.8	11.8
#299	Gemini 1.0 ProGoogle	11.7	11.7
#300	Llama 3.2 11B InstructMeta	11.2	11.2
#301	Phi-4-multimodal-instructMicrosoft	11	11
#302	LFM2-24B-A2BLiquid AI	10.9	10.9
#303	Phi 4 Mini InstructMicrosoft	10.8	10.8
#304	Olmo 3 7B InstructAllen Institute for AI	10.3	10.3
#305	Jamba 1.6 MiniAI21 Labs	10.1	10.1
#306	Granite 3.3 8BIBM	10.1	10.1
#307	Llama 3.1 Nemotron Nano 4B v1.1NVIDIA	10.1	10.1
#308	Nova MicroAmazon	9.4	9.4
#309	Jamba 1.7 MiniAI21 Labs	9.3	9.3
#310	Exaone 4.0 1.2BLG AI Research	9.3	9.3
#311	DeepHermes 3 - Llama-3.1 8BNous Research	9.1	9.1
#312	Phi-3 Mini Instruct 3.8BMicrosoft	9	9
#313	Granite 4.0 1BIBM	8.7	8.7
#314	Gemma 3n E4B InstructGoogle	8.6	8.6
#315	Granite 4.0 H 1BIBM	8.2	8.2
#316	OLMo 2 32BAllen Institute for AI	8	8
#317	Jamba 1.5 MiniAI21 Labs	8	8
#318	Gemma 3 4B InstructGoogle	7.3	7.3
#319	Qwen3.5 2BAlibaba	7.2	7.2
#320	LFM 40BLiquid AI	7.1	7.1
#321	Qwen3 1.7BAlibaba	6.9	6.9
#322	LFM2 8B A1BLiquid AI	6.8	6.8
#323	DeepSeek R1 Distill Qwen 1.5BDeepSeek	6.6	6.6
#324	Jamba Reasoning 3BAI21 Labs	5.9	5.9
#325	Apertus 70B InstructSwiss AI Initiative	5.7	5.7
#326	Gemma 3n E2B InstructGoogle	5.2	5.2
#327	Llama 3.2 3B InstructMeta	5.2	5.2
#328	LFM2.5-1.2B-ThinkingLiquid AI	4.2	4.2
#329	Qwen3 0.6BAlibaba	4.1	4.1
#330	Apertus 8B InstructSwiss AI Initiative	4.1	4.1
#331	OLMo 2 7BAllen Institute for AI	3.7	3.7
#332	Tiny Aya GlobalCohere	3.6	3.6
#333	Molmo 7B-DAllen Institute for AI	3.6	3.6
#334	LFM2.5-VL-1.6BLiquid AI	3	3
#335	Qwen3.5 0.8BAlibaba	2.9	2.9
#336	Mixtral 8x7B InstructMistral AI	2.8	2.8
#337	LFM2 1.2BLiquid AI	2.5	2.5
#338	LFM2 2.6BLiquid AI	2.5	2.5
#339	Mistral 7B InstructMistral AI	2.4	2.4
#340	LFM2.5-1.2B-InstructLiquid AI	2.3	2.3
#341	MiniCPM-V 4.6 1.3BOpenBMB	2.1	2.1
#342	Granite 4.0 H 350MIBM	1.7	1.7
#343	Llama 3.2 1B InstructMeta	1.7	1.7
#344	MiniCPM5-1BOpenBMB	1.4	1.4
#345	Granite 4.0 350MIBM	0.9	0.9
#346	Gemma 3 1B InstructGoogle	0.7	0.7
#347	Llama 2 Chat 7BMeta	0	0
#348	Gemma 3 270MGoogle	0	0

Related Coding benchmarks

Terminal-Bench292 LiveCodeBench282 HumanEval68 SWE-bench Verified51 MBPP31 Aider Polyglot21