Instruction-Following Eval
Tests whether models can follow explicit, verifiable instructions such as 'write exactly 3 paragraphs' or 'include the word ocean at least twice'
541 prompts with 25 types of verifiable instructions (length constraints, keyword inclusion, formatting requirements, etc). Evaluated by programmatic verification — no human or LLM judge needed. Two metrics: strict (all constraints met) and loose (partial credit).
Shows open-weight models only. Commercial API models (GPT-4o, Claude, Gemini) are not submitted to the Open LLM Leaderboard — their scores come from provider-reported benchmarks.
| # | Model | Score |
|---|---|---|
| 1 | Qwen3.7 Max | 94.3% |
| 2 | Llama 3.3 70B Instruct | 90.0% |
| 3 | Llama 3.1 70B Instruct | 86.7% |
| 4 | Qwen2.5 72B Instruct | 86.4% |
| 5 | Qwen2.5 14B Instruct 1M |
| 84.1% |
| 6 | Qwen2.5 32B Instruct | 83.5% |
| 7 | Qwen2.5 14B Instruct | 81.6% |
| 8 | EXAONE 3.5 7.8B Instruct | 81.4% |
| 9 | Meta Llama 3 70B Instruct | 81.0% |
| 10 | gemma 2 27b it | 79.8% |
| 11 | Hermes 3 Llama 3.1 70B | 76.6% |
| 12 | Falcon3 7B Instruct | 76.1% |
| 13 | Qwen2.5 7B Instruct | 75.9% |
| 14 | Qwen2.5 7B Instruct 1M | 74.5% |
| 15 | gemma 2 9b it | 74.4% |
| 16 | Meta Llama 3 8B Instruct | 74.1% |
| 17 | Llama 3.1 Tulu 3 8B SFT | 74.0% |
| 18 | Llama 3.2 3B Instruct | 73.9% |
| 19 | Phi 4 mini instruct | 73.8% |
| 20 | granite 3.2 8b instruct | 72.7% |
| 21 | Qwen2.5 Coder 32B Instruct | 72.7% |
| 22 | Meta Llama 3 8B Instruct abliterated v3 | 72.4% |
| 23 | OLMo 2 1124 7B Instruct | 72.4% |
| 24 | granite 3.1 8b instruct | 72.1% |
| 25 | Phi 3.5 MoE instruct | 69.2% |
| 26 | Qwen2.5 Coder 14B Instruct | 69.1% |
| 27 | OLMoE 1B 7B 0125 Instruct | 67.6% |
| 28 | Qwen2.5 3B Instruct | 64.7% |
| 29 | granite 3.1 2b instruct | 62.9% |
| 30 | Hermes 3 Llama 3.1 8B | 61.7% |
| 31 | Qwen2.5 Coder 7B Instruct | 61.0% |
| 32 | Phi 3 medium 128k instruct | 60.4% |
| 33 | openchat 3.5 1210 | 60.4% |
| 34 | Vikhr Nemo 12B Instruct R 21 09 24 | 60.0% |
| 35 | Phi 3 mini 128k instruct | 59.8% |
| 36 | openchat 3.5 | 59.3% |
| 37 | Nous Hermes 2 Mixtral 8x7B DPO | 59.0% |
| 38 | Phi 3.5 mini instruct | 57.7% |
| 39 | Llama 3.2 1B Instruct | 57.0% |
| 40 | gemma 2 2b it | 56.7% |
| 41 | Hermes 2 Pro Mistral 7B | 56.7% |
| 42 | Phi 3 mini 4k instruct | 56.1% |
| 43 | Bielik 11B v2.3 Instruct | 55.8% |
| 44 | OpenHermes 2.5 Mistral 7B | 55.7% |
| 45 | internlm2 5 7b chat | 55.4% |
| 46 | granite 3.1 3b a800m instruct | 55.2% |
| 47 | Mistral 7B Instruct v0.2 | 55.0% |
| 48 | SmolLM2 1.7B Instruct | 53.7% |
| 49 | gemma 2 2b it abliterated | 53.3% |
| 50 | granite 3.0 8b instruct | 53.1% |
| 51 | gemma 2 2b jpn it | 50.8% |
| 52 | gemma 1.1 7b it | 50.4% |
| 53 | Mistral NeMo Minitron 8B Instruct | 50.0% |
| 54 | WizardLM 70B V1.0 | 49.5% |
| 55 | zephyr 7b beta | 49.5% |
| 56 | Llama 3.1 8B Instruct | 49.2% |
| 57 | Qwen1.5 14B Chat | 47.7% |
| 58 | Yi 34B Chat | 47.0% |
| 59 | Mistral 7B Instruct v0.1 | 44.9% |
| 60 | Qwen2.5 1.5B Instruct | 44.8% |
| 61 | Llama 3 8B Instruct Gradient 1048k | 44.6% |
| 62 | DeepSeek R1 Distill Qwen 14B | 43.8% |
| 63 | Qwen2.5 Coder 32B | 43.6% |
| 64 | DeepSeek R1 Distill Llama 70B | 43.4% |
| 65 | DeepSeek R1 Distill Qwen 32B | 41.9% |
| 66 | deepseek llm 7b chat | 41.7% |
| 67 | Qwen2.5 72B | 41.4% |
| 68 | Qwen2.5 32B | 40.8% |
| 69 | DeepSeek R1 Distill Qwen 7B | 40.4% |
| 70 | QwQ 32B Preview | 40.4% |
| 71 | Llama 2 7b chat hf | 39.9% |
| 72 | Llama 2 13b chat hf | 39.8% |
| 73 | QwQ 32B | 39.8% |
| 74 | Hermes 3 Llama 3.2 3B | 38.2% |
| 75 | Qwen1.5 MoE A2.7B Chat | 38.0% |
| 76 | DeepSeek R1 Distill Llama 8B | 37.8% |
| 77 | Qwen2.5 14B | 36.9% |
| 78 | Qwen2.5 Coder 14B | 34.7% |
| 79 | DeepSeek R1 Distill Qwen 1.5B | 34.6% |
| 80 | Qwen2.5 Coder 7B | 34.5% |
| 81 | Yi 6B Chat | 34.0% |
| 82 | Qwen2.5 7B | 33.7% |
| 83 | Qwen2 1.5B Instruct | 33.7% |
| 84 | Qwen2.5 0.5B Instruct | 30.7% |
| 85 | gemma 1.1 2b it | 30.7% |
| 86 | Lucie 7B Instruct v1.1 | 30.4% |
| 87 | SmolLM2 135M Instruct | 28.8% |
| 88 | phi 2 | 27.4% |
| 89 | gemma 2b it | 26.9% |
| 90 | Qwen2.5 3B | 26.9% |
| 91 | Qwen1.5 7B | 26.8% |
| 92 | Qwen2.5 1.5B | 26.7% |
| 93 | Qwen1.5 MoE A2.7B | 26.6% |
| 94 | gemma 7b | 26.6% |
| 95 | Qwen2.5 Math 7B Instruct | 26.4% |
| 96 | gpt neo 2.7B | 25.9% |
| 97 | gpt neox 20b | 25.9% |
| 98 | gpt j 6b | 25.2% |
| 99 | Llama 2 7b hf | 25.2% |
| 100 | llama 7b | 25.0% |