Favur Evals
favur.dev

Agent role · weight 3/100 · rubric v15

🏗️ Build

Iterative compile-fix loop until zero errors — minimal fixes only, the only role allowed to install tooling.

What it's scored on

Output quality
Static code quality and deliverables of the build it produced.
Efficiency
Token volume, output per request, cost, context efficiency, request-error and hallucination rates.

Full scoring definition in the methodology → · Sample transcript

Model leaderboard

Every model ranked by Build score
#ModelScore / 10CoveragenLast seen
1Google Gemini Flash Lite 3.57.5399%52026-07-23
2Meta Muse Spark 1.17.53100%32026-07-19
3Xiaomi Mimo v2.5 †7.3580%32026-07-22
4xAI Grok 4.5 †7.0086%42026-07-22
5Qwen Plus qwen3.7 †6.8985%42026-07-22
6Stepfun Step Flash 3.7 †6.5573%22026-07-19
7Xiaomi Mimo Pro v2.5 †6.1964%52026-07-26
8Google Gemini Flash Lite 3.1 †5.9172%82026-07-19
9OpenAI Gpt Luna 5.6 †5.8445%12026-07-14
10Google Gemini Flash 3.65.7997%22026-07-24
11DeepSeek Deepseek Flash v4 †5.5652%62026-07-22
12Anthropic Claude Sonnet 5 †5.5169%42026-07-22
13Moonshot AI Kimi k2.6 †5.1861%32026-07-17
14Moonshot AI Kimi Code k2.7 †5.0845%22026-07-10
15Z.ai Glm 5.2 †5.0058%32026-07-17
16Google Gemini Flash 3 (preview) †4.9442%62026-07-10
17Google Gemini Flash 3.5 †4.7942%22026-07-08
18Thinkingmachines Inkling4.6797%22026-07-22
19DeepSeek Deepseek Pro v4 †4.6540%32026-07-05
20Google Gemini Pro 3.1 (preview) †4.5938%22026-07-03
21Anthropic Claude Sonnet 4.6 †3.9041%22026-07-03

† low confidence — sparse role or low data coverage (renormalized, never zero-filled).

Per-run history (newest first)

Every appearance of the Build role, newest first
DateRunSoWModelScore / 10
2026-07-26solarSystem__multi-xiaomi-mimo-v2.5-7.26.2026solarSystemXiaomi Mimo Pro v2.57.18
2026-07-24circles__xiaomi-multi-model-2.5-2.5-pro-7.24.2026circlesXiaomi Mimo Pro v2.57.51
2026-07-24circles__google-gemini-3.6-flash-7.23.2026circlesGoogle Gemini Flash 3.66.82
2026-07-23solarSystem/multi-anthropic-sonnet-gemini-3.5-flash-lite-7.22.2026solarSystemGoogle Gemini Flash Lite 3.57.33
2026-07-22circles__thinkingmachines-inkling-7.22.2026circlesThinkingmachines Inkling5.31
2026-07-22circles__x-ai-grok-4.5-7.22.2026circlesxAI Grok 4.57.44
2026-07-22circles__deepseek-deepseek-v4-flash-7.22.2026circlesDeepSeek Deepseek Flash v47.65
2026-07-22circles__anthropic-claude-sonnet-5-7.21.2026circlesAnthropic Claude Sonnet 56.66
2026-07-22the2048__gemini-multi-model-7-22-2026the2048Google Gemini Flash Lite 3.57.39
2026-07-22circles__qwen-qwen3.7-plus-7.22.2026circlesQwen Plus qwen3.77.02
2026-07-22circles__xiaomi-mimo-v2.5-7.22.2026circlesXiaomi Mimo v2.57.61
2026-07-21circles__google-gemini-3.6-flash-7.21.2026circlesGoogle Gemini Flash 3.64.76
2026-07-21solarSystem__google-multi-model-7.21.2026solarSystemGoogle Gemini Flash Lite 3.57.57
2026-07-21circles__x-ai-grok-4.5-7.21.2026circlesGoogle Gemini Flash Lite 3.57.56
2026-07-21circles__google-gemini-3.5-flash-lite-7.21.2026circlesGoogle Gemini Flash Lite 3.57.82
2026-07-19solarSystem__stepfun-step-3.7-flash-7.19.2026solarSystemStepfun Step Flash 3.77.26
2026-07-19circles__x-ai-grok-4.5-7.18.2026circlesxAI Grok 4.57.80
2026-07-19solarSystem__qwen-qwen3.7-plus-7.18.2026solarSystemQwen Plus qwen3.77.51
2026-07-19the2048__meta-muse-spark-1.1-7.19.2026the2048Meta Muse Spark 1.17.08
2026-07-19circles__google-gemini-3.1-flash-lite-7.19.2026circlesGoogle Gemini Flash Lite 3.15.45
2026-07-18circles__thinkingmachines-inkling-7.18.2026circlesThinkingmachines Inkling4.03
2026-07-18solarSystem__xiaomi-mimo-v2.5-7.18.2026solarSystemXiaomi Mimo v2.57.81
2026-07-17circles__z-ai-glm-5.2-7.17.2026circlesZ.ai Glm 5.27.19
2026-07-17circles__moonshotai-kimi-k2.6-7.16.2026circlesMoonshot AI Kimi k2.67.13
2026-07-17circles__x-ai-grok-4.5-7.16.2026circlesxAI Grok 4.57.49
2026-07-17circles__anthropic-claude-sonnet-5-7.16.2026circlesAnthropic Claude Sonnet 56.63
2026-07-17circles__qwen-qwen3.7-plus-7.16.2026circlesQwen Plus qwen3.77.59
2026-07-17solarSystem__meta-muse-spark-1.1-7.17.2026solarSystemMeta Muse Spark 1.17.60
2026-07-17circles__meta-muse-spark-1.1-7.16.2026circlesMeta Muse Spark 1.17.91
2026-07-16circles__google-multi-model-7.18.2026circlesGoogle Gemini Flash Lite 3.17.63
2026-07-16circles__google-multi-model-7.17.2026circlesGoogle Gemini Flash Lite 3.17.17
2026-07-15solarSystem__google-multi-model-7.15.2026solarSystemGoogle Gemini Flash Lite 3.15.11
2026-07-15circles__google-gemini-3.1-flash-lite-7.14.2026circlesGoogle Gemini Flash Lite 3.15.77
2026-07-15circles__google-multi-model-7.15.2026circlesGoogle Gemini Flash Lite 3.15.53
2026-07-14circles__openai-gpt-5.6-luna-7.14.2026circlesOpenAI Gpt Luna 5.65.84
2026-07-13circles__stepfun-step-3.7-flash-7.13.2026circlesStepfun Step Flash 3.75.84
2026-07-13solarSystem__google-gemini-3.1-flash-lite-7.12.2026solarSystemGoogle Gemini Flash Lite 3.14.62
2026-07-12circles__anthropic-claude-sonnet-5-7.11.2026circlesAnthropic Claude Sonnet 54.26
2026-07-10circles__google-gemini-3-flash-preview-7.9.2026circlesGoogle Gemini Flash 3 (preview)4.90
2026-07-10circles__moonshotai-kimi-k2.7-code-7.9.2026circlesMoonshot AI Kimi Code k2.75.08
2026-07-10circles__x-ai-grok-4.5-7.10.2026circlesxAI Grok 4.55.25
2026-07-10circles__deepseek-deepseek-v4-flash-7.10.2026circlesDeepSeek Deepseek Flash v46.38
2026-07-10circles__xiaomi-mimo-v2.5-pro-7.10.2026circlesXiaomi Mimo v2.56.64
2026-07-09circles__google-gemini-3-flash-preview-7.8.2026circlesGoogle Gemini Flash 3 (preview)5.25
2026-07-09circles__qwen-qwen3.7-plus-7.9.2026circlesQwen Plus qwen3.75.45
2026-07-09circles__google-gemini-3.1-flash-lite-7.9.2026circlesGoogle Gemini Flash Lite 3.16.00
2026-07-08circles__google-gemini-3.5-flash-7.8.2026circlesGoogle Gemini Flash 3.54.98
2026-07-08circles__google-gemini-3-flash-preview-7.7.2026circlesGoogle Gemini Flash 3 (preview)5.09
2026-07-08circles__moonshotai-kimi-k2.7-code-7.8.2026circlesMoonshot AI Kimi Code k2.75.07
2026-07-08circles__anthropic-claude-sonnet-5-7.8.2026circlesAnthropic Claude Sonnet 54.51
2026-07-05circles__google-gemini-3.5-flash-7.5.2026circlesGoogle Gemini Flash 3.54.59
2026-07-05circles__deepseek-deepseek-v4-pro-7.5.2026circlesDeepSeek Deepseek Pro v44.47
2026-07-05circles__google-gemini-3-flash-preview-7.5.2026circlesGoogle Gemini Flash 3 (preview)4.45
2026-07-05circles__moonshotai-kimi-k2.6-7.5.2026circlesMoonshot AI Kimi k2.64.13
2026-07-05circles__xiaomi-mimo-v2.5-pro-7.5.2026circlesXiaomi Mimo Pro v2.54.44
2026-07-04circles__deepseek-deepseek-v4-flash-7.3.2026circlesDeepSeek Deepseek Flash v45.41
2026-07-04circles__deepseek-deepseek-v4-flash-7.4.2026circlesDeepSeek Deepseek Flash v44.99
2026-07-03circles__anthropic-claude-sonnet-4.6-7.3.2026circlesAnthropic Claude Sonnet 4.64.06
2026-07-03circles__google-gemini-3.1-pro-preview-7.3.2026circlesGoogle Gemini Pro 3.1 (preview)4.42
2026-07-03circles__deepseek-deepseek-v4-pro-7.3.2026circlesDeepSeek Deepseek Pro v44.89
2026-07-03circles__z-ai-glm-5.2-7.3.2026circlesZ.ai Glm 5.24.51
2026-06-29solarSystem__xiaomi-mimo-v2.5-pro-6.29.2026solarSystemXiaomi Mimo Pro v2.55.79
2026-06-28circles__anthropic-claude-sonnet-4.6-6.28.2026circlesAnthropic Claude Sonnet 4.63.74
2026-06-28circles__google-gemini-3.1-pro-preview-6.28.2026circlesGoogle Gemini Pro 3.1 (preview)4.76
2026-06-28solarSystem__google-gemini-3-flash-preview-6.27.2026solarSystemGoogle Gemini Flash 3 (preview)4.23
2026-06-28circles__z-ai-glm-5.2-6.27.2026circlesZ.ai Glm 5.23.31
2026-06-28circles__xiaomi-mimo-v2.5-pro-6.28.2026circlesXiaomi Mimo Pro v2.56.02
2026-06-27circles__deepseek-deepseek-v4-pro-6.27.2026circlesDeepSeek Deepseek Pro v44.60
2026-06-27circles__moonshotai-kimi-k2.6-6.27.2026circlesMoonshot AI Kimi k2.64.27
2026-06-27solarSystem__deepseek-deepseek-v4-flash-6.27.2026solarSystemDeepSeek Deepseek Flash v44.57
2026-06-26circles__google-gemini-3-flash-preview-6.26.2026circlesGoogle Gemini Flash 3 (preview)5.71
2026-06-26the2048__deepseek-deepseek-v4-flash-6.26.2026the2048DeepSeek Deepseek Flash v44.37

Every run on Favur Evals is scored by the same deterministic engine, on the same Statements of Work. The benchmark is self-funded — no vendor sponsorship, credits, or grants.