Favur Evals
favur.dev

Agent role · weight 3/100 · rubric v15

🛠️ Platform

Infrastructure artifacts (CI/CD, Dockerfiles, configs) verified via shell execution before completion.

What it's scored on

Output quality
Static code quality and deliverables of the infrastructure it shipped.
Efficiency
Token volume, output per request, cost, context efficiency, request-error and hallucination rates.

Full scoring definition in the methodology → · Sample transcript

Model leaderboard

Every model ranked by Platform score
#ModelScore / 10CoveragenLast seen
1Stepfun Step Flash 3.7 †6.80100%12026-07-19
2Google Gemini Flash 3.66.75100%22026-07-24
3Google Gemini Flash Lite 3.56.4492%42026-07-22
4Qwen Plus qwen3.76.38100%22026-07-22
5Meta Muse Spark 1.16.3784%22026-07-19
6Google Gemini Flash Lite 3.15.8066%82026-07-16
7xAI Grok 4.55.8078%22026-07-17
8Thinkingmachines Inkling †5.78100%12026-07-22
9Anthropic Claude Sonnet 55.7578%22026-07-17
10Google Gemini Flash 3 (preview) †5.2052%62026-07-10
11Google Gemini Flash 3.55.1755%22026-07-08
12OpenAI Gpt Terra 5.6 †5.1755%12026-07-12
13DeepSeek Deepseek Pro v45.0155%22026-07-05
14OpenAI Gpt Luna 5.6 †4.9555%12026-07-14
15Moonshot AI Kimi Code k2.74.8955%22026-07-10
16DeepSeek Deepseek Flash v4 †4.8459%32026-07-22
17Google Gemini Pro 3.1 (preview) †4.4548%22026-07-03
18Moonshot AI Kimi k2.6 †4.4447%12026-06-27

† low confidence — sparse role or low data coverage (renormalized, never zero-filled).

Per-run history (newest first)

Every appearance of the Platform role, newest first
DateRunSoWModelScore / 10
2026-07-24circles__google-gemini-3.6-flash-7.23.2026circlesGoogle Gemini Flash 3.67.01
2026-07-22circles__thinkingmachines-inkling-7.22.2026circlesThinkingmachines Inkling5.78
2026-07-22circles__deepseek-deepseek-v4-flash-7.22.2026circlesDeepSeek Deepseek Flash v45.31
2026-07-22circles__qwen-qwen3.7-plus-7.22.2026circlesQwen Plus qwen3.76.18
2026-07-22the2048__gemini-multi-model-7-22-2026the2048Google Gemini Flash Lite 3.56.30
2026-07-21circles__google-gemini-3.6-flash-7.21.2026circlesGoogle Gemini Flash 3.66.48
2026-07-21solarSystem__google-multi-model-7.21.2026solarSystemGoogle Gemini Flash Lite 3.56.71
2026-07-21circles__x-ai-grok-4.5-7.21.2026circlesGoogle Gemini Flash Lite 3.56.21
2026-07-21circles__google-gemini-3.5-flash-lite-7.21.2026circlesGoogle Gemini Flash Lite 3.56.55
2026-07-19solarSystem__stepfun-step-3.7-flash-7.19.2026solarSystemStepfun Step Flash 3.76.80
2026-07-19the2048__meta-muse-spark-1.1-7.19.2026the2048Meta Muse Spark 1.16.10
2026-07-19solarSystem__qwen-qwen3.7-plus-7.18.2026solarSystemQwen Plus qwen3.76.59
2026-07-17circles__x-ai-grok-4.5-7.16.2026circlesxAI Grok 4.56.31
2026-07-17solarSystem__meta-muse-spark-1.1-7.17.2026solarSystemMeta Muse Spark 1.16.64
2026-07-17circles__anthropic-claude-sonnet-5-7.16.2026circlesAnthropic Claude Sonnet 56.64
2026-07-16circles__google-multi-model-7.17.2026circlesGoogle Gemini Flash Lite 3.17.16
2026-07-15solarSystem__google-multi-model-7.15.2026solarSystemGoogle Gemini Flash Lite 3.14.32
2026-07-15circles__google-gemini-3.1-flash-lite-7.14.2026circlesGoogle Gemini Flash Lite 3.16.35
2026-07-15circles__google-multi-model-7.16.2026circlesGoogle Gemini Flash Lite 3.16.65
2026-07-15circles__google-multi-model-7.15.2026circlesGoogle Gemini Flash Lite 3.15.57
2026-07-14circles__openai-gpt-5.6-luna-7.14.2026circlesOpenAI Gpt Luna 5.64.95
2026-07-14solarSystem__google-multi-model-7.14.2026solarSystemGoogle Gemini Flash Lite 3.15.73
2026-07-13solarSystem__google-gemini-3.1-flash-lite-7.12.2026solarSystemGoogle Gemini Flash Lite 3.15.07
2026-07-12circles__openai-gpt-5.6-terra-7.11.2026circlesOpenAI Gpt Terra 5.65.17
2026-07-10circles__google-gemini-3-flash-preview-7.9.2026circlesGoogle Gemini Flash 3 (preview)5.81
2026-07-10circles__moonshotai-kimi-k2.7-code-7.9.2026circlesMoonshot AI Kimi Code k2.74.46
2026-07-10circles__x-ai-grok-4.5-7.10.2026circlesxAI Grok 4.55.29
2026-07-10circles__deepseek-deepseek-v4-flash-7.10.2026circlesDeepSeek Deepseek Flash v45.81
2026-07-09circles__google-gemini-3-flash-preview-7.8.2026circlesGoogle Gemini Flash 3 (preview)5.04
2026-07-09circles__google-gemini-3.1-flash-lite-7.9.2026circlesGoogle Gemini Flash Lite 3.15.59
2026-07-08circles__google-gemini-3.5-flash-7.8.2026circlesGoogle Gemini Flash 3.55.70
2026-07-08circles__google-gemini-3-flash-preview-7.7.2026circlesGoogle Gemini Flash 3 (preview)5.59
2026-07-08circles__moonshotai-kimi-k2.7-code-7.8.2026circlesMoonshot AI Kimi Code k2.75.32
2026-07-08circles__anthropic-claude-sonnet-5-7.8.2026circlesAnthropic Claude Sonnet 54.86
2026-07-05circles__google-gemini-3.5-flash-7.5.2026circlesGoogle Gemini Flash 3.54.64
2026-07-05circles__deepseek-deepseek-v4-pro-7.5.2026circlesDeepSeek Deepseek Pro v45.19
2026-07-05circles__google-gemini-3-flash-preview-7.5.2026circlesGoogle Gemini Flash 3 (preview)5.97
2026-07-03circles__google-gemini-3.1-pro-preview-7.3.2026circlesGoogle Gemini Pro 3.1 (preview)4.59
2026-07-03circles__deepseek-deepseek-v4-pro-7.3.2026circlesDeepSeek Deepseek Pro v44.82
2026-06-28circles__google-gemini-3.1-pro-preview-6.28.2026circlesGoogle Gemini Pro 3.1 (preview)4.30
2026-06-28solarSystem__google-gemini-3-flash-preview-6.27.2026solarSystemGoogle Gemini Flash 3 (preview)3.88
2026-06-27circles__moonshotai-kimi-k2.6-6.27.2026circlesMoonshot AI Kimi k2.64.44
2026-06-27solarSystem__deepseek-deepseek-v4-flash-6.27.2026solarSystemDeepSeek Deepseek Flash v43.41
2026-06-26circles__google-gemini-3-flash-preview-6.26.2026circlesGoogle Gemini Flash 3 (preview)4.91

Every run on Favur Evals is scored by the same deterministic engine, on the same Statements of Work. The benchmark is self-funded — no vendor sponsorship, credits, or grants.