Favur Evals
favur.dev

Agent role · weight 4/100 · rubric v15

🧪 Test

Read-only verification — runs the suite, analyses failures and checks coverage against the requested floor.

What it's scored on

Output quality
Test pass rate, coverage above the 90% floor, structural test quality and deliverables.
Efficiency
Token volume, cost and context efficiency.

Full scoring definition in the methodology → · Sample transcript

Model leaderboard

Every model ranked by Test score
#ModelScore / 10CoveragenLast seen
1OpenAI Gpt Luna 5.6 †7.4060%12026-07-14
2Xiaomi Mimo Pro v2.57.2764%52026-07-26
3Xiaomi Mimo v2.57.1473%32026-07-22
4DeepSeek Deepseek Flash v47.0060%62026-07-22
5OpenAI Gpt Terra 5.6 †6.8960%12026-07-12
6Qwen Plus qwen3.76.7375%42026-07-22
7Stepfun Step Flash 3.76.6970%22026-07-19
8Google Gemini Flash 3.66.5480%22026-07-24
9Meta Muse Spark 1.16.4380%32026-07-19
10Moonshot AI Kimi k2.66.2563%32026-07-17
11Google Gemini Flash Lite 3.56.1980%52026-07-23
12Google Gemini Flash 3.56.1760%22026-07-08
13DeepSeek Deepseek Pro v46.0457%32026-07-05
14xAI Grok 4.55.9775%42026-07-22
15Google Gemini Flash Lite 3.1 †5.9068%82026-07-19
16Moonshot AI Kimi Code k2.75.8060%22026-07-10
17Google Gemini Pro 3.1 (preview)5.6455%22026-07-03
18Anthropic Claude Sonnet 55.5770%42026-07-22
19Z.ai Glm 5.25.5363%32026-07-17
20Thinkingmachines Inkling4.7480%22026-07-22
21Google Gemini Flash 3 (preview)4.6757%62026-07-10
22Anthropic Claude Sonnet 4.64.4155%22026-07-03

† low confidence — sparse role or low data coverage (renormalized, never zero-filled).

Per-run history (newest first)

Every appearance of the Test role, newest first
DateRunSoWModelScore / 10
2026-07-26solarSystem__multi-xiaomi-mimo-v2.5-7.26.2026solarSystemXiaomi Mimo Pro v2.57.10
2026-07-24circles__xiaomi-multi-model-2.5-2.5-pro-7.24.2026circlesXiaomi Mimo Pro v2.57.28
2026-07-24circles__google-gemini-3.6-flash-7.23.2026circlesGoogle Gemini Flash 3.66.28
2026-07-23solarSystem/multi-anthropic-sonnet-gemini-3.5-flash-lite-7.22.2026solarSystemGoogle Gemini Flash Lite 3.56.52
2026-07-22circles__thinkingmachines-inkling-7.22.2026circlesThinkingmachines Inkling4.81
2026-07-22circles__x-ai-grok-4.5-7.22.2026circlesxAI Grok 4.56.22
2026-07-22circles__deepseek-deepseek-v4-flash-7.22.2026circlesDeepSeek Deepseek Flash v46.85
2026-07-22circles__anthropic-claude-sonnet-5-7.21.2026circlesAnthropic Claude Sonnet 55.73
2026-07-22the2048__gemini-multi-model-7-22-2026the2048Google Gemini Flash Lite 3.53.93
2026-07-22circles__qwen-qwen3.7-plus-7.22.2026circlesQwen Plus qwen3.76.38
2026-07-22circles__xiaomi-mimo-v2.5-7.22.2026circlesXiaomi Mimo v2.57.21
2026-07-21circles__google-gemini-3.6-flash-7.21.2026circlesGoogle Gemini Flash 3.66.80
2026-07-21solarSystem__google-multi-model-7.21.2026solarSystemGoogle Gemini Flash Lite 3.57.18
2026-07-21circles__x-ai-grok-4.5-7.21.2026circlesGoogle Gemini Flash Lite 3.56.60
2026-07-21circles__google-gemini-3.5-flash-lite-7.21.2026circlesGoogle Gemini Flash Lite 3.56.72
2026-07-19solarSystem__stepfun-step-3.7-flash-7.19.2026solarSystemStepfun Step Flash 3.77.08
2026-07-19circles__x-ai-grok-4.5-7.18.2026circlesxAI Grok 4.55.87
2026-07-19solarSystem__qwen-qwen3.7-plus-7.18.2026solarSystemQwen Plus qwen3.76.89
2026-07-19the2048__meta-muse-spark-1.1-7.19.2026the2048Meta Muse Spark 1.15.15
2026-07-19circles__google-gemini-3.1-flash-lite-7.19.2026circlesGoogle Gemini Flash Lite 3.13.94
2026-07-18circles__thinkingmachines-inkling-7.18.2026circlesThinkingmachines Inkling4.68
2026-07-18solarSystem__xiaomi-mimo-v2.5-7.18.2026solarSystemXiaomi Mimo v2.57.41
2026-07-17circles__z-ai-glm-5.2-7.17.2026circlesZ.ai Glm 5.26.25
2026-07-17circles__moonshotai-kimi-k2.6-7.16.2026circlesMoonshot AI Kimi k2.67.32
2026-07-17circles__x-ai-grok-4.5-7.16.2026circlesxAI Grok 4.55.99
2026-07-17circles__anthropic-claude-sonnet-5-7.16.2026circlesAnthropic Claude Sonnet 56.01
2026-07-17circles__qwen-qwen3.7-plus-7.16.2026circlesQwen Plus qwen3.76.92
2026-07-17solarSystem__meta-muse-spark-1.1-7.17.2026solarSystemMeta Muse Spark 1.16.86
2026-07-17circles__meta-muse-spark-1.1-7.16.2026circlesMeta Muse Spark 1.17.28
2026-07-16circles__google-multi-model-7.18.2026circlesGoogle Gemini Flash Lite 3.16.72
2026-07-16circles__google-multi-model-7.17.2026circlesGoogle Gemini Flash Lite 3.17.03
2026-07-15solarSystem__google-multi-model-7.15.2026solarSystemGoogle Gemini Flash Lite 3.13.94
2026-07-15circles__google-gemini-3.1-flash-lite-7.14.2026circlesGoogle Gemini Flash Lite 3.17.17
2026-07-15circles__google-multi-model-7.15.2026circlesGoogle Gemini Flash Lite 3.16.89
2026-07-14circles__openai-gpt-5.6-luna-7.14.2026circlesOpenAI Gpt Luna 5.67.40
2026-07-13circles__stepfun-step-3.7-flash-7.13.2026circlesStepfun Step Flash 3.76.29
2026-07-13solarSystem__google-gemini-3.1-flash-lite-7.12.2026solarSystemGoogle Gemini Flash Lite 3.15.47
2026-07-12circles__openai-gpt-5.6-terra-7.11.2026circlesOpenAI Gpt Terra 5.66.89
2026-07-12circles__anthropic-claude-sonnet-5-7.11.2026circlesAnthropic Claude Sonnet 55.29
2026-07-10circles__google-gemini-3-flash-preview-7.9.2026circlesGoogle Gemini Flash 3 (preview)6.01
2026-07-10circles__moonshotai-kimi-k2.7-code-7.9.2026circlesMoonshot AI Kimi Code k2.75.84
2026-07-10circles__x-ai-grok-4.5-7.10.2026circlesxAI Grok 4.55.79
2026-07-10circles__deepseek-deepseek-v4-flash-7.10.2026circlesDeepSeek Deepseek Flash v47.74
2026-07-10circles__xiaomi-mimo-v2.5-pro-7.10.2026circlesXiaomi Mimo v2.56.79
2026-07-09circles__google-gemini-3-flash-preview-7.8.2026circlesGoogle Gemini Flash 3 (preview)6.68
2026-07-09circles__qwen-qwen3.7-plus-7.9.2026circlesQwen Plus qwen3.76.75
2026-07-09circles__google-gemini-3.1-flash-lite-7.9.2026circlesGoogle Gemini Flash Lite 3.16.02
2026-07-08circles__google-gemini-3.5-flash-7.8.2026circlesGoogle Gemini Flash 3.56.22
2026-07-08circles__google-gemini-3-flash-preview-7.7.2026circlesGoogle Gemini Flash 3 (preview)2.00
2026-07-08circles__moonshotai-kimi-k2.7-code-7.8.2026circlesMoonshot AI Kimi Code k2.75.77
2026-07-08circles__anthropic-claude-sonnet-5-7.8.2026circlesAnthropic Claude Sonnet 55.25
2026-07-05circles__google-gemini-3.5-flash-7.5.2026circlesGoogle Gemini Flash 3.56.13
2026-07-05circles__deepseek-deepseek-v4-pro-7.5.2026circlesDeepSeek Deepseek Pro v46.03
2026-07-05circles__google-gemini-3-flash-preview-7.5.2026circlesGoogle Gemini Flash 3 (preview)5.92
2026-07-05circles__moonshotai-kimi-k2.6-7.5.2026circlesMoonshot AI Kimi k2.66.00
2026-07-05circles__xiaomi-mimo-v2.5-pro-7.5.2026circlesXiaomi Mimo Pro v2.56.81
2026-07-04circles__deepseek-deepseek-v4-flash-7.3.2026circlesDeepSeek Deepseek Flash v46.90
2026-07-04circles__deepseek-deepseek-v4-flash-7.4.2026circlesDeepSeek Deepseek Flash v47.02
2026-07-03circles__anthropic-claude-sonnet-4.6-7.3.2026circlesAnthropic Claude Sonnet 4.64.78
2026-07-03circles__google-gemini-3.1-pro-preview-7.3.2026circlesGoogle Gemini Pro 3.1 (preview)7.20
2026-07-03circles__deepseek-deepseek-v4-pro-7.3.2026circlesDeepSeek Deepseek Pro v45.87
2026-07-03circles__z-ai-glm-5.2-7.3.2026circlesZ.ai Glm 5.26.01
2026-06-29solarSystem__xiaomi-mimo-v2.5-pro-6.29.2026solarSystemXiaomi Mimo Pro v2.58.18
2026-06-28circles__anthropic-claude-sonnet-4.6-6.28.2026circlesAnthropic Claude Sonnet 4.64.03
2026-06-28circles__google-gemini-3.1-pro-preview-6.28.2026circlesGoogle Gemini Pro 3.1 (preview)4.09
2026-06-28solarSystem__google-gemini-3-flash-preview-6.27.2026solarSystemGoogle Gemini Flash 3 (preview)0.53
2026-06-28circles__z-ai-glm-5.2-6.27.2026circlesZ.ai Glm 5.24.32
2026-06-28circles__xiaomi-mimo-v2.5-pro-6.28.2026circlesXiaomi Mimo Pro v2.56.99
2026-06-27circles__deepseek-deepseek-v4-pro-6.27.2026circlesDeepSeek Deepseek Pro v46.22
2026-06-27circles__moonshotai-kimi-k2.6-6.27.2026circlesMoonshot AI Kimi k2.65.44
2026-06-27solarSystem__deepseek-deepseek-v4-flash-6.27.2026solarSystemDeepSeek Deepseek Flash v47.34
2026-06-26circles__google-gemini-3-flash-preview-6.26.2026circlesGoogle Gemini Flash 3 (preview)6.87
2026-06-26the2048__deepseek-deepseek-v4-flash-6.26.2026the2048DeepSeek Deepseek Flash v46.16

Every run on Favur Evals is scored by the same deterministic engine, on the same Statements of Work. The benchmark is self-funded — no vendor sponsorship, credits, or grants.