Favur Evals
favur.dev

Agent role · weight 4/100 · rubric v15

🎛️ Orchestrator

Top-level lifecycle manager — directs phases, plans and dispatches the agent fleet, and gates development, sprint review and phase completion.

What it's scored on

Output quality
Whole-run code quality, test quality, deliverables completeness and process discipline (the work its fleet shipped under its direction).
Efficiency
Its own token volume, cost per request and context efficiency.

Full scoring definition in the methodology → · Sample transcript

Model leaderboard

Every model ranked by Orchestrator score
#ModelScore / 10CoveragenLast seen
1Xiaomi Mimo v2.5 †8.2788%52026-07-26
2Meta Muse Spark 1.17.93100%32026-07-19
3Google Gemini Flash Lite 3.5 †7.84100%12026-07-21
4xAI Grok 4.5 †7.8185%42026-07-22
5Google Gemini Flash 3.67.61100%52026-07-24
6Qwen Plus qwen3.7 †7.2985%42026-07-22
7Google Gemini Flash Lite 3.1 †7.1955%42026-07-19
8OpenAI Gpt Terra 5.6 †7.0940%12026-07-12
9OpenAI Gpt Luna 5.6 †7.0340%12026-07-14
10Stepfun Step Flash 3.7 †6.7570%22026-07-19
11Thinkingmachines Inkling6.69100%22026-07-22
12Google Gemini Flash 3.5 †6.6864%82026-07-16
13DeepSeek Deepseek Flash v4 †6.2047%62026-07-22
14Anthropic Claude Sonnet 5 †6.0276%52026-07-23
15Xiaomi Mimo Pro v2.5 †5.9435%32026-07-05
16Moonshot AI Kimi Code k2.7 †5.6340%22026-07-10
17Moonshot AI Kimi k2.6 †5.1357%32026-07-17
18Z.ai Glm 5.2 †5.0657%32026-07-17
19Google Gemini Flash 3 (preview) †5.0237%62026-07-10
20Google Gemini Pro 3.1 (preview) †4.3836%22026-07-03
21DeepSeek Deepseek Pro v4 †4.3737%32026-07-05
22Anthropic Claude Sonnet 4.6 †3.0036%22026-07-03

† low confidence — sparse role or low data coverage (renormalized, never zero-filled).

Per-run history (newest first)

Every appearance of the Orchestrator role, newest first
DateRunSoWModelScore / 10
2026-07-26solarSystem__multi-xiaomi-mimo-v2.5-7.26.2026solarSystemXiaomi Mimo v2.58.34
2026-07-24circles__google-gemini-3.6-flash-7.23.2026circlesGoogle Gemini Flash 3.67.64
2026-07-24circles__xiaomi-multi-model-2.5-2.5-pro-7.24.2026circlesXiaomi Mimo v2.58.28
2026-07-23solarSystem/multi-anthropic-sonnet-gemini-3.5-flash-lite-7.22.2026solarSystemAnthropic Claude Sonnet 55.89
2026-07-22circles__thinkingmachines-inkling-7.22.2026circlesThinkingmachines Inkling5.84
2026-07-22circles__x-ai-grok-4.5-7.22.2026circlesxAI Grok 4.58.14
2026-07-22circles__deepseek-deepseek-v4-flash-7.22.2026circlesDeepSeek Deepseek Flash v47.73
2026-07-22circles__anthropic-claude-sonnet-5-7.21.2026circlesAnthropic Claude Sonnet 56.98
2026-07-22the2048__gemini-multi-model-7-22-2026the2048Google Gemini Flash 3.67.71
2026-07-22circles__qwen-qwen3.7-plus-7.22.2026circlesQwen Plus qwen3.77.51
2026-07-22circles__xiaomi-mimo-v2.5-7.22.2026circlesXiaomi Mimo v2.58.43
2026-07-21solarSystem__google-multi-model-7.21.2026solarSystemGoogle Gemini Flash 3.67.53
2026-07-21circles__x-ai-grok-4.5-7.21.2026circlesGoogle Gemini Flash 3.67.81
2026-07-21circles__google-gemini-3.6-flash-7.21.2026circlesGoogle Gemini Flash 3.67.34
2026-07-21circles__google-gemini-3.5-flash-lite-7.21.2026circlesGoogle Gemini Flash Lite 3.57.84
2026-07-19solarSystem__stepfun-step-3.7-flash-7.19.2026solarSystemStepfun Step Flash 3.77.53
2026-07-19circles__x-ai-grok-4.5-7.18.2026circlesxAI Grok 4.57.99
2026-07-19circles__google-gemini-3.1-flash-lite-7.19.2026circlesGoogle Gemini Flash Lite 3.18.07
2026-07-19solarSystem__qwen-qwen3.7-plus-7.18.2026solarSystemQwen Plus qwen3.76.58
2026-07-19the2048__meta-muse-spark-1.1-7.19.2026the2048Meta Muse Spark 1.17.20
2026-07-18circles__thinkingmachines-inkling-7.18.2026circlesThinkingmachines Inkling7.54
2026-07-18solarSystem__xiaomi-mimo-v2.5-7.18.2026solarSystemXiaomi Mimo v2.58.41
2026-07-17circles__z-ai-glm-5.2-7.17.2026circlesZ.ai Glm 5.27.64
2026-07-17circles__moonshotai-kimi-k2.6-7.16.2026circlesMoonshot AI Kimi k2.67.28
2026-07-17circles__x-ai-grok-4.5-7.16.2026circlesxAI Grok 4.57.98
2026-07-17circles__anthropic-claude-sonnet-5-7.16.2026circlesAnthropic Claude Sonnet 57.09
2026-07-17circles__qwen-qwen3.7-plus-7.16.2026circlesQwen Plus qwen3.78.45
2026-07-17solarSystem__meta-muse-spark-1.1-7.17.2026solarSystemMeta Muse Spark 1.18.13
2026-07-17circles__meta-muse-spark-1.1-7.16.2026circlesMeta Muse Spark 1.18.45
2026-07-16circles__google-multi-model-7.18.2026circlesGoogle Gemini Flash 3.57.89
2026-07-16circles__google-multi-model-7.17.2026circlesGoogle Gemini Flash 3.57.76
2026-07-15circles__google-gemini-3.1-flash-lite-7.14.2026circlesGoogle Gemini Flash Lite 3.16.77
2026-07-15solarSystem__google-multi-model-7.15.2026solarSystemGoogle Gemini Flash 3.56.49
2026-07-15circles__google-multi-model-7.16.2026circlesGoogle Gemini Flash 3.57.74
2026-07-15circles__google-multi-model-7.15.2026circlesGoogle Gemini Flash 3.57.89
2026-07-14circles__openai-gpt-5.6-luna-7.14.2026circlesOpenAI Gpt Luna 5.67.03
2026-07-14solarSystem__google-multi-model-7.14.2026solarSystemGoogle Gemini Flash 3.56.83
2026-07-13circles__stepfun-step-3.7-flash-7.13.2026circlesStepfun Step Flash 3.75.96
2026-07-13solarSystem__google-gemini-3.1-flash-lite-7.12.2026solarSystemGoogle Gemini Flash Lite 3.17.35
2026-07-12circles__openai-gpt-5.6-terra-7.11.2026circlesOpenAI Gpt Terra 5.67.09
2026-07-12circles__anthropic-claude-sonnet-5-7.11.2026circlesAnthropic Claude Sonnet 55.51
2026-07-10circles__google-gemini-3-flash-preview-7.9.2026circlesGoogle Gemini Flash 3 (preview)5.79
2026-07-10circles__moonshotai-kimi-k2.7-code-7.9.2026circlesMoonshot AI Kimi Code k2.75.90
2026-07-10circles__x-ai-grok-4.5-7.10.2026circlesxAI Grok 4.57.11
2026-07-10circles__deepseek-deepseek-v4-flash-7.10.2026circlesDeepSeek Deepseek Flash v47.30
2026-07-10circles__xiaomi-mimo-v2.5-pro-7.10.2026circlesXiaomi Mimo v2.57.87
2026-07-09circles__google-gemini-3-flash-preview-7.8.2026circlesGoogle Gemini Flash 3 (preview)5.67
2026-07-09circles__google-gemini-3.1-flash-lite-7.9.2026circlesGoogle Gemini Flash Lite 3.16.57
2026-07-09circles__qwen-qwen3.7-plus-7.9.2026circlesQwen Plus qwen3.76.60
2026-07-08circles__google-gemini-3-flash-preview-7.7.2026circlesGoogle Gemini Flash 3 (preview)5.83
2026-07-08circles__moonshotai-kimi-k2.7-code-7.8.2026circlesMoonshot AI Kimi Code k2.75.37
2026-07-08circles__anthropic-claude-sonnet-5-7.8.2026circlesAnthropic Claude Sonnet 54.62
2026-07-08circles__google-gemini-3.5-flash-7.8.2026circlesGoogle Gemini Flash 3.54.95
2026-07-05circles__deepseek-deepseek-v4-pro-7.5.2026circlesDeepSeek Deepseek Pro v44.68
2026-07-05circles__google-gemini-3-flash-preview-7.5.2026circlesGoogle Gemini Flash 3 (preview)4.86
2026-07-05circles__moonshotai-kimi-k2.6-7.5.2026circlesMoonshot AI Kimi k2.63.43
2026-07-05circles__xiaomi-mimo-v2.5-pro-7.5.2026circlesXiaomi Mimo Pro v2.56.16
2026-07-05circles__google-gemini-3.5-flash-7.5.2026circlesGoogle Gemini Flash 3.53.92
2026-07-04circles__deepseek-deepseek-v4-flash-7.3.2026circlesDeepSeek Deepseek Flash v45.27
2026-07-04circles__deepseek-deepseek-v4-flash-7.4.2026circlesDeepSeek Deepseek Flash v45.52
2026-07-03circles__anthropic-claude-sonnet-4.6-7.3.2026circlesAnthropic Claude Sonnet 4.63.20
2026-07-03circles__google-gemini-3.1-pro-preview-7.3.2026circlesGoogle Gemini Pro 3.1 (preview)5.24
2026-07-03circles__deepseek-deepseek-v4-pro-7.3.2026circlesDeepSeek Deepseek Pro v44.27
2026-07-03circles__z-ai-glm-5.2-7.3.2026circlesZ.ai Glm 5.24.50
2026-06-29solarSystem__xiaomi-mimo-v2.5-pro-6.29.2026solarSystemXiaomi Mimo Pro v2.55.96
2026-06-28circles__anthropic-claude-sonnet-4.6-6.28.2026circlesAnthropic Claude Sonnet 4.62.80
2026-06-28circles__google-gemini-3.1-pro-preview-6.28.2026circlesGoogle Gemini Pro 3.1 (preview)3.52
2026-06-28solarSystem__google-gemini-3-flash-preview-6.27.2026solarSystemGoogle Gemini Flash 3 (preview)3.36
2026-06-28circles__z-ai-glm-5.2-6.27.2026circlesZ.ai Glm 5.23.03
2026-06-28circles__xiaomi-mimo-v2.5-pro-6.28.2026circlesXiaomi Mimo Pro v2.55.69
2026-06-27circles__deepseek-deepseek-v4-pro-6.27.2026circlesDeepSeek Deepseek Pro v44.15
2026-06-27circles__moonshotai-kimi-k2.6-6.27.2026circlesMoonshot AI Kimi k2.64.67
2026-06-27solarSystem__deepseek-deepseek-v4-flash-6.27.2026solarSystemDeepSeek Deepseek Flash v45.07
2026-06-26circles__google-gemini-3-flash-preview-6.26.2026circlesGoogle Gemini Flash 3 (preview)4.60
2026-06-26the2048__deepseek-deepseek-v4-flash-6.26.2026the2048DeepSeek Deepseek Flash v46.31

Every run on Favur Evals is scored by the same deterministic engine, on the same Statements of Work. The benchmark is self-funded — no vendor sponsorship, credits, or grants.