Favur Evals
favur.dev

Agent role · weight 2/100 · rubric v15

🔍 Scout

Read-only observer — produces a findings document of observations, insights and concerns (an honest null outcome is valid).

What it's scored on

Output quality
Finding quality of the report it produced and deliverables.
Efficiency
Token volume, output per request, cost, context efficiency, request-error and hallucination rates.

Full scoring definition in the methodology → · Sample transcript

Model leaderboard

Every model ranked by Scout score
#ModelScore / 10CoveragenLast seen
1OpenAI Gpt Luna 5.6 †7.8280%12026-07-14
2OpenAI Gpt Terra 5.6 †7.5180%12026-07-12
3DeepSeek Deepseek Pro v47.3477%32026-07-05
4Stepfun Step Flash 3.77.3175%22026-07-19
5DeepSeek Deepseek Flash v47.2381%62026-07-22
6xAI Grok 4.57.2295%42026-07-22
7Moonshot AI Kimi Code k2.77.0880%22026-07-10
8Z.ai Glm 5.27.0876%32026-07-17
9Xiaomi Mimo v2.57.0093%32026-07-22
10Google Gemini Flash 3 (preview)6.9077%62026-07-10
11Google Gemini Flash Lite 3.56.83100%52026-07-23
12Xiaomi Mimo Pro v2.56.6785%52026-07-26
13Google Gemini Flash 3.66.46100%22026-07-24
14Moonshot AI Kimi k2.6 †6.4367%32026-07-17
15Anthropic Claude Sonnet 4.66.1976%22026-07-03
16Google Gemini Flash 3.56.0380%22026-07-08
17Qwen Plus qwen3.76.03100%32026-07-22
18Meta Muse Spark 1.15.9885%32026-07-19
19Google Gemini Flash Lite 3.15.9189%92026-07-19
20Anthropic Claude Sonnet 55.8090%42026-07-22
21Thinkingmachines Inkling5.79100%22026-07-22

† low confidence — sparse role or low data coverage (renormalized, never zero-filled).

Per-run history (newest first)

Every appearance of the Scout role, newest first
DateRunSoWModelScore / 10
2026-07-26solarSystem__multi-xiaomi-mimo-v2.5-7.26.2026solarSystemXiaomi Mimo Pro v2.55.97
2026-07-24circles__xiaomi-multi-model-2.5-2.5-pro-7.24.2026circlesXiaomi Mimo Pro v2.55.86
2026-07-24circles__google-gemini-3.6-flash-7.23.2026circlesGoogle Gemini Flash 3.66.43
2026-07-23solarSystem/multi-anthropic-sonnet-gemini-3.5-flash-lite-7.22.2026solarSystemGoogle Gemini Flash Lite 3.56.77
2026-07-22circles__thinkingmachines-inkling-7.22.2026circlesThinkingmachines Inkling5.63
2026-07-22circles__x-ai-grok-4.5-7.22.2026circlesxAI Grok 4.57.16
2026-07-22circles__deepseek-deepseek-v4-flash-7.22.2026circlesDeepSeek Deepseek Flash v46.48
2026-07-22circles__anthropic-claude-sonnet-5-7.21.2026circlesAnthropic Claude Sonnet 55.49
2026-07-22the2048__gemini-multi-model-7-22-2026the2048Google Gemini Flash Lite 3.56.61
2026-07-22circles__qwen-qwen3.7-plus-7.22.2026circlesQwen Plus qwen3.75.88
2026-07-22circles__xiaomi-mimo-v2.5-7.22.2026circlesXiaomi Mimo v2.56.75
2026-07-21circles__google-gemini-3.6-flash-7.21.2026circlesGoogle Gemini Flash 3.66.48
2026-07-21solarSystem__google-multi-model-7.21.2026solarSystemGoogle Gemini Flash Lite 3.57.10
2026-07-21circles__x-ai-grok-4.5-7.21.2026circlesGoogle Gemini Flash Lite 3.56.55
2026-07-21circles__google-gemini-3.5-flash-lite-7.21.2026circlesGoogle Gemini Flash Lite 3.57.12
2026-07-19solarSystem__stepfun-step-3.7-flash-7.19.2026solarSystemStepfun Step Flash 3.77.30
2026-07-19circles__x-ai-grok-4.5-7.18.2026circlesxAI Grok 4.57.15
2026-07-19solarSystem__qwen-qwen3.7-plus-7.18.2026solarSystemQwen Plus qwen3.76.30
2026-07-19the2048__meta-muse-spark-1.1-7.19.2026the2048Meta Muse Spark 1.17.03
2026-07-19circles__google-gemini-3.1-flash-lite-7.19.2026circlesGoogle Gemini Flash Lite 3.16.79
2026-07-18circles__thinkingmachines-inkling-7.18.2026circlesThinkingmachines Inkling5.96
2026-07-18solarSystem__xiaomi-mimo-v2.5-7.18.2026solarSystemXiaomi Mimo v2.57.08
2026-07-17circles__z-ai-glm-5.2-7.17.2026circlesZ.ai Glm 5.26.09
2026-07-17circles__moonshotai-kimi-k2.6-7.16.2026circlesMoonshot AI Kimi k2.65.95
2026-07-17circles__x-ai-grok-4.5-7.16.2026circlesxAI Grok 4.57.36
2026-07-17circles__anthropic-claude-sonnet-5-7.16.2026circlesAnthropic Claude Sonnet 55.35
2026-07-17circles__qwen-qwen3.7-plus-7.16.2026circlesQwen Plus qwen3.75.92
2026-07-17solarSystem__meta-muse-spark-1.1-7.17.2026solarSystemMeta Muse Spark 1.16.32
2026-07-17circles__meta-muse-spark-1.1-7.16.2026circlesMeta Muse Spark 1.14.59
2026-07-16circles__google-multi-model-7.18.2026circlesGoogle Gemini Flash Lite 3.16.80
2026-07-16circles__google-multi-model-7.17.2026circlesGoogle Gemini Flash Lite 3.13.81
2026-07-15solarSystem__google-multi-model-7.15.2026solarSystemGoogle Gemini Flash Lite 3.15.88
2026-07-15circles__google-gemini-3.1-flash-lite-7.14.2026circlesGoogle Gemini Flash Lite 3.15.64
2026-07-15circles__google-multi-model-7.16.2026circlesGoogle Gemini Flash Lite 3.16.71
2026-07-15circles__google-multi-model-7.15.2026circlesGoogle Gemini Flash Lite 3.15.82
2026-07-14circles__openai-gpt-5.6-luna-7.14.2026circlesOpenAI Gpt Luna 5.67.82
2026-07-14solarSystem__google-multi-model-7.14.2026solarSystemGoogle Gemini Flash Lite 3.15.88
2026-07-13circles__stepfun-step-3.7-flash-7.13.2026circlesStepfun Step Flash 3.77.33
2026-07-13solarSystem__google-gemini-3.1-flash-lite-7.12.2026solarSystemGoogle Gemini Flash Lite 3.15.87
2026-07-12circles__openai-gpt-5.6-terra-7.11.2026circlesOpenAI Gpt Terra 5.67.51
2026-07-12circles__anthropic-claude-sonnet-5-7.11.2026circlesAnthropic Claude Sonnet 56.50
2026-07-10circles__google-gemini-3-flash-preview-7.9.2026circlesGoogle Gemini Flash 3 (preview)7.00
2026-07-10circles__moonshotai-kimi-k2.7-code-7.9.2026circlesMoonshot AI Kimi Code k2.77.22
2026-07-10circles__x-ai-grok-4.5-7.10.2026circlesxAI Grok 4.57.22
2026-07-10circles__deepseek-deepseek-v4-flash-7.10.2026circlesDeepSeek Deepseek Flash v48.48
2026-07-10circles__xiaomi-mimo-v2.5-pro-7.10.2026circlesXiaomi Mimo v2.57.16
2026-07-09circles__google-gemini-3-flash-preview-7.8.2026circlesGoogle Gemini Flash 3 (preview)6.72
2026-07-08circles__google-gemini-3.5-flash-7.8.2026circlesGoogle Gemini Flash 3.56.17
2026-07-08circles__google-gemini-3-flash-preview-7.7.2026circlesGoogle Gemini Flash 3 (preview)6.78
2026-07-08circles__moonshotai-kimi-k2.7-code-7.8.2026circlesMoonshot AI Kimi Code k2.76.94
2026-07-08circles__anthropic-claude-sonnet-5-7.8.2026circlesAnthropic Claude Sonnet 55.88
2026-07-05circles__google-gemini-3.5-flash-7.5.2026circlesGoogle Gemini Flash 3.55.89
2026-07-05circles__deepseek-deepseek-v4-pro-7.5.2026circlesDeepSeek Deepseek Pro v47.43
2026-07-05circles__google-gemini-3-flash-preview-7.5.2026circlesGoogle Gemini Flash 3 (preview)6.58
2026-07-05circles__moonshotai-kimi-k2.6-7.5.2026circlesMoonshot AI Kimi k2.67.04
2026-07-05circles__xiaomi-mimo-v2.5-pro-7.5.2026circlesXiaomi Mimo Pro v2.57.07
2026-07-04circles__deepseek-deepseek-v4-flash-7.3.2026circlesDeepSeek Deepseek Flash v47.65
2026-07-04circles__deepseek-deepseek-v4-flash-7.4.2026circlesDeepSeek Deepseek Flash v47.15
2026-07-03circles__anthropic-claude-sonnet-4.6-7.3.2026circlesAnthropic Claude Sonnet 4.65.91
2026-07-03circles__deepseek-deepseek-v4-pro-7.3.2026circlesDeepSeek Deepseek Pro v47.23
2026-07-03circles__z-ai-glm-5.2-7.3.2026circlesZ.ai Glm 5.27.54
2026-06-29solarSystem__xiaomi-mimo-v2.5-pro-6.29.2026solarSystemXiaomi Mimo Pro v2.56.92
2026-06-28circles__anthropic-claude-sonnet-4.6-6.28.2026circlesAnthropic Claude Sonnet 4.66.47
2026-06-28solarSystem__google-gemini-3-flash-preview-6.27.2026solarSystemGoogle Gemini Flash 3 (preview)6.69
2026-06-28circles__z-ai-glm-5.2-6.27.2026circlesZ.ai Glm 5.27.60
2026-06-28circles__xiaomi-mimo-v2.5-pro-6.28.2026circlesXiaomi Mimo Pro v2.57.52
2026-06-27circles__deepseek-deepseek-v4-pro-6.27.2026circlesDeepSeek Deepseek Pro v47.35
2026-06-27circles__moonshotai-kimi-k2.6-6.27.2026circlesMoonshot AI Kimi k2.66.30
2026-06-27solarSystem__deepseek-deepseek-v4-flash-6.27.2026solarSystemDeepSeek Deepseek Flash v46.65
2026-06-26circles__google-gemini-3-flash-preview-6.26.2026circlesGoogle Gemini Flash 3 (preview)7.63
2026-06-26the2048__deepseek-deepseek-v4-flash-6.26.2026the2048DeepSeek Deepseek Flash v46.95

Every run on Favur Evals is scored by the same deterministic engine, on the same Statements of Work. The benchmark is self-funded — no vendor sponsorship, credits, or grants.