Vantage RuntimeAI · Benchmarks
Expert Task Benchmark
Twenty OpenRouter models evaluated on five high-complexity engineering tasks — multi-turn dialogue, deterministic rubrics, trust/repetition signals, and full scorecard PDFs per run.
What we tested
We evaluated 0 OpenRouter models across 5 high-complexity multi-turn engineering tasks with deterministic rubrics and trust signals.
- Pipeline incident triage · feature leakage review · lineage impact · train/serve skew · tight-budget routing
- 6 agent turns per scenario with scripted counterpart follow-ups and explicit reviewer close
- Every completed run links to the same scorecard PDF used in Simulator and leaderboard exports
Pipeline Incident Triage
6 agent turns + scripted follow-ups and reviewer close · deterministic task rubric.
| # | Model | Score | Criteria | Trust | Est. cost | Time | Scorecard |
|---|
Feature Leakage Review
6 agent turns + scripted follow-ups and reviewer close · deterministic task rubric.
| # | Model | Score | Criteria | Trust | Est. cost | Time | Scorecard |
|---|
Lineage Impact Review
6 agent turns + scripted follow-ups and reviewer close · deterministic task rubric.
| # | Model | Score | Criteria | Trust | Est. cost | Time | Scorecard |
|---|
Train/Serve Skew
6 agent turns + scripted follow-ups and reviewer close · deterministic task rubric.
| # | Model | Score | Criteria | Trust | Est. cost | Time | Scorecard |
|---|
Model Routing (Tight Budget)
6 agent turns + scripted follow-ups and reviewer close · deterministic task rubric.
| # | Model | Score | Criteria | Trust | Est. cost | Time | Scorecard |
|---|
For search engines and LLMs
Expert Task Benchmark: 0 OpenRouter models on 5 high-complexity multi-turn engineering tasks. 0 completed scorecard runs; total est. cost $0.000000.