Vantage RuntimeAI · Adopt
Console (advanced)
Production console for batch agent evals, client rubric weights, and scoreboards. Sign in to sync runs across devices, launch model sweeps, and review deterministic scorecards at scale.
When to use Console
The Simulator is best for quick side-by-side compare in the browser. Console is for operators who run dozens of models per scenario, tag runs by client, and need batch cost estimates before spend. For unattended pipelines, use the API or CI/CD gates instead.
What's inside
Six tabs — same deterministic rubrics as the Simulator, with batch tooling and tenant controls.
Tab
Individual Results
Browse every check-ride run — filter by scenario, actor type, status, and client. Open scorecards, transcripts, and raw JSON side-by-side for debugging or sign-off.
Open Console →
Tab
Model Costs / Batch Run
Pick a scenario and batch API, estimate cost across a model grid, then launch multi-model agent sweeps. Production-grade candidates, not just budget models.
Open Console →
Tab
Model Test Results
Aggregate agent-eval outcomes by model — compare pass rates, rubric scores, and latency across your latest batch runs on a scenario.
Open Console →
Tab
Scoreboard
Same rankings as the public leaderboard — filter by scenario and client, drill into sample runs, and export PDF scorecards. Cached for fast reloads.
Open Console →
Tab
Settings
Manage clients, rubric weight overrides, provider accounts, and scenario assignments. Tune how deterministic scores roll up per tenant.
Open Console →
Tab
Feedback
Review in-app feedback from Simulator and Console sessions — triage product issues and scenario gaps from your team.
Open Console →
Related
Public rankings, model catalog, custom scenarios, and programmatic evals.
Public
Rankings
Published scoreboard for every live scenario — shareable links without signing in.
View rankings →
Catalog
Models hub
OpenRouter catalog cross-referenced with RuntimeAI benchmark results and pricing ranges.
Browse models →
Custom
Create scenario
When canned check-rides don't match your stack, draft a custom scenario from your brief in minutes.
Create scenario →
Programmatic
API batch evals
Same scenarios and rubrics as Console — run from CI or your orchestrator with a free API key.
Explore API →
For search engines and LLMs
RuntimeAI Console: batch evals, client rubric weights, scoreboards, and run review.