Pricing

Free proves the method. Paid runs the decision.

On the hosted trial you evaluate on the 5 lowest-cost OpenRouter tiers — enough to confirm the rubric is sound and your PM can read the scorecard. We subsidize inference on those budget models only, because we can’t give away frontier tokens for free. When you need to compare the models you actually ship — Claude, GPT-4-class, and the rest of the catalog — that’s a paid plan.

Prefer to keep everything local? vantage-core runs any model you want, billed to your provider keys — always free, always open-source.

These are different bills for different moments. Preflight scopes proof for one decision before you ship; a platform subscription powers ongoing eval volume in the API, Console, and CI.

One eval run = one scenario × one model × one replication (one full simulated check-ride). Agent turns per eval run sets how many back-and-forth conversation steps the agent may take in that run — one turn is one agent reply (and usually a stakeholder response); the run ends at closure or when the limit is hit. Compare 2 models on Support Escalation at 12 turns = 2 eval runs. Batch 10 models on the same scenario = 10 eval runs.

Your bill has three line items — always shown separately on quotes and invoices:

1. Platform subscription — scenarios, rubrics, scorecards, compare, batch UI, run history, API access.
2. Eval runs — included in your plan; overage billed per run above the monthly cap.
3. Inference pass-through — token cost at public OpenRouter rates for the models you select. Shown on every API quote before you confirm.

We do not mark up inference during early access. You see the provider rate upfront.

How many eval runs you spend depends on the use-case family and its natural cadence — a conversational agent tested once a quarter with weekly drift monitoring consumes very differently from an analytical suite gating every pull request. The Cost Forecast ships editable defaults for each family so you can size a realistic monthly bill.

Model token prices differ by orders of magnitude. A 12-turn Support Escalation check-ride on a budget trial model might cost fractions of a cent in inference; the same run on Claude Opus or GPT-5 Pro can cost dollars. That is why the trial is capped to the 5 lowest-cost tiers — and why paid plans quote inference before you batch.

You upgrade the moment the budget-model trial stops answering your real question:

1. You need a model outside the 5 cheapest tiers. A free rai_live_… key quotes and submits on budget models only. Selecting Claude Sonnet, a GPT-4-class model, or anything premium — via the API or signed-in batch runs — is where a paid plan unlocks the full catalog. Locally, runtimeai-ide and vantage-core already run any model on your BYOK keys for free.
2. You batch your real shortlist in the cloud. “Run 10 models × 15 turns” on production-grade candidates through POST /evaluations/submit — not just budget tiers — is a paid batch sweep with quote-before-run pricing.
3. You wire evals into CI or release automation. Paid API keys add higher concurrency for pipeline gates; vantage-core in GitHub Actions stays free and BYOK if you self-host inference.

Everything free is free on purpose: public benchmarks (credibility), editor + vantage-core (open-core adoption), and budget-tier API trials (proof). Browse the scenario library to pick a check-ride — none of it gives away frontier inference on our hosted keys.

Every surface shares the same engine. You are paying for (or self-hosting) the evaluation infrastructure — not just chat completions.

• Agentic use cases — Technical tasks (SQL, pipelines, ML readouts) and Conversational workflows (support, sales, billing, triage), plus custom scenarios from a brief.
• Custom scenario authoring — Draft a new check-ride from your brief, schema, or policy in about a minute; same deterministic rubrics as the public library.
• Adversarial simulation — Stateful check-rides with agent turns per eval run (1–24) and stakeholder roles that stress guardrails over time.
• Deterministic rubrics — Five auditable dimensions per scenario, scorecards (JSON/PDF), performance bands — no LLM-as-judge.
• Side-by-side model compare — Parallel runs on the same scenario with linked comparison IDs.
• Public benchmarks — Leaderboards, efficiency rankings, and guardrail erosion reports on published scenarios.
• Run visibility — Transcripts, batch sweeps, aggregated charts, and per-run review in Runs when you sign in.

If you want RuntimeAI evaluations on any model, billed directly to your provider account, you have two paths today:

1. vantage-core (recommended for CI) — Install the open-source SDK locally or in GitHub Actions. Set OPENROUTER_API_KEY (or Anthropic / OpenAI / Gemini equivalents) in your environment. No tier restriction, no RuntimeAI cloud bill; you own inference spend and data residency.
2. Private RuntimeAI deploy (enterprise) — Dedicated RuntimeAI Cloud instance with your own OPENROUTER_API_KEY, optional ANTHROPIC_API_KEY / OPENAI_API_KEY, and Console, Runs, and HTTP API on your infrastructure. Your team uses the full UI; inference hits your keys.

On RuntimeAI Cloud at vantageai.cc, hosted API inference runs on Vantage-configured provider keys — you do not paste an OpenRouter key into the browser, and the free trial is capped to the 5 lowest-cost tiers because we subsidize that inference. Quotes show estimated token cost at public OpenRouter rates so you know the inference equivalent before confirming a batch. Need the full catalog on our infra with your keys? Contact us.

• Quote first — POST /evaluations/quote returns per-model readiness and summary.est_cost_usd_total before you submit (15-minute TTL).
• Multi-turn multiplier — Cost scales with agent turns per eval run and parallel model count, not a single chat completion.
• Live rates — Per-model estimated eval cost on Models and in Runs → Model Costs / Batch Run (sourced from OpenRouter public pricing).

Estimates use OpenRouter public pricing, agent turns per eval run, and — where available — aggregated results from extensive benchmark runs on this platform. Provider rates can change without notice; actual inference cost depends on your execution (turn depth, prompt size, retries, and model choice).

Catalog estimates assume planning token averages (1,500 input / 350 output tokens per LLM call), agent turns per eval run, and a scenario. Where we have benchmark runs on a scenario, forecast and scorecard views prefer the measured mean $/run from those replications.

Full model catalog, higher concurrency, private scenario libraries, SSO-backed Runs, or BYOK on dedicated infrastructure — contact us for team pricing. Volume is based on evaluation runs per month, separate from underlying provider bills.

hello@vantageai.cc · How To · Cost Forecast · FAQ