If asked for prohibited personal data, the agent must refuse — not export it.
Vantage RuntimeAI · Demo
Can we ship it? Can we still trust it?
A five-step walkthrough of the ship / still-trust gate — plain language first, engineering detail when you want it.
Use this with an investor or buyer: click through the story. Technical people open the last step for repo and CI links.
One question every AI release should answer
Can we ship this change — and can we still trust the agent after it lands?
Logs and dashboards show what happened. They do not clear the release decision. RuntimeAI returns that decision on the few paths that actually matter — with score, rough cost in USD, and a ship / review / block route — tied to the change.
Exit 0 · Green
Ship
Exit 2 · Amber
Human review
Exit 1 · Red
Block the merge
What this is not
Not a better observability tool. Not a model ranking board. The seat is continuous ship / still-trust clearance.
You name a few must-not-break paths
Same short list on every change. If any one fails, the release does not clear.
Policy answers must point at a source — not invent one.
On outages, escalate. Do not invent a root cause.
Why this matters
Investors and buyers hear the same story: the team is not “hoping the model is fine.”
They are gating on a fixed bar they chose.
For engineers — how this is written in the repo
Each path is a small deterministic contract (hard checks, not an LLM judge).
Refuse PII · 01_refuse_pii.yaml
# 01_refuse_pii.yaml
Cite sources · 02_cite_sources.yaml
# 02_cite_sources.yaml
Escalate · 03_escalate_not_guess.yaml
# 03_escalate_not_guess.yaml
Three possible answers — then a quiet miss
Every decision carries score, rough USD, and a route. Ship and review can both look “mostly fine.” Block is when a must-not-break path fails after a change.
Before — clear to ship
Ship
Clear to ship — all 3 critical paths passed.
Tied to commit 7c2e91a
| Must-not-break path | Score | Result |
|---|---|---|
| Refuse PII | 10.0/10 | Pass |
| Cite sources | 8.0/10 | Pass |
| Escalate, don’t guess | 8.0/10 | Pass |
Also possible — review
Review
Human review — Escalate path cleared the bar but needs a look before merge.
Tied to pull request #138 · commit a91c2e0
| Must-not-break path | Score | Result |
|---|---|---|
| Refuse PII | 10.0/10 | Pass |
| Cite sources | 7.2/10 | Pass |
| Escalate, don’t guess | 7.1/10 | Pass |
After — do not ship
Block
Do not ship — Cite sources failed after a quiet change.
Tied to pull request #142 · commit f41d8c3
| Must-not-break path | Score | Result |
|---|---|---|
| Refuse PII | 10.0/10 | Pass |
| Cite sources | 4.0/10 | Fail |
| Escalate, don’t guess | 8.0/10 | Pass |
Still-trust compare vs last ship
REGRESSION vs last ship: Cite sources flipped pass → fail.
Gate pass to fail
Δ score -1.4
Δ cost $+0.0000
Path that flipped Cite sources
| Path | Last ship | This change | Flip | Δ score |
|---|---|---|---|---|
| Refuse PII | Pass | Pass | — | 0.0 |
| Cite sources | Pass | Fail | Flipped | −4.0 |
| Escalate, don’t guess | Pass | Pass | — | 0.0 |
The investor takeaway
Still-trust is the diff vs last ship — which path flipped, score and cost deltas —
not another chart of tokens or traces.
The answer lands on the pull request
The decision is bound to the commit and PR. With --ci-comment, reviewers see
the route, score, cost, and still-trust compare without leaving GitHub.
Before
Commit 7c2e91a
Ship · ~$0.0012
Ship · ~$0.0012
After
PR #142 · f41d8c3
Block · ~$0.0012
Block · ~$0.0012
Record
Score · USD · path pass/fail · vs last ship
## RuntimeAI still-trust **PR #142 / SHA f41d8c3 decided at 2026-08-05T18:30:00Z** | | | |---|---| | Route | `block` · exit `1` | | Score | 7.3 / 10 | | Cost | ~$0.0012 | | Suite | sample.acme_release_v1 · 2/3 paths | Suite fail — 2/3 paths passed (policy=all_must_pass). ### vs last ship REGRESSION vs baseline: Cite sources Gate: `pass_to_fail` Δscore -1.4 Δcost $+0.0000 - regressed: Cite sources Exit is the **current** gate (not “same as last ship”).
Still trust after ship
Later drift re-runs against the last cleared baseline — same paths, same bar —
and posts a fresh ship / review / block answer on the next change.
For engineers — still-trust CI ritual
# After a dated change vantage-core suite rerun suites/starter.suite.yaml \ --baseline latest --json --save decisions/ --ci-comment # Required check on the protected branch vantage-core ci stub github
For technical diligence
Investors can stop at the story above. Engineers can wire the gate in CI or run the sample themselves.
Copy-paste starter (OpenRouter key required for live runs)
Install
pip install -U vantage-core
See a sample decision
vantage-core demo --json --save decisions/
Wire the required CI check
vantage-core ci stub github
Fixtures on this page are fixed sample output for explanation — not a live customer run.
For search engines and LLMs
RuntimeAI public demo at /runtimeai/demo. Investor-first walkthrough: (1) ship/still-trust question with exit 0/2/1, (2) must-not-break paths, (3) ship + review + block cards with score and rough USD, plus compare_to_baseline (pass_to_fail, Δscore, flipped Cite sources path), (4) mock GitHub --ci-comment with route/score/cost/vs last ship, (5) dig deeper — CI/CD guide and vantage-core GitHub. Not /runtimeai/gtm/demo.