Provider health · last 72h — the shape, not just the current status
One cell per hour, oldest left. Grey = no traffic (not "healthy" — an idle hour and a clean hour are different facts). Window is capped at 72h because that is how long the hourly buckets live; anything longer would be invented history.
Estimates, not invoices — incremented with the failover engine's per-request estimate, so a threshold crossing is meaningful even if the absolute $ is ±30%.
Loading…
Grant plan · testing / comps
Manually set a user's plan (no payment). Use free to revoke.
Users
Email
Plan
Auto-renew
Joined
Set plan
Loading…
Page size
User detail
Loading…
Model usage & cost
Real input/output tokens per model, with an approximate cost (provider list prices; cached input billed ~20%). Bars show input (solid) vs output (light) volume.
Loading…
Edit success (per model)
Share of agent edit-tool calls that applied cleanly — detected server-side from edit results (the agent-quality north star). Compare the cheaper tool model vs the frontier one: if success % is comparable, keep the cheaper routing; if it drops, revert tool_call_model_cheap in Settings → Model routing.
Loading…
Skill-pack usage (the moat thesis-test)
Which domain skill packs the agent actually loads (counted server-side from the injected skill block — no Langfuse). The thesis test: if the packs are rarely invoked, the domain-depth bet isn't landing yet.
Loading…
Live in ~30 s — changes apply within the config cache TTL after saving.
Model routing
Plan daily limits — requests per day per plan
Monthly credits per plan — the depth-weighted credit pool per plan (react to a margin surprise without a redeploy)
Credit cost per tier — credits charged per request at each routing tier (all four required)
Defaults & rate limits
Spend guard — fleet-wide outage-amplification guard, not per-user billing
Estimated USD. These exist so an outage that re-routes the fleet onto an expensive fallback can be capped without a deploy — that is why they are editable here. The ceiling drops degraded fallbacks for free/trial only; paying tiers are never spend-blocked and no request is ever refused.
Downloads — fix the /download page without a redeploy
Auto-renew (Razorpay Subscriptions)
One-time setup: creates a monthly Razorpay plan per paid tier so auto-renew checkout works. Idempotent — safe to click again.
Live promo codes — add, edit, or remove instantly (no redeploy). percent = % off checkout; free_days = grants a plan free for N days with no payment. Removing a code stops it immediately; redemption counts are preserved.
Active coupons
Code
Type
Value
Plan
Expires
Used / Max
Loading…
Add / update a coupon
Measured model quality — the benchmark engine scores each tier/model on the coding-eval and ranks by pass@1. This is the source of truth for routing — measured, not vendor claims. Runs are stored versioned (never overwritten).
Coverage — do the models actually serving traffic have a score in this run?
Loading…
Routing recommendations
Advisory only. Nothing switches automatically — the engine ranks by pass@1, breaks ties on $/solve, and you flip the slot yourself in Settings → Model routing. Benchmark scores are compared against live production signals (TTFT p95, edit-success) because the offline eval is blind to latency.
Loading…
Leaderboard
#
Target
pass@1
Passed
Wall s
Loading…
Capability matrix — pass@1 per difficulty (measured)
Target
Overall
Easy
Medium
Hard
—
Run history — newest first, winner per run (regression tripwire)
When
Label
Winner
pass@1
Targets
—
Context Engine — is semantic_search actually being used? The IDE tool has no logging and falls back to grep silently, so a broken engine and an unused one look identical from the outside. This is the only thing that tells them apart.
Loading…
Live Razorpay mandates — we store ONE subscription id per user and overwrite it with whichever mandate charged last, so a second mandate made the first invisible: on no screen, cancellable through nothing, still charging. New ones are refused now; the ones already out there can only be found by asking Razorpay. Does not run on its own — it calls the Razorpay API.
Did anything break? — user-facing failures, so a silent breakage shows up here instead of waiting for a customer to report it. ide = reported by the app (the class the server can't see); server = what the backend knows (empty_response_healed = a user almost saw an error; _unhealed = they did).
Last 7 days
— a spike means investigate now
TOTAL ERRORS
—
FROM APP (IDE)
—
FROM BACKEND
—
By kind — what is actually failing
Kind
Count
Loading…
By model — which model produced the failure. A heal is a SECOND full provider call we pay for, so a model high on this list is costing twice per affected turn.
Model
Kind
Count
—
By release — which build regressed
Version
Errors
—
Ops error log — provider degraded/failed, reliability, and billing alerts land here instead of email. Each issue is deduped to once per hour, newest first. Email is off by default — flip ops_email_alerts in Settings to also email during a critical window.
Recent alerts
Loading…
What actually serves a request. These are the live values the router reads, not
the defaults baked into the deploy. An A/B rule sends a slice of one tier to a challenger model —
it was the only routing control with no page at all, so stopping one meant a database or a script.
A/B challenger
Loading…
Sets model_ab to {}. Takes effect on the next request.
Tier → model (live)
Edit these on the Settings tab. They are read on every request, so a change is live without a deploy.
Budgets
Voice is billed per audio second and vision per image, so each has its own daily ceiling per plan.
Did users LIKE it? — every other board measures machinery (edit-success = the edit applied; the eval = we pass tests). This is the only one that says the human was happy. A 👎 is also a real failure case worth turning into an eval task.
Last 30 days
SATISFACTION
—
👍 UP
—
👎 DOWN
—
BY TIER
—
What users said went wrong
— their own words; the most actionable list here (and real eval tasks)