Per-identity LLM spend cap and paid-surface gating¶
Verdict: IN-MVP-1 · Estimate: 2.5 half-days (Option C) · Status: DELIVERED — founder set the ceiling (DAILY_LLM_SPEND_CEILING_USD=5.00); shipped in PRs #247/#255 (spend ceiling, re-landed) and #253 (surface gating), released in v1.1.0
Serves gate: MVP-1 §1 — "Genfit and Polygon employees follow an authenticated link…". A named identity on an internet-reachable deployment with unmetered paid endpoints is the billing liability §7 Q2 attaches to the auth decision.
Scope contract: §9 — "Never cut, at any date: … the per-identity spend cap"; §2 line 116 — "Do not compensate by cutting authentication, provenance, stable feedback anchoring, spend control, or the production smoke gate."
Date: 2026-08-12
Why this is in scope¶
From product-manager, verbatim:
The framing offered — cap versus no cap — is not a live choice for this gate. §9 names the per-identity spend cap never-cut "at any date", and §2's checkpoint rule names spend control among the things a failed checkpoint may not be paid for with. Only the founder can remove it, under §6.1, and that requires cutting something else. What is open is the mechanism and its size, and that is out of my remit: the contract says "cap", not "
LlmSpendmodel, reserve/settle accounting and a model-pricing table".On the second branch: unmounting
/api/askin the client deployment is not an alternative to the cap — it is separately required. §6 lists conversational Ask as out of MVP-1 and §5 states the rule: a non-real surface "must be gated (unreachable from a client session) or deleted".frontend/src/mocks/chat-dupilumab.ts:245→AskPanel.tsx:163is on the §5 table, and §5 says wiring it to a real endpoint "is not an option". So gate the Ask surface and keep the cap.Reversal condition: a founder §6.1 amendment removing "the per-identity spend cap" from the §9 never-cut list, with the compensating cut recorded in §10. Nothing short of that.
What exists / what's new¶
The finding that reframes the question¶
PR #247's cap covers 2 of 7 client-reachable LLM-spending endpoints.
| Route | Handler | Model tier | Metered by #247? | UI caller |
|---|---|---|---|---|
POST /api/ask |
ogur/api/routes/query.py:10 |
Haiku | yes | AskPanel.tsx |
POST /api/explore/decompose |
ogur/api/routes/explore.py:85 |
Sonnet | yes | ResearchWizard.tsx:77,99 |
POST /api/briefing/{id}/drug/{drug}/generate |
ogur/api/routes/briefings.py:51 |
Sonnet | no | AssetDetail.tsx:381 via useAssetBriefing.ts:31 |
POST /api/briefing/{landscape_id} |
ogur/api/routes/briefings.py:79 |
Sonnet pipeline | no | none — useTriggerBriefing.ts:9 is imported by nothing; curl-reachable only |
POST …/overview/generate |
ogur/api/routes/briefings.py:196 |
Haiku/Sonnet | no | AssetDetail.tsx:371 |
POST …/trials/generate |
ogur/api/routes/briefings.py:235 |
Haiku/Sonnet | no | AssetDetail.tsx:1201 |
POST …/competitive/generate |
ogur/api/routes/briefings.py:277 |
Haiku/Sonnet | no | LandscapeTab.tsx:7 |
All five unmetered routes take no auth dependency and are reachable from /asset/:drugName (frontend/src/App.tsx:160). The most expensive path in the repo — full Sonnet synthesis at $0.05–0.20/run — is among them.
PR #247's split — interleaved, not separable¶
+1587 / −73 across 29 files: ≈ +679/−64 authorization (43%), ≈ +908/−9 spend (57%).
No commit is purely one half. 5d99fe58 ships UserLandscapeAccess and LlmSpend/store/spend.py together; 49715549 puts the grant tightening and reserve_llm_spend in the same ogur/api/deps.py diff and the same routes/query.py handler body; 671c4662 ships test_authorization.py and test_spend_ceiling.py and the conftest.py fixtures together. "Merge auth, drop spend" is a manual rewrite, not a cherry-pick.
Semantic coupling too: the ogur/api/schemas.py 4,000-char caps and the ogur/engine/query.py:98 60k context cap exist only to make the reservation a true bound.
Minimum cap vs what #247 ships¶
CostMeter is real — ogur/engine/discovery_modality/types.py:257, with add_llm (:278), usd_estimate (:324), _PRICE_PER_MTOK (:249) — but used only by the offline harnesses (scripts/eval/sirna_autonomous_discovery.py:691), never by the API. decompose_core already takes cost: (engine/discovery_modality/decompose.py:123); QueryEngine.ask does not.
A minimum cap (check-then-record, no reserve/settle, no worst-case bounds, no unpriced-model guards) is ~105 lines against #247's ~908. It drops protection against one concurrent request's actual spend (~$0.02 Haiku against a $5.00 ceiling, two identities, single-process uvicorn). Rebuilding it costs 2.5 half-days. Merging what exists costs 0.5. Reuse is strictly cheaper than right-sizing.
Option costs¶
| Option | Half-days | Satisfies §9? |
|---|---|---|
A — retarget #247 to main, merge |
0.5 | No — cap holds on 2 of 7 paid paths |
| B — split, merge auth, drop spend | 2.5, then 2.5 more | No, and costs more than A |
| C — A + gate all paid surfaces (backend flag + 5 briefings POSTs + frontend routes + tests) | 2.5 | Yes |
Retargeting is one command: gh pr edit 247 --base main. Verified — git merge-base origin/main b58e6925 = 33cd5642 (the ogur-69 head, already in main), and git diff --numstat origin/main...b58e6925 is byte-identical to the PR file list. No rebase, no conflict. mergeable: MERGEABLE, mergeStateStatus: CLEAN, 4/4 checks green.
Architectural seam & collisions¶
Seam: #247 respects the existing one. require_report_access (ogur/api/deps.py:28) was written by OGUR-69 explicitly as the seam OGUR-65 tightens, and #247 tightens it in place. The spend half adds one new seam — ogur/store/spend.py as admission control between deps.py and the engine, with a TYPE_CHECKING-only CostMeter import (store/spend.py:13) to avoid a store→engine runtime dependency. Correct inversion, small enough not to need an ADR.
Known trap: UserLandscapeAccess.user_id and LlmSpend.user_id are not FK-enforced (no PRAGMA foreign_keys=ON); deactivate_user deliberately does not cascade grants.
Collisions: none on the backend files #247 touches. Of 29 open PRs, only #230 overlaps (ogur/api/schemas.py, pack response models — disjoint from #247's request-constraint edits). Of 60 worktrees, the two that could collide (ogur-67-container, ogur-68) are both clean and already merged.
Sequencing constraint (not extra work): #247 makes /api/signals, /api/explore/classes, /api/explore/landscapes and report/figures 401 without a session cookie, and no login page exists — no Login* view, and rg "auth/login" frontend/src returns nothing outside tests. #251 (OGUR-71) is the identity surface. Merging #247 without a login path makes the deployed report unreachable in a browser.
Post-merge check (the #242 stranding failure): git branch -r --contains <merge-commit> | grep -q origin/main. Run it against the merge commit, not b58e6925.
Surface contract¶
There is no client-facing surface for the cap, and there must not be one for MVP-1. Once every paid path is gated, no client session can produce a 429, so a 429 renderer would render an unreachable state. The cap is a backend test assertion, not a screen.
Gating is removal, not a new component. No new component is introduced. <EmptyState> fits the shape but has no reachable trigger; <Badge> [DEMO]/SIMULATED (ux-spec §6.6) labels simulated data, not withheld capability.
| Component | Source | Change |
|---|---|---|
<AskDrawer> |
frontend/src/App.tsx:92 |
unmount |
| Ask pill + ⌘J + hint | App.tsx:98–106, :68–72, :40–53,108–139 |
remove |
<GlobalAsk>/<AskPanel> |
App.tsx:158 route |
drop route; keep files (gated, not deleted) |
<CommandPalette> |
CommandPalette.tsx:92 |
remove the navigate('/ask') item |
<ConversationHistory> |
App.tsx:159 |
drop route (it breadcrumbs into /ask) |
<LeftRail> |
LeftRail.tsx:29–34 |
drop /history nav item |
<ResearchWizard> |
App.tsx:152, WorkspacesTab.tsx:59 |
drop route + entry button |
<AssetDetail> generate buttons |
AssetDetail.tsx:371,381,1201; LandscapeTab.tsx:7 |
drop route or remove call sites |
<Navigate> |
already imported App.tsx:2 |
add <Route path="*" element={<Navigate to="/" replace/>}/> |
Data contract: none. No new endpoint, no capability flag. ogur/models/user.py:12 — "There is no role field: every active user has the same capabilities" — so a conditional route gate evaluates identically for every identity. That is a build-time removal wearing a runtime disguise. Remove the routes.
States¶
| State | What renders |
|---|---|
| Gated route entered by URL | Redirect to /. Today App.tsx:147–162 has no catch-all, and the SPA fallback (api/app.py:35–49) serves index.html with 200 — so a client typing /ask gets a blank white screen. The path="*" redirect is the one line without which gating ships a blank page. |
| Gated entry point | Absent — not disabled. A disabled "Ask AI" button advertises a capability we are withholding and invites the question in the meeting. |
| Loading / populated / error on the report | Unchanged — /explore/landscape/:landscapeId reads frozen pack JSON via useExploreReport, no LLM. |
| Spend ceiling hit | No client-rendered state by construction. |
Provenance¶
Gating improves both principles. AskPanel.tsx:374–390 renders <CitationChip> hydrated from mocks/chat-dupilumab.ts — fabricated source chips, exactly what §5 names. AskPanel.tsx:354–363 renders a hardcoded CONFIDENCE_BADGE[msg.confidence]; a fabricated <ConfidenceBadge> is worse than a missing one, because it asserts calibration never computed. Removing the reach removes both.
Standing constraint for any future 429 renderer: render as chrome (banner / <EmptyState>), never as prose in an answer slot. AskPanel.tsx:183–190 currently renders `Error: ${err.message}` — the raw 429 {"detail":…} JSON — into the assistant prose slot, which is orphan text under principle 1.
Acceptance criteria¶
Backend: uv run --extra dev python -m pytest. Frontend: cd frontend && npm test && npm run build.
| # | Criterion | Check | Layer |
|---|---|---|---|
| 1 | Identity at/over ceiling gets 429 on POST /api/ask and the Anthropic client is never called |
tests/unit/api/test_llm_spend.py::test_over_ceiling_returns_429_without_calling_the_model — assert mock.messages.create.call_count == 0 |
unit |
| 2 | Ceiling is per identity — A over, B under → B gets 200 | ::test_ceiling_is_per_identity |
unit |
| 3 | Rollup is per UTC day — 23:59:59Z yesterday excluded, 00:00:00Z today counted |
::test_daily_rollup_boundary_is_utc, frozen clock |
unit |
| 4 | Spend for a real call is > 0 for both configured models; unknown model raises rather than recording 0.0 |
::test_priced_models_record_nonzero (parametrised over settings.llm_synthesis_model, llm_classifier_model), ::test_unknown_model_is_an_error_not_zero |
unit |
| 5 | Under-ceiling call succeeds and increments the rollup | same test as #1 | unit |
| 6 | Every mutating route from create_app() is classified metered/gated/non-LLM against a frozen literal set |
tests/unit/api/test_paid_endpoint_gate.py::test_every_mutating_route_is_classified |
unit |
| 7 | At the default flag, all 7 LLM POST paths return exactly 404, content-type: application/json, anonymous and granted |
::test_paid_paths_are_404_when_disabled (7 paths × 2 identities) |
unit |
| 8 | enable_paid_llm_endpoints defaults False, asserted on the settings object |
::test_flag_defaults_off |
unit |
| 9 | OpenAPI omits the gated paths when off | ::test_openapi_omits_gated_paths |
unit |
| 10 | /ask, /history, /explore/research/new redirect to /, not a blank screen |
frontend/src/__tests__/App.routes.test.tsx::redirects removed routes to the mode selector |
frontend |
| 11 | App.tsx route set equals a frozen roster, each entry tagged real/gated/deleted |
::route roster matches the classified set |
frontend |
| 12 | No element named /Ask AI/i on any route; ⌘J opens nothing — mount once and navigate, direct-mount cannot catch a sticky useAIPanel leak |
frontend/src/__tests__/AppShell.gated.test.tsx |
frontend |
| 13 | AskPanel, AskDrawer, useAsk, useTriggerAssetBriefing, useTriggerOverview, useTriggerTrials, useTriggerCompetitive, useTriggerBriefing imported by no non-test source |
::no paid-surface imports remain, import.meta.glob on module path |
frontend |
| 14 | The test asserting the Ask pill exists is inverted, not deleted | frontend/src/__tests__/AppShell.askAI.test.tsx:64–83 replaced in a file that still exists |
frontend |
| 15 | npm run build green (tsc program includes test files) |
cd frontend && npm run build |
frontend |
| 16 | On the deployed URL, all 7 LLM POST paths return exactly 404 / application/json, granted and anonymous |
new block in scripts/smoke/smoke_production.sh, modelled on its criterion 8b (:158–163) |
e2e |
| 17 | Deployed GET /ask serves index.html and the app redirects |
smoke_production.sh + criterion 10 |
e2e |
| 18 | Container smoke passes with the flag unset | bash scripts/smoke/smoke_container.sh |
e2e |
| 19 | The merged code is on main |
git branch -r --contains <merge-commit> \| grep -q origin/main |
git |
| 20 | A browser path from / to a report exists — some non-test file posts to /api/auth/login |
rg -n "auth/login" frontend/src --glob '!**/__tests__/**' — fails today; the release-sequencing blocker |
frontend |
Regression risk¶
POST /api/explore/landscapes/{id}/feedback(routes/feedback.py:40) is non-LLM and must not be caught by the gate — an over-broadinclude_routerskip silently kills the §2 feedback gate item. Criterion 6 must classify itnon-LLM.POST /api/explore/landscapes/match(explore.py:325) is non-LLM but sharesexplore.routerwithdecompose— the gate must be per-route (404 dependency), not router-granularity.- SPA fallback (
app.py:35–50, GET/HEAD only,/api/*stays JSON) must stay green; criterion 7 depends on it. - Backend baseline ~1,932 → report the new total. A drop means tests were deleted rather than inverted. Never run
make fmton this branch.
False positives¶
| Looks like success | Actually is | Discriminating check |
|---|---|---|
| Gated path returns 404 | The SPA fallback answered, or the path was renamed | Assert content-type: application/json and a JSON detail |
| Gated path returns 401 | The route is still registered — a live paid surface for every real client | Require exactly 404; 401 = authenticated-not-gated |
| Gated path returns 405 | Path exists with other methods | Require 404 on the exact (method, path) pair |
| Ceiling test passes | No pricing entry for the configured model → every call records $0.00 and the ceiling is never reached |
Criterion 4 |
| 429 test passes | Request was rejected pre-dispatch by validation (query.py:12 422) or a broken mock — no spend path ran |
Criterion 5: under-ceiling call must return 200 and write a row |
/ask removed, Vitest green |
AskDrawer still mounted at App.tsx:92, ⌘J still opens it on the report route, fabricated <CitationChip> renders over the real report |
Criterion 12's keydown assertion on a surviving route |
| Vitest green after removal | AppShell.askAI.test.tsx was deleted, so nothing asserts absence |
Criterion 14 |
| Per-endpoint gate tests pass | A 7th or re-registered route bypasses the cap — exactly how briefings.py:51 was missed |
Criterion 6: set equality against a frozen literal, not spot checks |
| Unit tests prove the cap | Production runs with ENABLE_PAID_LLM_ENDPOINTS=true via fly secrets; the code default is irrelevant |
Criterion 16: curl the deployment |
| Briefings POSTs metered instead of gated, cap test green | They return 202 and spend in a BackgroundTasks job after the response; N requests fire before any spend is recorded, and the background task has no request identity |
If any paid endpoint stays enabled: reserve pre-dispatch keyed to the caller + a concurrency test. Gating removes the class |
git branch --contains b58e692 finds the branch |
It found the PR branch, not main — the #242 incident |
Criterion 19, on the merge commit |
| §5 "all surfaces gated" declared done | Only LLM surfaces were gated. /franchise (hand-written LOE constants), /deals, /foundations, CompetitiveTimeline, Inspector mockSummarize() are non-LLM fabricated surfaces still reachable — OGUR-62's rows, not closed here |
Criterion 11 must classify every route |
| Anonymous refused, gate green | Nobody can log in either | Criterion 20 |
Human judgement required¶
| Question | Who | Against what |
|---|---|---|
| ⚠ UX-SPEC CONFLICT. ux-spec §4 (131,143), §5.1 (165,232–244), §6.2 (291), §8 (384,393), §9 (410) describe Ask as a shipped primary surface; mvp-scope §6 lists conversational Ask as OUT. Amend ux-spec, or record a documented deviation? | Founder | ux-spec.md vs mvp-scope.md §5/§6. Blocks freezing criterion 11's roster |
| Gate (flag) or delete the 5 briefings POSTs and the Ask surface? Both pass §5; deletion is the stated end state | Founder + tech-lead | mvp-scope §5 |
Is enable_paid_llm_endpoints=true ever permitted on a client-facing deployment? If yes, the background-task metering hole becomes a required build |
Founder | §9 never-cut |
| What USD/day is the ceiling? Once all paid paths are gated it is unexercised in production — a defence for MVP-2, not for Aug 13 | Founder | Expected client usage |
| Merge #247 before a login page exists (criterion 20 failing), or land #251 first? | Founder | §2 item 1 + §2.3 |
Delete POST /api/briefing/{landscape_id} (:79) outright rather than flag-gate it — nothing imports useTriggerBriefing |
tech-lead | — |
Open questions¶
The two that block: the ux-spec conflict (founder), and gate-vs-delete (founder + tech-lead). Everything else in the table above can be decided during implementation.
Status is BLOCKED on those two, not on engineering. The implementation order is criteria 6, 7, 10, 12 first — those four are what make the remaining sixteen meaningful.