Deployment¶

How the OGUR delivery container runs in production, and every runbook needed to operate it: first deploy, secrets, user provisioning, backups, restore, and the smoke gate. The deployment shape is ratified in mvp-scope.md §8: Fly.io, EU region (cdg, Paris), single machine kept warm, ~$5–10/mo.

Issue: OGUR-68 (parent OGUR-59, MVP-1 release gate). Container: OGUR-67.

Architecture¶

One image (Dockerfile), three facts that drive everything else:

Everything immutable is baked read-only into the image: the built React frontend (/app/static), the API wheel (/opt/venv), both frozen report packs (/app/packs/*.json, mode 444), and the offline-seeded content database (/app/content/content.db, mode 444). Redeploying can never corrupt served content — the smoke gate's pack-digest check (criterion 10) proves the deployed bytes equal the repo bytes.
Everything mutable lives on one Fly volume mounted at /data. The image sets DATABASE_URL=sqlite:////data/ogur.db. At first boot docker/entrypoint.sh materializes /data/ogur.db from the baked content DB — atomically, and never overwriting an existing file. Identities (OGUR-64) and report feedback land here; this volume is the only unregenerable data in the system.
SQLite runs in WAL mode (ogur/store/database.py::_configure_sqlite): journal_mode=WAL, busy_timeout=5000, synchronous=NORMAL on every connection. Readers proceed during writes, and the backup runbook's sqlite3.backup() copy is consistent while the server is live.

Why a single machine: SQLite on a volume permits exactly one writer machine. fly.toml pins the shape — auto_stop_machines = "off" + min_machines_running = 1 (warm, no cold starts for clients), and deploys use --ha=false so Fly never adds a spare machine the volume can't follow. The single volume is a deliberate single point of failure; the mitigation is the snapshot + offsite-backup pair below.

Prerequisites¶

flyctl: brew install flyctl, then fly auth login (a Fly account with billing on file — cost envelope ~$5/mo: shared-cpu-1x/512MB + 1GB volume).
Repo checkout at the commit you intend to deploy (the packs under archived_data/ are inputs to the digest check).
Docker is not required locally: make fly-deploy builds remotely on Fly's x86_64 builders. (Apple-silicon local builds produce arm64 images Fly machines cannot run — always deploy with the remote build.)

First deploy¶

fly apps create ogur                 # app name = URL: https://ogur.fly.dev
fly volumes create ogur_data -a ogur --region cdg --size 1 --snapshot-retention 14 --yes
make fly-deploy                      # fly deploy --remote-only --ha=false

fly volumes create warns that a single volume is a single point of failure — accepted by design (see Architecture above); pass --yes.

Then provision users and run the smoke gate (next two sections).

Redeploying¶

git checkout main && git pull
make fly-deploy
curl -s -o /dev/null -w "%{http_code}\n" https://ogur.fly.dev/health

Redeploying replaces the image and leaves the volume alone. That single sentence decides whether a given fix reaches production, and the answer differs by which layer the change lives in:

Change	Reaches production on redeploy?
Report text — `briefs[].body_md` in a seed pack	Yes. Reports are served from the pack baked at `/app/packs/*.json`
Frontend, API code, dependencies	Yes. All baked into the image
Evidence rows — anything served from the database	No. See below
An ops script newly added to `/app/scripts`	Yes, and nothing works until it is — see the ordering note
Identities, sessions, grants, feedback	Untouched, by design

Why a database change does not ship. The image bakes a freshly seeded /app/content/content.db, but docker/entrypoint.sh materializes /data/ogur.db from it only when that file does not already exist. On any deployment that has run once, the baked copy is ignored — which is exactly what keeps identities and feedback alive across deploys, and equally what stops a corrected database row from ever arriving. Changing served database content on a live deployment is a migration against the volume, not a redeploy.

So a fix to a report claim ships by editing the pack and redeploying; a fix to an evidence row does not, however green its tests are.

Ordering. Deploy before running any make fly-* target that invokes a script the same change adds to the image. make fly-reset-feedback is the worked example: it takes the offsite backup first, so against an image that predates the script it backed up, failed to open the script, and reported a completed reset that deleted nothing.

Verifying. /health proves the machine is up, not that the new content is live. For a content change, sign in and read the changed claim, or diff the served pack digest — the smoke gate's criterion 10 does this and is the mechanical form of the same check.

Configuration and secrets¶

ogur/config.py reads settings from real environment variables; .env is gitignored, excluded from the image, and never leaves your machine.

Setting	Where	Value in production
`SESSION_COOKIE_SECURE`	defaults `true` in `ogur/config.py`; `fly.toml [env]` restates it	`"true"` — Fly forces HTTPS
`API_DOCS_ENABLED`	defaults `false` in `ogur/config.py`	unset — `/docs`, `/redoc`, `/openapi.json` are 404
`DATABASE_URL`	baked in the image	`sqlite:////data/ogur.db`
`EXPLORE_REPORT_PACK_PATH`, `FRONTEND_DIST_PATH`	baked in the image	both packs / `/app/static`
`ANTHROPIC_API_KEY`	`fly secrets set` — deferred	absent until OGUR-65 deploys
`DAILY_LLM_SPEND_CEILING_USD`	`fly secrets set` — with OGUR-65	set alongside the key

Secrets are injected at runtime, never baked:

fly secrets set -a ogur ANTHROPIC_API_KEY=sk-ant-...   # triggers a rolling restart

Why the key is deferred: mvp-scope §8 forbids exposing LLM endpoints without the per-day spend ceiling, and the ceiling ships with OGUR-65. Until then the deployment serves everything pack- and DB-backed (reports, figures, signals, auth) with the key unset; /api/ask and briefing synthesis fail closed. See "After OGUR-65 merges" below.

User provisioning¶

The image carries scripts/seed/seed_users.py for exactly this; there is no signup endpoint by design.

fly ssh console -a ogur -C "python /app/scripts/seed/seed_users.py --provisional"

--provisional idempotently creates genfit.provisional@ogur.local and polygon.provisional@ogur.local and prints each generated password exactly once — capture them from the terminal and hand them over out-of-band; they are stored only as scrypt hashes. Rotation: rerun with --deactivate / targeted flags (see the script's --help).

Note: once OGUR-65 is deployed, users also need per-landscape grants (--grant EMAIL LANDSCAPE_ID) — an ungranted user's report 404s.

Provisioning a named client reviewer¶

Real client identities are not recorded in this repository — not their names, not their email addresses. They are typed at provisioning time and live only in the production database. Substitute the actual values for the placeholders below; do not commit them back here, and do not paste a filled-in version of these commands into a commit message, an issue, or a PR.

make fly-create-user EMAIL=FIRST.LAST@CLIENT.COM NAME='FIRST LAST' ORG=CLIENT

Repeat per reviewer. Two things about the display name, because neither is recoverable: it is client-facing — it appears on every comment they leave — and a re-run will not correct a typo, since an existing email is skipped and never updated. Getting it wrong means --deactivate plus a fresh create.

Passwords are generated and printed exactly once — capture them from that terminal. There is no self-service reset and the email can never be reused (get_user_by_email does not filter on is_active, so even a deactivated address is skipped on re-create), so that password is the credential for the life of the account. To choose it yourself, call the script directly with --password-stdin, which the make target deliberately does not expose — a password must not pass through a make variable, where it lands in the process table and the shell history:

fly ssh console -a ogur --pty -C "python /app/scripts/seed/seed_users.py \
  --email FIRST.LAST@CLIENT.COM --name 'FIRST LAST' --org CLIENT --password-stdin"

Creating an identity grants nothing. Each one needs the landscape explicitly, or they sign in successfully and the report 404s:

make fly-grant EMAIL=FIRST.LAST@CLIENT.COM LANDSCAPE=gns561-cca-autophagy-001

Both targets refuse an invocation missing any variable, and refuse it before opening a session against the app — a half-specified grant that reaches production fails inside argparse, where the operator reads a stack trace instead of the name of the variable they forgot. Both take FLY_APP= to point at a staging app, and both are idempotent: an existing email is skipped rather than updated, and a repeated grant is a no-op.

Verify the grant landed. Nothing downstream tells you it did not — an ungranted identity signs in successfully and the report 404s, which is indistinguishable from a report this deployment does not hold. The release smoke catches it, but two criteria later and phrased as a roster problem:

make fly-prod-smoke

Do not grant them polygon-cd8-ami-001. Cross-client isolation at this layer is the grant and nothing else — require_report_access returns the same 404 for an ungranted report as for one this deployment does not hold, so a wrong grant is the whole of the exposure.

Backups and restore¶

Two complementary mechanisms; both restore paths below are exercised, not hypothetical.

Layer 1 — Fly volume snapshots (automatic, daily)¶

Fly snapshots every volume daily; retention is set to 14 days in fly.toml ([mounts] snapshot_retention = 14). On-demand snapshot before anything risky:

fly volumes list -a ogur                      # note the vol_... id
fly volumes snapshots create <vol-id>

Layer 2 — offsite copy (manual, run at least weekly and before deploys)¶

make fly-backup        # FLY_APP=ogur by default

This runs sqlite3.backup() inside the machine (consistent under WAL, no downtime), pulls the copy to backups/ogur-<UTC-stamp>.db via fly ssh sftp, and verifies PRAGMA integrity_check plus user/signal row counts locally. backups/ is gitignored — these files contain client identities and feedback; store them like credentials.

Resetting feedback¶

Handing the report to a new set of reviewers means they should open it to an empty thread, not to our test rows.

make fly-reset-feedback            # dry run: reports the count, deletes nothing
make fly-reset-feedback YES=1      # deletes

Both forms take an offsite backup first — it is the only undo. The target deletes rows from the feedback table only: identities, sessions and landscape grants are untouched, so a reviewer signed in while it runs stays signed in. That is why this exists rather than make clean-db, which on the production volume would destroy the accounts along with the comments.

Scope it to one report with --landscape, e.g. fly ssh console -a ogur -C "python /app/scripts/ops/reset_feedback.py --landscape gns561-cca-autophagy-001 --yes".

The local equivalent is make reset-feedback [YES=1], against ogur.db.

Restore path A — from an offsite copy (also the "is this backup real?" test)¶

Boot the delivery image locally with the backup as the live DB:

mkdir -p /tmp/ogur-restore && cp backups/ogur-<stamp>.db /tmp/ogur-restore/ogur.db
docker run -d -p 8400:8000 -v /tmp/ogur-restore:/data ogur     # image from make container-build
curl -fsS http://127.0.0.1:8400/health
curl -fsS "http://127.0.0.1:8400/api/signals?landscape_id=gns561-cca-autophagy-001&limit=1"

The entrypoint sees an existing /data/ogur.db and never overwrites it.

Restore path B — from a Fly snapshot¶

A restored volume cannot attach to an existing machine: restore always means a new volume plus a machine to mount it on. Which machine differs between a drill and a real disaster, and getting that wrong loses client writes — so the two procedures are separate below. Do not run the disaster path as a drill.

B1 — Drill (periodic verification, production untouched)¶

The verification machine must have no services, so it never joins the http_service pool. fly machine run without -p creates exactly that: a machine reachable over fly ssh on the private network and invisible to clients. A fly machine clone of the production machine would instead inherit its services, take a share of live traffic, and then have its volume destroyed by the cleanup — discarding any session or feedback write it accepted, and serving stale reads until then.

fly volumes snapshots list <vol-id>
fly volumes create ogur_drill -a ogur --region cdg --snapshot-id <vs_...> --size 1 --yes
IMAGE=$(fly machine list -a ogur --json | python3 -c 'import json,sys; print(json.load(sys.stdin)[0]["config"]["image"])')
fly machine run "$IMAGE" 3600 -a ogur --region cdg --name ogur-restore-drill \
  --volume <drill-vol-id>:/data --entrypoint /bin/sleep --restart no

# Confirm isolation BEFORE trusting the drill — this must print services= NONE:
fly machine list -a ogur --json | python3 -c 'import json,sys; [print(m["id"], m["name"], "services=", m["config"].get("services") or "NONE") for m in json.load(sys.stdin)]'

fly ssh console -a ogur --machine <drill-id> -C "python -c \"import sqlite3; c=sqlite3.connect('/data/ogur.db'); print(c.execute('PRAGMA integrity_check').fetchone()[0], c.execute('SELECT count(*) FROM user').fetchone()[0], c.execute('SELECT count(*) FROM signal').fetchone()[0])\""

fly machine destroy <drill-id> -a ogur --force && fly volumes destroy <drill-vol-id> -y

The drill volume is named ogur_drill, not ogur_data, so a stray volume can never be mistaken for the production one — and --entrypoint /bin/sleep ensures the app never boots against it, so nothing writes to the copy under inspection.

B2 — Real disaster (restoring service from a snapshot)¶

Here the restored machine should take traffic, and the damaged one must stop taking it first. Order matters:

fly volumes create ogur_data -a ogur --region cdg --snapshot-id <vs_...> --size 1 --yes
fly machine stop <old-machine-id> -a ogur          # stop serving from the bad volume first
fly machine clone <old-machine-id> -a ogur --region cdg --attach-volume <new-vol-id>:/data
BASE_URL=https://ogur.fly.dev FLY_APP=ogur make fly-prod-smoke
fly machine destroy <old-machine-id> -a ogur --force
fly volumes destroy <old-vol-id> -y                # only after the smoke passes

Cloning is correct here precisely because the clone inherits the production services — that is what restores the URL. Stopping the old machine first keeps the window where two machines answer for one SQLite database closed.

Smoke gate¶

# Before OGUR-65 — content routes are still public:
BASE_URL=https://ogur.fly.dev FLY_APP=ogur make fly-prod-smoke

# From OGUR-65 on — the release-gate form, one granted identity per landscape:
BASE_URL=https://ogur.fly.dev FLY_APP=ogur \
  GNS_EMAIL=genfit.provisional@ogur.local  GNS_PASSWORD=... \
  PLG_EMAIL=polygon.provisional@ogur.local PLG_PASSWORD=... \
  make fly-prod-smoke

scripts/smoke/smoke_production.sh asserts the same ten criteria as the local container smoke — health, both landscape reports + figures, the real React app, SPA deep links, JSON 404 on unmatched /api paths, the 401 auth seam, non-empty signals, and pack digests (local shasum -a 256 equals sha256sum inside the machine, both packs) — plus an eleventh: exactly one started machine and two sub-second /health probes 60 s apart (the warm requirement). Run it after every deploy; it is the release gate's evidence.

Two things it deliberately refuses to fake:

Credentials. OGUR-65 puts the roster, report, figures and signals routes behind per-identity landscape grants, so from that release on the content criteria must run as the identity actually granted each landscape — which is why the credentials are per-landscape, not one shared login. With them set, the script additionally asserts that an anonymous report request is refused, which tests the authorization gate rather than just the /auth/me seam. Without them it runs anonymously, says so on stderr, and prints PROD SMOKE PASS (PARTIAL) — it will not present itself as the release gate.
The warm interval. SKIP_WARM_WAIT=1 does not shorten criterion 11, it withdraws it: the 60-second gap is the criterion, and two back-to-back probes say nothing about a machine that would have stopped a minute later. The run then reports SKIP 11 and warm: not verified.

After OGUR-65 merges (release-gate closer)¶

Rebase/merge the deployment branch onto main with OGUR-65 included, and redeploy: make fly-deploy.
Inject the deferred secrets: fly secrets set -a ogur ANTHROPIC_API_KEY=... DAILY_LLM_SPEND_CEILING_USD=5.00
Grant landscape access: fly ssh console -a ogur -C "python /app/scripts/seed/seed_users.py --grant genfit.provisional@ogur.local gns561-cca-autophagy-001" (and the Polygon pair).
Rerun make fly-prod-smoke — this run closes the release gate (for MVP-1 this was OGUR-61; done 2026-08-13).

Cutting a tagged release¶

The tag stamps the repository with "this is what shipped." It is separate from deploying: make fly-deploy changes what the production URL serves, the tag records which commit was verified serving it. Cut it after the release gate is green, never before — a tag on unverified code is a false citation.

Preconditions¶

Do not tag until all four hold.

make fly-prod-smoke passes against the production URL with per-landscape credentials set — PROD SMOKE PASS, not PASS (PARTIAL), and not SKIP 11. Per docs/product/product-development-workflow.md, a merged PR is not evidence that a gate passed; this run is the evidence.
The Linear release-gate issue for the milestone is green against production (worked example — MVP-1: OGUR-61, which required OGUR-77's named-client smoke).
Every release-blocking PR is merged to main. PRs labelled *-Stretch are explicitly not release-blocking and do not hold the tag.
CHANGELOG.md has a dated section for the version, pyproject.toml, frontend/package.json and ogur/api/app.py all declare it, and docs/product/release-notes-<version>.md exists.

The two commands¶

Substitute the version and milestone; v1.1.0 (MVP-1, cut 2026-08-13) is the worked example.

git checkout main && git pull && git status --porcelain   # must print nothing
git tag -a v1.1.0 -m "Ogur v1.1.0 — MVP-1 authenticated auditable reports"

git push origin v1.1.0
gh release create v1.1.0 \
  --title "Ogur v1.1.0 — MVP-1 authenticated auditable reports" \
  --notes-file docs/product/release-notes-v1.1.0.md

No workflow is tag-triggered — .github/workflows/ fires on branch push and pull request only — so tagging deploys nothing and publishes no docs. The release is inert metadata by design.

After the tag¶

Put the release URL on the release issue as its completion record.
Advance the active milestone in docs/product/mvp-scope.md §1 and append the decision to §10. Per the 2026-08-10 decision-log entry, a founder release is what advances the milestone — an overdue gate otherwise stays active.
Run the release transition in docs/product/product-development-workflow.md (archive superseded plans, retire overlapping skills, reset the harness). Capped at two working days.