/landscape-loop-gt skill — contract¶
Project-local Claude Code skill at .claude/skills/landscape-loop-gt/SKILL.md. Invoked with /landscape-loop-gt <landscape-id> <gold-deck.csv> [thesis] to run a recall-gated self-improvement loop against a hand-built gold-standard landscape. GT = ground truth: this is the bounded variant, requiring a gold deck as the answer key. Its open-ended sibling — same methodology, no gold standard, no recall gate — is landscape-loop-explore.
Why this exists¶
Ogur's pitch is measurable coverage. The way we prove it is a gold-standard landscape: a competitive-intelligence deck an analyst built by hand, treated as an answer key, with recall against it as the fitness function. This workflow was run twice — cardiometabolic-rnai-001 (siRNA dual-targeting) and obesity-rnai-001 — and each run re-derived the same structure by hand: freeze the deck, scaffold a seed + scorer, measure recall, diagnose the gap, pull a lever, re-measure, guard against phantom binds, write it up. That is exactly the "extract from a proven workflow" bar the project's skill convention uses (cf. /new-source, /review-engine).
Two things a checklist can't capture, and this skill bakes in:
- The metric + gate + anti-gaming discipline. Recall alone is gameable — the siRNA
26/32episode was inflated by phantom matcher binds (BEBT-706"matching" a sterilizer patent). The loop pairs recall with a two-factor phantom guard and a cross-deck regression gate (a lever that lifts the new deck but regresses a prior one is auto-reverted), so it optimises generalisation, not one deck. - The hard-won source/cost gotchas. gpatents costs ~$1.40 batched in the seed but ~$87 per-query live; Lens trial tier 429-storms without pacing; SEC ingests filing metadata not body; patent-number rows bind two-factor not by doc-number string;
company_match_tokenscorrupts on commas; the matcher'sMODALITY_SUBCLASS_TOKENSis dual-targeting subclasses, not base modality. Each was learned the expensive way.
/landscape-loop-gt converts the workflow-you-must-remember into a procedure with a recall gate, a lever menu with attribution discipline, and a two-document report contract.
I/O expectations¶
| Input | <landscape-id>, a frozen <gold-deck.csv> (assets ± companies; any schema), a one-sentence [thesis]. |
| Output | closed-world scorecard (filled deck + core-tier recall + gate), off-deck in-thesis CSV, live gap-analysis + discovered CSVs, two pharma-readable reports with per-finding source-attribution tables, a pass log. |
| Never mutates | production ogur.db (copy DB only, verified byte-identical each pass), the shared matcher sirna_match.py, the gold deck. |
Supported landscape class¶
v1 is proven on the RNAi / modality-target class (cardiometabolic-rnai-001, obesity-rnai-001), and its shipped tooling — the shared matcher (sirna_match.py), the scope-fit classifier (sirna_discovery/scope_fit.py), and the live discovery runner (sirna_autonomous_discovery.py, hardcoded to that LANDSCAPE_ID) — encodes that class. A new deck within the class needs only the Phase-0 loader clones (matcher reused byte-identically — the anti-gaming invariant). A new landscape class (oncology, antibody, small-molecule, cell-therapy) is out of scope for v1 until it has its own matcher module (+ regression tests), cloned discovery runner, scope-fit classifier, and modality vocabulary — as named per-landscape siblings, never in-place edits to the siRNA files. The SKILL's "Supported landscape class" section is the operational contract; this is deliberately a scoping boundary, not a silent loader swap.
Validation metrics¶
- Closed-world gate (headline): core-asset recall AND core-company recall ≥
OGUR_GATE_MIN(env, default 0.85), on the deck's own scope tier. Core-asset recall is usually the binding constraint. - Anti-phantom: every bind two-factor (company+target or filer+code) + evidence-verified; a code-only collision must be refused.
- Cross-deck regression: re-score every prior deck in the benchmark registry (
archived_data/landscape_loop_benchmarks.md) each pass; any drop ⇒ auto-revert. - Live: sponsor / code-level / company recall + scope-fit split (core / honorable-mention / off-thesis). Rising off-thesis share alongside recall ⇒ gaming.
The two-phase shape¶
- Phase 0 (setup, once): freeze deck (sha256 provenance) → clone seed template (constants from the deck) → clone closed-world scorer (adapt loader, reuse matcher) → seed copy DB → optional
/looperhandoff. - Phase 1 (loop): per pass on
eval/<slug>-loop-pass-N— measure → gate-check → classify reachable/unreachable → Phase-0-test the lever → pull ONE lever (or one proposed structural edit to the loop's mechanics — decomposition, concept/target/company derivation — committed for later review; see the skill's "Adapting the loop itself") →make lint && make test+ cross-deck re-score → phantom guard → record. Stop at gate, all-unreachable residual, no-progress (K=2), or iteration cap. - Phase 2 (live): cold Explore-mode discovery from the thesis; honest scope refinement (chips, never company/asset names); report both LLM-scope and refined-scope numbers; the discovery leads are the value.
Report contract¶
Two documents, pharma-readable, mirroring the siRNA/obesity use-case Notion pages:
- Closed-world coverage — includes a source-attribution table (every finding →
source:count), the pass trajectory, the documented unreachable ceiling, and the anti-gaming integrity note. - Live discovery — honest two-run headline, a "what Ogur found that the deck didn't" attribution table, a deck-recoveries table, and the "40%→85% gap = seeded-Monitor value" framing.
Provenance is non-negotiable (core principle #1): every claim carries a source chip, one click from raw. No orphan findings.
Relationship to other skills / docs¶
- Uses
/new-source(Lever A) — a new source must pass its Phase-0 feasibility gate before scaffolding. - Uses
/looper(optional; an external environment skill — not checked into this repo, skip if unavailable) — formalises the gate/termination into a runnable handoff. - The shared matcher boundary is the same as siRNA source integration: feed the corpus, never fork
sirna_match.py. - Standing conventions (always run gpatents in the seed; always flag off-deck in-thesis findings) are enforced here.
Reference runs¶
cardiometabolic-rnai-001(siRNA) — closed-world + live use case.obesity-rnai-001— closed-world 6/14 → 12/14 core (85%) over 4 passes; live cold 40% sponsor recall + 13 core / 86 honorable-mention off-deck leads. Its reports (obesity_rnai_pass{0,1,4,5}.md+obesity_rnai_live_discovery.md) are tracked with the obesity workstream (PR #174), not in this PR's tree — see the benchmark registry for where its row lands once #174 merges.