Skip to content

/landscape-loop-gt skill — contract

Project-local Claude Code skill at .claude/skills/landscape-loop-gt/SKILL.md. Invoked with /landscape-loop-gt <landscape-id> <gold-deck.csv> [thesis] to run a recall-gated self-improvement loop against a hand-built gold-standard landscape. GT = ground truth: this is the bounded variant, requiring a gold deck as the answer key. Its open-ended sibling — same methodology, no gold standard, no recall gate — is landscape-loop-explore.

Why this exists

Ogur's pitch is measurable coverage. The way we prove it is a gold-standard landscape: a competitive-intelligence deck an analyst built by hand, treated as an answer key, with recall against it as the fitness function. This workflow was run twice — cardiometabolic-rnai-001 (siRNA dual-targeting) and obesity-rnai-001 — and each run re-derived the same structure by hand: freeze the deck, scaffold a seed + scorer, measure recall, diagnose the gap, pull a lever, re-measure, guard against phantom binds, write it up. That is exactly the "extract from a proven workflow" bar the project's skill convention uses (cf. /new-source, /review-engine).

Two things a checklist can't capture, and this skill bakes in:

  1. The metric + gate + anti-gaming discipline. Recall alone is gameable — the siRNA 26/32 episode was inflated by phantom matcher binds (BEBT-706 "matching" a sterilizer patent). The loop pairs recall with a two-factor phantom guard and a cross-deck regression gate (a lever that lifts the new deck but regresses a prior one is auto-reverted), so it optimises generalisation, not one deck.
  2. The hard-won source/cost gotchas. gpatents costs ~$1.40 batched in the seed but ~$87 per-query live; Lens trial tier 429-storms without pacing; SEC ingests filing metadata not body; patent-number rows bind two-factor not by doc-number string; company_match_tokens corrupts on commas; the matcher's MODALITY_SUBCLASS_TOKENS is dual-targeting subclasses, not base modality. Each was learned the expensive way.

/landscape-loop-gt converts the workflow-you-must-remember into a procedure with a recall gate, a lever menu with attribution discipline, and a two-document report contract.

I/O expectations

Input <landscape-id>, a frozen <gold-deck.csv> (assets ± companies; any schema), a one-sentence [thesis].
Output closed-world scorecard (filled deck + core-tier recall + gate), off-deck in-thesis CSV, live gap-analysis + discovered CSVs, two pharma-readable reports with per-finding source-attribution tables, a pass log.
Never mutates production ogur.db (copy DB only, verified byte-identical each pass), the shared matcher sirna_match.py, the gold deck.

Supported landscape class

v1 is proven on the RNAi / modality-target class (cardiometabolic-rnai-001, obesity-rnai-001), and its shipped tooling — the shared matcher (sirna_match.py), the scope-fit classifier (sirna_discovery/scope_fit.py), and the live discovery runner (sirna_autonomous_discovery.py, hardcoded to that LANDSCAPE_ID) — encodes that class. A new deck within the class needs only the Phase-0 loader clones (matcher reused byte-identically — the anti-gaming invariant). A new landscape class (oncology, antibody, small-molecule, cell-therapy) is out of scope for v1 until it has its own matcher module (+ regression tests), cloned discovery runner, scope-fit classifier, and modality vocabulary — as named per-landscape siblings, never in-place edits to the siRNA files. The SKILL's "Supported landscape class" section is the operational contract; this is deliberately a scoping boundary, not a silent loader swap.

Validation metrics

  • Closed-world gate (headline): core-asset recall AND core-company recall ≥ OGUR_GATE_MIN (env, default 0.85), on the deck's own scope tier. Core-asset recall is usually the binding constraint.
  • Anti-phantom: every bind two-factor (company+target or filer+code) + evidence-verified; a code-only collision must be refused.
  • Cross-deck regression: re-score every prior deck in the benchmark registry (archived_data/landscape_loop_benchmarks.md) each pass; any drop ⇒ auto-revert.
  • Live: sponsor / code-level / company recall + scope-fit split (core / honorable-mention / off-thesis). Rising off-thesis share alongside recall ⇒ gaming.

The two-phase shape

  • Phase 0 (setup, once): freeze deck (sha256 provenance) → clone seed template (constants from the deck) → clone closed-world scorer (adapt loader, reuse matcher) → seed copy DB → optional /looper handoff.
  • Phase 1 (loop): per pass on eval/<slug>-loop-pass-N — measure → gate-check → classify reachable/unreachable → Phase-0-test the lever → pull ONE lever (or one proposed structural edit to the loop's mechanics — decomposition, concept/target/company derivation — committed for later review; see the skill's "Adapting the loop itself") → make lint && make test + cross-deck re-score → phantom guard → record. Stop at gate, all-unreachable residual, no-progress (K=2), or iteration cap.
  • Phase 2 (live): cold Explore-mode discovery from the thesis; honest scope refinement (chips, never company/asset names); report both LLM-scope and refined-scope numbers; the discovery leads are the value.

Report contract

Two documents, pharma-readable, mirroring the siRNA/obesity use-case Notion pages:

  1. Closed-world coverage — includes a source-attribution table (every finding → source:count), the pass trajectory, the documented unreachable ceiling, and the anti-gaming integrity note.
  2. Live discovery — honest two-run headline, a "what Ogur found that the deck didn't" attribution table, a deck-recoveries table, and the "40%→85% gap = seeded-Monitor value" framing.

Provenance is non-negotiable (core principle #1): every claim carries a source chip, one click from raw. No orphan findings.

Relationship to other skills / docs

  • Uses /new-source (Lever A) — a new source must pass its Phase-0 feasibility gate before scaffolding.
  • Uses /looper (optional; an external environment skill — not checked into this repo, skip if unavailable) — formalises the gate/termination into a runnable handoff.
  • The shared matcher boundary is the same as siRNA source integration: feed the corpus, never fork sirna_match.py.
  • Standing conventions (always run gpatents in the seed; always flag off-deck in-thesis findings) are enforced here.

Reference runs

  • cardiometabolic-rnai-001 (siRNA) — closed-world + live use case.
  • obesity-rnai-001 — closed-world 6/14 → 12/14 core (85%) over 4 passes; live cold 40% sponsor recall + 13 core / 86 honorable-mention off-deck leads. Its reports (obesity_rnai_pass{0,1,4,5}.md + obesity_rnai_live_discovery.md) are tracked with the obesity workstream (PR #174), not in this PR's tree — see the benchmark registry for where its row lands once #174 merges.