# v1.8.0 — Adversarial Trap Categories Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Characterise dlPFC goal-stack mechanism behaviour on **adversarial trap categories** (≥3 new categories with lesson vocabulary at <0.30 Jaccard overlap vs the existing 10 v1.5 categories), under the v1.7.7 prereg's pre-committed escalation constraints AND the v1.7.9 `docs/RETRACTION.md` magnitude-smuggling guard. **NO magnitude claim, NO pre-registered ≥Xpp threshold, NO pass/fail decision rule on a magnitude.** Pure mechanism characterisation: report **(a)** workload-validity sanity verdict (PASS/FAIL on `mean ∈ [0.05, 0.50] AND ≥3 distinct seeds non-zero`), **(b)** C3 per-seed lattice histogram on the adversarial-late slice, **(c)** sign-only seed-pair direction count vs C2 (k of 20 seeds where C3 strictly lower / strictly higher / tied). Anything else in the result doc is forbidden by RETRACTION.md.

**Architecture:** Add 3 new adversarial trap categories to `benchmarks/sequential-learning/traps.mjs::TRAP_CATEGORIES` (10 → 13). Each has tags + lessons + recall queries with novel vocabulary that BM25 will NOT trivially match against the recall query. Expand the 50-task workload to 62 tasks. New trap-encounter positions distributed **uniformly across early/mid/late regions** (2 each at positions 11, 21, 33, 41, 51, 59) — NOT clustered in late. Preserve `--restrict-late-to 4` as the late-phase metric (preserves the v1.7.7 N=4 lattice gate as a workload-validity check). Run C0/C1/C2/C3 × 20 seeds × `--eval-strict`. Apply C2-before-C3 sanity preflight using the same `mean ∈ [0.05, 0.50] AND ≥3 distinct seeds non-zero` rule (now framed as a workload-validity / non-saturation check, NOT a magnitude criterion per `docs/RETRACTION.md`).

**Tech Stack:** Node.js 22.5+ (existing harness), Markdown for prereg / inventory / result docs, Vitest for unit tests on Jaccard helper + new category schema + BM25-mismatch sim. No new dependencies.

**Semver decision:** 1.8.0 (minor) because the trap-category set is a public benchmark surface — going from 10 → 13 categories changes the workload definition. v1.7.x already prefigured this with the "Deferred to v1.8.0" lines in v1.7.5/6/7 CHANGELOGs. Numerical values in machine-readable schema (`benchmarks/sequential-learning/results/...json`) gain three new keys.

---

## RETRACTION.md compliance (MANDATORY — read first)

This release operates under the v1.7.9 magnitude-smuggling guard at `docs/RETRACTION.md`. Every output artifact in this plan MUST satisfy:

1. **Reference `docs/RETRACTION.md`** in the prereg, the result doc, the CHANGELOG entry, and the README "What's new in v1.8.0" block.
2. **State explicitly: "This release does not re-assert the retracted −10pp magnitude."** Verbatim, in each artifact above.
3. **No magnitude framing in any new prose.** Forbidden: `Δ\s*=`, `\b\d+pp`, `−\d+pp`, "Xpp lift", "Xpp drop", "Xpp reduction", "Xpp improvement", "lift" without workload-validity framing, "discriminating", "discrimination", "discrimination room", "magnitude" applied to mechanism effect, "median paired diff" in pp units, "5-number summary" on paired-diff distributions in pp units, side-by-side per-condition late-mean reporting that invites mental subtraction (e.g. "C2_LATE = X, C3_LATE = Y" in adjacent lines).
4. **Replacement language (use verbatim):** "non-saturation check" (replaces "discrimination check"), "non-zero variance across seeds" (replaces "discrimination room"), "C3 per-seed lattice histogram" (replaces magnitude reporting), "sign-only seed-pair direction count" (replaces paired-diff median or 5-number summary), "characterise" with operational definition (see "Operational definition" below).
5. **Outside-voice review on whether framing satisfies the guard** before /publish-repo. Same as v1.7.9 Task 12: a senior-code-reviewer pass against the staged diff. Magnitude grep regex: `(Δ\s*=|\b\d+\s*pp|−\d+pp|magnitude|discriminating|discrimination)`. Allowlist: lines containing the word `retracted` within 10 chars of the match are LEGITIMATE retraction citations and do NOT count as findings.

### Operational definition of "characterise" (NEW per outside voice D2)

For v1.8 specifically, "characterise the mechanism" means producing exactly these three reports:

1. **Sanity verdict (PASS/FAIL).** From the C2 sanity preflight at `mean ∈ [0.05, 0.50] AND ≥3 distinct seeds non-zero`. This is a workload-validity / non-saturation check. NOT a magnitude criterion.
2. **C3 per-seed lattice histogram.** For the 20 seeds, count how many landed at each lattice value `{0, 0.25, 0.5, 0.75, 1.0}`. Report the histogram counts. NOT a magnitude — a frequency distribution over discrete values.
3. **Sign-only seed-pair direction count vs C2.** For each of 20 paired seeds, classify (C3 vs C2) as one of: `STRICTLY_LOWER` (C3 < C2), `STRICTLY_HIGHER` (C3 > C2), `TIED` (C3 = C2). Report the three counts. NOT a magnitude — direction without amplitude.

ANY other report (pp values, deltas, medians, means juxtaposed, "improvement", "reduction") is forbidden by RETRACTION.md.

---

## Pre-committed constraints inherited from v1.7.7 prereg

From `docs/evals/2026-05-09-v1.7.7-goal-stack-eval-prereg.md` lines 79-87 ("Pre-committed v1.8 escalation constraints"):

| # | Constraint | This plan satisfies via |
|---|------------|--------------------------|
| 1 | At least 3 new trap categories beyond the existing 10 | Task 1 authors 3 adversarial categories with full lesson + tags + recallQueries |
| 2 | Lessons use vocabulary with **<0.30** Jaccard overlap with v1.7.5 lessons (TIGHTENED from <0.40 per outside voice E1) | Task 2 implements + runs `tools/jaccard-overlap.mjs` (extended stop-words + Porter-stem step) against new vs existing lessons; commits the verification output; gates the commit |
| 3 | Target C2 band: `mean ∈ [0.05, 0.50] AND ≥3 distinct seeds non-zero` | Task 6 sanity preflight uses the same `analyze-v1.7.7.mjs::computeVerdict` sanity-gate logic, framed as workload-validity / non-saturation (not magnitude) |
| 4 | Same C2-before-C3 sanity preflight (no goal-stack data on SANITY_FAIL) | Task 6 enforces; Task 7 (C0/C1/C3) only runs IF Task 6 passes |
| 5 | Categories authored BEFORE running C3 (no choosing-after-seeing-results) | Tasks 1-5 commit categories + prereg + inventory + tests BEFORE Task 6 (C2 preflight) runs. Task 6 runs BEFORE Task 7 (C3). Pre-registration discipline preserved. |
| 6 | All 4 conditions × 20 seeds, hash-derived | Task 7 runs C0/C1/C3 with hash-derived seeds 0..19 (same `Math.imul(0x9E3779B9, 1000+i) >>> 0` formula as v1.7.5/6/7) |

**No constraint relaxation. No category swap-in / swap-out after seeing C2 results. No vocabulary tuning after seeing recall behaviour.**

---

## Pre-committed v1.9 direction (named BEFORE v1.8 runs — per outside voice C1)

If v1.8 SANITY_FAILs (workload-validity FAIL on the adversarial workload), the pre-committed escalation chain over **workload knobs in the sequential-learning benchmark** is exhausted. v1.9+ requires a different direction.

**Named NOW, before v1.8 results exist:** v1.9 will run the dlPFC goal-stack mechanism on the **LongMemEval R@5 corpus** (already named in v1.7.7 prereg line 23 as a separate eval line), as a cross-validation of the mechanism on a fundamentally different benchmark (different metric: R@5 not trap-rate; different corpus: 500-question public vs internal sequential learning; different mechanism stress: retrieval-on-fixed-corpus vs agent-improvement-over-time).

This pre-commitment exists so the v1.8 SANITY_FAIL outcome cannot be retroactively justified by choosing a convenient v1.9 direction afterwards. v1.9 PRE-REG will be drafted before v1.9 implementation begins (separate plan).

---

## Cumulative evidence chain (for context, not for claim)

| Release | Workload knob | Outcome |
|---------|---------------|---------|
| v1.7.5 | full-late = last 7 of 25 (chronological-third) | C2 SANITY_FAIL — saturation |
| v1.7.6 | budget sweep {200, 400, 600, 800, 1000} | B*=NULL — workload floor |
| v1.7.7 | `--restrict-late-to 4` (last 4 of 25) | C2 SANITY_FAIL — N=4 lattice gate |
| v1.7.9 | (retraction release) | −10pp magnitude **RETRACTED** publicly |
| v1.8.0 | adversarial categories (this release) | TBD — descriptive characterisation only |

---

## Workload size + slice math

| Quantity | v1.7.5/6/7 (existing 10) | v1.8 (10 + 3) |
|----------|---------------------------|---------------|
| Total trap categories | 10 | 13 |
| Total trap encounters per seed | 25 | 31 (existing 25 + new 6) |
| Total tasks per seed | 50 | 62 |
| Late-phase metric | `--restrict-late-to 4` (last 4 of 25 trap encounters) | `--restrict-late-to 4` (last 4 of 31 trap encounters) |
| Late-phase lattice | per-seed ∈ {0, 0.25, 0.5, 0.75, 1.0} | unchanged (still N=4) |
| Aggregate `lateMean` | `k/(20×4) = k/80` | unchanged |
| Sanity gate | `mean ∈ [0.05, 0.50] AND ≥3 seeds non-zero` | unchanged (workload-validity / non-saturation check) |

The N=4 lattice math is preserved end-to-end. The sanity gate values are reused verbatim. Only the workload's category space changes.

**New-category position assignment (UNIFORM distribution per outside voice B2):** the 6 new trap encounters are placed at positions 11, 21, 33, 41, 51, 59 — distributed uniformly across the 62-task sequence (2 per region of early/mid/late roughly defined as 1-22, 22-42, 42-62). All 6 positions are odd (clean in v1.7.x). **NOT clustered in the late region.** With this distribution, the last-4 trap encounters (positions 48, 50, 51, 59) include 2 existing categories + 2 adversarial categories — **late-4 is mixed, not pure-adversarial.** Density math: positions 50-62 contain 1 existing trap (position 50) + 2 new (positions 51, 59) = 3 traps in 13 slots = 23% density (lower than the existing-region 50% — consistent with "no late-clustering").

| Region (62-task) | Positions | Existing traps | New traps | Total density |
|-------------------|-----------|------------------|-----------|----------------|
| Early (1-22) | 22 slots | 11 (positions 2-22 even) | 2 (positions 11, 21) | 13/22 = 59% |
| Mid (22-42) | 20 slots | 10 (positions 24-42 even) | 2 (positions 33, 41) | 12/20 = 60% |
| Late (42-62) | 20 slots | 4 (positions 44, 46, 48, 50) | 2 (positions 51, 59) | 6/20 = 30% |

(Late region has lower density by design — preserves enough clean tasks for the 12 new clean descriptions in 51-62.)

---

## What the result is allowed to say (and not say) — REWRITTEN per outside voice A1, A2, A4, D2

### Allowed (descriptive characterisation only, per the operational definition above)

- **Sanity verdict (PASS/FAIL).** One word. Plus a one-sentence justification: "C2 hippo-base lateMean = X, k of 20 seeds non-zero" (X is a lattice rate, NOT pp).
- **C3 per-seed lattice histogram.** Counts at each of {0, 0.25, 0.5, 0.75, 1.0}. Reported as a 5-cell histogram. Read-only on adversarial-late-slice.
- **Sign-only seed-pair direction count.** k_lower / k_higher / k_tied (sums to 20).
- **Hook-failure count** (validity check).
- **Tie-degeneracy `tiePass` boolean** (informational only).
- **Reproducibility anchors** (versions, SHAs, timestamps).

### Two-section result structure (no juxtaposition — per outside voice A1)

The result.md MUST separate condition reporting into TWO non-adjacent sections:

**Section "Workload baseline" (C0, C1, C2 only):**
- C0 none: late mean = X (95% CI ± Y) — workload no-memory baseline.
- C1 static: late mean = X (95% CI ± Y) — pre-loaded memory baseline.
- C2 hippo-base: late mean = X (95% CI ± Y) — sanity-gate input.

**Section "C3 mechanism characterisation" (C3 only, READ ONLY IF C2 sanity passed):**
- C3 per-seed lattice histogram: |0=k0|0.25=k1|0.5=k2|0.75=k3|1.0=k4| (sums to 20).
- Sign-only seed-pair direction count (vs C2): STRICTLY_LOWER=N, STRICTLY_HIGHER=N, TIED=N.
- Hook failures: push=N, complete=N.
- Tie degeneracy: tiePass=true|false.

Each section MUST contain the verbatim sentence: "Do not subtract across sections. Per `docs/RETRACTION.md`, magnitude differences between conditions are not reported in this release."

### Forbidden (per RETRACTION.md, hardened)

- `Δ = X pp` framing.
- `Xpp lift` / `Xpp drop` / `Xpp reduction` / `Xpp improvement` — even if measured.
- Median paired diff in pp units.
- 5-number summary on paired diffs in pp units.
- Side-by-side per-condition late-mean reporting that invites mental subtraction.
- Variance comparisons ("C3 has higher variance than C2" — magnitude-shaped).
- Words "discriminating", "discrimination", "discrimination room", "lift", "improvement", "reduction" applied to the mechanism's effect — replace per the language map above.
- Any pre-registered numeric pass/fail threshold that is NOT the workload-validity sanity gate.
- "Supported / Not supported" verdict templates from v1.7.5/6/7.

### Pre-registered result template (use verbatim, fill in the {SLOTS})

```text
> **Mechanism characterisation on v1.8 adversarial trap categories — descriptive report.** v1.8 workload (`--budget 2000 --restrict-late-to 4`, 13 categories including 3 new adversarial), 4 conditions × 20 paired seeds, `--eval-strict`, hash-derived seeds (1000+i for i in 0..19). **This release does not re-assert the retracted −10pp magnitude.** Per `docs/RETRACTION.md`, this is mechanism characterisation only.

## Workload-validity sanity verdict

**{PASS or FAIL}** — C2 hippo-base lateMean = {C2_LATE} (lattice rate), {C2_SEEDS_NON_ZERO} of 20 seeds non-zero. Pre-registered N=4 lattice rule from v1.7.7: `mean ∈ [0.05, 0.50] AND ≥3 distinct seeds non-zero`. Framed as workload-validity / non-saturation check per `docs/RETRACTION.md`, NOT a magnitude criterion.

## Workload baseline (C0, C1, C2)

- C0 none: late mean = {C0_LATE} (95% CI ± {C0_CI}).
- C1 static: late mean = {C1_LATE} (95% CI ± {C1_CI}).
- C2 hippo-base: late mean = {C2_LATE} (95% CI ± {C2_CI}).

> **Do not subtract across sections.** Per `docs/RETRACTION.md`, magnitude differences between conditions are not reported in this release.

## C3 mechanism characterisation (only if sanity verdict = PASS)

C3 per-seed lattice histogram (over 20 seeds at adversarial-late N=4):

| 0/4 | 1/4 | 2/4 | 3/4 | 4/4 |
|-----|-----|-----|-----|-----|
| {k0} | {k1} | {k2} | {k3} | {k4} |

Sign-only seed-pair direction count (vs C2):
- STRICTLY_LOWER (C3 < C2): {N_LOWER}
- STRICTLY_HIGHER (C3 > C2): {N_HIGHER}
- TIED (C3 = C2): {N_TIED}

Hook failures: push={N_PUSH}, complete={N_COMPLETE}. Tie-degeneracy: tiePass={true|false}.

> **Do not subtract across sections.** Per `docs/RETRACTION.md`, magnitude differences between conditions are not reported in this release.

## Outside-voice questions answered (per pre-reg lock; required content)

[Verbatim answers to the 5 outside-voice questions named in the prereg's "Outside voice questions to brief" subsection. Each question gets 1-2 sentences. NO magnitude framing.]

## Reproducibility anchors

- `hippo --version` = ...
- `git rev-parse HEAD` = ...
- `node --version` = ...
- `npx vitest --version` = ...
- `run.mjs` blob SHA = ...
- `aggregate.mjs` blob SHA = ...
- `traps.mjs` blob SHA = ... (Task 1 commit; the audit anchor for "categories authored BEFORE C3")
- `tools/jaccard-overlap.mjs` blob SHA + verification output: `docs/evals/2026-05-09-v1.8.0-jaccard-verification.txt`
- workload knobs: `--budget 2000 --restrict-late-to 4`
- run start/end timestamps: ...

Pre-registration: `docs/evals/2026-05-09-v1.8.0-adversarial-eval-prereg.md`. Categories authored at commit {COMMIT_SHA_OF_TASK_1} BEFORE C3 ran.
```

---

## Branch + setup

```bash
git -C C:/Users/skf_s/hippo branch --show-current   # confirm master after v1.7.9 merge
git -C C:/Users/skf_s/hippo pull origin master       # ensure up-to-date
git -C C:/Users/skf_s/hippo checkout -b feat/v1.8.0-adversarial-categories
```

Expected: switched to `feat/v1.8.0-adversarial-categories` branch from current master HEAD (post-v1.7.9 merge).

---

## Task 1: Author 3 new adversarial trap categories (with pilot smoke test)

**Files:**
- Modify: `benchmarks/sequential-learning/traps.mjs::TRAP_CATEGORIES` (10 → 13 entries)
- Modify: `benchmarks/sequential-learning/traps.mjs::TRAP_PLACEMENTS` (positions for new categories — UNIFORM, not late-clustered)
- Modify: `benchmarks/sequential-learning/traps.mjs::CLEAN_DESCRIPTIONS` (add 6 new clean tasks for the expanded 62-task sequence in slots 51-62 not used by traps)

**Step 0 (NEW per outside voice B4): category-difficulty pilot smoke test**

Before committing the final categories, run a 1-seed pilot of C2 (hippo-base, no goal-stack) on the draft categories. This is part of authoring, NOT pre-registration. Goal: confirm at least one of the 3 categories produces non-zero late traps for C2 (hippo doesn't trivially recall all of them). If all 3 saturate at 0%, replace the easiest with a genuinely obscure trap.

```bash
# Draft the 3 categories in traps.mjs (uncommitted).
# Run pilot on seed=42 only (NOT the prereg seed range 0..19).
cd C:/Users/skf_s/hippo
node benchmarks/sequential-learning/run.mjs \
  --adapter hippo --n-seeds 1 --seed 42 --budget 2000 --restrict-late-to 4 \
  --output benchmarks/sequential-learning/results/v1.8-pilot/
# Inspect: jq '.conditions.hippo.phases.late' benchmarks/sequential-learning/results/v1.8-pilot/*.json
```

**Acceptance:** at least one new category produces ≥1 trap-hit on the pilot seed's late-4. If all 3 saturate at 0% trap-hits across the late-4 lattice, replace the easiest one. Suggested replacement: `seedstart_state_pollution` (Python `random.seed` × multiprocessing pickling: child workers re-derive the same seed and produce duplicate samples; lesson uses vocabulary like "fork", "pickling", "subprocess seeding", "child process state", "multiprocessing pool" — disjoint from existing 10).

**Iteration log:** if a replacement is required, document the decision in the prereg under "Category authoring iteration log" (which drafts were rejected, what the pilot data showed, what replaced it). The prereg locks ONLY the FINAL committed categories.

**The pilot results JSON is NOT the v1.8 result.** It is part of authoring. Per RETRACTION.md, the pilot output must NOT be cited as a magnitude claim or used to argue any v1.8 outcome. The prereg's "iteration log" is the only legitimate citation surface.

**Step 1: Author the 3 categories (DRAFTS — outside voice may revise)**

Each new category needs: `id`, `lesson`, `tags`, `recallQueries`. Vocabulary must satisfy <0.30 Jaccard with existing 10 lessons (verified in Task 2). Recall queries should NOT lexically match the lesson (BM25 should under-match on adversarial recall).

```javascript
{
  id: 'timezone_naive',
  lesson: 'Persist instants in UTC. Wall-clock arithmetic across daylight-saving boundaries silently shifts events one hour; conversion to local zone happens at presentation, never at storage.',
  tags: ['timezone', 'utc', 'daylight-saving', 'instants'],
  recallQueries: ['storing event times across regions', 'rendering schedules in user local zone'],
},
{
  id: 'idempotency_retry',
  lesson: 'Receivers must dedupe by request key. Network retries duplicate effects when the receiver lacks an idempotency token; pure best-effort APIs are not delivery guarantees.',
  tags: ['idempotency', 'retry', 'distributed', 'request-id'],
  recallQueries: ['handling network retries on payment processing', 'distributed task scheduler delivery'],
},
{
  id: 'float_accumulation',
  lesson: 'Loop-summed floats diverge from the analytic answer at high counts; Kahan summation or fixed-point integers preserve precision for monetary or scientific aggregates.',
  tags: ['floating-point', 'kahan', 'accumulation', 'precision'],
  recallQueries: ['summing transaction amounts in batch', 'aggregating sensor readings per second'],
},
```

**Why these 3 and not others** (decision-log entry, must appear in the prereg doc):
- **`timezone_naive`** — vocabulary (utc / instants / wall-clock / daylight-saving / persist) is entirely absent from v1.5 lessons.
- **`idempotency_retry`** — vocabulary (idempotency / dedupe / receiver / request-key / token / delivery) absent from v1.5.
- **`float_accumulation`** — vocabulary (kahan / accumulation / precision / monetary / scientific / aggregate) absent from v1.5.

**Position assignments — UNIFORM distribution (NOT late-clustered, per outside voice B2):**

To keep late-4 mixed (not pure-adversarial) and avoid optimising for magnitude visibility, distribute new traps across early/mid/late regions:

```javascript
{ category: 'timezone_naive',     positions: [11, 41] },
{ category: 'idempotency_retry',  positions: [21, 51] },
{ category: 'float_accumulation', positions: [33, 59] },
```

Per region (62-task):
- Early (1-22): 2 new traps (positions 11, 21)
- Mid (22-42): 2 new traps (positions 33, 41)
- Late (42-62): 2 new traps (positions 51, 59)

The last-4 trap encounters chronologically: positions 48, 50, 51, 59 — **2 existing + 2 new** (mixed slice, NOT pure-adversarial).

**Density math (corrected per outside voice B1):**
- Late region (50-62, 13 slots): 1 existing trap (position 50) + 2 new (51, 59) = **3 traps in 13 slots = 23% density** (lower than existing-region 50%).
- The earlier draft's "12 of 13 slots" claim was wrong. Corrected.

**Clean tasks for the expanded region:** add 6 new clean descriptions to `CLEAN_DESCRIPTIONS` for slots 52-58 (positions 52, 53, 54, 55, 56, 57, 58, 60, 61, 62 — 10 clean slots in 50-62, after subtracting trap positions 50, 51, 59).

**Step 2: Edit `traps.mjs`**

Append the 3 new category objects to `TRAP_CATEGORIES`. Append the 3 new placement entries to `TRAP_PLACEMENTS`. Add 6 new clean descriptions. Update any `traps.length` / iteration counts to reflect 13 categories and 31 trap encounters.

**Step 3: Update task generation (`generateTasks`)**

Confirm `generateTasks(seed)` correctly handles 62 tasks (not 50) and 31 trap encounters (not 25). The seeded category-to-slot variance algorithm should generalise; verify via unit test in Task 5.

**Step 4: Commit (categories + placements + clean tasks)**

```bash
git -C C:/Users/skf_s/hippo add benchmarks/sequential-learning/traps.mjs
git -C C:/Users/skf_s/hippo commit -m "feat(benchmark): add 3 adversarial trap categories with uniform-distribution positions (v1.8.0 pre-reg lock)"
```

**This commit must precede Task 6 (C2 preflight). It is the audit anchor for "categories authored BEFORE C3 ran."**

**Step 5 (NEW per outside voice E4): commit pilot result + iteration log**

```bash
git -C C:/Users/skf_s/hippo add benchmarks/sequential-learning/results/v1.8-pilot/ docs/evals/2026-05-09-v1.8.0-category-authoring-iteration-log.md
git -C C:/Users/skf_s/hippo commit -m "docs(benchmark): v1.8 pilot smoke test + category authoring iteration log"
```

The iteration log is part of pre-registration (declares the authoring process), NOT a result. Cite the pilot but do NOT use its values as a v1.8 outcome.

---

## Task 2: Verify <0.30 Jaccard overlap (TIGHTENED from 0.40 per outside voice E1)

**Files:**
- Create: `tools/jaccard-overlap.mjs` (one-off verification script; lives in `tools/`, not `benchmarks/`, since it's not part of the runner)
- Create: `tools/jaccard-overlap.test.ts` (unit test for the script's logic)
- Output: `docs/evals/2026-05-09-v1.8.0-jaccard-verification.txt` (committed audit artifact)

**Step 1: Define Jaccard overlap pre-registered method (extended per outside voice E1)**

For each new lesson L_new and each existing lesson L_existing:
- Tokenise: lowercase, split on `\W+`, drop stop-words, drop tokens ≤2 chars, **apply Porter-stem step** (treat "verify"/"verifying"/"verified" as the same stem `verifi`).
- **Extended stop-word set** (vs v1 draft):
  ```
  Common: a, the, an, is, are, was, were, be, been, being, of, in, on, at, to, for, with, from, by, as, and, or, not, no, do, does, did, will, would, should, could, can, may, must, this, that, these, those, it, its, has, have, had
  Modal/aux: shall, might, ought
  Engineering function words: use, uses, used, using, make, makes, made, get, gets, got, set, sets, run, runs, ran, run, find, finds, found, see, saw, seen, give, gives, gave, given, take, takes, took, taken
  ```
- Token set (after stem): `S_new = unique stems`, similarly `S_existing`.
- Jaccard = `|S_new ∩ S_existing| / |S_new ∪ S_existing|`.

Constraint: max Jaccard across all (3 × 10 = 30) pairs < **0.30** (TIGHTENED from 0.40 per outside voice E1).

**Justification for 0.30 threshold (per outside voice E1):** v1.7.7 prereg's 0.40 was an inherited default; tightening to 0.30 makes the adversarial claim stronger. With Porter-stem + extended stop-words, achievable thresholds drop ~5-10pp because stems collapse plurals/tenses; 0.30 post-stem is approximately equivalent to 0.40 pre-stem on the same lesson texts. This is documented in the prereg.

**Step 2: Implement the script**

```javascript
// tools/jaccard-overlap.mjs
import { TRAP_CATEGORIES } from '../benchmarks/sequential-learning/traps.mjs';

const STOP_WORDS = new Set([
  'a', 'the', 'an', 'is', 'are', 'was', 'were', 'be', 'been', 'being',
  'of', 'in', 'on', 'at', 'to', 'for', 'with', 'from', 'by', 'as',
  'and', 'or', 'not', 'no', 'do', 'does', 'did', 'will', 'would', 'should',
  'could', 'can', 'may', 'must', 'shall', 'might', 'ought',
  'this', 'that', 'these', 'those', 'it', 'its', 'has', 'have', 'had',
  'use', 'uses', 'used', 'using', 'make', 'makes', 'made', 'get', 'gets',
  'got', 'set', 'sets', 'run', 'runs', 'ran', 'find', 'finds', 'found',
  'see', 'saw', 'seen', 'give', 'gives', 'gave', 'given', 'take', 'takes',
  'took', 'taken',
]);

// Minimal Porter-style stem: strip common verb/noun suffixes.
function stem(word) {
  return word
    .replace(/ies$/, 'y')
    .replace(/(ing|ed|es|s)$/, '')
    .replace(/(tion|ment|ness|ity|sion)$/, '');
}

export function tokenize(text) {
  return new Set(
    text
      .toLowerCase()
      .split(/\W+/)
      .filter((w) => w.length > 2 && !STOP_WORDS.has(w))
      .map(stem)
  );
}

export function jaccard(setA, setB) {
  const intersection = [...setA].filter((x) => setB.has(x)).length;
  const union = new Set([...setA, ...setB]).size;
  return union === 0 ? 0 : intersection / union;
}

const NEW_CATEGORY_IDS = ['timezone_naive', 'idempotency_retry', 'float_accumulation'];
const newCategories = TRAP_CATEGORIES.filter((c) => NEW_CATEGORY_IDS.includes(c.id));
const existingCategories = TRAP_CATEGORIES.filter((c) => !NEW_CATEGORY_IDS.includes(c.id));

let maxJaccard = 0;
let maxPair = '';

for (const newCat of newCategories) {
  const newTokens = tokenize(newCat.lesson);
  for (const existing of existingCategories) {
    const existingTokens = tokenize(existing.lesson);
    const j = jaccard(newTokens, existingTokens);
    console.log(`${newCat.id} ↔ ${existing.id}: Jaccard = ${j.toFixed(4)} (${[...newTokens].length}/${[...existingTokens].length} stems, ∩ = ${[...newTokens].filter((x) => existingTokens.has(x)).length})`);
    if (j > maxJaccard) {
      maxJaccard = j;
      maxPair = `${newCat.id} ↔ ${existing.id}`;
    }
  }
}

console.log('---');
console.log(`MAX Jaccard: ${maxJaccard.toFixed(4)} (${maxPair})`);
console.log(`THRESHOLD: 0.30 (TIGHTENED from 0.40 per outside voice E1)`);
console.log(`PASS: ${maxJaccard < 0.30 ? 'YES' : 'NO'}`);
process.exit(maxJaccard < 0.30 ? 0 : 1);
```

**Step 3: Run + capture output**

```bash
node tools/jaccard-overlap.mjs > docs/evals/2026-05-09-v1.8.0-jaccard-verification.txt 2>&1
echo "Exit: $?"
```

**Expected:** exit code 0, max Jaccard < 0.30. **If FAIL** (max ≥ 0.30): rewrite the offending lesson(s) in Task 1 with more divergent vocabulary, re-run, until pass. Each rewrite attempt logs to the iteration log.

**Step 4: Unit test**

```javascript
// tools/jaccard-overlap.test.ts
import { describe, it, expect } from 'vitest';
import { tokenize, jaccard } from '../tools/jaccard-overlap.mjs';

describe('jaccard-overlap', () => {
  it('tokenize drops stop-words, short tokens, applies Porter-style stem', () => {
    const tokens = tokenize('The quickly running fox is fast and verifies retries.');
    // 'quickly'->'quick', 'running'->'runn', 'fox', 'fast', 'verifies'->'verifi', 'retries'->'retri'
    // 'the'/'is'/'and' dropped as stop-words
    expect(tokens.has('fox')).toBe(true);
    expect(tokens.has('fast')).toBe(true);
    expect(tokens.has('the')).toBe(false);
    expect(tokens.has('and')).toBe(false);
  });

  it('jaccard of identical sets is 1', () => {
    expect(jaccard(new Set(['a', 'b']), new Set(['a', 'b']))).toBe(1);
  });

  it('jaccard of disjoint sets is 0', () => {
    expect(jaccard(new Set(['a', 'b']), new Set(['c', 'd']))).toBe(0);
  });

  it('jaccard handles partial overlap', () => {
    expect(jaccard(new Set(['a', 'b', 'c']), new Set(['b', 'c', 'd']))).toBe(0.5);
  });

  it('jaccard of two empty sets is 0 (degenerate)', () => {
    expect(jaccard(new Set(), new Set())).toBe(0);
  });

  it('Porter-stem collapses plurals/tenses', () => {
    const a = tokenize('verifies retries running');
    const b = tokenize('verify retry run');
    // Both should produce identical stems after suffix-strip.
    expect([...a].sort()).toEqual([...b].sort());
  });
});
```

**Step 5: Commit**

```bash
git -C C:/Users/skf_s/hippo add tools/jaccard-overlap.mjs tools/jaccard-overlap.test.ts docs/evals/2026-05-09-v1.8.0-jaccard-verification.txt
git -C C:/Users/skf_s/hippo commit -m "tools(benchmark): jaccard-overlap with Porter-stem, threshold <0.30 (v1.8.0)"
```

---

## Task 3: BM25-mismatch sim test (NEW per outside voice E4 + B4)

**Files:**
- Create: `tests/sl-bm25-mismatch.test.ts`

**Goal:** independently verify that the new lessons are NOT trivially BM25-matched by the existing-10 recall queries (and vice versa). This is a complement to the Jaccard check — Jaccard catches lexical overlap; BM25 sim catches token-frequency-weighted match score.

**Step 1: Implement the test**

```javascript
// tests/sl-bm25-mismatch.test.ts
import { describe, it, expect } from 'vitest';
import { TRAP_CATEGORIES } from '../benchmarks/sequential-learning/traps.mjs';

const NEW_IDS = ['timezone_naive', 'idempotency_retry', 'float_accumulation'];

// Simple BM25-like score: shared-token count weighted by inverse frequency.
function bm25Score(query, docTokens) {
  const queryTokens = query.toLowerCase().split(/\W+/).filter((w) => w.length > 2);
  let score = 0;
  for (const qt of queryTokens) {
    if (docTokens.has(qt)) score += 1;
  }
  return score;
}

function tokenize(text) {
  return new Set(text.toLowerCase().split(/\W+/).filter((w) => w.length > 2));
}

describe('BM25-mismatch sim — adversarial categories should not be top-1 for existing recall queries', () => {
  it('each existing recall query does NOT top-1-rank a new-category lesson', () => {
    const newCats = TRAP_CATEGORIES.filter((c) => NEW_IDS.includes(c.id));
    const existingCats = TRAP_CATEGORIES.filter((c) => !NEW_IDS.includes(c.id));

    for (const ec of existingCats) {
      for (const query of ec.recallQueries) {
        const allCats = [...newCats, ...existingCats];
        const scored = allCats
          .map((c) => ({ id: c.id, score: bm25Score(query, tokenize(c.lesson)) }))
          .sort((a, b) => b.score - a.score);
        const top1 = scored[0];
        // For an existing-query, the top-1 should be the existing-category whose query it is, not a new adversarial one.
        expect(NEW_IDS.includes(top1.id)).toBe(false);
      }
    }
  });

  it('each new-category recall query does NOT top-1-rank an existing-category lesson', () => {
    const newCats = TRAP_CATEGORIES.filter((c) => NEW_IDS.includes(c.id));
    const existingCats = TRAP_CATEGORIES.filter((c) => !NEW_IDS.includes(c.id));

    for (const nc of newCats) {
      for (const query of nc.recallQueries) {
        const allCats = [...newCats, ...existingCats];
        const scored = allCats
          .map((c) => ({ id: c.id, score: bm25Score(query, tokenize(c.lesson)) }))
          .sort((a, b) => b.score - a.score);
        const top1 = scored[0];
        // For a new-query, the top-1 should be the new-category whose query it is, not an existing one.
        expect(NEW_IDS.includes(top1.id)).toBe(true);
      }
    }
  });
});
```

**Step 2: Run + commit**

```bash
cd C:/Users/skf_s/hippo
npx vitest run tests/sl-bm25-mismatch.test.ts
git -C C:/Users/skf_s/hippo add tests/sl-bm25-mismatch.test.ts
git -C C:/Users/skf_s/hippo commit -m "tests(benchmark): bm25-mismatch sim for adversarial categories (v1.8.0)"
```

---

## Task 4: Pre-registration document

**Files:**
- Create: `docs/evals/2026-05-09-v1.8.0-adversarial-eval-prereg.md`

**Step 1: Write the prereg**

Use the v1.7.7 prereg as a template, but **rewrite the hypothesis / decision-rule / templates sections to remove every magnitude term**. The prereg must:

- State the descriptive-characterisation framing explicitly. Reference `docs/RETRACTION.md`. Include the verbatim "This release does not re-assert the retracted −10pp magnitude."
- Include the **operational definition of "characterise"** verbatim from this plan (3 reports: sanity verdict, lattice histogram, sign-only direction count).
- Pre-register: workload definition (62 tasks, 13 categories, `--restrict-late-to 4`, **uniform position distribution**), conditions (C0/C1/C2/C3), seeds (0..19), eval-mode (`--eval-strict`), C2 sanity gate ([0.05, 0.50] AND ≥3 non-zero, framed as workload-validity / non-saturation).
- Pre-register the result template verbatim (the two-section structure from this plan).
- Pre-commit: NO SUPPORTED / NOT_SUPPORTED templates. Only "WORKLOAD-VALIDITY PASS / FAIL" + "MECHANISM CHARACTERISATION" / "SANITY_FAIL" templates.
- List the new 3 categories' IDs + lessons + Jaccard verification commit reference + BM25-mismatch test reference + pilot iteration log reference.
- Include the same C2-before-C3 hard-stop rule.
- Include the same hook-failure gate, seed-crash policy, outlier policy as v1.7.7.
- Reference Task 1 commit SHA as the "categories authored BEFORE C3" anchor.
- Include the **"Pre-committed v1.9 direction (named BEFORE v1.8 runs)"** subsection: LongMemEval R@5 cross-validation per v1.7.7 prereg line 23.
- Include **"Subset reporting lock"** (per outside voice A5): late-phase only, no subset slicing, no per-version (v1.7.x) cross-cite.
- Include **"Outside voice questions"** subsection (5 questions to answer in result.md).

**Step 2: Commit**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.8.0-adversarial-eval-prereg.md
git -C C:/Users/skf_s/hippo commit -m "docs(evals): pre-registration for v1.8.0 adversarial-categories eval (with v1.9 pre-commit)"
```

---

## Task 5: Claim inventory + retraction-compliance check

**Files:**
- Create: `docs/evals/2026-05-09-v1.8.0-claim-inventory.md`

**Step 1: Re-grep for any new magnitude phrases** introduced by the v1.8 plan / prereg / drafts:

```bash
git -C C:/Users/skf_s/hippo grep -nE "(\-10pp|10pp|10 percentage|magnitude|Δ\s*=|\b\d+pp|−\d+pp|pp lift|pp drop|≥\s*\d+pp|discriminating|discrimination|discrimination room)" -- \
  docs/plans/2026-05-09-v1.8.0-adversarial-categories.md \
  docs/evals/2026-05-09-v1.8.0-adversarial-eval-prereg.md \
  docs/evals/2026-05-09-v1.8.0-category-authoring-iteration-log.md \
  benchmarks/sequential-learning/traps.mjs \
  tools/jaccard-overlap.mjs
```

**Expected:** zero NEW hits in the v1.8 surfaces (only retraction prose mentioning "the retracted −10pp magnitude" or "no magnitude is currently claimed"). If any hit is a magnitude assertion, rewrite to characterisation framing. Apply the **retraction-citation allowlist** (per outside voice E2): grep finding is legitimate IFF the line contains the word `retracted` within 10 chars of the match.

**Step 2: Write inventory**

Same shape as the v1.7.9 retraction inventory: a literal grep block, a retraction-compliance table (per file: scanned / clean / re-scan needed), the explicit "This release does not re-assert the retracted −10pp magnitude" declaration. Plus a row asserting that **every artifact in this release contains the verbatim retraction sentence and references `docs/RETRACTION.md`**.

**Step 3: Commit**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.8.0-claim-inventory.md
git -C C:/Users/skf_s/hippo commit -m "docs(evals): v1.8.0 claim inventory + retraction-compliance check"
```

---

## Task 6: Update / add tests for the expanded workload

**Files:**
- Modify: any existing test that asserts `tasks.length === 50` or `trap encounters === 25`
- Add: `tests/sl-traps-v1.8.test.ts` — new unit tests for 13 categories, 31 trap encounters, 62 task slots, position uniqueness, **uniform distribution invariants**, **existing-10 PRNG-stability** (per outside voice E3)

**Step 1: Search for hardcoded 50 / 25 / 10 in tests**

```bash
git -C C:/Users/skf_s/hippo grep -nE "(50.{0,5}task|25.{0,5}trap|10.{0,5}categor)" -- tests/
```

For each hit, decide: parameterise (preferred), update to 62/31/13 (if v1.8-specific), or leave (if v1.7.7-historical and snapshot-locked).

**Step 2: New tests (TIGHTENED per outside voice B3, B2, E3)**

```javascript
// tests/sl-traps-v1.8.test.ts
import { describe, it, expect } from 'vitest';
import { TRAP_CATEGORIES, TRAP_PLACEMENTS, generateTasks } from '../benchmarks/sequential-learning/traps.mjs';

describe('v1.8 adversarial traps', () => {
  it('has 13 trap categories (10 + 3 adversarial)', () => {
    expect(TRAP_CATEGORIES.length).toBe(13);
  });

  it('includes the 3 new adversarial categories', () => {
    const ids = TRAP_CATEGORIES.map((c) => c.id);
    expect(ids).toContain('timezone_naive');
    expect(ids).toContain('idempotency_retry');
    expect(ids).toContain('float_accumulation');
  });

  it('total trap encounters = 31', () => {
    const total = TRAP_PLACEMENTS.reduce((sum, p) => sum + p.positions.length, 0);
    expect(total).toBe(31);
  });

  it('all positions unique across categories', () => {
    const allPositions = TRAP_PLACEMENTS.flatMap((p) => p.positions);
    expect(new Set(allPositions).size).toBe(allPositions.length);
  });

  // TIGHTENED per outside voice B3: exact counts, not >=
  it('new categories have EXACTLY 6 trap positions, distributed uniformly across 62-task sequence', () => {
    const newCats = ['timezone_naive', 'idempotency_retry', 'float_accumulation'];
    const newPositions = TRAP_PLACEMENTS
      .filter((p) => newCats.includes(p.category))
      .flatMap((p) => p.positions);

    expect(newPositions.length).toBe(6);

    // Per outside voice B2: uniform distribution = ~2 per region
    const earlyCount = newPositions.filter((pos) => pos >= 1 && pos <= 22).length;
    const midCount = newPositions.filter((pos) => pos >= 23 && pos <= 42).length;
    const lateCount = newPositions.filter((pos) => pos >= 43 && pos <= 62).length;

    expect(earlyCount).toBe(2);
    expect(midCount).toBe(2);
    expect(lateCount).toBe(2);
  });

  // EXACT positions (NOT clustered in late, per outside voice B2)
  it('new positions are exactly {11, 21, 33, 41, 51, 59}', () => {
    const newCats = ['timezone_naive', 'idempotency_retry', 'float_accumulation'];
    const newPositions = TRAP_PLACEMENTS
      .filter((p) => newCats.includes(p.category))
      .flatMap((p) => p.positions)
      .sort((a, b) => a - b);
    expect(newPositions).toEqual([11, 21, 33, 41, 51, 59]);
  });

  it('generateTasks(seed=0) produces 62 tasks with 31 trap encounters', () => {
    const tasks = generateTasks(0);
    expect(tasks.length).toBe(62);
    const trapTasks = tasks.filter((t) => t.trapCategory);
    expect(trapTasks.length).toBe(31);
  });

  it('generateTasks is deterministic for fixed seed', () => {
    const a = generateTasks(42).map((t) => t.trapCategory ?? null);
    const b = generateTasks(42).map((t) => t.trapCategory ?? null);
    expect(a).toEqual(b);
  });

  it('generateTasks(seed=0) and (seed=1) produce different category-to-slot mappings', () => {
    const a = generateTasks(0).map((t) => t.trapCategory ?? null);
    const b = generateTasks(1).map((t) => t.trapCategory ?? null);
    expect(a).not.toEqual(b);
  });

  // NEW per outside voice E3: existing-10 subset PRNG-stability across workload sizes.
  // If we ever cross-cite v1.7.5/6/7 results for the existing-10 subset (we won't in v1.8,
  // per the subset-reporting lock — but document explicitly that we COULDN'T even if we wanted to).
  it('existing-10 subset placement is PRNG-stable when workload grows 50 -> 62', () => {
    // Document: this test asserts that the existing-10 categories' positions in v1.8 are
    // exactly the v1.7.x positions [2, 4, ..., 50]. The PRNG state INSIDE the new categories'
    // slot-assignment may differ, so per-position content is NOT comparable across versions.
    // The subset-reporting lock in the prereg forbids cross-version cite anyway.
    const existingIds = TRAP_CATEGORIES.filter((c) =>
      !['timezone_naive', 'idempotency_retry', 'float_accumulation'].includes(c.id)
    ).map((c) => c.id);
    const existingPlacements = TRAP_PLACEMENTS.filter((p) => existingIds.includes(p.category));
    const existingPositions = existingPlacements.flatMap((p) => p.positions).sort((a, b) => a - b);
    // Existing 25 positions, all even, in 2..50.
    expect(existingPositions.length).toBe(25);
    expect(existingPositions[0]).toBe(2);
    expect(existingPositions[24]).toBe(50);
    expect(existingPositions.every((p) => p >= 2 && p <= 50 && p % 2 === 0)).toBe(true);
  });
});
```

**Step 3: Run tests**

```bash
cd C:/Users/skf_s/hippo
npx vitest run tests/sl-traps-v1.8.test.ts tests/sl-bm25-mismatch.test.ts tools/jaccard-overlap.test.ts
```

**Expected:** all new tests pass + existing sl-* tests still pass.

**Step 4: Commit**

```bash
git -C C:/Users/skf_s/hippo add tests/sl-traps-v1.8.test.ts
# plus any modified existing tests
git -C C:/Users/skf_s/hippo commit -m "tests(benchmark): v1.8 adversarial traps schema + uniform distribution + PRNG-stability"
```

---

## Task 7: C2 sanity preflight (workload-validity / non-saturation check)

**Files:** none modified; only runs harness with C2 only.
- Output: `benchmarks/sequential-learning/results/v1.8-eval-C2-hippo-base/benchmark-{ts}.json`

**Step 1: Run C2 only**

```bash
cd C:/Users/skf_s/hippo
mkdir -p benchmarks/sequential-learning/results/v1.8-eval-C2-hippo-base
node benchmarks/sequential-learning/run.mjs \
  --adapter hippo \
  --n-seeds 20 \
  --eval-strict \
  --budget 2000 \
  --restrict-late-to 4 \
  --output benchmarks/sequential-learning/results/v1.8-eval-C2-hippo-base/
```

Expected wall-time: ~8-10 min (similar to v1.7.7; +24% workload size offset by no C3).

**Step 2: Apply sanity gate (workload-validity / non-saturation check, NOT magnitude)**

Read the resulting JSON; compute `lateMean` and `≥3 distinct seeds non-zero`. Per `analyze-v1.7.7.mjs::computeVerdict`:

- `lateMean ∈ [0.05, 0.50]` AND `seedsNonZero ≥ 3` → **WORKLOAD VALIDITY PASS** (non-saturated). Proceed to Task 8.
- Else → **WORKLOAD VALIDITY FAIL** (saturated). Apply SANITY_FAIL branch: write result doc with workload-validity-fail prose. Per the **"Pre-committed v1.9 direction"** subsection in the prereg: v1.9 will explore LongMemEval R@5 cross-validation as a fundamentally different benchmark. Do NOT proceed to C3.

**Step 3: Commit C2 result**

```bash
git -C C:/Users/skf_s/hippo add benchmarks/sequential-learning/results/v1.8-eval-C2-hippo-base/
git -C C:/Users/skf_s/hippo commit -m "eval(v1.8): C2 sanity preflight result ({PASS|FAIL})"
```

---

## Task 8: Run C0 / C1 / C3 (only if Task 7 = PASS)

**Files:** none modified; only runs harness.
- Output: 3 result JSONs in `benchmarks/sequential-learning/results/v1.8-eval-C{0,1,3}-{adapter}/`

**Step 1: Run all three conditions in sequence**

```bash
cd C:/Users/skf_s/hippo

# C0 (no memory baseline)
mkdir -p benchmarks/sequential-learning/results/v1.8-eval-C0-none
node benchmarks/sequential-learning/run.mjs \
  --adapter baseline --n-seeds 20 --eval-strict --budget 2000 --restrict-late-to 4 \
  --output benchmarks/sequential-learning/results/v1.8-eval-C0-none/

# C1 (static)
mkdir -p benchmarks/sequential-learning/results/v1.8-eval-C1-static
node benchmarks/sequential-learning/run.mjs \
  --adapter static --n-seeds 20 --eval-strict --budget 2000 --restrict-late-to 4 \
  --output benchmarks/sequential-learning/results/v1.8-eval-C1-static/

# C3 (hippo + goal-stack)
mkdir -p benchmarks/sequential-learning/results/v1.8-eval-C3-hippo-goalstack
node benchmarks/sequential-learning/run.mjs \
  --adapter hippo --n-seeds 20 --eval-strict --budget 2000 --restrict-late-to 4 \
  --use-goal-stack \
  --output benchmarks/sequential-learning/results/v1.8-eval-C3-hippo-goalstack/
```

Expected wall-times: C0 ~1 min, C1 ~1 min, C3 ~10-12 min (with goal-stack hooks).

**Step 2: Hook-failure gate**

After C3 completes, check `hookFailures.push` and `hookFailures.complete` in the JSON. If either > 0, the run is contaminated. STOP, fix root cause, restart C3 (do NOT proceed with contaminated data).

**Step 3: Commit results**

```bash
git -C C:/Users/skf_s/hippo add benchmarks/sequential-learning/results/v1.8-eval-C0-none/ \
  benchmarks/sequential-learning/results/v1.8-eval-C1-static/ \
  benchmarks/sequential-learning/results/v1.8-eval-C3-hippo-goalstack/
git -C C:/Users/skf_s/hippo commit -m "eval(v1.8): C0 + C1 + C3 results on adversarial workload (hooks {OK|N failures})"
```

---

## Task 9: Write the result doc (descriptive characterisation, two-section format, sign-only diff reporting)

**Files:**
- Create: `docs/evals/2026-05-09-v1.8.0-adversarial-eval-result.md`

**Step 1: Compute workload baseline (C0/C1/C2) phase rates + C3 lattice histogram + sign-only direction count**

Use existing `aggregate.mjs::aggregatePhases` for per-condition phase rates. For C3 lattice histogram and sign-only direction counts, write a small script `tools/v1.8-summarise.mjs`:

```javascript
// tools/v1.8-summarise.mjs
import { readFileSync } from 'node:fs';

const c2 = JSON.parse(readFileSync('benchmarks/sequential-learning/results/v1.8-eval-C2-hippo-base/<latest>.json'));
const c3 = JSON.parse(readFileSync('benchmarks/sequential-learning/results/v1.8-eval-C3-hippo-goalstack/<latest>.json'));

// Per-seed late values (lattice ∈ {0, 0.25, 0.5, 0.75, 1.0})
const c2Late = c2.seeds.map((s) => s.phases.late);
const c3Late = c3.seeds.map((s) => s.phases.late);

// C3 per-seed lattice histogram
const histogram = { 0: 0, 0.25: 0, 0.5: 0, 0.75: 0, 1.0: 0 };
for (const v of c3Late) histogram[v]++;
console.log('C3 lattice histogram:', histogram);

// Sign-only seed-pair direction count
let lower = 0, higher = 0, tied = 0;
for (let i = 0; i < c2Late.length; i++) {
  if (c3Late[i] < c2Late[i]) lower++;
  else if (c3Late[i] > c2Late[i]) higher++;
  else tied++;
}
console.log('Sign-only direction:', { STRICTLY_LOWER: lower, STRICTLY_HIGHER: higher, TIED: tied });
```

Run + capture output for the result doc.

**Step 2: Fill in the pre-registered result template**

Use the template verbatim from this plan (the **two-section** structure). Fill {SLOTS} from JSON outputs and aggregate computations. The two sections MUST be non-adjacent — separate them with the verbatim sentence "Do not subtract across sections" each time.

**No magnitude framing.** Pre-publish self-check (must run before Task 12 outside voice): grep the result doc for forbidden language:

```bash
grep -nE "(Δ\s*=|\b\d+pp|−\d+pp|pp lift|pp drop|≥\s*\d+pp|discriminating|discrimination|magnitude|median.{0,20}pp|lift|reduction|improvement)" docs/evals/2026-05-09-v1.8.0-adversarial-eval-result.md | grep -v -i "retracted"
```

Expected: zero hits. (Lines mentioning "retracted" are allowlisted.)

**Step 3: Answer the 5 outside-voice questions verbatim** (per outside voice D3) in the dedicated subsection of the result doc. Each answer 1-2 sentences. NO magnitude framing.

**Step 4: Reproducibility anchors**

Include all anchors per the plan template.

**Step 5: Commit**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.8.0-adversarial-eval-result.md
git -C C:/Users/skf_s/hippo commit -m "docs(evals): v1.8.0 descriptive characterisation result (sign-only, two-section)"
```

---

## Task 10: Update CHANGELOG / README / ROADMAP-RESEARCH / TODOS / docs/RETRACTION.md

**Files:**
- Modify: `CHANGELOG.md` (insert v1.8.0 entry above v1.7.9)
- Modify: `README.md` (add "What's new in v1.8.0" above v1.7.9 block + update sequential-learning benchmark section to mention 13 categories with 3 adversarial)
- Modify: `ROADMAP-RESEARCH.md` (B3 Status update: v1.8.0 result; close the pre-committed escalation chain over workload knobs; cite v1.9 LongMemEval pre-commitment)
- Modify: `TODOS.md` (close `[ ] v1.8.0 adversarial trap categories` line; spawn v1.9 LongMemEval cross-validation `[ ]` line)
- Modify: `docs/RETRACTION.md` (append "## v1.8.0 outcome" subsection: did the guard hold? + name the v1.9 direction)

CHANGELOG / README / ROADMAP-RESEARCH / TODOS / RETRACTION updates use sign-only direction reporting + workload-validity verdict ONLY. **No "lift", no pp, no median, no juxtaposition.**

**Commit:**

```bash
git -C C:/Users/skf_s/hippo add CHANGELOG.md README.md ROADMAP-RESEARCH.md TODOS.md docs/RETRACTION.md
git -C C:/Users/skf_s/hippo commit -m "docs: v1.8.0 outcome across CHANGELOG/README/ROADMAP-RESEARCH/TODOS/RETRACTION"
```

---

## Task 11: Bump version 1.7.9 → 1.8.0 across 5 manifests

Same pattern as v1.7.9 Task 10. Edit `package.json`, `openclaw.plugin.json`, `extensions/openclaw-plugin/package.json`, `extensions/openclaw-plugin/openclaw.plugin.json`, `src/version.ts`. Sync `package-lock.json` via `npm install`. Verify no stragglers.

```bash
git -C C:/Users/skf_s/hippo add package.json package-lock.json openclaw.plugin.json extensions/openclaw-plugin/package.json extensions/openclaw-plugin/openclaw.plugin.json src/version.ts
git -C C:/Users/skf_s/hippo commit -m "chore: bump to v1.8.0 (minor — adversarial-categories release)"
```

---

## Task 12: Run full test suite

```bash
cd C:/Users/skf_s/hippo
npm run build
npx vitest run 2>&1 | tee /tmp/v1.8.0-vitest.log
```

Expected: ALL passing (1480 + new tests = ~1495+), 0 regressions vs v1.7.9. Capture actual count.

---

## Task 13: Pre-publish staged-diff outside voice (RETRACTION.md compliance audit, regex hardened)

**Step 1: Stage diff for review**

```bash
git -C C:/Users/skf_s/hippo log --oneline master..feat/v1.8.0-adversarial-categories
git -C C:/Users/skf_s/hippo diff master..feat/v1.8.0-adversarial-categories --stat
```

**Step 2: Dispatch one `senior-code-reviewer` agent against the staged diff**

Brief (verbatim, magnitude grep regex hardened per outside voice E2):

> Verify v1.8.0 adversarial-categories execution matches the plan at `docs/plans/2026-05-09-v1.8.0-adversarial-categories.md` AND satisfies the `docs/RETRACTION.md` magnitude-smuggling guard.
>
> 1. **No magnitude framing in any new prose.** Grep with hardened regex:
>    `(Δ\s*=|\b\d+\s*pp|−\d+pp|pp lift|pp drop|≥\s*\d+pp|magnitude|discriminating|discrimination|median.{0,20}pp|lift|reduction|improvement|5-number summary|paired diff)`.
>    **Allowlist:** any line containing the word `retracted` within 10 characters of the match is a legitimate retraction citation and is NOT a finding.
>    **Any other hit in NEW prose is P0.**
> 2. **Each new artifact contains the verbatim "This release does not re-assert the retracted −10pp magnitude."** Required surfaces: prereg, claim inventory, result.md, CHANGELOG entry, README "What's new in v1.8.0", docs/RETRACTION.md "## v1.8.0 outcome" subsection.
> 3. **Each new artifact references `docs/RETRACTION.md`.** Same surfaces.
> 4. **Pre-registration anchored before C3 ran.** Verify Task 1 commit (categories) precedes Task 7 (C2 preflight) and Task 8 (C3) in the branch's git log.
> 5. **Jaccard verification output committed.** `docs/evals/2026-05-09-v1.8.0-jaccard-verification.txt` exists with literal output. Max Jaccard < 0.30 (TIGHTENED from 0.40).
> 6. **BM25-mismatch test exists and passes.** `tests/sl-bm25-mismatch.test.ts` is in the diff and `npx vitest run` includes its results.
> 7. **Workload size/encounters consistent.** 13 categories × correct trap encounters per category = 31. 62 task slots. tests/sl-traps-v1.8.test.ts asserts these PLUS uniform distribution PLUS exact positions {11, 21, 33, 41, 51, 59} PLUS existing-10 PRNG-stability.
> 8. **No SUPPORTED / NOT_SUPPORTED templates** in any v1.8 artifact (those were magnitude-decision-rule terms).
> 9. **C3 results MUST exist iff C2 sanity passed.** If Task 7 SANITY_FAILed, no C3 result JSONs in the diff.
> 10. **Version bump hits all 5 manifests, no 1.7.9 stragglers.**
> 11. **Result doc uses two-section format** (workload baseline / C3 characterisation) with the verbatim "Do not subtract across sections" sentence in EACH section.
> 12. **C3 reporting is sign-only.** Lattice histogram + STRICTLY_LOWER/STRICTLY_HIGHER/TIED counts ONLY. NO median pp, NO 5-number summary, NO Δ.
> 13. **v1.9 direction pre-committed in the prereg** (LongMemEval R@5 cross-validation per v1.7.7 prereg line 23).
> 14. **Pilot smoke test + iteration log committed before pre-registration locked.**
> 15. **5 outside-voice questions answered verbatim in the result doc.**
>
> Cap response at 400 words. Format: numbered findings list. If clean: "Clean. No findings. /publish-repo authorised." Read-only verification.

**Step 3: Address P0/P1 findings before /publish-repo.**

---

## Task 14: /self-review → /review → /ship-check → /publish-repo

**Step 1: /self-review** (manual walk through commits, verify each criterion of plan-execution match).

**Step 2: /review** (full skill flow — covered by Task 13 staged-diff outside voice).

**Step 3: /ship-check** (formal ship-verdict block, 8/8 due-diligence checklist).

**Step 4: /publish-repo:**

- Bump confirmed at 1.8.0 across 5 manifests.
- `npm run build` succeeds; `npx vitest run` passes; capture actual test count.
- Commit chain merged to `master` via `--no-ff`.
- `git tag v1.8.0 && git push origin master && git push origin v1.8.0`.
- `npm publish`.
- `gh release create v1.8.0 --title "v1.8.0 — adversarial trap categories (descriptive characterisation; no magnitude)" --notes-file /tmp/v1.8.0-notes.md --latest`.
- `npm install -g hippo-memory@1.8.0` then `hippo --version` → `1.8.0`.

**Step 5: Dual `hippo remember`**

```bash
hippo remember "v1.8.0 shipped: adversarial trap categories release; mechanism characterisation only (no magnitude per RETRACTION.md guard); workload-validity verdict {PASS|FAIL}; v1.9 pre-committed to LongMemEval R@5 cross-validation" --tags workflow-discipline,retraction-compliance,v1.8

hippo remember "decision precedent: 'mechanism characterisation' must be operationally defined (sanity verdict + lattice histogram + sign-only direction count) and result reports must be two-section / non-juxtaposed / sign-only — otherwise paired-diff median in pp units smuggles magnitude back. Outside voice catches this by default; do not skip the staged-diff RETRACTION.md compliance audit." --tags decision-precedent,retraction-design
```

**Step 6: Decision-log update commit** (append "## Outside voice resolutions (2026-05-09)" subsection to this plan).

---

## Risk register

| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Outside voice flags v1.8 framing as soft magnitude smuggling | Low | High | Plan pre-bakes RETRACTION.md compliance into every artifact: forbidden-language list, two-section result template, sign-only C3 reporting, hardened grep regex with retraction-citation allowlist. Task 13 staged-diff outside voice catches drift. |
| 3 chosen adversarial categories fail the <0.30 Jaccard threshold | Low | Medium | Task 2 runs the verification BEFORE Task 7 C2 preflight. If fails, rewrite lessons; iterate until pass. Iteration log lives in the prereg. |
| Categories too easy → C2 saturates at 0% → SANITY_FAIL chain-exhausted | Medium | Medium | Task 1 Step 0 pilot smoke test runs BEFORE pre-registration locks. If all 3 saturate at 0%, replace easiest with `seedstart_state_pollution` or similar. v1.9 LongMemEval pre-commitment provides a defined next direction (not face-saving). |
| C3 hook failures (push or complete > 0) | Low | High | `--eval-strict` mode enforces fail-fast. v1.7.8 instance-state hoist (P1-4) makes the adapter race-condition-safe. Hook-failure gate post-C3 stops the run if contaminated. |
| Late-region position bias toward magnitude visibility | Low | Medium | Per outside voice B2: positions {11, 21, 33, 41, 51, 59} are uniformly distributed across early/mid/late. Last-4 trap encounters are mixed (2 existing + 2 new), NOT pure-adversarial. Test asserts exact positions. |
| BM25 trivially matches new lessons via shared engineering jargon despite low Jaccard | Low | Low | Task 3 BM25-mismatch sim test asserts existing recall queries do NOT top-1-rank new-category lessons. Independent verification beyond Jaccard. |
| Pre-commit hook failures | Low | Medium | Per CLAUDE.md "NEVER use `--no-verify`". If hook fails, fix the failure; never bypass. |
| Subset cherry-picking (existing-10 vs all-13 vs late-4) | Low | High | Per outside voice A5: prereg's "Subset reporting lock" pre-registers late-phase only, no subset slicing, no per-version cite. Result doc enforces. |
| Result doc median-pp / 5-number summary / juxtaposition leak | Low | High | Pre-publish self-check grep (Task 9 Step 2) + Task 13 staged-diff outside voice both flag any leak. Two-section template + sign-only reporting prevent at authoring time. |
| v1.9 direction post-hoc justification | Low | Medium | Per outside voice C1: prereg's "Pre-committed v1.9 direction" subsection names LongMemEval R@5 cross-validation BEFORE v1.8 runs. v1.9 cannot be retroactively justified by v1.8 outcome. |

---

## Decision log

This plan operates under TWO strict pre-commitment regimes:

1. **v1.7.7 prereg "Pre-committed v1.8 escalation constraints"** — 6 numbered constraints inherited verbatim and mapped to plan tasks.
2. **v1.7.9 `docs/RETRACTION.md` magnitude-smuggling guard** — descriptive characterisation framing only; sign-only diff reporting; no SUPPORTED / NOT_SUPPORTED templates; no magnitude verdicts; no median-pp; no 5-number summary in pp units; no juxtaposition; no "discriminating".

**Why these 3 adversarial categories:** vocabulary disjointness from v1.5; common real-world traps; pilot smoke test confirms at least one produces non-zero late traps for C2.

**Why uniform position distribution (not late-clustered):** late-clustering optimises for magnitude visibility, which is exactly what "characterisation only" should NOT do. Positions {11, 21, 33, 41, 51, 59} distribute new traps uniformly across early/mid/late.

**Why <0.30 Jaccard threshold (TIGHTENED from 0.40):** v1.7.7 prereg's 0.40 was an inherited default; tightening to 0.30 makes the adversarial claim stronger. Justification documented in prereg.

**What this plan does NOT do:**
- Does NOT re-assert the retracted −10pp magnitude.
- Does NOT pre-register a magnitude pass/fail decision rule.
- Does NOT use SUPPORTED / NOT_SUPPORTED templates.
- Does NOT cross-cite v1.7.5/6/7 late-phase rates as comparable to v1.8.
- Does NOT report median paired diffs, 5-number summaries on diffs, or any juxtaposed per-condition late means.
- Does NOT swap out existing 10 categories.
- Does NOT use "discriminating" / "discrimination room" / "lift" / "improvement" / "reduction" applied to the mechanism's effect.

**Outside voice questions to brief** (5 questions, answered verbatim in result.md per outside voice D3):
1. Are the 3 chosen categories sufficiently adversarial (per pilot smoke test outcome)?
2. Did uniform position distribution actually hold in the executed task generation?
3. Did the descriptive characterisation framing hold across all artifacts (prereg + inventory + result + CHANGELOG + README + RETRACTION update)?
4. Does the workload-validity verdict (PASS or FAIL) inform v1.9 direction (is LongMemEval still the right pre-committed next step)?
5. Did Task 13 staged-diff outside voice find any RETRACTION.md compliance leaks?

---

## Time budget

- Task 1 (3 categories + pilot + uniform positions + commit): ~45 min (incl. pilot iteration cycle).
- Task 2 (Jaccard with Porter-stem + extended stop-words + tests + commit): ~30 min.
- Task 3 (BM25-mismatch test + commit): ~20 min.
- Task 4 (prereg with 3 new subsections: v1.9 commitment, subset lock, OV questions): ~40 min.
- Task 5 (claim inventory + retraction-compliance grep with allowlist): ~20 min.
- Task 6 (test additions + sweep + PRNG-stability): ~30 min.
- Task 7 (C2 preflight): ~10 min subprocess.
- Task 8 (C0 + C1 + C3): ~12-15 min subprocess.
- Task 9 (result doc with two-section + sign-only + 5 OV answers): ~45 min.
- Task 10 (CHANGELOG + README + ROADMAP-RESEARCH + TODOS + RETRACTION): ~45 min.
- Task 11 (version bump): ~5 min.
- Task 12 (test suite): ~3 min subprocess.
- Task 13 (staged-diff outside voice): ~15 min subprocess.
- Task 14 (ship chain + dual hippo remember + decision-log update): ~30 min.
- **Total wall time: ~5-6 hours.** Subprocess work: ~40-50 min.

---

## Pre-registration commit ordering (CRITICAL — pre-commit lock)

1. Branch from master at HEAD (post-v1.7.9).
2. Task 1 (a) — adversarial categories committed → commit (Task 1 SHA = "categories authored BEFORE C3" anchor).
3. Task 1 (b) — pilot smoke test result + iteration log → commit.
4. Task 2 — Jaccard script + verification output → commit.
5. Task 3 — BM25-mismatch test → commit.
6. Task 4 — prereg → commit (references Task 1 SHA + Task 2 SHA + Task 3 SHA).
7. Task 5 — claim inventory + retraction-compliance grep → commit.
8. Task 6 — tests → commit.
9. Task 7 — C2 sanity preflight result → commit (PASS or FAIL).
10. Task 8 — C0 + C1 + C3 results (only if Task 7 PASS) → commit.
11. Task 9 — result.md → commit.
12. Task 10 — CHANGELOG / README / ROADMAP-RESEARCH / TODOS / RETRACTION.md → commit.
13. Task 11 — version bump → commit.
14. Task 12 — test suite passes (no commit; verification only).
15. Task 13 — staged-diff outside voice (no commit; gating only).
16. Task 14 — `/self-review` → `/review` → `/ship-check` → `/publish-repo` (+ dual `hippo remember` + decision-log update).

**14 atomic commits + decision-log update commit = 15 commits total** on `feat/v1.8.0-adversarial-categories`. Merge to `master` via `--no-ff` after `/ship-check` returns "Ship it".

**The Task 1 SHA is load-bearing.** It is the audit proof that adversarial categories were authored BEFORE C3 ran. The result doc, the prereg, and the CHANGELOG entry must all cite this SHA.

---

## Outside voice resolutions (2026-05-09, post-publish appendix)

### Pre-execution outside voice (Task 0)

Two parallel `senior-code-reviewer` agents on plan v1: in-house arch + adversarial RETRACTION-compliance proxy. **22 findings**: 7 P0, 8 P1, 6 P2, 1 P3. All accepted into plan v2. Disposition table summary:

| Group | Items | Disposition |
|-------|-------|-------------|
| A. Magnitude smuggling via reporting structure | A1-A5 | **All accepted** — result template rewritten: sign-only direction count + lattice histogram only; two-section format with verbatim "Do not subtract"; "discriminating"/"discrimination" globally replaced with "non-saturation". |
| B. Workload design | B1-B4 | **All accepted** — math fix (7 traps in 50-62 not 12); uniform position distribution {11, 21, 33, 41, 51, 59} (not late-clustered); Task 1 Step 0 pilot smoke test added (passed at 25%); Task 5 tests tightened to exact assertions. |
| C. v1.9 not pre-registered | C1 | **Accepted** — "Pre-committed v1.9 direction" subsection added: LongMemEval R@5 cross-validation. |
| D. Operational definitions | D1-D3 | **All accepted** — "characterise" operationally defined; outside voice questions answered verbatim in result.md. |
| E. Audit/verification | E1-E4 | **All accepted** — Jaccard threshold tightened 0.40→0.30; Porter-stem + extended stop-words; magnitude grep retraction-citation allowlist; existing-10 PRNG-stability test; BM25-mismatch test. |
| F. Cosmetic | F1 | **Accepted** — commit count reconciled. |

### Pre-publish staged-diff outside voice (Task 13)

One `senior-code-reviewer` against `feat/v1.8.0-adversarial-categories` (13 commits, 24 files, 2168 insertions / 46 deletions). Verdict: **"Clean. No findings. /publish-repo authorised."** All 16 plan-match criteria + non-negotiables satisfied.

### Ship outcome (2026-05-09)

- Branch merged to master via `--no-ff` (commit `cb4fbfa`).
- Tag `v1.8.0` pushed.
- npm: `hippo-memory@1.8.0` published.
- GitHub Release: `https://github.com/kitfunso/hippo-memory/releases/tag/v1.8.0` (latest).
- Global install: `hippo --version` → `1.8.0` ✓.
- Tests: 1508 passing, 2 skipped, 0 failed, 0 regressions vs v1.7.9.
- Dual `hippo remember`: workflow-discipline + decision-precedent entries captured.

### Pre-committed v1.9 direction (named BEFORE v1.8 ran)

LongMemEval R@5 cross-validation. The v1.8 PASS verdict + C3=C2 sign-only result does not change this pre-commitment. v1.9 PRE-REG to be drafted in a separate plan before v1.9 implementation begins.
