# v1.7.5 Sequential-Learning Adapter Contract Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Extend the sequential-learning benchmark adapter contract with `pushGoal` / `completeGoal` hooks, build a multi-seed eval harness with REAL seed-driven variance, run a pre-registered 20-seed eval, and either DEMONSTRATE or HONESTLY RETIRE the −10pp trap-rate lift claim from the v0.38 B3 dlPFC ship.

**Architecture (revised post outside-voice review, 2026-05-07):** Three logical pieces. (1) Adapter contract extension — `interface.mjs` gains optional `pushGoal/completeGoal` methods; `hippo.mjs` implements both via the existing `hippo goal push|complete` CLI; `run.mjs` simulator wraps each trap task. **Critical fix:** `simulate()` stores memories with `[category.id, ...category.tags, 'error']` so the goal name `category.id` matches a real tag (without this fix the boost would never fire). (2) Multi-seed harness with MEANINGFUL seed variance — `run.mjs` accepts `--seed N` and `--n-seeds N`. Variance source: random assignment of CATEGORIES to position-slots within each phase bucket (preserving the "trap appears early-then-later" invariant), NOT shuffling clean-vs-trap positions (which would be statistical theatre). Permutation-based paired-Δ CI in `aggregate.mjs`. (3) Pre-registered 20-seed eval with 4 conditions (none, static, hippo, hippo+goal-stack) — both CHANGELOG/README templates pre-committed, decision rule mechanical, retraction sweep covers `README.md`, `CHANGELOG.md`, `RESEARCH.md`, `ROADMAP-RESEARCH.md`, `TODOS.md`, and historical `docs/plans/2026-04-29-b3-dlpfc-depth.md`.

**Tech Stack:** Node 22+, ESM, no npm deps in benchmark code; `hippo` CLI on PATH for the `hippo` adapter; vitest for unit tests on the harness.

**Branch:** `feat/v1.7.5-sl-adapter` off `master`. Do not commit on master.

**Source item:** TODOS.md line 139-143 (`B3 follow-up: sequential-learning adapter contract → v1.7.5`). Last contract-shaped B3 follow-up. After this ships, only vlPFC interference remains in the B3 basket (deferred to v1.8.0).

**Pre-investigated facts (anchor before code; revised post-review):**

- `benchmarks/sequential-learning/adapters/interface.mjs` has 5 required methods today. New OPTIONAL hooks: `pushGoal`, `completeGoal`. Paired — supply both or neither.
- `benchmarks/sequential-learning/adapters/hippo.mjs` drives the CLI via `execSync` with `HOME`/`USERPROFILE` overrides. **Codex finding (P0):** isolation hole — `getGlobalRoot()` prefers `HIPPO_HOME` and falls back to `XDG_DATA_HOME` BEFORE `HOME`. If user has either set globally, eval contaminates real store. Adapter must explicitly set `HIPPO_HOME=<temp>` and unset `XDG_DATA_HOME` in child env.
- `benchmarks/sequential-learning/run.mjs::simulate` loops at line 110-158. For each trap task: recall → matched? → outcome(±) + store(lesson, tags+'error'). **Critical mechanism gap:** stored tags are `category.tags` (e.g. `['error-handling','exception','python']`) — they do NOT include the category id. So pushing goal name `'bare_except'` matches against `goalsByTag` keyed on memory tags and finds NOTHING. Fix: store with `[category.id, ...category.tags, 'error']` so the goal name matches.
- `task.trapCategory` IS the category id (e.g. `'bare_except'`). After the fix above, also push it as the goal name so id-as-tag matches.
- CLI `hippo goal push <name>` prints just the goal id (`g_<16-hex>`) via `console.log(goal.id)` (`src/cli.ts:4907`). One-line regex parse.
- CLI reads session via `resolveGoalSession(flags)` → `--session-id` flag OR `HIPPO_SESSION_ID` env. Adapter sets `HIPPO_SESSION_ID` on every exec from pushGoal through completeGoal AND the recall in between.
- v1.7.4 lifted the boost into `applyGoalStackBoost`. CLI recall (`src/cli.ts:988-1010`) applies it via `HIPPO_SESSION_ID`. With the tag fix above, the boost will fire.
- Today's benchmark is deterministic single-run. **Senior + codex both flag:** `traps.mjs` already places traps at FIXED positions (early/mid/late thirds preserved by construction). Naive "shuffle within thirds" reshuffles trap-vs-clean positions (clean = no-op) — variance is artificially zero. Real seed variance source: randomly assign which CATEGORY appears at which trap-slot within each phase, OR vary `recallQuery` per encounter, OR perturb the input `category.lesson` text slightly per seed. Choose one; document the choice.
- README's "78%→14%" is hippo WITHOUT goal-stack. The −10pp claim is an ADDITIONAL drop on top.
- **Power analysis** (codex + senior): late phase has ~7-9 trap encounters per seed → single-seed binomial SE ≈ 11pp. n=10 paired seeds is borderline. **Decision: N=20 default** (estimated +5 min runtime, well within budget).
- **Other locations of −10pp / goal-stack-lift claim** (must update on retraction): `ROADMAP-RESEARCH.md` (around line 156), `docs/plans/2026-04-29-b3-dlpfc-depth.md` (around line 1776), plus any matches surfaced by `grep -rn "10pp\|−10pp\|goal-stack lift" --include='*.md' .`. Pre-eval claim-inventory artifact captures all locations.

**Pre-flight (run once before Task 1):**

```bash
git status --short
git switch -c feat/v1.7.5-sl-adapter
npm install
hippo --version  # should be 1.7.4 (just shipped)
node benchmarks/sequential-learning/run.mjs --adapter hippo  # baseline check, single run, no goal-stack
```

Expected: `hippo` reports 1.7.4, baseline run reports SOME hit rate (the existing 78→14% number — confirm the harness still functions before changing it). **STOP if `hippo` is missing from PATH or the baseline run errors.**

---

## Task 1: Adapter contract extension — `pushGoal` / `completeGoal`

**Why:** Without these hooks, the goal-stack mechanism (the ENTIRE point of B3 dlPFC) cannot be exercised on the public sequential-learning benchmark. The boost lives in `applyGoalStackBoost` (shipped v1.7.4) but no benchmark adapter has wired it.

**Files:**
- Modify: `benchmarks/sequential-learning/adapters/interface.mjs` — extend type docstrings + `createAdapter` validator
- Modify: `benchmarks/sequential-learning/adapters/hippo.mjs` — implement `pushGoal` / `completeGoal` + thread `HIPPO_SESSION_ID` env
- Modify: `benchmarks/sequential-learning/run.mjs` — `simulate()` calls hooks when adapter has them; new `--use-goal-stack on/off` flag
- Create: `tests/sequential-learning-adapter-contract.test.ts` — unit tests for the new contract

### Step 1: Write the failing contract test

Create `tests/sequential-learning-adapter-contract.test.ts`:

```ts
/**
 * v1.7.5 — sequential-learning adapter contract gains optional
 * pushGoal/completeGoal hooks for B3 dlPFC goal-stack exercising.
 * Optional so non-goal-aware adapters keep working unchanged.
 */

import { describe, it, expect } from 'vitest';
import { createAdapter } from '../benchmarks/sequential-learning/adapters/interface.mjs';

describe('sequential-learning adapter contract (v1.7.5)', () => {
  const baseAdapter = {
    name: 'test',
    init: async () => {},
    store: async () => {},
    recall: async () => [],
    outcome: async () => {},
    cleanup: async () => {},
  };

  it('accepts an adapter without pushGoal/completeGoal (back-compat)', () => {
    expect(() => createAdapter(baseAdapter)).not.toThrow();
  });

  it('accepts an adapter with both pushGoal and completeGoal', () => {
    expect(() => createAdapter({
      ...baseAdapter,
      pushGoal: async () => 'g_1234567890abcdef',
      completeGoal: async () => {},
    })).not.toThrow();
  });

  it('rejects adapters that supply only pushGoal without completeGoal', () => {
    expect(() => createAdapter({
      ...baseAdapter,
      pushGoal: async () => 'g_x',
    })).toThrow(/completeGoal/);
  });

  it('rejects adapters that supply completeGoal without pushGoal', () => {
    expect(() => createAdapter({
      ...baseAdapter,
      completeGoal: async () => {},
    })).toThrow(/pushGoal/);
  });
});
```

Run: `npx vitest run tests/sequential-learning-adapter-contract.test.ts`
Expected: FAIL — current `createAdapter` doesn't enforce the pair invariant.

### Step 2: Update `interface.mjs`

In `benchmarks/sequential-learning/adapters/interface.mjs`:

```js
/**
 * @typedef {Object} MemoryAdapter
 * @property {string} name
 * @property {() => Promise<void>} init
 * @property {(content: string, tags: string[]) => Promise<void>} store
 * @property {(query: string) => Promise<RecallResult[]>} recall
 * @property {(good: boolean) => Promise<void>} outcome
 * @property {() => Promise<void>} cleanup
 *
 * v1.7.5 — optional B3 dlPFC goal-stack hooks. Either supply BOTH or NEITHER.
 * @property {((name: string) => Promise<string>)} [pushGoal] - Push an active
 *   goal at task start. Returns the goal id (opaque string). Adapter must
 *   thread the goal's session id into subsequent recall calls so the
 *   goal-stack boost activates.
 * @property {((id: string, good: boolean) => Promise<void>)} [completeGoal] -
 *   Complete the goal at task end. `good` is the outcome (true = trap
 *   avoided, false = trap hit). Adapter MAY propagate strength multipliers
 *   to recalled memories per its own internal contract.
 */

export function createAdapter(adapter) {
  const required = ['name', 'init', 'store', 'recall', 'outcome', 'cleanup'];
  for (const key of required) {
    if (!(key in adapter)) {
      throw new Error(`Adapter missing required field: ${key}`);
    }
  }
  for (const key of required.slice(1)) {
    if (typeof adapter[key] !== 'function') {
      throw new Error(`Adapter.${key} must be a function`);
    }
  }
  // v1.7.5 -- pushGoal/completeGoal must be supplied together.
  const hasPush = 'pushGoal' in adapter;
  const hasComplete = 'completeGoal' in adapter;
  if (hasPush !== hasComplete) {
    throw new Error(
      hasPush
        ? 'Adapter supplies pushGoal but missing completeGoal -- the v1.7.5 B3 hooks are paired'
        : 'Adapter supplies completeGoal but missing pushGoal -- the v1.7.5 B3 hooks are paired',
    );
  }
  if (hasPush && typeof adapter.pushGoal !== 'function') {
    throw new Error('Adapter.pushGoal must be a function');
  }
  if (hasComplete && typeof adapter.completeGoal !== 'function') {
    throw new Error('Adapter.completeGoal must be a function');
  }
  return adapter;
}
```

Run: `npx vitest run tests/sequential-learning-adapter-contract.test.ts`
Expected: PASS — 4 cases.

### Step 3: Implement `pushGoal` / `completeGoal` in `hippo.mjs`

In `benchmarks/sequential-learning/adapters/hippo.mjs`. **Three codex/senior fixes baked in:** (a) isolate `HIPPO_HOME` + unset `XDG_DATA_HOME`, (b) hard-fail (no swallow) on push errors, (c) clear `_sessionId` in `cleanup()` so cross-task state can never leak.

```js
// v1.7.5 -- session id stays stable across the task lifespan. Set via env
// on every hippo exec so goal push/complete and recall share state.
let _sessionId = null;
let _pushedCount = 0;
let _completedCount = 0;

function hippoExec(storeDir, args) {
  try {
    const result = execSync(`hippo ${args}`, {
      cwd: storeDir,
      env: {
        ...process.env,
        // v1.7.5 codex P0 -- override HIPPO_HOME (wins over HOME in
        // getGlobalRoot) and clear XDG_DATA_HOME so neither can leak the
        // user's real global store into the eval.
        HIPPO_HOME: storeDir,
        HOME: storeDir,
        USERPROFILE: storeDir,
        XDG_DATA_HOME: '',
        ...(_sessionId ? { HIPPO_SESSION_ID: _sessionId } : {}),
      },
      encoding: 'utf-8',
      timeout: 15_000,
      stdio: ['pipe', 'pipe', 'pipe'],
    });
    return result.trim();
  } catch (err) {
    if (err.stdout) return err.stdout.trim();
    return null;
  }
}

export default createAdapter({
  name: 'Hippo',

  async pushGoal(name) {
    _sessionId = `bench-${Date.now()}-${Math.floor(Math.random() * 1e6)}`;
    const out = hippoExec(this._storeDir, `goal push ${name}`);
    if (!out) {
      _sessionId = null;
      // v1.7.5 codex P1 -- HARD FAIL. Do not swallow. Eval mode wants the
      // run to abort if the mechanism can't even fire.
      throw new Error(`hippo goal push failed for name='${name}'`);
    }
    const match = out.match(/g_[0-9a-f]{16}/);
    if (!match) {
      _sessionId = null;
      throw new Error(`hippo goal push output did not contain a goal id: '${out}'`);
    }
    _pushedCount++;
    return match[0];
  },

  async completeGoal(id, good) {
    const outcome = good ? '1.0' : '0.0';
    hippoExec(this._storeDir, `goal complete ${id} --outcome ${outcome}`);
    _completedCount++;
    _sessionId = null;
  },

  async cleanup() {
    // v1.7.5 codex P1 -- always clear cross-task state, even after exceptions.
    _sessionId = null;
    if (this._storeDir && existsSync(this._storeDir)) {
      rmSync(this._storeDir, { recursive: true, force: true });
    }
    this._storeDir = null;
  },

  // Expose counters so the runner can assert non-zero in eval mode.
  _stats() { return { pushed: _pushedCount, completed: _completedCount }; },

  // ... existing init/store/recall/outcome ...
});
```

**Pre-flight before this step:** `grep -n "getGlobalRoot\|HIPPO_HOME\|XDG_DATA_HOME" src/store.ts src/db.ts` to confirm the precedence (`HIPPO_HOME > XDG_DATA_HOME > HOME/.hippo`). Adjust the env shape above if precedence differs.

### Step 4: Update `run.mjs` simulator to call hooks (with tag-fix + finally cleanup + hard-fail)

**THREE post-review fixes baked in here:**
- (P0) Store memories with `[category.id, ...category.tags, 'error']` so the goal name `category.id` matches a real tag (without this the boost matches zero memories).
- (P1) `completeGoal` lives in a `finally` so it always fires, even after exceptions in recall/outcome.
- (P1) In eval mode (when `useGoalStack`), hook failures HARD-FAIL with a count threshold rather than swallow.

```js
async function simulate(adapter, tasks, opts = {}) {
  await adapter.init();
  const useGoalStack = opts.useGoalStack === true && typeof adapter.pushGoal === 'function';
  const evalStrict = opts.evalStrict === true; // hard-fail on hook errors

  const results = [];
  let pushFailures = 0;
  let completeFailures = 0;

  for (const task of tasks) {
    if (!task.trapCategory) {
      results.push({ taskId: task.id, trapCategory: null, trapHit: false, memoryRecalled: false });
      continue;
    }

    const category = getTrapCategory(task.trapCategory);

    let goalId = null;
    if (useGoalStack) {
      try {
        goalId = await adapter.pushGoal(task.trapCategory);
      } catch (err) {
        pushFailures++;
        if (evalStrict) throw new Error(`evalStrict: pushGoal failed task ${task.id}: ${err.message}`);
        console.error(`pushGoal failed for task ${task.id}: ${err.message}`);
      }
    }

    let matched = false;
    try {
      const recalled = await adapter.recall(task.recallQuery);
      const top5 = recalled.slice(0, 5);
      matched = isRecalled(top5, category);

      if (matched) {
        await adapter.outcome(true);
        results.push({ taskId: task.id, trapCategory: task.trapCategory, trapHit: false, memoryRecalled: true });
      } else {
        await adapter.outcome(false);
        // v1.7.5 P0 -- include category.id as a tag so the goal-stack boost
        // can match it. Pre-v1.7.5 stored only category.tags which lacked
        // the id, making the goal-name->tag match return zero.
        await adapter.store(category.lesson, [task.trapCategory, ...category.tags, 'error']);
        results.push({ taskId: task.id, trapCategory: task.trapCategory, trapHit: true, memoryRecalled: false });
      }
    } finally {
      // v1.7.5 P1 -- complete the goal even if recall/outcome threw.
      if (useGoalStack && goalId) {
        try {
          await adapter.completeGoal(goalId, matched);
        } catch (err) {
          completeFailures++;
          if (evalStrict) throw new Error(`evalStrict: completeGoal failed task ${task.id}: ${err.message}`);
          console.error(`completeGoal failed for task ${task.id}: ${err.message}`);
        }
      }
    }
  }

  await adapter.cleanup();
  // v1.7.5 P1 -- if eval-strict, fail hard if any hook failed silently
  // before the strict gate would have caught it.
  if (evalStrict && (pushFailures > 0 || completeFailures > 0)) {
    throw new Error(`evalStrict: ${pushFailures} push failures, ${completeFailures} complete failures`);
  }
  return { results, hookFailures: { push: pushFailures, complete: completeFailures } };
}
```

**Caller change:** `runAdapter` reads `simulate()` return `{results, hookFailures}` instead of just `results`. Failure counts written into the output JSON per-seed.

Add `--use-goal-stack` CLI flag in `parseArgs()`:

```js
} else if (args[i] === '--use-goal-stack') {
  opts.useGoalStack = true;
}
```

Pass `useGoalStack` from `opts` into the `simulate(adapter, tasks, { useGoalStack: opts.useGoalStack })` call.

### Step 5: Smoke run both modes

```bash
node benchmarks/sequential-learning/run.mjs --adapter hippo \
  --output results/v1.7.5-task1-no-goalstack/
node benchmarks/sequential-learning/run.mjs --adapter hippo --use-goal-stack \
  --output results/v1.7.5-task1-with-goalstack/
```

Expected: both runs complete cleanly. Phase rates differ (don't have to differ in the right direction yet — Task 3 is where the eval happens). Confirm `--use-goal-stack` actually pushes goals: grep stderr for "pushGoal failed" → should be empty.

### Step 6: Commit

```bash
git add benchmarks/sequential-learning/adapters/interface.mjs \
  benchmarks/sequential-learning/adapters/hippo.mjs \
  benchmarks/sequential-learning/run.mjs \
  tests/sequential-learning-adapter-contract.test.ts
git commit -m "feat: sequential-learning adapter pushGoal/completeGoal hooks (v1.7.5 task 1)"
```

**Success criteria:**
- `interface.mjs` accepts adapters with both hooks OR neither; rejects partial.
- `hippo.mjs` implements both. `cleanup()` clears `_sessionId` and removes temp dir even after exceptions.
- `run.mjs::simulate` stores memories with `[category.id, ...category.tags, 'error']` (the boost-firing fix).
- `run.mjs::simulate` returns `{results, hookFailures}` and supports `evalStrict` mode for hard-fail on hook errors.
- Smoke run with `--use-goal-stack` produces non-zero `goal_recall_log` rows in the temp store. Add a smoke-step assertion: after a smoke run, exec `hippo --json memory list --tag goal-recall` (or query the temp DB directly) and confirm at least one row exists. **If zero rows: STOP — the boost is not firing and the eval cannot run.**
- 4 contract tests pass + 1 boost-fires integration test pass (push goal `bare_except`, recall, assert `goal_recall_log` has a row).
- One atomic commit.

---

## Task 2: Multi-seed eval harness

**Why:** A single deterministic run produces one number. Any "10pp lift" claim from a single run is anecdote, not evidence. The eval needs ≥10 seeded runs per condition, mean ± 95% CI.

**Files:**
- Modify: `benchmarks/sequential-learning/traps.mjs` — `generateTasks(seed?)` accepts an optional seed, shuffles deterministically when set
- Modify: `benchmarks/sequential-learning/run.mjs` — `--seed N`, `--n-seeds N`, multi-run loop, aggregate output JSON
- Create: `benchmarks/sequential-learning/aggregate.mjs` — pure aggregation helpers (mean, std, CI)
- Create: `tests/sequential-learning-aggregate.test.ts` — unit tests for the aggregator

### Step 1: Write failing tests for the aggregator

Create `tests/sequential-learning-aggregate.test.ts`:

```ts
/**
 * v1.7.5 -- multi-seed aggregation helpers. Pure math, no DB.
 */

import { describe, it, expect } from 'vitest';
import {
  mean, stdDev, ciHalfWidth95, aggregatePhases,
} from '../benchmarks/sequential-learning/aggregate.mjs';

describe('aggregate helpers (v1.7.5)', () => {
  it('mean of [0.1, 0.2, 0.3] = 0.2', () => {
    expect(mean([0.1, 0.2, 0.3])).toBeCloseTo(0.2, 5);
  });

  it('stdDev of identical values is 0', () => {
    expect(stdDev([0.5, 0.5, 0.5])).toBeCloseTo(0, 5);
  });

  it('ciHalfWidth95 for 10-sample [0.10..0.20] ≈ 0.0315 (t-crit 2.262 / sqrt(10))', () => {
    // Hand-verified: mean=0.15, var=0.0175/9≈0.001944, sd≈0.04410,
    // CI = 2.262 * 0.04410 / sqrt(10) ≈ 0.0315 (NOT 0.026 — that was wrong in plan v1).
    expect(ciHalfWidth95([0.10, 0.15, 0.20, 0.10, 0.15, 0.20, 0.10, 0.15, 0.20, 0.15])).toBeCloseTo(0.032, 2);
  });

  it('ciHalfWidth95 returns 0 for n=1 (no sample variance estimable)', () => {
    expect(ciHalfWidth95([0.5])).toBe(0);
  });

  it('ciHalfWidth95 rejects samples below n=5 by returning 0 (codex P2 — n=2/3/4 t-crit gives nonsense CI)', () => {
    expect(ciHalfWidth95([0.5, 0.6])).toBe(0);
    expect(ciHalfWidth95([0.5, 0.6, 0.7])).toBe(0);
    expect(ciHalfWidth95([0.5, 0.6, 0.7, 0.8])).toBe(0);
    // n=5 is the floor where CI is reportable
    expect(ciHalfWidth95([0.5, 0.5, 0.5, 0.5, 0.5])).toBe(0); // zero variance still works
  });

  it('aggregatePhases averages early/mid/late across seeds', () => {
    const seeds = [
      { early: 0.80, mid: 0.20, late: 0.10 },
      { early: 0.70, mid: 0.30, late: 0.15 },
      { early: 0.75, mid: 0.25, late: 0.05 },
    ];
    const agg = aggregatePhases(seeds);
    expect(agg.early.mean).toBeCloseTo(0.75, 5);
    expect(agg.mid.mean).toBeCloseTo(0.25, 5);
    expect(agg.late.mean).toBeCloseTo(0.10, 5);
    expect(agg.late.ci95).toBeGreaterThan(0);
  });
});
```

Run: `npx vitest run tests/sequential-learning-aggregate.test.ts`
Expected: FAIL — `aggregate.mjs` does not exist.

### Step 2: Implement `aggregate.mjs`

```js
// benchmarks/sequential-learning/aggregate.mjs
// v1.7.5 -- pure aggregation helpers for multi-seed runs. No imports beyond
// Node built-ins. Keep this file dependency-free so the benchmark runs on a
// vanilla Node 22+ install.

export function mean(xs) {
  if (xs.length === 0) return 0;
  return xs.reduce((a, b) => a + b, 0) / xs.length;
}

export function stdDev(xs) {
  if (xs.length < 2) return 0;
  const m = mean(xs);
  const sumSq = xs.reduce((a, b) => a + (b - m) ** 2, 0);
  return Math.sqrt(sumSq / (xs.length - 1)); // sample stddev
}

// t-distribution critical values for 95% two-sided, df = n - 1.
// Hardcoded for n in [2..30]; falls back to 1.960 (z) for n > 30.
const T_CRIT_95 = {
  2: 12.706, 3: 4.303, 4: 3.182, 5: 2.776, 6: 2.571, 7: 2.447,
  8: 2.365, 9: 2.306, 10: 2.262, 11: 2.228, 12: 2.201, 13: 2.179,
  14: 2.160, 15: 2.145, 16: 2.131, 17: 2.120, 18: 2.110, 19: 2.101,
  20: 2.093, 21: 2.086, 22: 2.080, 23: 2.074, 24: 2.069, 25: 2.064,
  26: 2.060, 27: 2.056, 28: 2.052, 29: 2.048, 30: 2.045,
};

export function ciHalfWidth95(xs) {
  // v1.7.5 codex P2 -- reject n<5 by returning 0. n=2 t-crit=12.706 gives
  // nonsense CIs; eval requires n>=10 anyway. Smoke runs labelled
  // exploratory, no CI reported.
  if (xs.length < 5) return 0;
  const t = T_CRIT_95[xs.length] ?? 1.960;
  return (t * stdDev(xs)) / Math.sqrt(xs.length);
}

/**
 * v1.7.5 -- exact paired permutation CI for mean(deltaA - deltaB).
 * Used in Task 3 instead of paired t-test: phase rates are bounded
 * binomial-like and t-test is fragile at n=20. Permutation makes no
 * normality assumption.
 *
 * @param {number[]} xsA per-seed late-rate for condition A
 * @param {number[]} xsB per-seed late-rate for condition B (same seeds)
 * @param {number} alpha 0.05 for 95% CI
 * @param {number} nResamples e.g. 10_000
 * @returns {{deltaMean: number, ciLow: number, ciHigh: number}}
 */
export function pairedPermutationCI(xsA, xsB, alpha = 0.05, nResamples = 10_000) {
  if (xsA.length !== xsB.length) throw new Error('pairedPermutationCI: lengths differ');
  const n = xsA.length;
  if (n < 5) throw new Error('pairedPermutationCI: n<5');
  const diffs = xsA.map((a, i) => a - xsB[i]);
  const observed = mean(diffs);

  // Sign-flip permutation: each resample flips signs randomly, then re-means.
  // Use mulberry32 internally so this is deterministic given a seed env.
  const rng = mulberry32(0x9E3779B9);
  const resampledMeans = [];
  for (let r = 0; r < nResamples; r++) {
    let s = 0;
    for (let i = 0; i < n; i++) s += diffs[i] * (rng() < 0.5 ? -1 : 1);
    resampledMeans.push(s / n);
  }
  // Bias-corrected percentile bounds around the observed mean.
  resampledMeans.sort((a, b) => a - b);
  const loIdx = Math.floor((alpha / 2) * nResamples);
  const hiIdx = Math.ceil((1 - alpha / 2) * nResamples) - 1;
  const ciLow = observed + (resampledMeans[loIdx] - mean(resampledMeans));
  const ciHigh = observed + (resampledMeans[hiIdx] - mean(resampledMeans));
  return { deltaMean: observed, ciLow, ciHigh };
}

// mulberry32 lives here so aggregate.mjs has a deterministic RNG dep-free.
function mulberry32(seed) {
  let s = seed >>> 0;
  return function () {
    s = (s + 0x6D2B79F5) >>> 0;
    let t = s;
    t = Math.imul(t ^ (t >>> 15), t | 1);
    t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

/**
 * Aggregate a list of per-seed phase rates into mean/stddev/CI per phase.
 * @param {Array<{early: number, mid: number, late: number}>} seedResults
 * @returns {{early: {mean, std, ci95}, mid: {...}, late: {...}}}
 */
export function aggregatePhases(seedResults) {
  const result = {};
  for (const phase of ['early', 'mid', 'late']) {
    const xs = seedResults.map((r) => r[phase]);
    result[phase] = { mean: mean(xs), std: stdDev(xs), ci95: ciHalfWidth95(xs) };
  }
  return result;
}
```

### Step 3: Add seeded RNG + MEANINGFUL variance to `traps.mjs`

**Reviewer finding:** Plan v1's "shuffle within thirds" was MISCONCEIVED — `traps.mjs` already places traps at fixed positions (e.g. `bare_except` at 4, 28, 46), so within-thirds shuffling only reshuffles trap-vs-clean positions (clean = no-op). The "seed" varies almost nothing real and CIs would be artificially tiny.

**Real seed variance source (chosen):** Randomly assign WHICH category appears at each trap-slot within each phase bucket, while preserving (a) the slot positions themselves (early/mid/late distribution unchanged), (b) the count of trap encounters per category (each category still appears 2-3 times across the sequence), (c) the early-then-later structure (each category's first encounter is in early or mid, second in mid or late, third if any in late).

Implementation:

```js
// v1.7.5 -- mulberry32 RNG, dep-free.
function mulberry32(seed) { /* same as aggregate.mjs */ }

/**
 * Deterministic Fisher-Yates shuffle scoped to an index list.
 * @param {number[]} arr indices to shuffle in place
 */
function shuffleIndices(arr, rng) {
  for (let i = arr.length - 1; i > 0; i--) {
    const j = Math.floor(rng() * (i + 1));
    [arr[i], arr[j]] = [arr[j], arr[i]];
  }
  return arr;
}

/**
 * Generate the 50-task sequence, optionally with seed-driven category-to-slot
 * randomization. Seed varies WHICH category lands at each trap slot within
 * each phase bucket. Preserves: total trap count, slot positions, per-category
 * encounter count, early-then-later structure.
 *
 * @param {number} [seed] - integer; same seed produces identical output
 */
export function generateTasks(seed) {
  // 1. Build the canonical fixed-position structure: { earlySlots: [pos...],
  //    midSlots: [pos...], lateSlots: [pos...] } from the existing TRAPS
  //    array (which encodes category -> [pos1, pos2, pos3]).
  // 2. For each phase bucket, collect (category, encounter_index) pairs that
  //    land in that phase. Shuffle the assignment of pairs to slots within
  //    the phase using mulberry32(seedForPhase). Each phase uses its own
  //    sub-seed derived from `seed` to avoid correlated streams across phases.
  // 3. Re-emit the 50-task array with the new (slot, category) assignments.
  // 4. Filler positions (no trap) stay filler.

  if (typeof seed !== 'number') {
    return generateCanonicalTasks(); // existing fixed ordering
  }

  // codex P2 -- hash-derive sub-seeds to avoid correlation across phases.
  const earlyRng = mulberry32((0x9E3779B9 * (seed + 1)) >>> 0);
  const midRng   = mulberry32((0x9E3779B9 * (seed + 2)) >>> 0);
  const lateRng  = mulberry32((0x9E3779B9 * (seed + 3)) >>> 0);

  // ... assign categories to slots per phase using each rng ...
  // (full implementation reads TRAPS structure + emits tasks array)
}
```

**Pre-flight before writing this code:** read `benchmarks/sequential-learning/traps.mjs` in full to understand the canonical TRAPS structure (likely `{ id, tags, lesson, recallQueries[] }` per category, plus an array mapping category to fixed positions). The category-to-slot algorithm depends on this exact shape.

**Determinism + variance tests** in `tests/sequential-learning-aggregate.test.ts`:

```ts
it('generateTasks(seed) is deterministic for a given seed', () => {
  const a = generateTasks(42);
  const b = generateTasks(42);
  expect(a).toEqual(b);
});

it('generateTasks(seed) varies meaningfully across different seeds', () => {
  // Same number of trap tasks (deterministic structure preserved)
  const s1 = generateTasks(1000);
  const s2 = generateTasks(1001);
  const trapCount1 = s1.filter((t) => t.trapCategory).length;
  const trapCount2 = s2.filter((t) => t.trapCategory).length;
  expect(trapCount1).toBe(trapCount2);
  // But which category lands at each trap position differs at >half the slots
  const trapPositions = s1.map((t, i) => (t.trapCategory ? i : null)).filter((i) => i !== null);
  let differingSlots = 0;
  for (const i of trapPositions) {
    if (s1[i].trapCategory !== s2[i].trapCategory) differingSlots++;
  }
  expect(differingSlots).toBeGreaterThanOrEqual(Math.floor(trapPositions.length / 2));
});

it('generateTasks(seed) preserves per-category encounter count', () => {
  const tasks = generateTasks(42);
  const byCat = {};
  for (const t of tasks) {
    if (t.trapCategory) byCat[t.trapCategory] = (byCat[t.trapCategory] ?? 0) + 1;
  }
  // each category still appears 2-3 times (verified against canonical TRAPS)
  for (const [cat, count] of Object.entries(byCat)) {
    expect(count).toBeGreaterThanOrEqual(2);
    expect(count).toBeLessThanOrEqual(3);
  }
});

it('generateTasks(seed) preserves early-then-later structure', () => {
  // For each category, first encounter must be in positions 0-16 (early third).
  const tasks = generateTasks(42);
  const firstSeen = {};
  tasks.forEach((t, i) => {
    if (t.trapCategory && firstSeen[t.trapCategory] === undefined) {
      firstSeen[t.trapCategory] = i;
    }
  });
  for (const pos of Object.values(firstSeen)) {
    expect(pos).toBeLessThan(17); // each category first-encountered in early third
  }
});
```

**Stop condition:** If reading `traps.mjs` reveals the canonical TRAPS structure does NOT support the assignment algorithm (e.g., categories' fixed positions intertwine in a way that prevents per-phase reassignment), STOP and report. The seed variance plan needs a different shape.

### Step 4: Add `--seed` and `--n-seeds` to `run.mjs`

```js
function parseArgs() {
  const args = process.argv.slice(2);
  const opts = {
    adapter: 'all',
    output: join(__dirname, 'results'),
    useGoalStack: false,
    seed: undefined,
    nSeeds: 1,
  };
  for (let i = 0; i < args.length; i++) {
    if (args[i] === '--adapter' && args[i + 1]) opts.adapter = args[++i];
    else if (args[i] === '--output' && args[i + 1]) opts.output = args[++i];
    else if (args[i] === '--use-goal-stack') opts.useGoalStack = true;
    else if (args[i] === '--seed' && args[i + 1]) opts.seed = parseInt(args[++i], 10);
    else if (args[i] === '--n-seeds' && args[i + 1]) opts.nSeeds = parseInt(args[++i], 10);
    // ...
  }
  return opts;
}
```

Multi-seed loop in `runAdapter` (or wherever the per-adapter run lives):

```js
async function runAdapter(name, adapter, opts) {
  const seeds = opts.nSeeds > 1
    ? Array.from({ length: opts.nSeeds }, (_, i) => 1000 + i)
    : [opts.seed ?? 0];
  const seedResults = [];
  for (const seed of seeds) {
    const tasks = generateTasks(seed);
    const results = await simulate(adapter, tasks, { useGoalStack: opts.useGoalStack });
    seedResults.push({ seed, phases: phaseRates(results), overall: trapHitRate(results) });
  }
  const phaseAgg = aggregatePhases(seedResults.map((r) => r.phases));
  return { name, seedResults, phaseAgg };
}
```

Output JSON shape:

```json
{
  "benchmark": "hippo-sequential-learning",
  "version": "1.7.5",
  "config": { "useGoalStack": true, "nSeeds": 10 },
  "conditions": {
    "hippo": {
      "seeds": [{ "seed": 1000, "phases": {...}, "overall": 0.30 }, ...],
      "phaseAggregate": { "early": {"mean": ..., "std": ..., "ci95": ...}, ... }
    }
  }
}
```

### Step 5: Run tests + smoke

```bash
npx vitest run tests/sequential-learning-aggregate.test.ts
node benchmarks/sequential-learning/run.mjs --adapter hippo --n-seeds 3 \
  --output results/v1.7.5-task2-smoke/
```

Expected: aggregator tests pass; 3-seed smoke run produces a JSON with `phaseAggregate` containing finite mean/std/ci95. Hand-check the math against a calculator on one seed's phase rates.

### Step 6: Commit

```bash
git add benchmarks/sequential-learning/aggregate.mjs \
  benchmarks/sequential-learning/traps.mjs \
  benchmarks/sequential-learning/run.mjs \
  tests/sequential-learning-aggregate.test.ts
git commit -m "feat: multi-seed eval harness for sequential-learning benchmark (v1.7.5 task 2)"
```

**Success criteria:**
- `aggregate.mjs` exports `mean`, `stdDev`, `ciHalfWidth95`, `aggregatePhases`. 4 unit tests pass.
- `traps.mjs::generateTasks(seed)` is deterministic; same seed → same task ordering.
- `run.mjs --n-seeds N` runs N times and produces aggregated output.
- One atomic commit.

---

## Task 3: Pre-registered eval — DEMONSTRATE or RETIRE

**Why:** This is the eval that decides whether the −10pp goal-stack lift claim is real or vapour. The decision rule and BOTH report templates are committed BEFORE the run so confirmation bias has nowhere to hide.

**Files:**
- Create: `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md` — pre-registered protocol + decision rule + both report templates (commit BEFORE Step 4 below)
- Create: `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-result.md` — actual measured numbers + chosen decision (commit AFTER the run)

### Step 1: Write the pre-registration doc

Create `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md`:

```markdown
# v1.7.5 Goal-Stack Eval — Pre-Registration

**Pre-registered:** 2026-05-07, BEFORE running the eval.
**Author:** Keith So
**Mechanism under test:** B3 dlPFC goal-stack boost (v0.38.0 ship; v1.7.4 lift into shared helper).
**Public claim being tested:** Goal-stack provides ≥10pp ADDITIONAL drop in late-phase trap-hit-rate on top of base hippo (78% → 14% baseline). Source of claim: TODOS.md v0.39.0 B3 follow-ups, plus general references in CHANGELOG/RESEARCH.md.

## Protocol

- Benchmark: `benchmarks/sequential-learning` (v1.7.5 multi-seed harness with category-to-slot variance).
- **Conditions (4, not 2):**
  - **C0 — none:** `baseline` adapter, no memory.
  - **C1 — static:** `static` adapter, all lessons pre-loaded.
  - **C2 — hippo-base:** `hippo` adapter, `--use-goal-stack` OFF.
  - **C3 — hippo+goalstack:** `hippo` adapter, `--use-goal-stack` ON.
- **Seeds: 20.** Hash-derived (`0x9E3779B9 * (1000+i) >>> 0` for i in 0..19) to avoid correlated mulberry32 streams.
- **Eval mode:** `--eval-strict` flag → simulator hard-fails on any pushGoal/completeGoal hook error. Hook-failure counts written to result JSON; non-zero counts disqualify the run.
- **Primary metric (PRE-LOCKED):** late-phase trap-hit-rate. **"Late" definition (PRE-LOCKED):** positions 34-49 inclusive (last third, 0-indexed). Measured per seed by counting `trapHit === true` among trap-tasks at those positions, divided by trap-task count at those positions.
- **Top-5 cap interaction (PRE-LOCKED):** a task is "trap hit" iff `isRecalled(top5, category) === false`. The top-5 cap is part of the metric definition; right-lesson-at-rank-6 counts as a hit, by design.
- **Reporting:** mean ± 95% CI per phase, per condition. Plus paired Δ (C2 vs C3) via permutation CI.

## Statistical method (PRE-LOCKED)

- **Estimand:** mean late-phase Δ in percentage points = `mean_late(C2) − mean_late(C3)` over the 20 paired seeds.
- **CI method:** exact paired sign-flip permutation, 10,000 resamples (`pairedPermutationCI` in `aggregate.mjs`). NOT t-test — phase rates are bounded discrete and t-test is fragile at n=20 with binomial-like variance.
- **Robustness check:** Wilcoxon signed-rank p-value reported alongside (informational; primary CI is permutation).

## Decision rule (mechanical)

- **Δ ≥ 0.10 AND lower permutation-CI bound > 0** → **DEMONSTRATED.** Ship with "DEMONSTRATED" template.
- **Δ < 0.10 OR CI overlaps 0** → **RETRACTED.** Ship with "RETRACTED" template. Apply the retraction sweep below to ALL claim locations.
- **Δ < 0 (goal-stack HARMS performance)** → **RETRACTED + flag for v1.8 followup.** Mechanism may need design review before further use.
- **Any condition fails to run cleanly** (seed crash, hook failure, baseline drift) → STOP, fix, re-run. No partial-data ship.

## Stop conditions (mid-run)

- If C2 (hippo-base) late-phase mean diverges from README headline by >±10pp (i.e. >24% or <4%): harness is broken or headline is stale. STOP, investigate before scoring C3.
- If any seed crashes in any condition: drop that seed across ALL conditions (paired) and run a replacement seed with index 1020+. Do NOT cherry-pick.
- If `hookFailures.push > 0` or `hookFailures.complete > 0` in C3: the run is contaminated. STOP, fix, restart. (eval-strict mode prevents partial completions reaching the aggregator.)
- If any single seed's late-phase rate is >3 stddev from the rest in either condition: investigate (FS error, network blip, etc.). Drop or include based on investigation; document the choice.

## Retraction sweep (apply ONLY if RETRACTED)

Before writing the result doc, run:

```bash
grep -rn "10pp\|−10pp\|goal-stack lift\|goal stack lift" --include='*.md' . | grep -v node_modules | grep -v results/
```

Update EVERY surfaced location, specifically:
- `README.md`
- `CHANGELOG.md`
- `RESEARCH.md` (the dlPFC sections, around line 218-240 — NOT the vlPFC line)
- `ROADMAP-RESEARCH.md` (around line 156)
- `TODOS.md` (any open or closed B3 references)
- `docs/plans/2026-04-29-b3-dlpfc-depth.md` (mark as superseded with header note pointing to v1.7.5 result)

Commit the inventory artifact at `docs/evals/2026-05-07-v1.7.5-claim-inventory.md` BEFORE the run, listing every grep hit. After the run, if RETRACTED, every listed location must be touched.

## Pre-committed report templates

### Template "DEMONSTRATED" (use IFF the decision rule above selects it)

> **Goal-stack lift on sequential-learning benchmark (DEMONSTRATED).** 20
> paired seeds, exact permutation CI. Late-phase trap-hit-rate dropped by
> Δ = {DELTA}pp (95% CI [{CI_LOW}pp, {CI_HIGH}pp], Wilcoxon p={WILCOXON_P}).
> Goal-stack ON: mean late = {C3_LATE}% (CI ±{C3_CI}pp). Goal-stack OFF:
> mean late = {C2_LATE}% (CI ±{C2_CI}pp). Sanity anchors — none: {C0_LATE}%,
> static: {C1_LATE}%. Pre-registration:
> docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md. Claim inventory:
> docs/evals/2026-05-07-v1.7.5-claim-inventory.md.

### Template "RETRACTED" (use IFF the decision rule above selects it)

> **Goal-stack lift on sequential-learning benchmark — claim withdrawn.** 20
> paired seeds. Measured late-phase Δ = {DELTA}pp (95% permutation CI
> [{CI_LOW}pp, {CI_HIGH}pp], Wilcoxon p={WILCOXON_P}). Does not meet the
> pre-registered ≥10pp threshold with lower-CI bound above zero. Pre-v1.7.5
> CHANGELOG / README / RESEARCH.md / ROADMAP-RESEARCH.md / TODOS.md
> references to a −10pp goal-stack lift updated to reflect the measured
> number. Mechanism shipped, claim withdrawn. Future eval may revisit.
> Pre-registration: docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md.
> Claim inventory: docs/evals/2026-05-07-v1.7.5-claim-inventory.md.
```

Commit:

```bash
git add docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md
git commit -m "docs: pre-registered protocol for v1.7.5 goal-stack eval"
```

### Step 1.5: Build the claim inventory artifact (commit BEFORE running the eval)

```bash
grep -rn "10pp\|−10pp\|goal-stack lift\|goal stack lift" --include='*.md' . \
  | grep -v node_modules | grep -v results/ | grep -v docs/evals/ \
  > /tmp/v1.7.5-claim-inventory.txt
```

Curate `/tmp/v1.7.5-claim-inventory.txt` into `docs/evals/2026-05-07-v1.7.5-claim-inventory.md`. List file path + line + verbatim text. Commit:

```bash
git add docs/evals/2026-05-07-v1.7.5-claim-inventory.md
git commit -m "docs: pre-eval claim inventory for v1.7.5 retraction sweep"
```

This commits the surface area BEFORE the eval result is known so the retraction sweep can't selectively miss a location.

### Step 2: Run the eval (4 conditions × 20 seeds, eval-strict mode)

```bash
# C0 — none (free statistical anchor)
node benchmarks/sequential-learning/run.mjs --adapter none --n-seeds 20 --eval-strict \
  --output results/v1.7.5-eval-C0-none/

# C1 — static (free statistical anchor)
node benchmarks/sequential-learning/run.mjs --adapter static --n-seeds 20 --eval-strict \
  --output results/v1.7.5-eval-C1-static/

# C2 — hippo-base
node benchmarks/sequential-learning/run.mjs --adapter hippo --n-seeds 20 --eval-strict \
  --output results/v1.7.5-eval-C2-hippo-base/

# C3 — hippo+goalstack
node benchmarks/sequential-learning/run.mjs --adapter hippo --n-seeds 20 --use-goal-stack --eval-strict \
  --output results/v1.7.5-eval-C3-hippo-goalstack/
```

**Time budget:** C0/C1 are fast (no subprocess). C2 ≈ 4-6 min (20 seeds × 25 trap tasks × ~200ms recall). C3 ≈ 8-12 min (20 × 25 × ~400ms with push+complete). Total ≈ 15-20 min. Smoke first with `--n-seeds 1` to confirm shapes.

**Sanity gate before scoring:** check `hookFailures.push === 0 && hookFailures.complete === 0` in C3's output JSON. If non-zero, eval-strict should have already aborted; if not, the harness has a bug — STOP and investigate.

### Step 3: Apply decision rule, write result doc

Compute paired-Δ via `pairedPermutationCI` across the 20 seeds. Compute Wilcoxon signed-rank as robustness check. Apply decision rule from prereg. Create `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-result.md`:

```markdown
# v1.7.5 Goal-Stack Eval — Result

**Run:** 2026-05-07
**Pre-registration:** docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md
**Claim inventory:** docs/evals/2026-05-07-v1.7.5-claim-inventory.md
**Decision applied:** {DEMONSTRATED | RETRACTED}

## Measured numbers (20 seeds, 4 conditions)

| Condition | Early mean (CI) | Mid mean (CI) | Late mean (CI) |
|-----------|-----------------|---------------|----------------|
| C0 — none           | {C0_EARLY} (±{C0_EARLY_CI}) | {C0_MID} (±{C0_MID_CI}) | {C0_LATE} (±{C0_LATE_CI}) |
| C1 — static         | {C1_EARLY} (±{C1_EARLY_CI}) | {C1_MID} (±{C1_MID_CI}) | {C1_LATE} (±{C1_LATE_CI}) |
| C2 — hippo-base     | {C2_EARLY} (±{C2_EARLY_CI}) | {C2_MID} (±{C2_MID_CI}) | {C2_LATE} (±{C2_LATE_CI}) |
| C3 — hippo+goalstack | {C3_EARLY} (±{C3_EARLY_CI}) | {C3_MID} (±{C3_MID_CI}) | {C3_LATE} (±{C3_LATE_CI}) |

Paired Δ on late-phase (C2 vs C3): **Δ = {DELTA}pp** (95% permutation CI [{CI_LOW}pp, {CI_HIGH}pp]).
Wilcoxon signed-rank robustness check: p = {WILCOXON_P}.

Hook failures (must be 0 for valid run): C3.push={C3_PUSH_FAIL}, C3.complete={C3_COMPLETE_FAIL}.

Sanity gate: C2 late-phase mean = {C2_LATE}% (must be within 4-24% of README headline 14%; otherwise harness/headline drift).

## Decision

{One sentence: "Δ = {DELTA}pp ≥ 10pp AND lower-CI {CI_LOW}pp > 0pp → DEMONSTRATED" OR "Δ = {DELTA}pp does not meet ≥10pp threshold OR CI overlaps 0 → RETRACTED" — exact-quote the rule.}

## Raw artifacts

- `results/v1.7.5-eval-C0-none/benchmark-*.json`
- `results/v1.7.5-eval-C1-static/benchmark-*.json`
- `results/v1.7.5-eval-C2-hippo-base/benchmark-*.json`
- `results/v1.7.5-eval-C3-hippo-goalstack/benchmark-*.json`
```

Commit:

```bash
git add docs/evals/2026-05-07-v1.7.5-goal-stack-eval-result.md \
  results/v1.7.5-eval-C0-none/ \
  results/v1.7.5-eval-C1-static/ \
  results/v1.7.5-eval-C2-hippo-base/ \
  results/v1.7.5-eval-C3-hippo-goalstack/
git commit -m "docs: v1.7.5 goal-stack eval result ({DEMONSTRATED|RETRACTED})"
```

**If RETRACTED:** apply the retraction sweep from prereg. For each location in the claim inventory, update the text. Commit each file group separately:

```bash
git add README.md CHANGELOG.md
git commit -m "docs: retract −10pp goal-stack lift claim per v1.7.5 eval (README + CHANGELOG)"

git add RESEARCH.md ROADMAP-RESEARCH.md
git commit -m "docs: retract −10pp goal-stack lift claim per v1.7.5 eval (research)"

git add TODOS.md docs/plans/2026-04-29-b3-dlpfc-depth.md
git commit -m "docs: retract −10pp goal-stack lift claim per v1.7.5 eval (plans + todos)"
```

**Success criteria:**
- Both pre-reg and result docs present + committed.
- Decision matches the pre-registered rule mechanically (no re-interpretation).
- Raw JSON artifacts in `results/` so anyone can re-derive the aggregates.

---

## Task 4: Ship phase

After all 3 tasks committed:

### S1: `/self-review`

Re-read brief, verify each commit traces to a TODOS.md item AND to the pre-registered protocol. No drift.

### S2: `/review`

Pre-landing review against `master`. Pay attention to: paired-CI math (easy to get wrong), seed determinism (mulberry32 is fine but pin it), interface backward-compatibility (new optional fields don't break existing custom adapters).

### S3: `/ship-check`

Confirm: 1 mechanical task + 1 harness task + 1 eval+report task all closed. Full suite green. Decision applied per pre-reg.

### S4: `/publish-repo`

Bump 1.7.4 → 1.7.5. CHANGELOG content depends on Task 3's outcome:

#### IF DEMONSTRATED:

```markdown
## 1.7.5 (YYYY-MM-DD)

### Added

- **Sequential-learning adapter contract — `pushGoal` / `completeGoal` hooks.** `benchmarks/sequential-learning/adapters/interface.mjs` accepts optional paired hooks. `hippo.mjs` adapter implements both via the existing `hippo goal push|complete` CLI commands. Closes the v0.39 B3 follow-up that gated public exercising of the dlPFC goal-stack mechanism.
- **Multi-seed eval harness.** `--seed N` and `--n-seeds N` flags on `run.mjs`. `benchmarks/sequential-learning/aggregate.mjs` provides mean / stddev / 95% CI helpers. Output JSON gains a `phaseAggregate` block.
- **`--use-goal-stack` opt-in flag** on `run.mjs`. When set AND the adapter supplies `pushGoal/completeGoal`, the simulator wraps each trap task in goal-push / goal-complete so the dlPFC boost activates during recall.

### Eval

- **Goal-stack lift on sequential-learning benchmark — DEMONSTRATED.** 20-seed paired comparison, exact permutation CI. Late-phase trap-hit-rate dropped by Δ = {DELTA}pp (95% CI [{CI_LOW}pp, {CI_HIGH}pp], Wilcoxon p={WILCOXON_P}). Pre-registration committed BEFORE the run: `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md`. Full result: `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-result.md`.
```

README "What's new in v1.7.5" (DEMONSTRATED variant):

```markdown
### What's new in v1.7.5

- **Sequential-learning benchmark now exercises the goal-stack.** New optional `pushGoal/completeGoal` adapter hooks; `--use-goal-stack` flag on the runner. Goal-stack lift demonstrated: late-phase trap-hit-rate drops by Δ = {DELTA}pp (95% CI [{CI_LOW}pp, {CI_HIGH}pp], Wilcoxon p={WILCOXON_P}) over 20 paired seeds. Pre-registered protocol in `docs/evals/`.
- **Multi-seed eval harness.** `run.mjs` gains `--seed N` / `--n-seeds N` / `--eval-strict`; new `aggregate.mjs` provides mean / std / CI / paired-permutation helpers.
```

#### IF RETRACTED:

```markdown
## 1.7.5 (YYYY-MM-DD)

### Added

- **Sequential-learning adapter contract — `pushGoal` / `completeGoal` hooks.** `benchmarks/sequential-learning/adapters/interface.mjs` accepts optional paired hooks. `hippo.mjs` adapter implements both via the existing `hippo goal push|complete` CLI commands. Closes the v0.39 B3 follow-up that gated public exercising of the dlPFC goal-stack mechanism.
- **Multi-seed eval harness.** `--seed N` and `--n-seeds N` flags on `run.mjs`. `benchmarks/sequential-learning/aggregate.mjs` provides mean / stddev / 95% CI helpers. Output JSON gains a `phaseAggregate` block.
- **`--use-goal-stack` opt-in flag** on `run.mjs`. When set AND the adapter supplies `pushGoal/completeGoal`, the simulator wraps each trap task in goal-push / goal-complete so the dlPFC boost activates during recall.

### Eval — claim withdrawn

- **Goal-stack lift on sequential-learning benchmark — claim withdrawn.** 20-seed paired comparison, exact permutation CI. Late-phase trap-hit-rate Δ = {DELTA}pp (95% CI [{CI_LOW}pp, {CI_HIGH}pp], Wilcoxon p={WILCOXON_P}) does NOT meet the pre-registered ≥10pp threshold with lower-CI bound above zero. References to a −10pp goal-stack lift in `README.md`, `CHANGELOG.md`, `RESEARCH.md`, `ROADMAP-RESEARCH.md`, `TODOS.md`, and `docs/plans/2026-04-29-b3-dlpfc-depth.md` updated to reflect the measured number. Mechanism shipped, claim withdrawn. Future eval may revisit. Pre-registration: `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-prereg.md`. Full result: `docs/evals/2026-05-07-v1.7.5-goal-stack-eval-result.md`. Claim inventory: `docs/evals/2026-05-07-v1.7.5-claim-inventory.md`.
```

README "What's new in v1.7.5" (RETRACTED variant):

```markdown
### What's new in v1.7.5

- **Sequential-learning benchmark now exercises the goal-stack** via new optional `pushGoal/completeGoal` adapter hooks and the `--use-goal-stack` runner flag.
- **Goal-stack lift claim withdrawn.** A pre-registered 20-seed paired eval (exact permutation CI) did not show the previously claimed −10pp late-phase improvement (measured Δ = {DELTA}pp, 95% CI [{CI_LOW}pp, {CI_HIGH}pp]). Mechanism shipped, claim withdrawn. Pre-registration + claim inventory + full result in `docs/evals/`.
- **Multi-seed eval harness.** `run.mjs` gains `--seed N` / `--n-seeds N` / `--eval-strict`; new `aggregate.mjs` provides mean / std / CI / paired-permutation helpers.
```

**On retraction, the inventory artifact (Step 1.5) drives the sweep.** Every location listed there must be touched. Each file group commits separately so the diff is easy to follow and revert if needed.

### S5: `hippo capture` + close TODOS

```bash
hippo capture --stdin <<< 'v1.7.5 shipped: sequential-learning adapter contract + multi-seed eval. Decision: {DEMONSTRATED|RETRACTED}. Pre-registered protocol prevented confirmation-bias drift.'
```

Edit `TODOS.md` line 139-143: mark the v1.7.5 item closed with reference to the eval result doc.

---

## Risk register

| Risk | Likelihood | Mitigation |
|------|-----------|------------|
| **Boost mechanism doesn't fire — eval RETRACTS for the wrong reason** (P0 from outside-voice review) | High (was the dominant P0) | `simulate()` stores memories with `[category.id, ...category.tags, 'error']` so the goal name `category.id` matches a real tag. Task 1 success criteria includes a smoke assertion that `goal_recall_log` has rows after a smoke run. STOP if zero rows. |
| **Seed variance is statistical theatre** (within-thirds shuffle was misconceived) | High (was P0) | Real variance source: random assignment of CATEGORIES to position-slots within each phase. Determinism + variance + per-category-encounter-count + early-then-later tests pin the algorithm. |
| Single deterministic run produces a number; we publish "10pp lift" without statistical defensibility | High (prior pattern) | Multi-seed harness mandatory. Pre-registered decision rule. BOTH templates pre-committed. |
| Confirmation bias — running the eval, wanting a positive result | High (human nature) | Pre-registered decision rule + claim-inventory artifact committed BEFORE eval. Decision mechanical. |
| HIPPO_HOME / XDG_DATA_HOME isolation hole — eval contaminates real store | Medium (codex P0) | Adapter sets `HIPPO_HOME=<temp>` and clears `XDG_DATA_HOME` in child env. Pre-flight grep verifies precedence. |
| Hook failures swallow silently — eval looks null but boost never fired | Medium (codex P1) | `--eval-strict` mode + finally cleanup + nonzero hookFailures aborts. Sanity gate post-run. |
| Paired t-test fragile for bounded discrete rates at n=20 | Medium (codex P1) | `pairedPermutationCI` (exact sign-flip, 10k resamples) is the primary CI. Wilcoxon signed-rank as robustness check. No t-test. |
| Late-phase has only ~7-9 trap encounters per seed → coarse 14pp steps | Medium (senior P1) | N=20 (not 10). Document residual variance clearly in the result doc. |
| Confounding — goal-stack adds CLI calls; differential overhead bias | Low | Paired-on-same-seed A/B controls for it. Permutation CI requires no further assumption. |
| Subprocess overhead inflates wall time | Low | Smoke first. Time budget accounts for ~15-20 min total. |
| Eval reveals goal-stack is HARMFUL (Δ < 0) | Low | Pre-reg has explicit "Δ < 0 → RETRACTED + flag for v1.8 followup". |
| Headline 78%→14% baseline doesn't reproduce — harness is stale | Low | Pre-reg stop condition: C2 late-phase must be within 4-24% (±10pp of 14%) or STOP. |
| Retraction sweep misses a claim location | Medium (codex P1) | Pre-eval `claim-inventory.md` artifact lists every grep hit. Retraction sweep must touch every listed location. |
| RETRACTED template soft-frames the result | Medium (was REJECT trigger) | Template stripped of "mechanism still useful as steering signal" framing. Says "claim withdrawn. Future eval may revisit." Period. |

---

## Constraints (from CLAUDE.md + project memory)

- **Real DB for tests.** Aggregator unit tests are pure math (no DB). Adapter contract tests load `interface.mjs` directly. No mocks.
- **Never `--no-verify`** on commits.
- **Atomic commit per task slice** + a separate commit for the prereg doc + a separate commit for the result doc.
- **TDD: failing test first** on Task 1 (contract validator) and Task 2 (aggregator math). Task 3 is an eval, not a code change — TDD doesn't apply.
- **Branch off master.**
- **Honest reporting (CRITICAL):** if the eval comes back null or worse-than-baseline, retract the claim. No soft framing. No "directional signal". The pre-registered template is what ships.
- **No public API change** in this release — adapter interface gains optional methods (additive). Multi-seed flags are CLI-only.
- **`run.mjs` and `aggregate.mjs` stay zero-dependency** (no npm imports beyond Node 22+ built-ins). The benchmark must run on a vanilla Node install.

## Estimated time

| Task | Time |
|------|------|
| Pre-flight + branch | 5 min |
| Task 1 (adapter contract + tag-fix + isolation + 5 tests + smoke + boost-fires assertion) | 75-90 min |
| Task 2 (category-to-slot assignment + permutation CI + aggregator + 6+ tests + smoke) | 90-120 min |
| Task 3 Step 1 (prereg doc) | 20 min |
| Task 3 Step 1.5 (claim-inventory artifact) | 10 min |
| Task 3 Step 2 (4 conditions × 20 seeds eval) | 15-20 min runtime |
| Task 3 Step 3 (result doc + retraction sweep if RETRACTED) | 30-60 min (sweep is heavier than write) |
| Ship phase (S1-S5) | 30-45 min |
| **Total** | **~5-6 hours** |

Caveat: total grew vs v1 estimate because outside-voice review added (a) the boost-fires assertion + tag-fix (Task 1), (b) category-to-slot assignment + permutation CI (Task 2), (c) 4 conditions instead of 2 + N=20 instead of 10 + claim-inventory artifact + retraction sweep discipline (Task 3). The growth is mostly statistical rigor, not feature scope. Without these, the eval would have been pseudo-replication that retracted a real mechanism.

---

## Plan history

- **2026-05-07 v1** — initial draft.
- **2026-05-07 v2 (this version)** — post outside-voice review. 13 consolidated revisions. P0 reshapes: (a) tag mismatch fix — store with `[category.id, ...category.tags, 'error']` so the boost can match, (b) within-thirds shuffle replaced with category-to-slot assignment (real seed variance), (c) CI test expected value 0.026 → 0.032, (d) HIPPO_HOME / XDG_DATA_HOME isolation, (e) finally-cleanup + eval-strict + hard-fail on hook errors, (f) permutation CI replaces paired t-test (fragile for bounded discrete rates), (g) 4 conditions instead of 2 (free statistical anchor), (h) N=20 instead of 10 (power), (i) claim-inventory artifact pre-committed, (j) RETRACTED template stripped of soft framing, (k) hash-derived seeds, (l) determinism + per-category-encounter-count + early-then-later tests, (m) pre-reg gaps closed (seed crash policy, "late" definition, top-5 cap interaction).
