# v1.9.0 — LongMemEval R@5 Cross-Validation Implementation Plan (v2 — REWRITE post outside-voice)

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Why this is v2:** v1 had 4 architectural P0s flagged by outside voice: (a) targeted `retrieve.py` which is explicitly NOT canonical (the file's own banner says "DO NOT compare numbers from retrieve.py against retrieve_inprocess.mjs"); (b) used a fictional `--use-goal-stack` flag that doesn't exist in hippo CLI; (c) conflated `--goal <tag>` single-tag override with the `applyGoalStackBoost` mechanism; (d) used `question_type` meta-categories as goal-tag, which would never appear as exact session tags → boost guaranteed no-op → designed-to-be-null. v2 corrects all four + 9 framing/reporting fixes.

**Goal:** Cross-validate the dlPFC `applyGoalStackBoost` mechanism (cli.ts:987-1005, fires when `sessionId && goalTag === ''`) on the LongMemEval R@5 corpus via the **canonical** `retrieve_inprocess.mjs` harness. Per-question, push a session-scoped goal with content-keyword tags extracted from the question; run recall; complete the goal. Compare C2 (goal-stack ON) vs C1 (goal-stack OFF) on the same `hippo_store`. **NO magnitude claim, NO ≥Xpp threshold, NO pass/fail decision rule on a magnitude.** Per `docs/RETRACTION.md`, mechanism characterisation only: workload-validity sanity verdict, sign-only per-question direction count (`STRICTLY_HIGHER` / `STRICTLY_LOWER` / `SAME`, NOT `STRICTLY_BETTER` / `WORSE` per outside voice B8), per-question rank-shift descriptive (operational footprint of the boost), boost-firing-rate (mechanism non-degeneracy check).

**Architecture:**
- **Harness:** `benchmarks/longmemeval/retrieve_inprocess.mjs` (canonical; in-process; budget=1M; min-results=10).
- **Mechanism path:** session-scoped `pushGoal(contentKeywordTag)` before each per-question recall; `completeGoal()` after. Drives `applyGoalStackBoost` because `sessionId !== ''` and `goalTag === ''`. NOT `--goal <tag>` override.
- **Goal-tag source:** top-K (default K=3) content-keyword stems extracted from each question via Porter-stem + extended-stop-words tokenizer (reused from v1.8's `tools/jaccard-overlap.mjs::tokenize`). Both ingest-side session-tags AND recall-side goal-tags content-derived → real token overlap structurally possible.
- **Conditions:** C1 (no goal-stack), C2 (session-scoped goal-stack via pushGoal). Both on same `hippo_store`. C0 (BM25-only) dropped from active comparison; cited prose-only with explicit "different-store provenance" caveat.
- **Workload size:** all 500 questions × 2 conditions.

**Tech Stack:** Node.js (existing `retrieve_inprocess.mjs`), Python (existing `evaluate_retrieval.py`), Vitest for new tests on the keyword-extractor + sign-only summariser. No new package dependencies.

**Semver decision:** 1.9.0 (minor). LongMemEval harness gains `--use-goal-stack` flag (in MJS, not Python) + `--goal-tag-source` flag. Additive feature; no breaking change.

---

## Task 0 — PRE-INVESTIGATION GATE (read-only; runs BEFORE pre-registration locks)

This gate exists per outside voice A3: the v1 plan's mental model of the boost mechanism was wrong (`tags?.includes(goalTag)` is exact-equality, not token-overlap). Before locking the v2 design, **read the actual hippo source** to confirm the design assumptions hold. If any assumption fails, plan needs a v3.

**Required reads (use Read tool, not summary):**

1. `src/cli.ts` lines 980-1010 (or wherever `applyGoalStackBoost` is invoked from `recall`). Verify:
   - The exact gating condition (claim: `sessionId && goalTag === ''`).
   - How `goalsByTag` is constructed from active goals (claim: tag-set per active goal, exact membership test on memory tags).
   - How `pushGoal` populates `goal_stack` table and which tags survive into the boost path.

2. `src/goals.ts::applyGoalStackBoost` (helper exported per v1.7.4 CHANGELOG). Verify:
   - The exact match predicate (claim: `goalsByTag.has(memoryTag)` exact string equality).
   - Any tokenisation or normalisation applied (claim: none).

3. Hippo's auto-tag system. Specifically: when `hippo remember` is invoked from `benchmarks/longmemeval/ingest.py` (or `ingest_direct.py`), what tags get assigned to the session memory? Read whichever code path produces the tags. Verify:
   - Are tags content-derived (BM25-token-style words from session content)?
   - Or schema_fit-derived (numeric / categorical)?
   - Or only user-supplied via `--tags`?
   - Result determines whether content-keyword goal-tags can plausibly match session tags.

4. `benchmarks/longmemeval/retrieve_inprocess.mjs`. Verify:
   - Does the harness already pass `sessionId` to recall? Or does it need a flag added?
   - Does it call `api.recall` or `physicsSearch`/`hybridSearch`? (Different code paths trigger different boost gates.)
   - What does its current recall call shape look like?

**Step 1: Read all 4 sources, capture findings in `docs/evals/2026-05-09-v1.9.0-pre-investigation.md`.**

**Step 2: Decision gate (must be answered explicitly):**

- **(a) Auto-tag system produces content-derived tokens that could overlap with question keywords?** YES → proceed. NO → STOP, escalate: option-A modify ingest.py to add content-keyword auto-tags; option-B re-pivot v1.9 entirely.
- **(b) `applyGoalStackBoost` gating matches the v2 design (sessionId set, goalTag='', goals pushed)?** YES → proceed. NO → adjust the mechanism path in this plan and re-dispatch outside voice on the revised design.
- **(c) `retrieve_inprocess.mjs` already supports session_id, OR can be extended without breaking existing baselines?** YES (either case) → proceed. NO → cite the obstacle, plan a separate harness fix as Task 1a.

**Step 3: Commit pre-investigation document.**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.9.0-pre-investigation.md
git -C C:/Users/skf_s/hippo commit -m "docs(longmemeval): pre-investigation gate for v1.9.0 v2 (auto-tag system, boost path, harness shape)"
```

**This commit must precede Task 1.** It is the audit anchor for "design assumptions verified BEFORE pre-registration locked."

---

## RETRACTION.md compliance (MANDATORY — carried forward from v1.8 with hardened terminology)

This release operates under the v1.7.9 magnitude-smuggling guard at `docs/RETRACTION.md`. Every output artifact MUST satisfy:

1. Reference `docs/RETRACTION.md` in the prereg, the result doc, the CHANGELOG entry, the README "What's new in v1.9.0" block, and the `docs/RETRACTION.md` "## v1.9.0 outcome" subsection.
2. State explicitly: **"This release does not re-assert the retracted −10pp magnitude."** Verbatim, in each artifact above.
3. **Forbidden language (hardened per outside voice B8):** `Δ\s*=`, `\b\d+\s*pp`, `−\d+pp`, "pp lift", "pp drop", "pp reduction", "pp improvement", "lift", "reduction", "improvement", "STRICTLY_BETTER", "STRICTLY_WORSE", "discriminating", "discrimination", "median pp", "5-number summary on Δ", side-by-side per-condition R@5 reporting that invites mental subtraction, per-category direction-with-attribution reporting (e.g. "C2 better on category X").
4. **Required terminology:** `STRICTLY_HIGHER` (C2 retrieved answer-in-top-5, C1 missed) / `STRICTLY_LOWER` (C1 retrieved, C2 missed) / `SAME` (both retrieved OR both missed). NEUTRAL on direction. Match v1.8 vocabulary exactly.
5. Outside-voice review on the staged retraction diff before /publish-repo. Same gate as v1.8 Task 13.

### Operational definition of "characterise" (carried from v1.8, expanded per outside voice C6)

For v1.9 specifically:
1. **Workload-validity verdict (PASS/FAIL).** From the C1 sanity preflight (procedural gate per outside voice B2; explicitly acknowledged as procedural-not-discriminating).
2. **Boost-firing-rate (NEW per outside voice B4 mitigation):** of 500 questions, count how many had ≥1 memory boosted by the goal-stack mechanism (mechanism non-degeneracy check). Reported as integer count. NOT a magnitude — frequency of mechanism activation.
3. **C2 per-question outcome distribution:** binary lattice {answer_in_top_5: k_yes, answer_not_in_top_5: k_no}.
4. **Sign-only per-question direction count vs C1:** STRICTLY_HIGHER / STRICTLY_LOWER / SAME counts.
5. **Per-question rank-shift descriptive (NEW per outside voice C6):** of 500 questions, count how many had a different rank-1 retrieval between C1 and C2 (mechanism's operational footprint, not magnitude).

ANY other report (Δ R@5, per-category direction-with-attribution, median pp, mean change, "improvement on X") is forbidden by RETRACTION.md.

---

## Pre-committed cumulative-null retraction-escalation rule (NEW per outside voice B4)

**Trigger:** If v1.9 returns SAME ≥ 495/500 (98% TIED) AND boost-firing-rate ≥ 50/500 (boost path exercised but no detectable behavioural effect), append a "## Mechanism-effect status (cumulative null escalation)" subsection to `docs/RETRACTION.md` acknowledging:

> Cumulative null evidence (v1.7.5 SANITY_FAIL, v1.7.6 B*=NULL, v1.7.7 SANITY_FAIL, v1.8.0 SAME=20/20 on adversarial workload, v1.9.0 SAME=K/500 on LongMemEval with content-keyword goal-tags). The dlPFC goal-stack mechanism remains shipped from v1.7.4. The cumulative evidence does not support the mechanism producing a detectable behavioural effect on these workloads at this scale. The mechanism is preserved in code; no specific effect is currently claimed.

This pre-commitment exists so v1.9's null result (if it occurs) cannot be silently absorbed without escalating the public retraction guard. Pre-committed BEFORE v1.9 runs.

---

## Pre-committed v1.10 direction (CORRECTED per outside voice B5)

**Was (v1):** "embeddings + goal-stack composition." Outside voice rejected this — embeddings is a retrieval-substrate change, not a mechanism test fix; v1.10 would test the same broken mapping under different retrieval and confound the mechanism characterisation.

**Now (v2):** v1.10 will iterate the goal-tag mapping itself. Specifically: try alternative content-keyword extractors (top-K with K ∈ {1, 5, 10}; NER-based extraction; n-gram extraction). v1.10 PRE-REG drafted before v1.10 implementation.

Embeddings + goal-stack composition is deferred to v1.11.

This is named NOW. v1.10 cannot be retroactively justified by v1.9's outcome.

---

## Branch + setup

```bash
git -C C:/Users/skf_s/hippo branch --show-current   # confirm master after v1.8.0 merge
git -C C:/Users/skf_s/hippo pull origin master
git -C C:/Users/skf_s/hippo checkout -b feat/v1.9.0-longmemeval-crossval
```

LongMemEval data must be staged in `benchmarks/longmemeval/` (download from HuggingFace per existing README). If not present, plan execution stops until data is staged.

---

## Task 1: Add `--use-goal-stack` + `--goal-tag-source` + `--max-questions` flags to `retrieve_inprocess.mjs`

**Files:**
- Modify: `benchmarks/longmemeval/retrieve_inprocess.mjs` (canonical harness)
- Create: `benchmarks/longmemeval/keyword-extractor.mjs` (content-keyword tokenizer, reused from v1.8 Jaccard tool's tokenize logic)

**Step 1: Read current `retrieve_inprocess.mjs` shape**

Already done at Task 0. Confirm flag-parsing pattern + recall-call shape.

**Step 2: Create the keyword-extractor module**

Reuse the Porter-stem + extended-stop-words tokenizer from `tools/jaccard-overlap.mjs`. Extract top-K stems by frequency (or, if all stems are unique within the question, top-K by position).

```javascript
// benchmarks/longmemeval/keyword-extractor.mjs
import { tokenize } from '../../tools/jaccard-overlap.mjs';

/**
 * Extract top-K content-keyword stems from a question.
 * Reuses v1.8 Jaccard tokenizer: lowercase, split on \W+, drop stop-words,
 * drop short tokens, Porter-stem.
 *
 * @param {string} questionText
 * @param {number} k - max stems to return (default 3)
 * @returns {string[]} stems, ordered by first-occurrence position
 */
export function extractKeywordStems(questionText, k = 3) {
  const stems = [...tokenize(questionText)];
  return stems.slice(0, k);
}
```

**Step 3: Add flags to `retrieve_inprocess.mjs::parseArgs`**

```javascript
// New flags (additive; preserve existing behaviour)
'--use-goal-stack': boolean (default false) — enable session-scoped pushGoal/completeGoal per question
'--goal-tag-source': enum (default 'question-keywords') — source for goal-tags. 'question-keywords' uses extractKeywordStems(question, 3).
'--max-questions': integer (default null) — limit to first N questions (for pilot/dev). null = all 500.
```

**Step 4: Plumb the goal-stack hooks per question**

For each question in the eval loop:

```javascript
if (opts.useGoalStack) {
  const stems = extractKeywordStems(question, 3);
  const sessionId = `lme-${question_id}`; // unique per question for isolation
  for (const stem of stems) {
    await hippoApi.pushGoal({ tenantId, sessionId, tag: stem });
  }
  // recall WITHOUT explicit goalTag (so applyGoalStackBoost fires per cli.ts:1005 gate)
  const memories = await hippoApi.recall({ query: question, sessionId, /* no goalTag */ });
  // record boost-firing flag in result
  const boostFired = memories.some((m) => m._goalBoosted === true);
  // complete the goal stack (drains the active goals so next question starts fresh)
  await hippoApi.completeGoal({ tenantId, sessionId });
  // ...
} else {
  const memories = await hippoApi.recall({ query: question /* no sessionId, no goalTag */ });
  // ...
}
```

**Note on boost-firing telemetry:** `applyGoalStackBoost` per cli.ts:990-1005 modifies the score per memory. To capture which memories were boosted, either (a) attach a `_goalBoosted` flag to the returned memory objects (requires harness modification), OR (b) infer from rank-shifts between C1 and C2 (compute post-hoc). Pick whichever is cheaper; document choice in Task 1 commit message.

**Step 5: Record per-question metadata in retrieval JSONL**

Add to each retrieval record:
- `goal_tags_pushed`: string[] (or null if --use-goal-stack=false)
- `boost_fired`: boolean (or null if telemetry path (b) chosen)
- `top_1_memory_id`: string (used for rank-shift descriptive at result-write time)

**Step 6: Tests**

```javascript
// tests/longmemeval-keyword-extractor.test.ts
import { describe, it, expect } from 'vitest';
import { extractKeywordStems } from '../benchmarks/longmemeval/keyword-extractor.mjs';

describe('LongMemEval keyword extractor', () => {
  it('extracts top-3 content stems from a question', () => {
    const stems = extractKeywordStems('What did the user say about the meeting schedule yesterday?', 3);
    expect(stems.length).toBeLessThanOrEqual(3);
    expect(stems).toContain('user'); // not stop-word
    // 'said' (verb), 'meeting', 'schedule', 'yesterday' candidates
  });

  it('respects k parameter', () => {
    const stems = extractKeywordStems('one two three four five six seven', 2);
    expect(stems.length).toBeLessThanOrEqual(2);
  });

  it('returns empty array for stop-word-only input', () => {
    const stems = extractKeywordStems('a is to the and of for', 3);
    expect(stems).toEqual([]);
  });

  it('Porter-stems plurals/tenses', () => {
    const a = extractKeywordStems('users meetings schedules', 3);
    const b = extractKeywordStems('user meeting schedule', 3);
    expect(a).toEqual(b);
  });
});
```

**Step 7: Commit (Task 1 SHA = audit anchor — "goal-stack hooks added BEFORE C2 ran")**

```bash
git -C C:/Users/skf_s/hippo add benchmarks/longmemeval/retrieve_inprocess.mjs benchmarks/longmemeval/keyword-extractor.mjs tests/longmemeval-keyword-extractor.test.ts
git -C C:/Users/skf_s/hippo commit -m "feat(longmemeval): add session-scoped goal-stack hooks + content-keyword extractor (v1.9.0 v2)"
```

---

## Task 2: Pilot smoke test on 10 questions (NOT pre-registration; authoring iteration)

**Step 1: Run pilot with `--use-goal-stack --max-questions 10`**

```bash
cd C:/Users/skf_s/hippo/benchmarks/longmemeval
node retrieve_inprocess.mjs --data longmemeval_oracle.json --store-dir hippo_store \
  --use-goal-stack --max-questions 10 --output results/v1.9-pilot/retrieval.jsonl
```

**Step 2: Verify**

- All 10 retrievals have `goal_tags_pushed` populated (non-null, non-empty).
- `boost_fired = true` on at least 1 of 10 questions (mechanism non-degeneracy: boost path is reachable on this corpus).
- No crashes, no hook failures, no degenerate empty retrievals.

**Step 3: Acceptance**

- IF `boost_fired = true` on ≥ 1 of 10 → mechanism path is plumbed correctly. ACCEPT.
- IF `boost_fired = false` on all 10 → boost path is structurally inert (no overlap between question-keywords and session-tags). STOP. Escalate per Task 0 decision gate (option-A: modify ingest.py to auto-tag content; option-B: pivot v1.9).

**Step 4: Iteration log + commit**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.9.0-pilot-iteration-log.md
git -C C:/Users/skf_s/hippo commit -m "docs(longmemeval): v1.9 pilot smoke test (10 questions, boost-firing-rate K/10)"
```

---

## Task 3: Pre-registration document

**Files:**
- Create: `docs/evals/2026-05-09-v1.9.0-longmemeval-eval-prereg.md`

**Required content (use v1.8 prereg as template):**

- Hypothesis: NONE (per RETRACTION.md). Two questions: workload-validity + sign-only direction count + boost-firing-rate.
- **Operational definition of "characterise"** (verbatim from this plan; 5 outputs).
- **Cumulative-null retraction-escalation trigger** (verbatim from this plan).
- **Pre-committed v1.10 direction** (goal-tag mapping iteration).
- Workload-validity sanity gate: C1 R@5 ∈ [0.55, 0.90] (TIGHTENED from v1's [0.40, 0.95] per outside voice B2; centred on 74% historical with explicit acknowledgement that this is procedural, not discriminating).
- Discrimination check: boost-firing-rate ≥ 50/500 (mechanism non-degeneracy gate; if boost fires on <10% of questions, the mechanism is functionally inert on this workload — flag in result doc).
- Conditions: C1 (no goal-stack), C2 (session-scoped goal-stack via pushGoal). Same `hippo_store`.
- Reporting lock: 5 outputs from "characterise" definition. NO per-category breakdown (per outside voice B1). NO C0 in the workload-baseline section (per outside voice B6); C0 cited prose-only with provenance caveat.
- Audit anchor: Task 0 commit SHA (pre-investigation) + Task 1 commit SHA (mechanism plumbing) — both precede C2 run.
- Outside voice questions (5 named below).
- RETRACTION.md compliance declaration with hardened terminology list.

**Outside voice questions to brief (must answer verbatim in result.md):**

1. Did Task 0 pre-investigation confirm the auto-tag system permits content-keyword overlap with session tags?
2. Did the workload-validity gate (C1 R@5 ∈ [0.55, 0.90]) hold? What was the procedural-vs-discrimination distinction in practice?
3. Did the boost-firing-rate exceed the 50/500 non-degeneracy threshold? If not, what does that imply about the mechanism's reach on this corpus?
4. Did the descriptive characterisation framing hold across all artifacts (sign-only, no per-category breakdown, no juxtaposition)?
5. Did the cumulative-null retraction-escalation trigger fire? If so, was the `docs/RETRACTION.md` "Mechanism-effect status" subsection appended verbatim?

**Step 2: Commit**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.9.0-longmemeval-eval-prereg.md
git -C C:/Users/skf_s/hippo commit -m "docs(evals): pre-registration for v1.9.0 LongMemEval cross-validation (v2 with cumulative-null trigger)"
```

---

## Task 4: Claim inventory + retraction-compliance check

Same shape as v1.8 Task 5. Hardened magnitude grep including the new forbidden terms (`STRICTLY_BETTER`, `STRICTLY_WORSE`, "lift", "reduction", "improvement"). Retraction-citation allowlist applies.

**Commit:**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.9.0-claim-inventory.md
git -C C:/Users/skf_s/hippo commit -m "docs(evals): v1.9.0 claim inventory + retraction-compliance check (hardened)"
```

---

## Task 5: Run C1 (no goal-stack) — sanity preflight + full eval

**Step 1: Full retrieval**

```bash
cd C:/Users/skf_s/hippo/benchmarks/longmemeval
node retrieve_inprocess.mjs --data longmemeval_oracle.json --store-dir hippo_store \
  --output results/v1.9-eval-C1-hippo-base/retrieval.jsonl
# (no --use-goal-stack)
python evaluate_retrieval.py --retrieval results/v1.9-eval-C1-hippo-base/retrieval.jsonl \
  --data longmemeval_oracle.json --output results/v1.9-eval-C1-hippo-base/evaluation.json
```

**Step 2: Capture hippo_store SHA (per outside voice B9)**

```bash
git -C hippo_store rev-parse HEAD 2>&1 || sha256sum hippo_store/memories.sqlite | cut -d ' ' -f 1 > /tmp/v1.9-c1-store.sha
```

(If the store is a SQLite file, hash it. Used to assert C2 runs on the SAME store.)

**Step 3: Apply sanity gate**

- Sanity PASS: C1 R@5 ∈ [0.55, 0.90]. Proceed to Task 6.
- Sanity FAIL: outside band. Apply SANITY_FAIL template + retarget to v1.10. Do NOT run C2.

**Step 4: Commit verdict**

```bash
git -C C:/Users/skf_s/hippo commit --allow-empty -m "eval(v1.9): C1 sanity preflight verdict ({PASS|FAIL}) — R@5={value}, store_sha={...}"
```

---

## Task 6: Run C2 (session-scoped goal-stack) — only if Task 5 PASS

**Step 1: Verify hippo_store SHA stability**

```bash
git -C hippo_store rev-parse HEAD 2>&1 || sha256sum hippo_store/memories.sqlite | cut -d ' ' -f 1 > /tmp/v1.9-c2-store.sha
diff /tmp/v1.9-c1-store.sha /tmp/v1.9-c2-store.sha && echo "STABLE" || echo "DIFFERENT — STOP"
```

If different, STOP. Comparison is corrupted. Re-run C1 + C2 in close sequence on a frozen store.

**Step 2: Full retrieval with goal-stack**

```bash
node retrieve_inprocess.mjs --data longmemeval_oracle.json --store-dir hippo_store \
  --use-goal-stack --output results/v1.9-eval-C2-hippo-goalstack/retrieval.jsonl
python evaluate_retrieval.py --retrieval results/v1.9-eval-C2-hippo-goalstack/retrieval.jsonl \
  --data longmemeval_oracle.json --output results/v1.9-eval-C2-hippo-goalstack/evaluation.json
```

**Step 3: Compute summary stats**

Write `tools/v1.9-summarise.mjs`:
1. Load C1 + C2 retrieval JSONLs.
2. Per question_id, compute: was answer in top-5 for each? what was rank-1 memory_id for each?
3. Sign-only direction: STRICTLY_HIGHER (C2 hit, C1 miss) / STRICTLY_LOWER (C1 hit, C2 miss) / SAME (both hit OR both miss).
4. Boost-firing-rate: count C2 questions with `boost_fired = true` (or, if telemetry path (b), questions where C2 top_1 ≠ C1 top_1).
5. Per-question rank-shift: count questions where top_1 differs between C1 and C2.
6. C2 outcome distribution: {answer_in_top_5: k_yes, answer_not_in_top_5: k_no}.

Output the 5 numbers + JSON for the result-doc fill-in.

**Step 4: Commit**

```bash
git -C C:/Users/skf_s/hippo commit --allow-empty -m "eval(v1.9): C2 vs C1 — sign-only HIGHER/LOWER/SAME = {N}/{N}/{N}, boost-fired={N}/500, rank-shift={N}/500"
```

---

## Task 7: Write the result doc (descriptive characterisation, two-section format)

**Files:**
- Create: `docs/evals/2026-05-09-v1.9.0-longmemeval-eval-result.md`

Use v1.8 result.md as template. Two-section structure:
- Workload-validity sanity verdict.
- **Workload baseline section: C1 only** (per outside voice B6 — drop C0 from this section; cite prose-only with "different-store provenance" caveat in a separate prose paragraph).
- C2 mechanism characterisation section: sign-only direction count + lattice histogram + boost-firing-rate + rank-shift descriptive.
- Verbatim "Do not subtract across sections" sentence in EACH section.
- Verbatim "This release does not re-assert the retracted −10pp magnitude" sentence.
- 5 outside-voice questions answered verbatim.
- Reproducibility anchors.
- IF cumulative-null trigger fires (SAME ≥ 495/500 AND boost-firing-rate ≥ 50/500), append verbatim escalation note to `docs/RETRACTION.md` "## Mechanism-effect status (cumulative null escalation)" subsection.

**Forbidden language self-check (hardened):**

```bash
grep -nE "(Δ\s*=|\b\d+\s*pp|−\d+pp|pp lift|pp drop|≥\s*\d+pp|magnitude|discriminating|discrimination|median.{0,20}pp|5-number summary|paired diff|lift|reduction|improvement|STRICTLY_BETTER|STRICTLY_WORSE|better on (single-session|multi-session|temporal|knowledge-update|abstention))" \
  docs/evals/2026-05-09-v1.9.0-longmemeval-eval-result.md | grep -v -i "retracted"
```

Expected: zero hits.

**Commit:**

```bash
git -C C:/Users/skf_s/hippo add docs/evals/2026-05-09-v1.9.0-longmemeval-eval-result.md
git -C C:/Users/skf_s/hippo commit -m "docs(evals): v1.9.0 LongMemEval result (two-section, sign-only, boost-firing-rate)"
```

---

## Task 8: Update CHANGELOG / README / ROADMAP-RESEARCH / TODOS / docs/RETRACTION.md

Same shape as v1.8 Task 10. v1.9.0 entry uses sign-only + boost-firing-rate + rank-shift descriptive ONLY. NO per-category breakdown. NO C0 juxtaposition. Append `## v1.9.0 outcome` to docs/RETRACTION.md. If cumulative-null trigger fires, ALSO append `## Mechanism-effect status` subsection.

---

## Task 9: Bump version 1.8.0 → 1.9.0 across 5 manifests

Same pattern as v1.8 Task 11.

---

## Task 10: Run full test suite + build

```bash
cd C:/Users/skf_s/hippo
npm run build
npx vitest run
```

Expected: ALL passing (1508+ from v1.8 + new keyword-extractor tests = ~1512+), 0 regressions.

---

## Task 11: Pre-publish staged-diff outside voice

Same brief shape as v1.8 Task 13. Hardened magnitude grep with forbidden terms list extended (per outside voice B8).

---

## Task 12: /self-review → /review → /ship-check → /publish-repo

Same pattern as v1.8 Task 14. Plus dual `hippo remember` + decision-log update commit.

---

## Risk register

| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Task 0 pre-investigation finds auto-tag system not content-derived | Medium | High | STOP before pre-registration. Decision gate spells out option-A (ingest.py change) vs option-B (pivot). |
| Pilot (Task 2) shows boost_fired=false on all 10 → boost path inert | Medium | High | STOP at pilot, before pre-reg locks. Don't proceed with designed-to-be-null eval. |
| Outside voice flags v2 framing as still-soft | Low | High | Hardened forbidden-language list + Task 11 staged-diff outside voice + retraction-citation allowlist all guard. |
| `hippo_store` SHA changes between C1 and C2 | Low | High | Task 5 + 6 capture SHA; Task 6 Step 1 asserts equality, hard-stops if mismatched. |
| C1 sanity preflight FAILs because retrieve_inprocess.mjs result differs from README's 74.0% baseline (different code path / config) | Medium | Medium | If FAIL, document the discrepancy + retarget to v1.10. Do NOT silently re-tune. |
| Cumulative-null retraction-escalation trigger fires | Medium | Medium | Pre-committed; appendage to docs/RETRACTION.md is part of the plan, not a surprise. |
| RETRACTION.md compliance leak via per-category direction-with-attribution | Low | High | Plan forbids per-category breakdown entirely. Pre-publish self-grep + Task 11 outside voice both verify. |
| Pre-commit hook failures | Low | Medium | NEVER use `--no-verify`. |

---

## Decision log

This plan v2 is a deliberate rewrite after outside voice flagged 4 architectural P0s in v1. Documented:

**v1 problems (corrected):**
- Wrong harness (`retrieve.py` non-canonical) → v2 uses `retrieve_inprocess.mjs`.
- Fictional `--use-goal-stack` flag → v2 invents the flag in MJS, plumbs to actual `applyGoalStackBoost` path via session-scoped pushGoal/completeGoal.
- Conflated `--goal` single-tag override with `applyGoalStackBoost` → v2 explicitly drives the latter (sessionId set, goalTag empty).
- `question_type` meta-categories as goal-tag (designed-to-be-null) → v2 uses content-keyword stems (Porter-stem from v1.8 Jaccard tokenizer).

**v1 framing fixes (corrected):**
- Per-category breakdown leaks magnitude through clustering → v2 drops it entirely.
- Sanity gate `[0.40, 0.95]` calibrated to PASS → v2 uses `[0.55, 0.90]` AND adds boost-firing-rate non-degeneracy gate.
- C0 + C1 side-by-side invites mental subtraction → v2 reports C1 only in baseline section; C0 cited prose-only.
- "STRICTLY_BETTER" / "STRICTLY_WORSE" value-loaded → v2 uses "STRICTLY_HIGHER" / "STRICTLY_LOWER" / "SAME" matching v1.8 vocabulary.
- Cumulative-null trigger missing → v2 pre-commits the trigger BEFORE v1.9 runs.
- v1.10 = embeddings (substrate change, not mechanism fix) → v2 v1.10 = goal-tag mapping iteration; embeddings deferred to v1.11.
- C0 provenance unchecked → v2 cites with explicit "different-store provenance" caveat.

**What this plan does NOT do:**
- Does NOT re-assert the retracted −10pp magnitude.
- Does NOT pre-register a magnitude pass/fail decision rule.
- Does NOT use SUPPORTED / NOT_SUPPORTED templates.
- Does NOT use STRICTLY_BETTER / STRICTLY_WORSE terminology.
- Does NOT report per-category direction-with-attribution.
- Does NOT cross-cite v1.7.5/6/7/8 numbers as comparable to LongMemEval.
- Does NOT pre-tag sessions with question_type at ingest (would leak).

**Outside voice questions (5 named, must answer verbatim in result.md per outside voice C6):** see Task 3.

---

## Time budget

- Task 0 (pre-investigation read + decision gate): ~45 min careful reading + 15 min decision doc.
- Task 1 (flag plumbing + keyword extractor + tests): ~60 min code work.
- Task 2 (pilot smoke test on 10 questions): ~10 min subprocess.
- Task 3 (prereg with hardened terminology + cumulative-null trigger + v1.10 correction): ~50 min doc work.
- Task 4 (claim inventory + retraction-compliance grep): ~20 min doc work.
- Task 5 (C1 full eval + sanity preflight + store SHA): ~20-30 min subprocess.
- Task 6 (C2 full eval — only if C1 PASS, includes summariser script): ~30-40 min subprocess.
- Task 7 (result doc, two-section, sign-only, OV answers): ~50 min.
- Task 8 (CHANGELOG/README/ROADMAP-RESEARCH/TODOS/RETRACTION.md): ~40 min doc work.
- Task 9 (version bump): ~5 min.
- Task 10 (test suite): ~3 min subprocess.
- Task 11 (staged-diff outside voice): ~15 min subprocess.
- Task 12 (ship chain + dual hippo remember + decision-log update): ~30 min.
- **Total wall time: ~5-6 hours.** Subprocess work: ~80-100 min.

---

## Pre-registration commit ordering (CRITICAL — pre-commit lock + double audit anchor)

1. Branch from master at HEAD (post-v1.8.0).
2. **Task 0 — pre-investigation gate** → commit (Task 0 SHA = "design assumptions verified BEFORE pre-reg").
3. **Task 1 — flag plumbing + keyword extractor + tests** → commit (Task 1 SHA = "mechanism hooks added BEFORE C2 ran").
4. Task 2 — pilot smoke test result + iteration log → commit.
5. Task 3 — prereg → commit (references Task 0 + Task 1 SHAs as audit anchors).
6. Task 4 — claim inventory + retraction-compliance grep → commit.
7. Task 5 — C1 sanity preflight result + store SHA → commit.
8. Task 6 — C2 result (only if Task 5 PASS) + store-stability assertion → commit.
9. Task 7 — result.md → commit.
10. Task 8 — CHANGELOG/README/ROADMAP-RESEARCH/TODOS/RETRACTION.md → commit.
11. Task 9 — version bump → commit.
12. Task 10 — test suite passes (no commit; verification only).
13. Task 11 — staged-diff outside voice (no commit; gating only).
14. Task 12 — ship chain.

**~13 atomic commits + decision-log update commit on `feat/v1.9.0-longmemeval-crossval`.** Merge to master via `--no-ff` after `/ship-check` returns "Ship it".

**Two load-bearing audit anchors:** Task 0 SHA (pre-investigation) AND Task 1 SHA (mechanism plumbing). Both precede C2 run. Both must be cited in prereg, result.md, and CHANGELOG.
