# v1.7.6 Calibration — Result

**Run:** 2026-05-09
**Verdict:** **B\* = NULL — calibration SANITY_FAIL.** No budget candidate (200, 400, 600, 800, 1000) puts C2 hippo-base into the band [4%, 24%] late-phase trap-hit-rate. Workload's late-phase floor effect is structural, not budget-tunable.

**Pre-registration anchors (post-audit honesty fix, v1.7.8):** v1.7.6 did NOT have a separate prereg or claim-inventory document — the calibration was a parameter sweep with a mechanical selection rule, not a hypothesis run with verdict templates. The "pre-registration" was the rule itself, committed in code BEFORE the calibration ran:
- **Plan v2** (commit `c670ac9`, 2026-05-07): `docs/plans/2026-05-07-v1.7.6-discriminating-workload.md` defines the band `[0.04, 0.24]`, the `selectBStar` rule, the seed offset 10000, and the SANITY_FAIL escalation to v1.7.7.
- **`calibrate.mjs`** (commit `9cd83de`, 2026-05-07): `BAND_LOW`, `BAND_HIGH`, and `selectBStar` codify the rule mechanically, exported and unit-tested.

The calibration sweep ran at commit `9cd83de`+, with both anchor commits already merged on the feature branch. The verdict (B*=NULL) was applied by the rule in code, not by hand. There was no separate `goal-stack-eval-prereg.md` because the calibration step had no hypothesis to test — just a parameter sweep that returned NULL.

**Selection rule (committed in code BEFORE calibration ran):** B* = LARGEST budget where (mean late ∈ [4%, 24%]) AND (lower-CI > 0) AND (not starved). Source: `calibrate.mjs::selectBStar` at commit `9cd83de`.

**Calibration seeds (n=10):** hash-derived from `1000 + 10000 + i` for i in 0..9 — distinct from hypothesis seeds 0..19 by `tests/sl-calibrate.test.ts` pairwise non-collision check.

## Reproducibility anchors

- `hippo --version` = `1.7.5`
- `git rev-parse HEAD (start)` = `9cd83de4d51e9d4474a315892eaaa8a988d9b665`
- `node --version` = `v24.13.0`
- run start = 2026-05-09T06:06:39Z
- run end (calibration sweep) = 2026-05-09T06:25:38Z
- wall-time = ~19 minutes

## Per-budget late-phase trap rate (C2 only, n=10)

| Budget | Late mean | ±95% CI | Lower-CI | In band? |
|--------|-----------|---------|----------|----------|
| 200 | 0.00% | ±0.00pp | 0.00pp | no (mean < 4% lower band) |
| 400 | 0.00% | ±0.00pp | 0.00pp | no (mean < 4% lower band) |
| 600 | 0.00% | ±0.00pp | 0.00pp | no (mean < 4% lower band) |
| 800 | 0.00% | ±0.00pp | 0.00pp | no (mean < 4% lower band) |
| 1000 | 0.00% | ±0.00pp | 0.00pp | no (mean < 4% lower band) |

All 50 single-seed runs (5 budgets × 10 seeds) returned identical `phases.late = 0.0`. Late-phase trap suppression is total at every budget tested.

## B\* selected

**B\* = NULL** — "No candidate satisfies (mean ∈ [0.04, 0.24]) AND (lower-CI > 0)."

## Calibration-script bug discovered + fixed in v1.7.6

The starvation guard (post-review P1-4) was structurally inert. `runOneBudget` read `j.conditions[condName].results[]` to count per-task `memoryRecalled` flags, but `run.mjs::buildOutput` (line 393) does not serialize the per-task `results` array in single-seed JSON — it only writes `overall_trap_hit_rate`, `phases`, `learns`, `improvement_pct`, and `hook_failures`. The extraction returned `[]`, the guard false-positive flagged every candidate as `starved=true`.

**The bug did not affect the verdict.** `lateMean=0%` across all 5 budgets is the load-bearing rejection signal — no candidate would satisfy the band even with a working starvation guard. The bug was masked by the structural floor.

**Fix shipped in v1.7.6:** `runOneBudget` no longer attempts the broken extraction. Returned row shape: `{budget, seedRates, lateMean, lateCI}`. `selectBStar` still respects an optional `starved` field defensively, but rows no longer carry it. `calibration-result.json` was re-derived from the existing per-seed JSONs after the fix; phase data was unchanged.

**Tracked for v1.7.7+:** "Expose per-task results in single-seed JSON" + "re-enable starvation guard" with the corrected schema.

## Why every budget produces 0% late

Per-seed early/mid/late breakdown (samples from stdout):
- budget-200 seed 1: early=78%, mid=22%, late=0%
- budget-1000 seed 6: early=89%, mid=11%, late=0%
- budget-800 seed 2: early=100%, mid=0%, late=0%

Early- and mid-phase trap rates do vary across budgets (suggesting `--budget` IS being honored), but by the late phase (last 8 of 25 trap encounters), accumulated lesson memories make recall reliable at every budget tested. The floor is structural to the 50-task workload, not a budget parameter.

This confirms the pre-registered escalation:

> Budget reduction (the v1.7.6 workload knob) does not produce a discriminating workload for the goal-stack hypothesis. The hypothesis remains untested.

## Pre-committed escalation

Per the v1.7.6 plan and pre-reg P2-4:

- **v1.7.7 (next):** `--restrict-late-to last-4` — narrow the late-phase metric to the last 4 trap encounters where the floor effect is sharpest. Single-line workload tweak; preserves the metric definition shift as the only knob.
- **v1.8 (later):** adversarial trap categories (requires new lesson authoring + harness changes).
- **Separate eval line:** LongMemEval R@5 corpus (different metric; not a substitute).

## What v1.7.6 ships

1. The `--budget` parameter end-to-end through `run.mjs` → `simulate()` → `adapter.recall(query, budget)` (Task 1, commit `fac46b1`). Reusable by future eval variants.
2. `calibrate.mjs` with mechanical `selectBStar` (Task 2, commit `9cd83de`). Re-usable for any future budget-sweep style calibration.
3. This calibration result documenting the workload floor and the bug-fix.
4. Pre-committed escalation note in TODOS.md / CHANGELOG / RESEARCH.md.

## What v1.7.6 does NOT ship

- The −10pp goal-stack hypothesis test on a discriminating workload. **Untested.** The mechanism (B3 dlPFC goal-stack boost) remains shipped from v1.7.4. The magnitude claim is still untested by a public benchmark variant. Tracked as a v1.7.7+ followup with the workload-knob escalation.

## Why we're shipping despite the inconclusive eval (same as v1.7.5)

The pre-registration here is **less formal** than v1.7.5/v1.7.7 — see "Pre-registration anchors" above. v1.7.6 was a parameter sweep with a mechanical rule in code, not a hypothesis run with verdict templates. What was committed before the calibration ran was the plan v2 (`c670ac9`) + the `selectBStar` rule (`9cd83de`); both define the band [0.04, 0.24] and the SANITY_FAIL escalation. We:

- committed the band + rule in code BEFORE running the calibration sweep,
- captured per-seed phase data across 5 budgets × 10 seeds,
- mechanically applied the B* rule (B* = NULL — code, not hand),
- discovered + fixed a `calibrate.mjs` bug (broken starvation guard reading non-existent JSON field) that did NOT change the verdict (lateMean=0% was load-bearing),
- documented the floor effect investigation as the structural reason,
- did NOT run a hypothesis eval (no B* to use, no C3 collected, no goal-stack data leak),
- shipped the calibration finding so the next person can pick up at v1.7.7 escalation.

Honest reporting > brand maintenance. v1.7.6 ships the calibration finding + bug fix. The −10pp magnitude remains untested.

## Raw artifacts

- `benchmarks/sequential-learning/results/v1.7.6-calibration/calibration-result.json` (re-derived post bug-fix)
- `benchmarks/sequential-learning/results/v1.7.6-calibration/budget-{200,400,600,800,1000}/seed-{...}/benchmark-*.json` (50 per-seed JSONs)
- Plan: `docs/plans/2026-05-07-v1.7.6-discriminating-workload.md`
- Calibration script: `benchmarks/sequential-learning/calibrate.mjs`
- Selection rule: `benchmarks/sequential-learning/calibrate.mjs::selectBStar`

Re-derive the JSON from existing seed runs:
```bash
node -e "import('./benchmarks/sequential-learning/calibrate.mjs').then(m => { /* ... */ })"
```
(See repo for the inline regeneration script used post bug-fix.)

## Update 2026-05-09 (forward pointer to v1.7.7)

The "Pre-committed escalation" section above named v1.7.7 → `--restrict-late-to last-4` as the next workload knob. v1.7.7 acts on that escalation:

- Plan: `docs/plans/2026-05-09-v1.7.7-restricted-late-window.md`
- Pre-registration: `docs/evals/2026-05-09-v1.7.7-goal-stack-eval-prereg.md`
- Claim inventory: `docs/evals/2026-05-09-v1.7.7-claim-inventory.md`
- Result (post-eval): `docs/evals/2026-05-09-v1.7.7-goal-stack-eval-result.md`

Verdict will be one of SUPPORTED / NOT_SUPPORTED / SANITY_FAIL per the pre-registered decision rule.

**v1.7.7 result (2026-05-09): SANITY_FAIL.** C2 hippo-base mean = 0.00% across 20 seeds at `--restrict-late-to 4`; 0 of 20 seeds non-zero at the N=4 lattice gate. C3 (goal-stack ON) was NOT collected — no goal-stack data leak. Adapter not starved (early=77.28%, mid=4.50%). Floor effect persists at last-4 just as it did at last-7 (v1.7.5) and across all budgets {200..1000} (v1.7.6). v1.8 retargets to adversarial trap categories with pre-committed constraints. See `docs/evals/2026-05-09-v1.7.7-goal-stack-eval-result.md`.

---

## Update 2026-05-09 (v1.7.9 release) — retraction (forward-pointer)

The −10pp goal-stack lift magnitude tested by this eval is **RETRACTED publicly in v1.7.9**, based on cumulative evidence from three pre-registered workload variants (v1.7.5 SANITY_FAIL, v1.7.6 B*=NULL, v1.7.7 SANITY_FAIL — every C2 hippo-base late mean returned 0% across every seed).

v1.7.9 retracts on cumulative evidence rather than waiting for v1.8. The v1.7.7 prereg's SANITY_FAIL ≠ NOT_SUPPORTED distinction was wrong: three SANITY_FAILs across distinct workload knobs is meaningful negative evidence regardless of formal verdict label. The mechanism (`pushGoal`/`completeGoal` hooks, `--use-goal-stack`, `applyGoalStackBoost`) remains shipped from v1.7.4. **The magnitude is RETRACTED. The mechanism is shipped; no magnitude is currently claimed.** v1.8.0 (queued) explores adversarial trap categories as mechanism characterisation under the magnitude-smuggling guard in `docs/RETRACTION.md`.

See:
- `CHANGELOG.md` v1.7.9 entry
- `docs/evals/2026-05-09-v1.7.9-retraction-inventory.md`
- `docs/RETRACTION.md`
- `README.md` "What's new in v1.7.9"
