# NPC Reason — Pre-registration of lift criteria (FROZEN before any training)

Committed in Dispatch 1, before any SFT / distillation / RL exists. Same discipline as
BigBugAI v2: the success bars are fixed in advance so the eventual before/after delta means
something. These bars are deltas/targets chosen on principle; they do NOT depend on the
measured baseline values (recorded separately in `reports/baseline.md` / `BASELINE.lock`),
so there is no possibility of tuning the bars to the baseline.

The comparator for the primary bar is the base model's **FORMAT-PROMPT** number — the honest
question is "did training add something that prompting the base alone could not?", not
"did training beat a base that was never asked for the format?".

## Metrics (all from the frozen mechanical verifier, VERIFIER.lock)

- **verifiable-rate** — fraction of chains that are mechanically VERIFIABLE
  (≥1 load-bearing `<<EXPR=RESULT>>` assertion, all assertions verify, final answer composes).
- **accuracy** — fraction whose final answer equals gold (independent of verifiable).
- **verified-and-correct** — fraction that are BOTH verifiable AND correct. This is the headline.

## Committed criteria

1. **Primary (lift).** SFT NPC Reason must raise **verified-and-correct** by
   **≥ +15 percentage points absolute** over the base model's **format-prompt** verified-and-correct rate.

2. **Verifiable-rate target.** NPC Reason must reach **verifiable-rate ≥ 90%**. The project's
   purpose is near-total mechanical grounding of load-bearing steps; a specialized model should
   emit the checkable format almost always.

3. **Accuracy guard (no regression trade).** NPC Reason **accuracy** must not regress by more
   than **5 percentage points** versus the base model's **format-prompt** accuracy. Verifiability
   must not be bought by sacrificing correctness — both axes matter.

   SFT passes Dispatch 2 only if (1) AND (2) AND (3) all hold.

4. **RL gating (later dispatch).** The RL phase is gated on SFT meeting criteria 1–3. RL's own
   criteria will be pre-registered at that time, evaluated against a held-out slice never used
   for SFT model selection. The verifier (VERIFIER.lock) is the RL reward and must remain
   byte-identical.

## Honesty clause

Report whatever the numbers say. A **null** — training fails to beat the format-prompt
baseline on the primary bar — is a valid, publishable finding, consistent with this being a
real experiment and not a foregone conclusion. The bars above are not moved after results are
seen; if they are missed, that is the result.

## Frozen references

- Verifier: `verifier/VERIFIER.lock`
- Eval set: `eval/EVAL.lock`
- Baseline: `baseline/BASELINE.lock`
- This pre-registration: `reports/PREREG.lock` (sha256 of this file).