54 lines
3.1 KiB
Markdown
54 lines
3.1 KiB
Markdown
|
# NPC Reason — SFT result vs frozen baseline + PREREG (Dispatch 2, Stage 4)
|
|||
|
|
|
|||
|
|
NPC-Reason-SFT = R1-Distill-1.5B + LoRA (r64), distilled on **7,546 V4 chains that the FROZEN
|
|||
|
|
verifier confirmed verifiable AND correct**. Evaluated on the frozen EVAL.lock (500 held-out
|
|||
|
|
GSM8K+MATH), same greedy decoding as the baseline (apples-to-apples), scored by the frozen
|
|||
|
|
verifier (VERIFIER.lock d5d146cf).
|
|||
|
|
|
|||
|
|
## Result (format-prompt = the committed comparator)
|
|||
|
|
|
|||
|
|
| | base (format) | NPC-Reason-SFT | delta |
|
|||
|
|
|------------------------|--------------:|---------------:|--------:|
|
|||
|
|
| verifiable-rate | 0.0% | **76.8%** | +76.8pp |
|
|||
|
|
| accuracy | 61.6% | **65.8%** | +4.2pp |
|
|||
|
|
| verified-and-correct | 0.0% | **58.0%** | +58.0pp |
|
|||
|
|
| mean assertions/chain | 0.0 | 3.44 | — |
|
|||
|
|
|
|||
|
|
Plain-prompt (native, no format instruction): verifiable 0.0%, accuracy 64.8%. The model emits
|
|||
|
|
the `<<EXPR=RESULT>>` format under the format prompt it was trained on, not unprompted — expected,
|
|||
|
|
and the format prompt is the committed comparator.
|
|||
|
|
|
|||
|
|
## PREREG verdict (committed bars, frozen b5a49437 — graded honestly)
|
|||
|
|
|
|||
|
|
| Criterion | Bar | Result | Verdict |
|
|||
|
|
|---|---|---|---|
|
|||
|
|
| 1. Primary lift (verified-and-correct over format baseline) | ≥ +15pp | **+58.0pp** | **SUPPORTED** |
|
|||
|
|
| 2. Verifiable-rate | ≥ 90% | 76.8% | **NOT SUPPORTED** |
|
|||
|
|
| 3. Accuracy guard (no regression > 5pp) | ≥ −5pp | **+4.2pp** (improved) | **SUPPORTED** |
|
|||
|
|
|
|||
|
|
**2 of 3 SUPPORTED.** The all-three gate is not met (criterion 2 missed).
|
|||
|
|
|
|||
|
|
## Honest read
|
|||
|
|
|
|||
|
|
- **The central thesis is validated, strongly.** Distilling *only mechanically-verified* teacher
|
|||
|
|
chains moved a model that produced **zero** checkable reasoning (0% verifiable, even when asked)
|
|||
|
|
to **76.8% verifiable / 58% verified-and-correct** — a +58pp jump on the headline metric, far
|
|||
|
|
past the +15pp bar. The grounding transferred.
|
|||
|
|
- **Accuracy did not regress — it rose** (+4.2pp). Verifiability was not bought by sacrificing
|
|||
|
|
correctness; the strict-basis corpus improved both. (Some of the rise is reduced greedy
|
|||
|
|
repetition vs the base; attributed, not overclaimed.)
|
|||
|
|
- **The one missed bar is verifiable-rate (76.8% vs 90%).** ~23% of format-prompt chains still
|
|||
|
|
fail full verification — a step that doesn't verify, doesn't compose, or the model reverts to
|
|||
|
|
prose on harder MATH items. This is exactly the gap RL-with-verifier-reward is designed to close.
|
|||
|
|
|
|||
|
|
## RL gate
|
|||
|
|
|
|||
|
|
The committed gate is: RL proceeds if SFT clears its bar **OR** produces a high verifiable-rate
|
|||
|
|
so RL has signal to sharpen. All-three bar: **NOT cleared** (verifiable 76.8 < 90). High-signal
|
|||
|
|
condition: **MET** — 76.8% verifiable / 58% verified-and-correct gives RL (verifier = reward, the
|
|||
|
|
same frozen code) dense signal to push toward ≥90%. So the RL phase is **warranted**, entered with
|
|||
|
|
eyes open that criterion 2 is the explicit target. A null at SFT would have been valid; this is not
|
|||
|
|
a null — it is a large, real lift that misses one of three bars.
|
|||
|
|
|
|||
|
|
Frozen: `sft/SFT_RESULT.lock`. Raw per-item outputs: `eval/eval_raw.jsonl`.
|