37 lines
2.4 KiB
Markdown
37 lines
2.4 KiB
Markdown
|
=== NPC Reason — Dispatch 2: V4 Corpus + SFT Distill ===
|
||
|
|
|
||
|
|
Corpus: 13,245 V4 (deepseek-v4-flash) chains from TRAIN splits (GSM8K-train + MATH-train 7 subj)
|
||
|
|
-> mechanically filtered against FROZEN verifier (verifiable AND correct)
|
||
|
|
-> 7,546 kept (keep rate 56.97%). TRAINSET.lock frozen.
|
||
|
|
reject breakdown: no_assertions 3894 (29%, mostly non-arithmetic MATH) ·
|
||
|
|
bad_arithmetic 1211 (9%) · non_composing 387 · wrong_answer 206 · dup 1
|
||
|
|
decontam vs EVAL.lock: 0 overlap (train/test disjoint by construction)
|
||
|
|
Finding: the strong teacher emits ungrounded/wrong chains ~43% of the time on this
|
||
|
|
distribution; all rejected before training (the strict-basis move).
|
||
|
|
|
||
|
|
SFT: R1-Distill-1.5B + LoRA r64 (7 modules, 73.9M = 3.99% trainable), batch1 x accum16,
|
||
|
|
3 epochs, seq 1024 constant-pad (0.03% truncated), sdpa + gradient checkpointing.
|
||
|
|
train loss 0.538 -> 0.075, eval 0.311, runtime 2h31m. Merged model saved (3.5GB).
|
||
|
|
|
||
|
|
Result: (format-prompt, vs frozen baseline, greedy, frozen verifier)
|
||
|
|
verifiable 76.8% (base 0.0) · accuracy 65.8% (base 61.6) · verified+correct 58.0% (base 0.0)
|
||
|
|
mean assertions/chain 3.44 (base 0.0)
|
||
|
|
|
||
|
|
PREREG: lift +58.0pp verified-and-correct -> SUPPORTED (bar +15pp)
|
||
|
|
verifiable-rate 76.8% -> NOT SUPPORTED (bar 90%)
|
||
|
|
accuracy delta +4.2pp (improved) -> SUPPORTED (bar: no regress >5pp)
|
||
|
|
=> 2 of 3 SUPPORTED; all-three gate not met.
|
||
|
|
|
||
|
|
RL gate: WARRANTED via the high-signal condition (76.8% verifiable / 58% v&c gives the
|
||
|
|
verifier-reward RL dense signal); all-three bar NOT cleared (verifiable < 90 is the
|
||
|
|
explicit RL target). Not a null — a large real lift missing one of three bars.
|
||
|
|
|
||
|
|
Honest: Thesis validated — distilling ONLY verifier-confirmed chains took the model from 0%
|
||
|
|
mechanically-checkable reasoning to 76.8% verifiable / 58% verified-and-correct, with
|
||
|
|
accuracy rising not falling. The grounding transferred; the remaining ~23% unverifiable
|
||
|
|
tail is the RL phase's job.
|
||
|
|
|
||
|
|
Decisions np-007..np-010 · all frozen locks intact · eval decontaminated · SFT_RESULT.lock frozen
|
||
|
|
Next: Dispatch 3 = RLVR/GRPO with the frozen verifier as reward, target verifiable-rate >=90%,
|
||
|
|
RL criteria pre-registered against a held-out slice never used for SFT selection.
|