# NPC Reason - GGUF decision-fidelity (Dispatch 4, Stage 1)

GGUFs of the shipped RL model (`rl/merged`), built with llama.cpp (CUDA). Fidelity is validated on
DECISIONS, not just structure (the Fin Nano lesson: a structurally-valid quant can still drift the
model's decisions). Each quant ran the SAME 100 frozen-eval problems greedy via llama-cpp-python
(`create_chat_completion`, the model's embedded chat template), scored by the FROZEN verifier
(d5d146cf), and compared to the bf16 model on those exact 100 problems.

Tolerance: a quant is fidelity-OK if verified-and-correct is within +/-2pp of bf16.

| model   | size   | verifiable | verified-and-correct | dVC vs bf16 | fidelity |
|---------|-------:|-----------:|---------------------:|------------:|----------|
| bf16    | 3.5 GB |     93.0%  |               71.0%  |      ref    | ref      |
| f16     | 3.6 GB |     95.0%  |               71.0%  |     +0.0    | OK       |
| **q8_0**| 1.9 GB |     95.0%  |               71.0%  |     +0.0    | **OK (recommended)** |
| q4_K_M  | 1.1 GB |     95.0%  |               77.0%  |     +6.0    | DRIFTS (flagged) |

(The 100-problem subset is GSM8K-heavy, so the absolute v&c is higher than the full n=500 number
of 59.6%; fidelity here is the per-quant DELTA, which is the meaningful quantity.)

## Recommendation: q8_0

- **q8_0 is behaviorally lossless** on this product (verified-and-correct delta 0.0 vs bf16). It is
  half the size of f16 and reproduces the model's decisions. Use it.
- **q4_K_M DRIFTS and is flagged, not recommended.** Its verified-and-correct moved +6pp (well past
  the +/-2pp tolerance). The direction happens to be upward on these 100, but that is a
  decision CHANGE under quant noise, not fidelity. Per the Fin Nano lesson, do not choose a quant on
  file size alone; q4_K_M's chains are not the bf16 model's chains. Use only if a 1.1 GB footprint
  is required and the decision drift is acceptable for the use case.
- f16 is provided as the lossless reference.

Validation harness: `gguf/fidelity_eval.py` (frozen verifier, greedy, n=100). Raw:
`gguf/fidelity_result.json`.