4eea584c9d
Sub509 docker logs show 99.98-100% NaN rate in ALL GatedDeltaNet layers. nan_to_num(nan=0.0) replaces them with zeros — entire DeltaNet layers produce zero output, crippling model quality. This is root cause of: - d03_tool_call FAIL (model too impaired to output <tool_call> XML) - d01 content[0] (no content output, only 1085 reasoning tokens) - d04 content[0] (same: reasoning but no content) - 11x slower than opponent (model generates excessive tokens) Five-layer fix based on CCCL overflow_cast_t pattern: 1. Pre-cumsum clamp: g.clamp(-0.5, 0.5) before cumsum (was: no pre-clamp) - Limits cumsum growth to ±32 for chunk_size=64 - Post-cumsum clamp tightened from ±20 to ±12 2. A_log clamp tightened: [-20,20] → [-5,5] - exp(5) ≈ 148 vs exp(20) ≈ 4.9e8 - Prevents extreme decay rates that feed into g 3. Forward substitution per-row clamp: ±1e4 - _forward_sub_lower was the primary NaN amplifier - Each x[i] = rhs[i] + A[i,:i]@x[:i] now clamped 4. Cross-chunk state clamp: ±1e4 - last_state *= g_last_exp can blow up across many chunks - Both prefill and decode paths protected 5. Decode temporal_state in-place clamp: ±1e4 - ts_flat.clamp_() after baddbmm_ state update Also adds SUB509_DEEP_DIAGNOSIS.md with full root cause analysis.