c17c490e06f745fd375eb71e2a95b0311bc02bac
ROOT CAUSE: g.clamp(-5,2) before cumsum corrupted gate values. The GDN algorithm computes decay_mask = exp(g_i - g_j) which is numerically stable via subtraction cancelling cumsum growth. Pre-clamping g distorts these differences → wrong decay rates → NaN. xllm reference: qwen3_gated_delta_net_base.cpp lines 170-238 - cumsum first (no pre-clamp) - difference form: (g_i_last - g[:, i]).exp() for state update - k_cumdecay uses g.exp() directly (not clamped) Removed: g.clamp(-5,2), g.clamp(-20,20), g_exp_cache, g_clamped Added: xllm-style g_i_last/g_exp_term/k_g_exp state update
project_6
Description
Languages
C++
41.8%
Cuda
31.6%
Python
22.2%
C
2.1%
CMake
1.1%
Other
1.1%