EX Engine 8eba1750fa fix(GDN): clamp gate [-5,2] + state [-65504,65504] to prevent inf/NaN
Root cause from real machine test: gdn_forward.cu output abs mean = inf
- gate_raw can be positive → exp(gate) > 1 → state grows exponentially
- Over 64 tokens: exp(2.0)^64 = inf
- PyTorch ref clamps g ∈ [-5, 2] but CUDA kernel did not

Fix:
  gdn_forward.cu: clamp gate_raw ∈ [-5, 2] before exp (both kernel variants)
  gdn_forward.cu: clamp state ∈ [-65504, 65504] after update (fp16 safe range)
  qwen3_5.py: clamp g_3d before passing to SM70 kernel (belt + suspenders)
  qwen3_5.py: clamp temporal_state after decode update
2026-08-10 03:38:46 +00:00
2026-08-07 08:43:51 +00:00
2026-07-30 17:03:23 +08:00
2026-08-07 07:47:01 +00:00

project_6

Description
No description provided
Readme 427 MiB
Languages
C++ 41.8%
Cuda 31.6%
Python 22.2%
C 2.1%
CMake 1.1%
Other 1.1%