fix(CRITICAL): max_model_len 100000→80000 (KV cache only 88112) + NaN fix

Docker log proves two fatal issues:

1. max_model_len=100000 > KV cache capacity 88112 → ValueError crash
   'max seq len (100000) is larger than maximum number of tokens
    that can be stored in KV cache (88112)'
   Fix: set max_model_len=80000 (safe margin below 88112)

2. NaN in GatedDeltaNet layers 34,36,37,38 (frac=1.0000)
   Root cause: g.cumsum() → g.exp() overflow to inf → inf*0 = NaN
   Fix: clamp all g values to [-80,80] before exp() calls
   (max safe float32 exp input ~88, use 80 for margin)
   Applied to: cumsum result, k_cumdecay, attn_inter, last_state update

3. CoreX modules confirmed NOT in base image:
   'CoreX GDN module not found'
   'CoreX MoE module not found'
   → pure PyTorch is the only path, must be numerically stable
This commit is contained in:
Claude
2026-08-08 15:08:00 +00:00
parent c1065aaf2c
commit ff971686d4
2 changed files with 8 additions and 5 deletions

View File

@@ -8,7 +8,7 @@ command:
- --served-model-name
- llm
- --max-model-len
- '100000'
- '80000'
- --gpu-memory-utilization
- '0.9'
- --trust-remote-code