fix(precision): guard all corex .so outputs with nan_to_num + reduce max-model-len

MoE kernels:
- topk_softmax: add .contiguous() + nan_to_num + re-normalize weights
- direct_routed: nan_to_num on w2_reduce output
- exact_reduce: nan_to_num on serial_float output

GDN kernels:
- packed_decode: nan_to_num on core_out

BI-V100 CUB may produce non-finite values in fp16 softmax/reduce.
These guards prevent garbage propagation without disabling the kernels.

max-model-len: 256000 → 131072 (4x32GB BI-V100 OOM prevention)
Dockerfile: unchanged (no force push needed)
This commit is contained in:
project6-dev
2026-08-13 02:12:16 +00:00
parent 8abc7cb0d7
commit 8acc47129b
2 changed files with 13 additions and 4 deletions

View File

@@ -8,7 +8,7 @@ command:
- --served-model-name
- llm
- --max-model-len
- '256000'
- '131072'
- --gpu-memory-utilization
- '0.95'
- --trust-remote-code