CCCL source: kernel_segmented_scan.cuh (675 lines)
Core design: segmented scan with three-way dispatch:
1. Fixed-size segments → direct division (fast path)
2. Variable-size → branchless search
3. Fallback → basic scan
Applied to qwen3_5.py — three critical fixes:
FIX#1: Remove nan_to_num(nan=0.0) from both prefill and decode paths.
This was the double disaster: it hid NaN (making model look alive while
outputting garbage) AND filled all outputs with zeros (making every
layer's input all-zeros → semantically dead model → 0 points).
Now: NaN is logged but propagated for honest failure detection.
FIX#2: Add corex_gdn native dispatch in GatedDeltaNet.forward.
Sub168 docker log proves: corex_gdn.py:56 loads libcorex_gdn.so,
corex_gdn.py:228 uses fused prefill operator → zero NaN, 17.35GB weights.
Our code never called this module. Now we try to import and use it.
FIX#3: Add corex_moe native dispatch in MoeSparseBlock.forward.
Sub168 docker log: corex_moe.py:339 Using CoreX fused MoE prefill
operator: expert-grouped-wmma. Our code only tried ixformer.functions
which lacks MoE kernels. Now we also check for corex_moe.py.
FIX#4: MoE native retry instead of permanent abandon after first failure.
Fallback analysis:
#3 (ixformer import → all-False) + #4 (nan_to_num) + #5 (permanent MoE abandon)
= the exact combination that produced Sub508's 0 score.
4daa30a267