4daa30a267
CCCL source: kernel_segmented_scan.cuh (675 lines) Core design: segmented scan with three-way dispatch: 1. Fixed-size segments → direct division (fast path) 2. Variable-size → branchless search 3. Fallback → basic scan Applied to qwen3_5.py — three critical fixes: FIX #1: Remove nan_to_num(nan=0.0) from both prefill and decode paths. This was the double disaster: it hid NaN (making model look alive while outputting garbage) AND filled all outputs with zeros (making every layer's input all-zeros → semantically dead model → 0 points). Now: NaN is logged but propagated for honest failure detection. FIX #2: Add corex_gdn native dispatch in GatedDeltaNet.forward. Sub168 docker log proves: corex_gdn.py:56 loads libcorex_gdn.so, corex_gdn.py:228 uses fused prefill operator → zero NaN, 17.35GB weights. Our code never called this module. Now we try to import and use it. FIX #3: Add corex_moe native dispatch in MoeSparseBlock.forward. Sub168 docker log: corex_moe.py:339 Using CoreX fused MoE prefill operator: expert-grouped-wmma. Our code only tried ixformer.functions which lacks MoE kernels. Now we also check for corex_moe.py. FIX #4: MoE native retry instead of permanent abandon after first failure. Fallback analysis: #3 (ixformer import → all-False) + #4 (nan_to_num) + #5 (permanent MoE abandon) = the exact combination that produced Sub508's 0 score.