d15dcea7c6
Source: qwen3_6_scripts/xformers.py (competition-specific)
CCCL ref: agent_reduce.cuh ConsumeFullTile (GQA broadcast)
block_load_to_shared.cuh (loop invariant hoisting)
agent_sub_warp_merge_sort.cuh (buffer reuse)
CRITICAL: Qwen3.6 uses head_dim=256. ixformer flash attention only
supports head_dim<=128. Without this fallback, base xformers.py would
try ixformer flash on head_dim=256 -> crash or wrong results.
SDPA fallback features (CCCL-driven):
1. Q-tiling with _Q_CHUNK=256: O(chunk*seq) memory, not O(seq^2)
2. GQA broadcast matmul: K/V as [kv_h,1,seq,d], broadcast over gqa
groups -> 6x memory savings vs repeat_interleave for Qwen3.6
3. Pre-allocated loop invariants (k_pos, qc_q_pos_base)
4. Float32 softmax to prevent fp16 overflow
This directly impacts all 50+ functional test cases that use prefill.