Files
project_6/qwen3_6_scripts
muh 082ded7d69 [ENGINE] xformers.py: CCCL GQA broadcast eliminates 6x repeat_interleave in sdpa_fallback
Qwen3.6 head_dim=256 forces sdpa_fallback path (head_size > 128).
Old code: repeat_interleave(6, dim=0) expands KV from [4, seq, 256]
to [24, seq, 256] — 6x memory copy every prefill Q-chunk.

New code: CCCL agent_reduce.cuh ConsumeFullTile broadcast pattern.
K/V stay at [kv_h, 1, seq, d], Q reshaped to [kv_h, gqa, chunk, d].
matmul broadcasts K over gqa dim without materializing the expansion.

For Qwen3.6 (kv_h=4, gqa=6, d=256, q_chunk=256):
  Old: 6 × 4 × seq × 256 × 4B = 24 × seq × 1KB expanded per chunk
  New: 4 × 1 × seq × 256 × 4B = 4 × seq × 1KB (no expansion)

CCCL source: agent_reduce.cuh VectorT striped access pattern,
catch2_test_device_find_env.cu find_tuning<BlockSize> injection.
2026-08-06 00:59:40 +00:00
..