e69c46d0b7
This is the key differentiator vs parameter brute-force.
Everyone else hardcodes BLOCK_SIZE=64, NUM_WARPS=4, PARTITION_SIZE=512.
muh_dispatch replaces these with type-dispatched values derived from
CCCL's policy_selector architecture.
Dispatch axes (matching CCCL type_t × op_kind_t × offset_size):
- dtype → determines accum_size, SMEM per element
- head_dim → determines tile width, SMEM constraint
- max_seq_len → determines V1/V2 threshold (single_tile vs multi_tile)
- num_kv_heads → determines GQA ratio (memory access pattern)
Output: AttentionConfig struct with all kernel parameters.
CCCL reference: ReducePolicy{multi_tile, single_tile} pattern.
Example type dispatches for Qwen3.6 on BI-V100:
bf16 h128 100K → partition=512, v1_thresh=8192, reduce(512,16,vec=4)
bf16 h256 100K → partition=256, v1_thresh=8192, reduce(512,16,vec=4)
fp32 h128 32K → partition=256, v1_thresh=8192, reduce(512,16,vec=4)
bf16 h128 2K → v1_thresh=2049 (always V1, skip V2 overhead)