ab81329cb4
Source input for this commit:
- CCCL bench/adjacent_difference/subtract_left.cu (randomly selected)
→ Learned: %RANGE% parameter search + policy_selector_t override pattern
- CCCL bench/reduce/sum.cu + base.cuh
→ Learned: scale_mem_bound adapts (threads, items, vec) to hardware
→ 3 search dims: ipt 7:24, tpb 128:1024, ipv 1:2
- CCCL bench/scan/exclusive/sum.cu
→ Learned: 7 search dims including delay_ns, L2_write_latency
→ This is why nobody wins by guessing — NVIDIA searches 7D space
- CCCL thrust/examples/summary_statistics.cu
→ Welford parallel merge = paged_attention_v2 partition merge pattern
- Base engine: vllm/worker/cache_engine.py (already has CCCL layout/slot)
- Base engine: vllm/attention/ops/paged_attn.py (V1/V2 dispatch)
- Base engine: vllm/attention/ops/prefix_prefill.py (Triton prefill)
Changes:
prefix_prefill.py:
- Replaced hardcoded BLOCK=64/NUM_WARPS=4 with CCCL-informed
SMEM-aware policy selection
- Documents the actual SMEM model: BLOCK_N * Lk * elem_bytes * 2
- For BI-V100: derives BLOCK from smem_limit dynamically
- NUM_WARPS follows CCCL pattern: fewer warps when SM count is low
- Search space documented: BLOCK ∈ {16,32,64}, NUM_WARPS ∈ {2,4,8}
paged_attn.py:
- Enriched _PARTITION_SIZE documentation with CCCL scan benchmark
7-dimensional parameter space reference
- Added scale_mem_bound analysis for future float16 vs float32
partition size differentiation
- Connected GridEvenShare dispatch to scan delay parameters
NOT changed (correctly):
- _PARTITION_SIZE value stays 512 (precompiled .so constraint)
- V1/V2 threshold logic stays max_num_partitions == 1
- These require .so recompilation to change