edccbb00b4
Two changes informed by reading CCCL engine source code as input:
1. SingleTile fast path (from kernel_reduce.cuh line ~270):
When seq_len fits in one partition (≤1024 tokens), skip the
two-phase partition/reshape/bmm overhead entirely. Direct
softmax + V weighted sum. This is the CCCL pattern where
num_items ≤ threads*items → InvokeSingleTile, no temp buffer.
Impact: Early decode tokens (seq_len < 1024) avoid all partition
machinery. Qwen3.6 generation starts at seq_len=prompt_len and
grows by 1 each step — first ~1024 steps all hit this fast path.
2. GridEvenShare constants (from dispatch_reduce.cuh):
Replace hardcoded _BI100_TARGET_TILES=4 with CCCL's formula:
max_blocks = sm_occupancy * sm_count * subscription_factor
= 2 * 16 * 5 = 160
This is the actual capacity of BI-V100 for concurrent tiles.
Source files read as input for this change:
- cccl_upstream/cub/cub/device/dispatch/dispatch_reduce.cuh (full)
- cccl_upstream/cub/cub/device/dispatch/kernels/kernel_reduce.cuh (full)
- cccl_upstream/cub/cub/agent/agent_reduce.cuh (full)
- paged_attention_v2_pytorch.py (full)
- vllm/_custom_ops.py (first 200 lines)