7552365c7f
CCCL dispatch_reduce.cuh uses: max_blocks = sm_occupancy * sm_count * subscription_factor BI-V100: 1 * 16 * 5 = 80 max CTAs But paged_attn._forward_decode_pytorch runs in Python (torch.matmul), not as CUDA CTA launches. Python loop overhead >> kernel launch overhead. Each iteration = torch.matmul + online softmax update (2-3 CUDA launches). Change: TARGET_TILES 4→2, MIN_TILE_BLOCKS 64→128, MAX_TILE_BLOCKS 4096→8192 Effect: For seq_len=100K (6250 blocks), tile_blocks goes from ceil(6250/4)=1563 → ceil(6250/2)=3125 blocks per tile = 2 Python iterations instead of 4 = 50% fewer torch.matmul launches for long contexts Memory check: 3125 blocks × 16 tokens/block = 50K tokens per tile Score: 4 kv_heads × 6 gqa × 50K × 4B = 4.8 MB ✓ (fits in 48KB SMEM for the matmul kernel; actual memory is HBM-allocated by PyTorch) Source: CCCL grid_even_share.cuh DispatchInit + subscription_factor=5