fix: critical config + tuning corrections from CCCL source analysis

computility-run.yaml:
  max-num-seqs 1→256: benchmark sweeps [128,256] concurrent seqs,
    current config processes 1 while 127 queue. KV cache budget:
    256 seqs × 2048 tokens × 80KB/token = 41.9GB < 45GB available.
  max-num-batched-tokens 8192→32768: support 256 concurrent prefills.
  gpu-memory-utilization 0.9→0.95: provide KV cache headroom.

Dockerfile:
  Deploy paged_attention_v2_triton.py to vllm package path so
  try-triton-first logic in _custom_ops.py can find it. Falls back
  to PyTorch V2 automatically if Triton V2 fails (SMEM/runtime).

muh/tuning/common.cuh:
  scale_mem_bound max_smem now a parameter (default 48KB). Allows
  policy_selectors to pass hw.max_shared_memory_per_block if actual
  SMEM differs from CCCL 48KB assumption.

muh/tuning/tuning_transform.cuh:
  bytes_in_flight 16KB→32KB. Old derivation used 900/50=18 GB/s/SM
  (wrong, SM=16 confirmed). Actual per-SM BW = 56 GB/s.
  32KB is estimate pending benchmark sweep.

SM count 50→16 corrections across all affected files.
This commit is contained in:
Claude
2026-08-03 06:45:54 +00:00
parent 071fa361a3
commit cdc01bbc6a
7 changed files with 33 additions and 14 deletions

View File

@@ -8,7 +8,7 @@ BI-V100 hardware:
SMEM per block: 48 KB
Warp size: 32 (assumed)
Max threads/block: 1024
SM count: 50
SM count: 16 (confirmed via ixsmi, not 50 from spec sheet)
HBM bandwidth: 900 GB/s
Qwen3.6-35B-A3B attention:
@@ -30,11 +30,14 @@ BLOCK_M analysis:
More work per thread = better instruction-level parallelism (ILP).
Fewer warps = more blocks can run concurrently per SM = better occupancy.
BI-V100 has 50 SMs. With batch_size=1, num_heads~24-28:
BI-V100 has 16 SMs (confirmed, not 50 from spec sheet).
With batch_size=1, num_heads~24-28:
grid = (batch=1, heads≈24, ceil(seq_len/BLOCK_M))
For seq_len=100K: grid_z = 1563 blocks.
Total blocks = 1 × 24 × 1563 = 37,512 blocks.
Blocks per SM = 37512/50 = 750 — plenty of parallelism.
Blocks per SM = 37512/16 = 2344 — plenty of parallelism.
NOTE: with max-num-seqs=256 (benchmark config), batch_size >> 1,
grid is even larger. Parallelism is never the bottleneck.
Reducing NUM_WARPS from 8→4:
- Each SM can run more blocks concurrently (limited by registers/SMEM)