After reading the full baseline (enginex-vllm-bi100-qwen36-main.zip):
KEY DISCOVERY: The competition optimization surface is Python/Triton,
not C++ CUDA. There is no csrc/ directory. All CUDA kernels are
precompiled in vllm._C and ixformer .so files. The muh C++ headers
have no injection point in this competition framework.
What CAN be optimized:
1. Triton kernel parameters (prefix_prefill.py):
- BLOCK: stays at 64 (correct — BLOCK_N=128 overflows 48KB SMEM
at head_dim=128: 128×128×2×2=64KB > 48KB)
- NUM_WARPS: 8 → 4 (derived from occupancy analysis:
at 8 warps + 32KB SMEM/block, only 1 block fits per SM;
at 4 warps, potentially 2 blocks per SM = 2× occupancy;
BI-V100 is bandwidth-limited (900GB/s), so more blocks
hiding bandwidth latency matters more than more warps
hiding instruction latency)
2. computility-run.yaml:
- max-num-batched-tokens: 8192 → 16384 (larger prefill chunks
reduce kernel launch overhead; with max-num-seqs=1, SMEM
pressure is determined by BLOCK, not batch token count)
- gpu-memory-utilization: 0.9 → 0.95 (model uses ~17.5GB/GPU,
KV cache for 100K tokens ≈ 1.38GB, plenty of headroom)
3. Added Dockerfile with patch_triton_tuning.py step.
4. Analysis document in optimizations/prefix_prefill_patch.py
with full SMEM/register/occupancy derivation.
4463e9ccee