[FIX] Deploy _custom_ops.py SMEM 32KB→48KB fix — was in repo but never deployed

Source: cccl_upstream/cub/test/catch2_test_grid_even_share.cu (random pick)

GridEvenShare test validates: grid_size = min(max_grid, ceil_div(N, tile_size))
If SMEM is reported as 32KB instead of 48KB, tile_size is 33% smaller,
grid_size is 50% larger, and every kernel launch wastes occupancy.

Base image _custom_ops.py: get_max_shared_memory_per_block → 32*1024 = 32768
Our fix: → 49152 (confirmed 48KB via ixsmi on Phanthy Cloud)

This affects ALL kernel launches that query SMEM limits:
  - Triton JIT tile sizing (prefix_prefill, flash_attn)
  - ixformer internal SMEM allocation
  - paged_attention block_size calculations

Was modified in vllm/_custom_ops.py but NEVER added to qwen3_6_scripts/
for Docker deployment. Now deployed.
This commit is contained in:
dylanyunlon
2026-08-05 08:38:36 +00:00
parent 81972a05c6
commit b902090fb2
2 changed files with 1139 additions and 0 deletions

File diff suppressed because it is too large Load Diff

View File

@@ -22,6 +22,17 @@ fi
echo "[patch_ops] vllm path: $V"
# --- _custom_ops.py: SMEM 48KB fix + hardware ops bindings -------------------
# Base image returns 32KB (32768) for get_max_shared_memory_per_block, but
# BI-V100 actually has 48KB (49152) confirmed via ixsmi. This limits Triton
# tile sizes and ixformer internal allocations if not corrected.
# CCCL GridEvenShare test (catch2_test_grid_even_share.cu) validates that
# work distribution depends on correct hardware parameters — wrong SMEM
# means wrong tile_size means wrong grid_size.
# FULL FILE REPLACEMENT.
cp ./_custom_ops.py $V/_custom_ops.py
echo "[patch_ops] _custom_ops.py → / (SMEM 32KB→48KB fix)"
# --- paged_attn.py: pure-PyTorch attention fallback --------------------------
# Bypasses Triton context_attention_fwd (hangs BI-V100 permanently).
# Uses K-tiling Flash Attention online softmax for prefix attention.