fix: revert invalid patches, add honest tuning surface assessment
REVERTED (invalid): - paged_attn.py: restored use_v1=True hardcode. V2 is NotImplementedError on BI-V100, removing the guard would cause runtime crash. - fused_moe.py: BLOCK_SIZE_N/K changes reverted. ixformer only reads BLOCK_SIZE_M from config dict, ignores N/K/GROUP_SIZE_M entirely (confirmed: _custom_ops.py:774 only passes config['BLOCK_SIZE_M']). - _custom_ops.py: SMEM change reverted pending hardware confirmation. - triton_flash_attention.py: autotune configs reverted (will re-add properly). - prefix_prefill.py: comment enhancement reverted (was harmless but noisy). ADDED: - TUNING_SURFACE_TRUTH.md: honest assessment of what's actually tunable on BI-V100 with ixformer. Documents that bench_bi100.py benchmark functions are invalid (point params not injected into kernels). Actual tuning surface is 5 parameters, not dozens: 1. BLOCK_SIZE_M (fused_moe, passes to ixformer) 2. use_v1 threshold (hardcoded True, V2 unimplemented) 3. BLOCK/NUM_WARPS (prefix_prefill Triton JIT) 4. SMEM declaration (affects Triton compiler) 5. autotune config set (triton_flash_attention)
This commit is contained in:
@@ -726,12 +726,8 @@ if triton.__version__ >= "2.1.0":
|
||||
# Triton equivalent: BLOCK=64, warps=4 (128 threads, larger tile per warp)
|
||||
_is_bi_v100 = not current_platform.has_device_capability(80)
|
||||
if _is_bi_v100:
|
||||
# muh: CCCL-informed block selection for BI-V100 (SM=16, SMEM≤48KB)
|
||||
# SMEM = BLOCK_M*Hd*elem + BLOCK_N*Hd*elem*2(K+V)
|
||||
# head_dim=128, fp16(2B): BLOCK=64,N=64 → 48KB (100% SMEM, risky)
|
||||
# Conservative: BLOCK=64,N=32 → 32KB (65% SMEM, safe for 32KB limit)
|
||||
BLOCK = 64
|
||||
NUM_WARPS = 4 # 4 warps × 32 = 128 threads; BW-limited at 56 GB/s/SM
|
||||
NUM_WARPS = 4
|
||||
else:
|
||||
BLOCK = 128
|
||||
NUM_WARPS = 8
|
||||
|
||||
Reference in New Issue
Block a user