fix: revert invalid patches, add honest tuning surface assessment
REVERTED (invalid): - paged_attn.py: restored use_v1=True hardcode. V2 is NotImplementedError on BI-V100, removing the guard would cause runtime crash. - fused_moe.py: BLOCK_SIZE_N/K changes reverted. ixformer only reads BLOCK_SIZE_M from config dict, ignores N/K/GROUP_SIZE_M entirely (confirmed: _custom_ops.py:774 only passes config['BLOCK_SIZE_M']). - _custom_ops.py: SMEM change reverted pending hardware confirmation. - triton_flash_attention.py: autotune configs reverted (will re-add properly). - prefix_prefill.py: comment enhancement reverted (was harmless but noisy). ADDED: - TUNING_SURFACE_TRUTH.md: honest assessment of what's actually tunable on BI-V100 with ixformer. Documents that bench_bi100.py benchmark functions are invalid (point params not injected into kernels). Actual tuning surface is 5 parameters, not dozens: 1. BLOCK_SIZE_M (fused_moe, passes to ixformer) 2. use_v1 threshold (hardcoded True, V2 unimplemented) 3. BLOCK/NUM_WARPS (prefix_prefill Triton JIT) 4. SMEM declaration (affects Triton compiler) 5. autotune config set (triton_flash_attention)
This commit is contained in:
@@ -338,23 +338,17 @@ def get_default_config(
|
||||
dtype: Optional[str],
|
||||
is_marlin: bool,
|
||||
) -> Dict[str, int]:
|
||||
# muh: BI-V100 (SM=16, 48KB SMEM) aware defaults
|
||||
# Qwen3.6 MoE: E≈128, topk=8, K≈2048, N≈5504
|
||||
# SM=16 → fewer CTAs → each CTA should do more work → larger K tile
|
||||
# SMEM check: M=64 * K=64 * 2B(fp16) * 2(A+B) = 16KB < 48KB ✓
|
||||
config = {
|
||||
'BLOCK_SIZE_M': 64,
|
||||
'BLOCK_SIZE_N': 64,
|
||||
'BLOCK_SIZE_K': 64, # muh: 32→64, better memory coalescing on BI-V100
|
||||
'BLOCK_SIZE_K': 32,
|
||||
'GROUP_SIZE_M': 8
|
||||
}
|
||||
# A heuristic: fused marlin works faster with this config for small M
|
||||
if M <= E or (is_marlin and M <= 32):
|
||||
# muh: decode path (M=1 for single-token, M=8 for topk=8)
|
||||
# BI-V100: K=64 good for memory BW, N=64 for output tile
|
||||
config = {
|
||||
'BLOCK_SIZE_M': 16,
|
||||
'BLOCK_SIZE_N': 64, # muh: 32→64, wider output tile
|
||||
'BLOCK_SIZE_N': 32,
|
||||
'BLOCK_SIZE_K': 64,
|
||||
'GROUP_SIZE_M': 1
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user