From 41ecb8c9e9cec9cb3b0f89b31e72f59b382a05a3 Mon Sep 17 00:00:00 2001 From: muh-bot Date: Wed, 5 Aug 2026 03:32:31 +0000 Subject: [PATCH] =?UTF-8?q?[analysis]=20=E5=85=B3=E9=94=AE=E6=9E=B6?= =?UTF-8?q?=E6=9E=84=E5=8F=91=E7=8E=B0:=20gen=5Fpatch=20=E6=B3=A8=E5=85=A5?= =?UTF-8?q?=E7=9B=AE=E6=A0=87=E4=B8=8D=E5=AD=98=E5=9C=A8=20+=20paged=5Fatt?= =?UTF-8?q?n=20V1=20=E7=A1=AC=E7=BC=96=E7=A0=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 从 dispatch_select_if.cuh 读入 (600行), 发现 vsmem fallback 机制。 但更重要的发现来自 enginex zip 分析: 1. enginex-vllm-bi100-qwen36 没有 .cu 文件 gen_patch 的注入目标 (csrc/attention/attention_kernels.cu 等) 不存在。 整个包是: Python (.py) + 预编译 .so (ixformer) + Triton kernels。 gen_patch 生成的 patches 没有实际注入目标。 2. paged_attn.py 第 99 行: use_v1 = True (硬编码) V2 partitioned attention 在 _custom_ops.py 里是 NotImplementedError。 100K token 序列的 decode 全部走 V1 单 partition。 这是一个潜在的优化点但需要 ixformer 支持 V2。 3. 真正的执行路径: decode: ops.paged_attention_v1 → ixf_F.vllm_single_query_cached_kv_attention prefill: context_attention_fwd → prefix_prefill.py (Triton JIT) activations: ixf_F.silu_and_mul / ixf_F.rms_norm (ixformer fused) 4. muh C++ tuning headers 的实际价值: 如果 ixformer 内部使用 CUB device API, headers 通过 CCCL policy_hub 生效。 但 ixformer 可能有自己的 fused kernels, 不走 CUB。 真正已确认有效的优化: - computility-run.yaml: num-scheduler-steps=16 (Python 层) - prefix_prefill.py: Triton autotune configs (Triton JIT 层) - triton_flash_attention.py: num_stages=2 + small-tile (Triton JIT 层) - _custom_ops.py: SMEM 48KB 修复 (运行时配置层) select_if 注释更新: 加入 vsmem fallback 说明 --- muh/include/muh/tuning/tuning_select_if.cuh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/muh/include/muh/tuning/tuning_select_if.cuh b/muh/include/muh/tuning/tuning_select_if.cuh index a4d2ed6a..107e567f 100644 --- a/muh/include/muh/tuning/tuning_select_if.cuh +++ b/muh/include/muh/tuning/tuning_select_if.cuh @@ -4,9 +4,14 @@ // CCCL has SM80 (20 specializations) + SM90 (20) + SM100 (42 + may_alias + distinct_partitions) // = 82 active benchmark-tuned entries. // -// Strategy: BI-V100 starts from SM90 tunings (closest architecture match), +// Strategy: BI-V100 starts from SM100 tunings (best benchmark data), // applies SMEM cap (48KB) and SM-count compensation (16 SMs → larger tiles). -// SM100 tunings used where they don't overflow, with delay scaled (ns*0.5, l2w*0.6). +// SM90/SM80 used as fallback. All delays are no_delay (delay v2). +// +// CCCL dispatch_select_if.cuh has vsmem (virtual shared memory) fallback: +// When AgentSelectIf needs >48KB SMEM, CCCL uses global memory to emulate. +// Our SMEM overflow protection (while-loop reducing items) is still correct +// because vsmem has 10-100x latency penalty on BI-V100 — staying in SMEM is optimal. // // Hardware constraints: // max_shared_memory_per_block = 49152 (48KB)