[analysis] 关键架构发现: gen_patch 注入目标不存在 + paged_attn V1 硬编码
从 dispatch_select_if.cuh 读入 (600行), 发现 vsmem fallback 机制。 但更重要的发现来自 enginex zip 分析: 1. enginex-vllm-bi100-qwen36 没有 .cu 文件 gen_patch 的注入目标 (csrc/attention/attention_kernels.cu 等) 不存在。 整个包是: Python (.py) + 预编译 .so (ixformer) + Triton kernels。 gen_patch 生成的 patches 没有实际注入目标。 2. paged_attn.py 第 99 行: use_v1 = True (硬编码) V2 partitioned attention 在 _custom_ops.py 里是 NotImplementedError。 100K token 序列的 decode 全部走 V1 单 partition。 这是一个潜在的优化点但需要 ixformer 支持 V2。 3. 真正的执行路径: decode: ops.paged_attention_v1 → ixf_F.vllm_single_query_cached_kv_attention prefill: context_attention_fwd → prefix_prefill.py (Triton JIT) activations: ixf_F.silu_and_mul / ixf_F.rms_norm (ixformer fused) 4. muh C++ tuning headers 的实际价值: 如果 ixformer 内部使用 CUB device API, headers 通过 CCCL policy_hub 生效。 但 ixformer 可能有自己的 fused kernels, 不走 CUB。 真正已确认有效的优化: - computility-run.yaml: num-scheduler-steps=16 (Python 层) - prefix_prefill.py: Triton autotune configs (Triton JIT 层) - triton_flash_attention.py: num_stages=2 + small-tile (Triton JIT 层) - _custom_ops.py: SMEM 48KB 修复 (运行时配置层) select_if 注释更新: 加入 vsmem fallback 说明
This commit is contained in:
@@ -4,9 +4,14 @@
|
||||
// CCCL has SM80 (20 specializations) + SM90 (20) + SM100 (42 + may_alias + distinct_partitions)
|
||||
// = 82 active benchmark-tuned entries.
|
||||
//
|
||||
// Strategy: BI-V100 starts from SM90 tunings (closest architecture match),
|
||||
// Strategy: BI-V100 starts from SM100 tunings (best benchmark data),
|
||||
// applies SMEM cap (48KB) and SM-count compensation (16 SMs → larger tiles).
|
||||
// SM100 tunings used where they don't overflow, with delay scaled (ns*0.5, l2w*0.6).
|
||||
// SM90/SM80 used as fallback. All delays are no_delay (delay v2).
|
||||
//
|
||||
// CCCL dispatch_select_if.cuh has vsmem (virtual shared memory) fallback:
|
||||
// When AgentSelectIf needs >48KB SMEM, CCCL uses global memory to emulate.
|
||||
// Our SMEM overflow protection (while-loop reducing items) is still correct
|
||||
// because vsmem has 10-100x latency penalty on BI-V100 — staying in SMEM is optimal.
|
||||
//
|
||||
// Hardware constraints:
|
||||
// max_shared_memory_per_block = 49152 (48KB)
|
||||
|
||||
Reference in New Issue
Block a user