5e618bf480
基础引擎结构 (排除 pkgs 预编译包):
顶层: Dockerfile, computility-run.yaml, paged_attn.py, prefix_prefill.py, attention.py
qwen3_6_scripts/: 我们的全部适配代码 (模型定义, attention, API, parsers, patches)
vllm/: vllm 0.6.3 BI-V100 适配版 (985 files)
_custom_ops.py: ixformer 绑定层 (V1 decode, silu_and_mul, rms_norm...)
attention/selector.py: backend 选择 (BI-V100 → XFORMERS → _run_sdpa_fallback)
model_executor/models/: 200+ 模型定义 (无 qwen3_5)
关键发现:
1. 基础引擎 Dockerfile 只有 4 行: FROM + COPY + RUN patch_ops.sh
2. vllm/_custom_ops.py 硬编码 SMEM=32KB (已在我们版本修为 49152)
3. attention selector: BI-V100 → XFORMERS (not FlashAttn, SM<80)
4. Prefill 路径: xformers._run_sdpa_fallback (PyTorch matmul+softmax)
不是 paged_attn.forward_prefix (那是 context attention for cached tokens)
CCCL source read: dispatch_batch_memcpy.cuh (large_buffer vs small_buffer paths)
KV block size = 16×4×256×2 = 32KB → large_buffer path in CCCL
8 lines
105 B
Plaintext
8 lines
105 B
Plaintext
muh/__pycache__/
|
|
enginex-vllm-bi100-qwen36-main.zip
|
|
cccl_upstream/
|
|
muh/
|
|
baseline.muh
|
|
pkgs/
|
|
enginex_base/
|