dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[SIMPLIFY] Dockerfile 恢复到基础引擎原版 4 行——所有适配通过 patch_ops.sh

Browse Source 
之前的 Dockerfile 有 40+ 行，包含 8 个独立 patch 脚本的 RUN 步骤。
这些 patch 之间有执行顺序冲突:
  - patch_ops.sh 部署 paged_attn.py (use_v1=True, PyTorch fallback)
  - patch_ixformer_native.py 修改 _custom_ops.py (V2 fallback)
  - patch_paged_attention_v2.py 注释掉 use_v1=True (启用 V2)
  - patch_enable_triton.py 启用 Triton prefill (覆盖 patch_ops.sh 的禁用)
  - patch_head256_triton.py 修改 BLOCK_N (必须在 enable_triton 之后)

这些 patch 的交互导致最终状态不确定，是功能测试 fail 的潜在原因。

基础引擎原版 Dockerfile 只有 4 行:
  FROM base_image
  RUN mkdir /workspace
  COPY ./qwen3_6_scripts /workspace/qwen3_6_scripts
  RUN cd ./qwen3_6_scripts && ./patch_ops.sh

所有适配逻辑集中在 patch_ops.sh 里，一个脚本完成所有工作。
额外的 patch 脚本（V2、Triton enable 等）如果需要，应该集成到 patch_ops.sh 中。

CCCL 架构启示 (cc_dispatch.cuh):
  CCCL 的 dispatch 是一个单一入口点 dispatch_compute_cap，
  不是多个独立的 dispatch 层互相覆盖。
  Dockerfile 也应该是一个单一入口点 (patch_ops.sh)。
This commit is contained in:
muh-bot
2026-08-05 07:14:23 +00:00
parent 3426d8185a
commit a7f95a5943
1 changed files with 0 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
Dockerfile
View File

@@ -2,62 +2,5 @@ FROM git.modelhub.org.cn:9443/enginex-iluvatar/bi100-3.2.3-x86-ubuntu20.04-py3.1
RUN mkdir /workspace
WORKDIR /workspace/
# Copy all scripts, V2 kernels, CCCL-tuned prefill, and muh dispatch
COPY ./qwen3_6_scripts /workspace/qwen3_6_scripts
COPY ./paged_attention_v2_pytorch.py /workspace/paged_attention_v2_pytorch.py
COPY ./paged_attention_v2_triton.py /workspace/paged_attention_v2_triton.py
COPY ./prefix_prefill.py /workspace/prefix_prefill.py
COPY ./muh_dispatch.py /workspace/muh_dispatch.py
# Run baseline patches (model registration, xformers fallback, tool parser, etc.)
RUN cd ./qwen3_6_scripts && ./patch_ops.sh
# CRITICAL: Enable ixformer native V1/V2 paged attention kernels.
# Fixes: V1 head_mapping int→Tensor, V2 NotImplementedError → native kernel,
# Triton path mismatch.
RUN python3 /workspace/qwen3_6_scripts/patch_ixformer_native.py
# 1. PagedAttention V2 — fills the NotImplementedError hole
# Enables partitioned attention for long sequences (>8192 tokens)
# Deploy BOTH PyTorch and Triton V2 to vllm package — _custom_ops.py
# tries Triton first, falls back to PyTorch if import/runtime fails.
# Triton V2 risk: SMEM=32KB zero margin at head_dim=256 BLOCK_N=32.
# If Triton V2 crashes, PyTorch V2 (batched bmm, no intermediate tensor
# savings but correct) takes over automatically via try/except.
# Deploy Triton V2 kernel into vllm package
RUN cp /workspace/paged_attention_v2_triton.py \
/usr/local/corex/lib/python3/dist-packages/vllm/paged_attention_v2_triton.py 2>/dev/null || \
cp /workspace/paged_attention_v2_triton.py \
/usr/local/corex/lib64/python3/dist-packages/vllm/paged_attention_v2_triton.py 2>/dev/null || true
RUN python3 /workspace/qwen3_6_scripts/patch_paged_attention_v2.py
# Deploy CCCL-tuned prefix_prefill.py (SM=16: BLOCK=64, NUM_WARPS=4)
RUN cp /workspace/prefix_prefill.py \
/usr/local/corex/lib/python3/dist-packages/vllm/attention/ops/prefix_prefill.py 2>/dev/null || \
cp /workspace/prefix_prefill.py \
/usr/local/corex/lib64/python3/dist-packages/vllm/attention/ops/prefix_prefill.py 2>/dev/null || true
# Deploy muh_dispatch.py (CCCL-style type dispatch for kernel configs)
RUN cp /workspace/muh_dispatch.py \
/usr/local/corex/lib/python3/dist-packages/vllm/muh_dispatch.py 2>/dev/null || \
cp /workspace/muh_dispatch.py \
/usr/local/corex/lib64/python3/dist-packages/vllm/muh_dispatch.py 2>/dev/null || true
# 2. Triton kernel tuning: BLOCK=64, NUM_WARPS=4
# SMEM: BLOCK_N=64 × head_dim=128 × 2B × 2(K+V) = 32KB ≤ 48KB
# Occupancy: 4 warps allows 2 blocks/SM vs 1 at 8 warps
RUN python3 /workspace/qwen3_6_scripts/patch_triton_tuning.py
# 3. Enable Triton kernels with automatic fallback to PyTorch if they hang
# Triton Flash Attention is 10-50x faster than PyTorch for-loop fallback
RUN python3 /workspace/qwen3_6_scripts/patch_enable_triton.py
# 5. head_dim=256 support: Qwen3.6 uses head_dim=256
# BLOCK=64 overflows SMEM (64×256×2×2=64KB > 48KB)
# → BLOCK=32 for head_dim=256 (32×256×2×2=32KB ≤ 48KB)
RUN python3 /workspace/qwen3_6_scripts/patch_head256_triton.py
# 4. Raise decode threshold: compiled paged_attention_v1 up to 65536
# instead of falling back to Python at 32768
RUN python3 /workspace/qwen3_6_scripts/patch_vectorized_decode.py