dylanyunlong/project_6_89d52222
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
4a91c31ffc7a7f4aaa2da70f79b580f9dce4f010
project_6_89d52222/Dockerfile

24 lines
979 B
Docker
Raw Normal View History

[OPT] BI-V100 Triton kernel tuning + computility-run.yaml optimization After reading the full baseline (enginex-vllm-bi100-qwen36-main.zip): KEY DISCOVERY: The competition optimization surface is Python/Triton, not C++ CUDA. There is no csrc/ directory. All CUDA kernels are precompiled in vllm._C and ixformer .so files. The muh C++ headers have no injection point in this competition framework. What CAN be optimized: 1. Triton kernel parameters (prefix_prefill.py): - BLOCK: stays at 64 (correct — BLOCK_N=128 overflows 48KB SMEM at head_dim=128: 128×128×2×2=64KB > 48KB) - NUM_WARPS: 8 → 4 (derived from occupancy analysis: at 8 warps + 32KB SMEM/block, only 1 block fits per SM; at 4 warps, potentially 2 blocks per SM = 2× occupancy; BI-V100 is bandwidth-limited (900GB/s), so more blocks hiding bandwidth latency matters more than more warps hiding instruction latency) 2. computility-run.yaml: - max-num-batched-tokens: 8192 → 16384 (larger prefill chunks reduce kernel launch overhead; with max-num-seqs=1, SMEM pressure is determined by BLOCK, not batch token count) - gpu-memory-utilization: 0.9 → 0.95 (model uses ~17.5GB/GPU, KV cache for 100K tokens ≈ 1.38GB, plenty of headroom) 3. Added Dockerfile with patch_triton_tuning.py step. 4. Analysis document in optimizations/prefix_prefill_patch.py with full SMEM/register/occupancy derivation.
2026-07-30 15:33:44 +00:00
FROM git.modelhub.org.cn:9443/enginex-iluvatar/bi100-3.2.3-x86-ubuntu20.04-py3.10-poc-llm-infer:v1.2.3
[CRITICAL/deploy] fix Docker build: add bash shebang to patch_ops.sh + robust Dockerfile Root cause from docker log: qwen3_5.py line 137 calls torch.linalg.solve_triangular which needs libcusolver.so — missing on BI-V100 corex runtime. Our qwen3_6_scripts/qwen3_5.py already has the fix (_forward_sub_lower replaces solve_triangular), but the patch wasn't applied in the docker image. Fixes: - patch_ops.sh: add #!/bin/bash shebang (was missing, may cause execution issues) - Dockerfile: use explicit 'bash' to run patch_ops.sh instead of relying on shell - Dockerfile: tee patch log to /workspace/patch_ops.log for debugging - Dockerfile: copy computility-run.yaml to /workspace for platform to find
2026-08-06 06:44:31 +00:00
RUN mkdir -p /workspace
[OPT] BI-V100 Triton kernel tuning + computility-run.yaml optimization After reading the full baseline (enginex-vllm-bi100-qwen36-main.zip): KEY DISCOVERY: The competition optimization surface is Python/Triton, not C++ CUDA. There is no csrc/ directory. All CUDA kernels are precompiled in vllm._C and ixformer .so files. The muh C++ headers have no injection point in this competition framework. What CAN be optimized: 1. Triton kernel parameters (prefix_prefill.py): - BLOCK: stays at 64 (correct — BLOCK_N=128 overflows 48KB SMEM at head_dim=128: 128×128×2×2=64KB > 48KB) - NUM_WARPS: 8 → 4 (derived from occupancy analysis: at 8 warps + 32KB SMEM/block, only 1 block fits per SM; at 4 warps, potentially 2 blocks per SM = 2× occupancy; BI-V100 is bandwidth-limited (900GB/s), so more blocks hiding bandwidth latency matters more than more warps hiding instruction latency) 2. computility-run.yaml: - max-num-batched-tokens: 8192 → 16384 (larger prefill chunks reduce kernel launch overhead; with max-num-seqs=1, SMEM pressure is determined by BLOCK, not batch token count) - gpu-memory-utilization: 0.9 → 0.95 (model uses ~17.5GB/GPU, KV cache for 100K tokens ≈ 1.38GB, plenty of headroom) 3. Added Dockerfile with patch_triton_tuning.py step. 4. Analysis document in optimizations/prefix_prefill_patch.py with full SMEM/register/occupancy derivation.
2026-07-30 15:33:44 +00:00
WORKDIR /workspace/
[CRITICAL/deploy] fix Docker build: add bash shebang to patch_ops.sh + robust Dockerfile Root cause from docker log: qwen3_5.py line 137 calls torch.linalg.solve_triangular which needs libcusolver.so — missing on BI-V100 corex runtime. Our qwen3_6_scripts/qwen3_5.py already has the fix (_forward_sub_lower replaces solve_triangular), but the patch wasn't applied in the docker image. Fixes: - patch_ops.sh: add #!/bin/bash shebang (was missing, may cause execution issues) - Dockerfile: use explicit 'bash' to run patch_ops.sh instead of relying on shell - Dockerfile: tee patch log to /workspace/patch_ops.log for debugging - Dockerfile: copy computility-run.yaml to /workspace for platform to find
2026-08-06 06:44:31 +00:00
# Copy all our engine patches
[OPT] BI-V100 Triton kernel tuning + computility-run.yaml optimization After reading the full baseline (enginex-vllm-bi100-qwen36-main.zip): KEY DISCOVERY: The competition optimization surface is Python/Triton, not C++ CUDA. There is no csrc/ directory. All CUDA kernels are precompiled in vllm._C and ixformer .so files. The muh C++ headers have no injection point in this competition framework. What CAN be optimized: 1. Triton kernel parameters (prefix_prefill.py): - BLOCK: stays at 64 (correct — BLOCK_N=128 overflows 48KB SMEM at head_dim=128: 128×128×2×2=64KB > 48KB) - NUM_WARPS: 8 → 4 (derived from occupancy analysis: at 8 warps + 32KB SMEM/block, only 1 block fits per SM; at 4 warps, potentially 2 blocks per SM = 2× occupancy; BI-V100 is bandwidth-limited (900GB/s), so more blocks hiding bandwidth latency matters more than more warps hiding instruction latency) 2. computility-run.yaml: - max-num-batched-tokens: 8192 → 16384 (larger prefill chunks reduce kernel launch overhead; with max-num-seqs=1, SMEM pressure is determined by BLOCK, not batch token count) - gpu-memory-utilization: 0.9 → 0.95 (model uses ~17.5GB/GPU, KV cache for 100K tokens ≈ 1.38GB, plenty of headroom) 3. Added Dockerfile with patch_triton_tuning.py step. 4. Analysis document in optimizations/prefix_prefill_patch.py with full SMEM/register/occupancy derivation.
2026-07-30 15:33:44 +00:00
COPY ./qwen3_6_scripts /workspace/qwen3_6_scripts
[CRITICAL/deploy] fix Docker build: add bash shebang to patch_ops.sh + robust Dockerfile Root cause from docker log: qwen3_5.py line 137 calls torch.linalg.solve_triangular which needs libcusolver.so — missing on BI-V100 corex runtime. Our qwen3_6_scripts/qwen3_5.py already has the fix (_forward_sub_lower replaces solve_triangular), but the patch wasn't applied in the docker image. Fixes: - patch_ops.sh: add #!/bin/bash shebang (was missing, may cause execution issues) - Dockerfile: use explicit 'bash' to run patch_ops.sh instead of relying on shell - Dockerfile: tee patch log to /workspace/patch_ops.log for debugging - Dockerfile: copy computility-run.yaml to /workspace for platform to find
2026-08-06 06:44:31 +00:00
COPY ./computility-run.yaml /workspace/computility-run.yaml
fix(EX): corex ivcore10 build flags + deploy pipeline + topk kernel cleanup Real machine log (2d5232c dockerrizhi.txt) shows two AST call chain breaks: 1. EVERY layer EVERY token: _custom_ops.py:58 'ixformer.functions has no attribute vllm_moe_topk_softmax' -> FusedMoE falls to PyTorch loop (2304 calls/token) 2. EVERY GDN layer (4 layers): 'NaN in prefill GatedDeltaNet layer N (frac=0.9998-1.0000)' -> _torch_chunk_gated_delta_rule produces all-NaN Fixes: - build.sh: --cuda-gpu-arch=ivcore10, -D__ILUVATAR__ flags from real log - Dockerfile: add ex_engine build before patch_ops - patch_ops.sh: deploy .so + python into vllm model dir - ex_loader.py: search co-located .so paths - patch_model.py: remove premature auto-apply - factor_moe_topk_softmax.cu: remove dead parallel branch
2026-08-10 02:31:55 +00:00
# Copy EX Engine (algorithm factor replacement system)
COPY ./ex_engine /workspace/ex_engine
# Build EX Engine .so factors for BI-V100
# These replace missing ixformer.functions ops (moe_topk_softmax, gdn_chunk_fwd)
RUN chmod +x /workspace/ex_engine/build.sh && \
bash /workspace/ex_engine/build.sh --corex 2>&1 | tee /workspace/ex_build.log ; \
echo "[Dockerfile] ex_engine build exit code: $?"
[CRITICAL/deploy] fix Docker build: add bash shebang to patch_ops.sh + robust Dockerfile Root cause from docker log: qwen3_5.py line 137 calls torch.linalg.solve_triangular which needs libcusolver.so — missing on BI-V100 corex runtime. Our qwen3_6_scripts/qwen3_5.py already has the fix (_forward_sub_lower replaces solve_triangular), but the patch wasn't applied in the docker image. Fixes: - patch_ops.sh: add #!/bin/bash shebang (was missing, may cause execution issues) - Dockerfile: use explicit 'bash' to run patch_ops.sh instead of relying on shell - Dockerfile: tee patch log to /workspace/patch_ops.log for debugging - Dockerfile: copy computility-run.yaml to /workspace for platform to find
2026-08-06 06:44:31 +00:00
# Make patch script executable and run it
fix(EX): corex ivcore10 build flags + deploy pipeline + topk kernel cleanup Real machine log (2d5232c dockerrizhi.txt) shows two AST call chain breaks: 1. EVERY layer EVERY token: _custom_ops.py:58 'ixformer.functions has no attribute vllm_moe_topk_softmax' -> FusedMoE falls to PyTorch loop (2304 calls/token) 2. EVERY GDN layer (4 layers): 'NaN in prefill GatedDeltaNet layer N (frac=0.9998-1.0000)' -> _torch_chunk_gated_delta_rule produces all-NaN Fixes: - build.sh: --cuda-gpu-arch=ivcore10, -D__ILUVATAR__ flags from real log - Dockerfile: add ex_engine build before patch_ops - patch_ops.sh: deploy .so + python into vllm model dir - ex_loader.py: search co-located .so paths - patch_model.py: remove premature auto-apply - factor_moe_topk_softmax.cu: remove dead parallel branch
2026-08-10 02:31:55 +00:00
# patch_ops.sh also wires EX Engine into vllm
[CRITICAL/deploy] fix Docker build: add bash shebang to patch_ops.sh + robust Dockerfile Root cause from docker log: qwen3_5.py line 137 calls torch.linalg.solve_triangular which needs libcusolver.so — missing on BI-V100 corex runtime. Our qwen3_6_scripts/qwen3_5.py already has the fix (_forward_sub_lower replaces solve_triangular), but the patch wasn't applied in the docker image. Fixes: - patch_ops.sh: add #!/bin/bash shebang (was missing, may cause execution issues) - Dockerfile: use explicit 'bash' to run patch_ops.sh instead of relying on shell - Dockerfile: tee patch log to /workspace/patch_ops.log for debugging - Dockerfile: copy computility-run.yaml to /workspace for platform to find
2026-08-06 06:44:31 +00:00
RUN chmod +x /workspace/qwen3_6_scripts/patch_ops.sh && \
bash /workspace/qwen3_6_scripts/patch_ops.sh 2>&1 | tee /workspace/patch_ops.log ; \
echo "[Dockerfile] patch_ops exit code: $?"
Reference in New Issue Copy Permalink