b75965d4ea
Real machine log (2d5232c dockerrizhi.txt) shows two AST call chain breaks:
1. EVERY layer EVERY token:
_custom_ops.py:58 'ixformer.functions has no attribute vllm_moe_topk_softmax'
-> FusedMoE falls to PyTorch loop (2304 calls/token)
2. EVERY GDN layer (4 layers):
'NaN in prefill GatedDeltaNet layer N (frac=0.9998-1.0000)'
-> _torch_chunk_gated_delta_rule produces all-NaN
Fixes:
- build.sh: --cuda-gpu-arch=ivcore10, -D__ILUVATAR__ flags from real log
- Dockerfile: add ex_engine build before patch_ops
- patch_ops.sh: deploy .so + python into vllm model dir
- ex_loader.py: search co-located .so paths
- patch_model.py: remove premature auto-apply
- factor_moe_topk_softmax.cu: remove dead parallel branch
24 lines
979 B
Docker
24 lines
979 B
Docker
FROM git.modelhub.org.cn:9443/enginex-iluvatar/bi100-3.2.3-x86-ubuntu20.04-py3.10-poc-llm-infer:v1.2.3
|
|
|
|
RUN mkdir -p /workspace
|
|
WORKDIR /workspace/
|
|
|
|
# Copy all our engine patches
|
|
COPY ./qwen3_6_scripts /workspace/qwen3_6_scripts
|
|
COPY ./computility-run.yaml /workspace/computility-run.yaml
|
|
|
|
# Copy EX Engine (algorithm factor replacement system)
|
|
COPY ./ex_engine /workspace/ex_engine
|
|
|
|
# Build EX Engine .so factors for BI-V100
|
|
# These replace missing ixformer.functions ops (moe_topk_softmax, gdn_chunk_fwd)
|
|
RUN chmod +x /workspace/ex_engine/build.sh && \
|
|
bash /workspace/ex_engine/build.sh --corex 2>&1 | tee /workspace/ex_build.log ; \
|
|
echo "[Dockerfile] ex_engine build exit code: $?"
|
|
|
|
# Make patch script executable and run it
|
|
# patch_ops.sh also wires EX Engine into vllm
|
|
RUN chmod +x /workspace/qwen3_6_scripts/patch_ops.sh && \
|
|
bash /workspace/qwen3_6_scripts/patch_ops.sh 2>&1 | tee /workspace/patch_ops.log ; \
|
|
echo "[Dockerfile] patch_ops exit code: $?"
|