feat(CRITICAL): ix_bridge call chain + upstream xllm/ds_vllm sync
3 changes that close the MoE performance gap:
1. _custom_ops.py: Add ix_bridge as Priority 0 for topk_softmax
- Before: tries our .cu kernel (fails) → PyTorch fallback (1-3 TPS)
- After: tries ix_bridge → ixformer::infer::topk_softmax() → FAST
- Call chain: _custom_ops.topk_softmax() → ix_bridge.topk_softmax()
→ ix_moe_bridge.so → ixformer::infer::topk_softmax()
2. Dockerfile: Add ix_moe_bridge.cpp precompile step
- This was the missing link: code existed but was never compiled
- Uses torch.utils.cpp_extension.load() to link against libixformer.so
3. upstream_ref sync from GitHub (cloned, not rewritten):
- xLLM-AI/xllm: ILU kernels + CUDA MoE + GDN fp32 state mgmt
- Deep-Spark/vllm: latest MoE kernel sources
This commit is contained in:
@@ -26,7 +26,13 @@ RUN chmod +x /workspace/qwen3_6_scripts/patch_ops.sh && \
|
||||
bash /workspace/qwen3_6_scripts/patch_ops.sh 2>&1 | tee /workspace/patch_ops.log ; \
|
||||
echo "[Dockerfile] patch_ops exit code: $?"
|
||||
|
||||
# Step 5: Precompile GDN kernel (needs vllm in path, so after patch_ops)
|
||||
# Step 5: Precompile ix_moe_bridge.cpp → links to ixformer::infer::topk_softmax()
|
||||
# This is the C++ pybind bridge that makes ixformer SDK callable from Python.
|
||||
# Source: upstream_ref/xllm/xllm/core/kernels/ilu/fused_moe.cpp call pattern
|
||||
RUN python3 /workspace/ex_engine/precompile_ix_bridge.py 2>&1 | tee -a /workspace/ex_build.log ; \
|
||||
echo "[Dockerfile] ix_bridge precompile exit code: $?"
|
||||
|
||||
# Step 6: Precompile GDN kernel (needs vllm in path, so after patch_ops)
|
||||
RUN python3 /workspace/qwen3_6_scripts/precompile_gdn.py \
|
||||
/workspace/qwen3_6_scripts/flash_qla_sm70 2>&1 | tee -a /workspace/ex_build.log ; \
|
||||
echo "[Dockerfile] gdn precompile exit code: $?"
|
||||
|
||||
Reference in New Issue
Block a user