dylanyunlong/project_6_89d52222
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

feat: 10-file MoE bridge pipeline — compile, dispatch, patch, test

Browse Source 
The complete chain to replace 180 Python fallback calls/token with C++:

BUILD:
  1. moe_ops_impl.cu (489L) — 5 MoE functions in ixformer::infer namespace
     - topk_softmax: dynamic num_experts (128 for Qwen3.5), shared-mem
     - moe_compute_token_index: histogram + prefix_sum + scatter
     - moe_expand_input: gather kernel
     - moe_w16a16_group_gemm: per-expert cuinferCustomGemm loop
     - moe_output_reduce_sum: weighted combine
  2. ix_full_bridge_v2.cpp (461L) — pybind11 bridge, 14+1 functions
  3. build_moe_bridge.sh — torch.utils.cpp_extension compile, link cuinfer+ixformer

DISPATCH:
  4. moe_dispatch.py — 3-tier fallback (fused → individual → PyTorch)
  5. patch_moe_hot_path.py — monkey-patch Qwen3_5MoE.forward()

CONFIG:
  6. computility-run.yaml — max_num_seqs 1→2 (match sub168 baseline)
  7. patch_ops.sh — add build + deploy steps for MoE bridge

VERIFY:
  8. probe_moe_symbols.sh — nm -D .so to confirm 5 MoE symbols present
  9. test_moe_bridge.py — random-tensor integration test (no weights needed)

DEPLOY:
 10. Dockerfile — COPY ex_engine sources for in-container compilation
This commit is contained in:
project_6
2026-08-16 17:46:53 +00:00
parent c54923a17e
commit 49034d1d09
9 changed files with 774 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
Dockerfile
View File

@@ -4,6 +4,12 @@ WORKDIR /workspace/
# Copy all our engine patches
COPY ./qwen3_6_scripts /workspace/qwen3_6_scripts
COPY ./computility-run.yaml /workspace/computility-run.yaml
# Copy ex_engine source for MoE bridge compilation
COPY ./ex_engine/csrc/moe_ops_impl.cu /workspace/qwen3_6_scripts/ex_engine_src/csrc/moe_ops_impl.cu
COPY ./ex_engine/csrc/ix_full_bridge_v2.cpp /workspace/qwen3_6_scripts/ex_engine_src/csrc/ix_full_bridge_v2.cpp
COPY ./ex_engine/build_moe_bridge.sh /workspace/qwen3_6_scripts/ex_engine_src/build_moe_bridge.sh
COPY ./ex_engine/python/moe_dispatch.py /workspace/qwen3_6_scripts/ex_engine_src/python/moe_dispatch.py
COPY ./ex_engine/python/patch_moe_hot_path.py /workspace/qwen3_6_scripts/ex_engine_src/python/patch_moe_hot_path.py
# Make patch script executable and run it
RUN chmod +x /workspace/qwen3_6_scripts/patch_ops.sh && \
bash /workspace/qwen3_6_scripts/patch_ops.sh 2>&1 | tee /workspace/patch_ops.log ; \