dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

feat(SO): ix_moe_bridge.cpp — dlopen bridge for 12 ixformer::infer functions

Browse Source 
THE CORE .so: ix_moe_bridge.cpp compiles to ix_moe_bridge.so which:
  - Links against base image's libixformer.so at load time
  - Exposes 12 functions to Python via pybind11:

  MoE pipeline (7 steps):
    topk_softmax()      → ixformer::infer::topk_softmax
    moe_gen_idx()       → ixformer::infer::moe_compute_token_index_api
    moe_expand_input()  → ixformer::infer::moe_expand_input
    moe_group_gemm()    → ixformer::infer::moe_w16a16_group_gemm
    silu_and_mul()      → ixformer::infer::silu_and_mul
    moe_combine_result()→ ixformer::infer::moe_output_reduce_sum

  Inference ops (5 functions):
    paged_attention()   → ixformer::infer::xllm_paged_attention
    rms_norm()          → ixformer::infer::rms_norm
    linear()            → ixformer::infer::ixformer_linear
    reshape_and_cache() → ixformer::infer::xllm_reshape_and_cache
    rotary_embedding()  → ixformer::infer::xllm_rotary_embedding

Build chain:
  Dockerfile → build.sh → precompile_ix_bridge.py
    → torch.utils.cpp_extension.load(ix_moe_bridge.cpp, -lixformer)
      → ix_moe_bridge.cpython-310.so

Load chain:
  Python: from ex_engine.python.ix_bridge import topk_softmax
    → ix_bridge.py loads ix_moe_bridge.so
      → dlopen links to libixformer.so
        → CUDA kernel on BI-V100

Interface source: upstream_ref/xllm_latest/core/kernels/ilu/ixformer.h
This commit is contained in:
project6-dev
2026-08-11 02:36:59 +00:00
parent 0eab333fb0
commit d1c5e992aa
5 changed files with 540 additions and 561 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
Dockerfile
View File

@@ -3,22 +3,25 @@ FROM git.modelhub.org.cn:9443/enginex-iluvatar/bi100-3.2.3-x86-ubuntu20.04-py3.1
RUN mkdir -p /workspace
WORKDIR /workspace/
# Copy all sources
# Copy sources
COPY ./qwen3_6_scripts /workspace/qwen3_6_scripts
COPY ./computility-run.yaml /workspace/computility-run.yaml
COPY ./ex_engine /workspace/ex_engine
# Step 1: Compile _moe_C (CUB-based topk_softmax + moe_align_block_size)
# Proven on real BI-V100: WARP_SIZE=64, -cl-fast-relaxed-math, cub/block/block_reduce.cuh
RUN python3 /workspace/ex_engine/precompile_moe_kernels.py 2>&1 | tee /workspace/ex_build.log ; \
echo "[Dockerfile] _moe_C precompile exit code: $?"
# Step 1: Compile ix_moe_bridge.so — dlopen bridge to libixformer.so
# This is THE critical .so: it exposes topk_softmax + 11 other ixformer::infer
# functions that the base image's Python binding doesn't expose.
RUN chmod +x /workspace/ex_engine/build.sh && \
bash /workspace/ex_engine/build.sh 2>&1 | tee /workspace/build.log ; \
echo "[Docker] build exit code: $?"
# Step 2: Deploy patches (serving + engine fixes)
# Step 2: Deploy patches (serving layer + conditional model layer)
# patch_ops.sh v2: does NOT overwrite base qwen3_5.py (comp 168 strategy)
RUN chmod +x /workspace/qwen3_6_scripts/patch_ops.sh && \
bash /workspace/qwen3_6_scripts/patch_ops.sh 2>&1 | tee /workspace/patch_ops.log ; \
echo "[Dockerfile] patch_ops exit code: $?"
echo "[Docker] patch_ops exit code: $?"
# Step 3: Precompile GDN kernel (needs vllm in path, so after patch_ops)
# Step 3: Precompile GDN kernel (needs vllm in path)
RUN python3 /workspace/qwen3_6_scripts/precompile_gdn.py \
/workspace/qwen3_6_scripts/flash_qla_sm70 2>&1 | tee -a /workspace/ex_build.log ; \
echo "[Dockerfile] gdn precompile exit code: $?"
/workspace/qwen3_6_scripts/flash_qla_sm70 2>&1 | tee -a /workspace/build.log ; \
echo "[Docker] gdn precompile exit code: $?"