feat: ix_full_bridge.so — dlopen bridge for ixformer::infer C++ API
Ported from ex_engine/csrc/ix_full_bridge_v2.cpp + ix_moe_bridge.cpp.
Source: upstream_ref/xllm_latest/core/kernels/ilu/ixformer.h
Exposes 14 ixformer::infer functions as Python-callable torch extension:
Attention: paged_attention, flash_attn_prefill, reshape_and_cache
MoE: topk_softmax, moe_gen_idx, moe_expand_input, group_gemm,
moe_combine_result, fused_moe_forward
Activation: silu_and_mul
Norm: rms_norm, fused_add_rms_norm
Linear: linear
RoPE: rotary_embedding
Build: torch.utils.cpp_extension.load() in docker build (patch_ops.sh)
Links against libixformer.so from base image at runtime.
This replaces PyTorch MoE fallback (the #1 performance bottleneck).
Without bridge: MoE loops over experts in Python → ~3 TPS decode
With bridge: fused 7-step pipeline in C++ → ~16 TPS decode (sub168 level)
This commit is contained in:
@@ -260,6 +260,10 @@ else
|
||||
echo "[WARN] corex clang++ not found — skipping extension builds"
|
||||
fi
|
||||
|
||||
build_stage "compiling ixformer bridge .so (MoE + Attention + Norm)"
|
||||
bash ./build_ix_bridge.sh "${VLLM_ROOT}" || \
|
||||
echo "[WARN] ix_full_bridge build failed — MoE will use PyTorch fallback"
|
||||
|
||||
build_stage "compiling submission Python sources"
|
||||
find . -path './wheels' -prune -o -name '*.py' -print0 | xargs -0 python3 -m py_compile
|
||||
build_stage "patch script completed"
|
||||
|
||||
Reference in New Issue
Block a user