project_6 49034d1d09 feat: 10-file MoE bridge pipeline — compile, dispatch, patch, test ...

The complete chain to replace 180 Python fallback calls/token with C++:

BUILD:
  1. moe_ops_impl.cu (489L) — 5 MoE functions in ixformer::infer namespace
     - topk_softmax: dynamic num_experts (128 for Qwen3.5), shared-mem
     - moe_compute_token_index: histogram + prefix_sum + scatter
     - moe_expand_input: gather kernel
     - moe_w16a16_group_gemm: per-expert cuinferCustomGemm loop
     - moe_output_reduce_sum: weighted combine
  2. ix_full_bridge_v2.cpp (461L) — pybind11 bridge, 14+1 functions
  3. build_moe_bridge.sh — torch.utils.cpp_extension compile, link cuinfer+ixformer

DISPATCH:
  4. moe_dispatch.py — 3-tier fallback (fused → individual → PyTorch)
  5. patch_moe_hot_path.py — monkey-patch Qwen3_5MoE.forward()

CONFIG:
  6. computility-run.yaml — max_num_seqs 1→2 (match sub168 baseline)
  7. patch_ops.sh — add build + deploy steps for MoE bridge

VERIFY:
  8. probe_moe_symbols.sh — nm -D .so to confirm 5 MoE symbols present
  9. test_moe_bridge.py — random-tensor integration test (no weights needed)

DEPLOY:
 10. Dockerfile — COPY ex_engine sources for in-container compilation

2026-08-16 17:46:53 +00:00

..

csrc

feat: 10-file MoE bridge pipeline — compile, dispatch, patch, test

2026-08-16 17:46:53 +00:00

fla_kernels

feat: import CUDA kernels from xllm/CCCL/FLA upstream repos

2026-08-14 07:48:52 +00:00

include

fix(CRITICAL): engine death on image request + stop overwriting base corex modules

2026-08-10 09:12:05 +00:00

kernels

fix: group_gemm format "TN" + Layer 3 ops_api dispatch from xllm upstream

2026-08-16 16:09:15 +00:00

moe

data: port complete MoE + xllm layer call chains from upstream repos

2026-08-15 14:26:24 +00:00

python

feat: 10-file MoE bridge pipeline — compile, dispatch, patch, test

2026-08-16 17:46:53 +00:00

xllm_kernels

fix: group_gemm format "TN" + Layer 3 ops_api dispatch from xllm upstream

2026-08-16 16:09:15 +00:00

xllm_layers

data: port complete MoE + xllm layer call chains from upstream repos

2026-08-15 14:26:24 +00:00

xllm_models

fix(critical): fold max_completion_tokens + max_num_seqs=2 + max_model_len=80000 + xllm_latest layer import

2026-08-13 03:19:39 +00:00

__init__.py

fix: add ex_engine/__init__.py for Python package import

2026-08-15 13:08:24 +00:00

build_ix_bridge.sh

feat: ILU kernel pipeline — ix_full_bridge_v2 build + deploy + 7-step MoE dispatch

2026-08-15 14:15:47 +00:00

build_moe_bridge.sh

feat: 10-file MoE bridge pipeline — compile, dispatch, patch, test

2026-08-16 17:46:53 +00:00

build_xllm_ilu_kernels.sh

feat: ILU kernel pipeline — ix_full_bridge_v2 build + deploy + 7-step MoE dispatch

2026-08-15 14:15:47 +00:00

build_xllm_kernels.sh

feat: xllm_ops NO-FALLBACK kernel loader + 6 missing .so build targets + hot-path patcher

2026-08-15 14:13:13 +00:00

build.sh

fix(build): 回退qwen3_6_scripts+ex_engine到26e6cb40(能得分版本)

2026-08-12 01:33:24 +00:00

deploy_corex_modules.sh

feat(EX): corex_fa2.py — third dlopen module from comp 168 AST chain

2026-08-10 04:01:21 +00:00

deploy_ilu_pipeline.sh

feat: ILU kernel pipeline — ix_full_bridge_v2 build + deploy + 7-step MoE dispatch

2026-08-15 14:15:47 +00:00

deploy_ix_bridge.sh

feat: ix_ops integration layer — wire ix_full_bridge.so into vllm hot path

2026-08-15 06:15:17 +00:00

precompile_moe_kernels.py

fix(build): 回退qwen3_6_scripts+ex_engine到26e6cb40(能得分版本)

2026-08-12 01:33:24 +00:00

precompile_moe_topk.py

fix(MoE): robust CUDA kernel loading + no-GPU precompile

2026-08-10 07:50:28 +00:00

probe_moe_symbols.sh

feat: 10-file MoE bridge pipeline — compile, dispatch, patch, test

2026-08-16 17:46:53 +00:00

test_moe_bridge.py

feat: 10-file MoE bridge pipeline — compile, dispatch, patch, test

2026-08-16 17:46:53 +00:00