dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
841 Commits 1 Branch 0 Tags
main
Commit Graph

7 Commits

Author SHA1 Message Date
claude
051b02d3cd feat: ix_moe_bridge + ix_attn_bridge — dlopen bridges for full ixformer::infer API 
Bridge architecture (from xllm/core/kernels/ilu/ixformer.h):

ix_moe_bridge.so (MoE 7-step fused pipeline):
  - topk_softmax → moe_compute_token_index_api → moe_expand_input
  - moe_w16a16_group_gemm (x2) → silu_and_mul → moe_output_reduce_sum
  - fused_moe_forward(): replaces entire Python expert loop
  - Fix: group_gemm format NT→TN (match xllm trans_b=true)

ix_attn_bridge.so (attention + linear):
  - ixinfer_flash_attn_unpad_with_block_tables (fused prefill)
  - xllm_paged_attention (fused paged decode)
  - ixformer_linear (matmul + activation)
  - residual_rms_norm (fused residual + norm)

Integration:
  - ix_fused_moe.py: Python loader (prebuilt .so → JIT → unavailable)
  - qwen3_5.py: Tier 0 dispatch in _pure_pytorch_experts()
  - patch_ops.sh: deploys ix_fused_moe.py + all prebuilt/*.so

Source: jd-opensource/xllm (fresh clone, all ILU kernels verified SAME)
Sync: upstream_ref/xllm_latest/models/llm/qwen3_next_hybrid_base.h (+32 lines)

Build on real machine:
  bash qwen3_6_scripts/build_ix_moe_bridge.sh
  bash qwen3_6_scripts/build_ix_attn_bridge.sh
 2026-08-14 07:32:31 +00:00
project6-dev
d025b08a95 upstream(xllm): sync to jd-opensource/xllm latest + revert serving_chat.py 
搬运 jd-opensource/xllm 最新代码到 upstream_ref/xllm_latest/:
- core/kernels/ilu/ 10 files (ixformer.h API 不变)
- core/layers/ilu/ 4 files (fused_moe.cpp config 访问从 FLAGS→singleton)
- core/layers/npu_torch/ 14 files (qwen3_gated_delta_net_base.cpp 576→1164行,
  新增 repeat_tensor_heads, checkpoint_stride, spec_verify 等 GDN 功能)
- models/llm/ 5 files (qwen3_5.h 模型注册重构, 新增 qwen3_5_mtp_base.h)
- models/vlm/ 1 file (qwen3_5.h 218→440行)

serving_chat.py: 还原到 8030a11b 原版,删掉 6dcf3590 的语法错误 min(8192,
(缺右括号导致 py_compile 失败)
 2026-08-12 04:22:34 +00:00
Claude
32ee28122e ref(upstream): 搬运 xllm ilu kernel+layer 完整源码 — 2089行 14个API声明 
来源: Deep-Spark/xllm core/kernels/ilu/ + core/layers/ilu/
  ixformer.h: 14个ixformer::infer API完整声明
  kernel wrappers: activation(32) attention(162) fused_moe(99) group_gemm(39)
                   matmul(73) norm(50) rope(31) + headers
  layer dispatch: fused_moe.cpp(797行) attention.cpp(189行) + headers

覆盖状态 (ix_moe_bridge.cpp vs ixformer.h 14个API):
  已覆盖 13/14: silu_and_mul, rms_norm, residual_rms_norm, ixformer_linear,
    ixformer_linear_ex, topk_softmax, moe_compute_token_index, moe_expand_input,
    moe_w16a16_group_gemm, moe_output_reduce_sum, xllm_paged_attention,
    xllm_reshape_and_cache, xllm_rotary_embedding
  缺失 1/14: ixinfer_flash_attn_unpad_with_block_tables

dlopen 调用链验证:
  12个 prebuilt .so → 9个 qwen3_5.py + 2个 paged_attn.py + 1个 block_major_kv_cache.py
  辅助模块: bi100_env, bi100_profile, gdn_prefix, block_major_kv_cache 全部到位
 2026-08-11 04:41:29 +00:00
Claude
6cdf2ec87b ref(upstream): 搬运 3 大 GDN 上游仓库 — FLA naive ops + vllm GDN 子树 + xllm C++ 参考 
来源:
  1. fla-org/flash-linear-attention (5538 stars)
     → upstream_ref/fla/ops/gated_delta_rule/naive.py (正确的纯 PyTorch GDN)
     → upstream_ref/fla/ops/gated_delta_rule/chunk.py (Triton chunk kernel)
     → upstream_ref/fla/layers/gated_deltanet.py (层集成)

  2. vllm-project/vllm main (88717 stars)
     → upstream_ref/vllm_gdn/gdn/qwen_gdn_linear_attn.py (1751行, Qwen3.5 原生 GDN)
     → upstream_ref/vllm_gdn/ops/causal_conv1d.py (1289行, 正确的 Conv1d)
     → upstream_ref/vllm_gdn/third_party/ops/ (FLA Triton ops vendored)
     → upstream_ref/vllm_gdn/models/qwen3_5.py (vllm 最新 Qwen3.5 模型)

  3. Deep-Spark/xllm (BI-V100 硬件厂商)
     → upstream_ref/xllm_latest/core/layers/npu_torch/qwen3_gated_delta_net_base.cpp (576行)
     → upstream_ref/xllm_latest/core/kernels/npu/npu_causal_conv1d.cpp
     → upstream_ref/xllm_latest/core/kernels/npu/npu_recurrent_gated_delta_rule.cpp

目的: 修复 corex_gdn.py Conv1d groups 接口不匹配问题
  错误: conv1d_weight shape (2560,1,4) 被当成 (num_k_heads,1,4) 索引
  conv_dim = key_dim*2 + value_dim = 10240, TP=4 后 2560
  FLA naive.py 和 vllm qwen_gdn_linear_attn.py 有正确的实现可直接对接
 2026-08-11 03:55:59 +00:00
project6-dev
87a19d2d00 feat(CRITICAL): 从 GitHub 扫描搬运 ixformer SDK + xllm 完整 GDN/MoE 代码 
来源:
  1. Chranos/ixformer (GitHub) → ixformer_sdk/ (230 files, 70K lines)
     - inference/functions/vllm.py: vllm_moe_topk_softmax 完整实现 (2033 lines)
     - inference/functions/moe.py: MoE ops 完整实现 (1380 lines)
     - contrib/vllm_flash_attn/: FA2 Python 接口 (1018 lines)
     - contrib/tgi/fused_moe.py: TGI fused MoE (429 lines)
     - csrc/include/ixformer/: C++ kernel headers + cmake

  2. Deep-Spark/xllm (GitHub) → upstream_ref/xllm_latest/ (+15 files)
     - npu_torch/qwen3_5_decoder_layer_impl.cpp/.h
     - npu_torch/qwen3_5_gated_delta_net.cpp/.h
     - npu_torch/qwen3_next_*.cpp/.h (6 files)
     - npu_torch/attention.cpp/.h + fused_moe.cpp/.h + CMakeLists.txt
     - models/llm/qwen3_5.h + qwen3_5_mtp.h + qwen3_next.h
     - models/vlm/qwen3_5.h

调用链完整性:
  ixformer_sdk/inference/functions/vllm.py
    → ops.infer.moe_topk_softmax() (C++ 层)
    → 这就是 base 镜像 libixformer.so 里的实现

  upstream_ref/xllm_latest/core/layers/ilu/fused_moe.cpp
    → ixformer::infer::topk_softmax() (直接 C++ 调用)
    → ixformer::infer::group_gemm() → 完整 7-step MoE pipeline
 2026-08-11 02:32:06 +00:00
project6-dev
56146f8130 feat(CRITICAL): ix_bridge call chain + upstream xllm/ds_vllm sync 
3 changes that close the MoE performance gap:

1. _custom_ops.py: Add ix_bridge as Priority 0 for topk_softmax
   - Before: tries our .cu kernel (fails) → PyTorch fallback (1-3 TPS)
   - After: tries ix_bridge → ixformer::infer::topk_softmax() → FAST
   - Call chain: _custom_ops.topk_softmax() → ix_bridge.topk_softmax()
     → ix_moe_bridge.so → ixformer::infer::topk_softmax()

2. Dockerfile: Add ix_moe_bridge.cpp precompile step
   - This was the missing link: code existed but was never compiled
   - Uses torch.utils.cpp_extension.load() to link against libixformer.so

3. upstream_ref sync from GitHub (cloned, not rewritten):
   - xLLM-AI/xllm: ILU kernels + CUDA MoE + GDN fp32 state mgmt
   - Deep-Spark/vllm: latest MoE kernel sources
 2026-08-11 01:27:33 +00:00
project6-dev
2aedf7377b ref(upstream): add Deep-Spark/vllm latest + xllm ILU kernel sources 
Cloned from GitHub:
  - Deep-Spark/vllm (latest): qwen3_5.py with multimodal support,
    transformers configs, multimodal registry, model registry
  - jd-opensource/xllm (latest): ILU kernel implementations
    (attention, fused_moe, group_gemm, activation, norm, rope, matmul)
    + GatedDeltaNet layer for Qwen3.5

These are the REAL upstream implementations that the base Docker image
is compiled from. Our dlopen modules should match these interfaces:
  - ilu_ops_api.h: 14 functions in xllm::kernel::ilu namespace
  - ixformer.h: 15 functions in ixformer::infer namespace

Key interface signatures for dlopen targets:
  batch_prefill()  → ixinfer_flash_attn_unpad_with_block_tables
  batch_decode()   → xllm_paged_attention
  moe_active_topk()→ topk_softmax
  moe_gen_idx()    → moe_compute_token_index_api
  group_gemm()     → moe_w16a16_group_gemm
  silu_and_mul()   → silu_and_mul
  rms_norm()       → rms_norm + residual_rms_norm
 2026-08-10 09:44:11 +00:00