dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
436 Commits 1 Branch 0 Tags
490ff98ad640dafadff52ed678a06cbb57b00a9b
Commit Graph

7 Commits

Author SHA1 Message Date
Claude
32ee28122e ref(upstream): 搬运 xllm ilu kernel+layer 完整源码 — 2089行 14个API声明 
来源: Deep-Spark/xllm core/kernels/ilu/ + core/layers/ilu/
  ixformer.h: 14个ixformer::infer API完整声明
  kernel wrappers: activation(32) attention(162) fused_moe(99) group_gemm(39)
                   matmul(73) norm(50) rope(31) + headers
  layer dispatch: fused_moe.cpp(797行) attention.cpp(189行) + headers

覆盖状态 (ix_moe_bridge.cpp vs ixformer.h 14个API):
  已覆盖 13/14: silu_and_mul, rms_norm, residual_rms_norm, ixformer_linear,
    ixformer_linear_ex, topk_softmax, moe_compute_token_index, moe_expand_input,
    moe_w16a16_group_gemm, moe_output_reduce_sum, xllm_paged_attention,
    xllm_reshape_and_cache, xllm_rotary_embedding
  缺失 1/14: ixinfer_flash_attn_unpad_with_block_tables

dlopen 调用链验证:
  12个 prebuilt .so → 9个 qwen3_5.py + 2个 paged_attn.py + 1个 block_major_kv_cache.py
  辅助模块: bi100_env, bi100_profile, gdn_prefix, block_major_kv_cache 全部到位
 2026-08-11 04:41:29 +00:00
Claude
6cdf2ec87b ref(upstream): 搬运 3 大 GDN 上游仓库 — FLA naive ops + vllm GDN 子树 + xllm C++ 参考 
来源:
  1. fla-org/flash-linear-attention (5538 stars)
     → upstream_ref/fla/ops/gated_delta_rule/naive.py (正确的纯 PyTorch GDN)
     → upstream_ref/fla/ops/gated_delta_rule/chunk.py (Triton chunk kernel)
     → upstream_ref/fla/layers/gated_deltanet.py (层集成)

  2. vllm-project/vllm main (88717 stars)
     → upstream_ref/vllm_gdn/gdn/qwen_gdn_linear_attn.py (1751行, Qwen3.5 原生 GDN)
     → upstream_ref/vllm_gdn/ops/causal_conv1d.py (1289行, 正确的 Conv1d)
     → upstream_ref/vllm_gdn/third_party/ops/ (FLA Triton ops vendored)
     → upstream_ref/vllm_gdn/models/qwen3_5.py (vllm 最新 Qwen3.5 模型)

  3. Deep-Spark/xllm (BI-V100 硬件厂商)
     → upstream_ref/xllm_latest/core/layers/npu_torch/qwen3_gated_delta_net_base.cpp (576行)
     → upstream_ref/xllm_latest/core/kernels/npu/npu_causal_conv1d.cpp
     → upstream_ref/xllm_latest/core/kernels/npu/npu_recurrent_gated_delta_rule.cpp

目的: 修复 corex_gdn.py Conv1d groups 接口不匹配问题
  错误: conv1d_weight shape (2560,1,4) 被当成 (num_k_heads,1,4) 索引
  conv_dim = key_dim*2 + value_dim = 10240, TP=4 后 2560
  FLA naive.py 和 vllm qwen_gdn_linear_attn.py 有正确的实现可直接对接
 2026-08-11 03:55:59 +00:00
project6-dev
87a19d2d00 feat(CRITICAL): 从 GitHub 扫描搬运 ixformer SDK + xllm 完整 GDN/MoE 代码 
来源:
  1. Chranos/ixformer (GitHub) → ixformer_sdk/ (230 files, 70K lines)
     - inference/functions/vllm.py: vllm_moe_topk_softmax 完整实现 (2033 lines)
     - inference/functions/moe.py: MoE ops 完整实现 (1380 lines)
     - contrib/vllm_flash_attn/: FA2 Python 接口 (1018 lines)
     - contrib/tgi/fused_moe.py: TGI fused MoE (429 lines)
     - csrc/include/ixformer/: C++ kernel headers + cmake

  2. Deep-Spark/xllm (GitHub) → upstream_ref/xllm_latest/ (+15 files)
     - npu_torch/qwen3_5_decoder_layer_impl.cpp/.h
     - npu_torch/qwen3_5_gated_delta_net.cpp/.h
     - npu_torch/qwen3_next_*.cpp/.h (6 files)
     - npu_torch/attention.cpp/.h + fused_moe.cpp/.h + CMakeLists.txt
     - models/llm/qwen3_5.h + qwen3_5_mtp.h + qwen3_next.h
     - models/vlm/qwen3_5.h

调用链完整性:
  ixformer_sdk/inference/functions/vllm.py
    → ops.infer.moe_topk_softmax() (C++ 层)
    → 这就是 base 镜像 libixformer.so 里的实现

  upstream_ref/xllm_latest/core/layers/ilu/fused_moe.cpp
    → ixformer::infer::topk_softmax() (直接 C++ 调用)
    → ixformer::infer::group_gemm() → 完整 7-step MoE pipeline
 2026-08-11 02:32:06 +00:00
project6-dev
56146f8130 feat(CRITICAL): ix_bridge call chain + upstream xllm/ds_vllm sync 
3 changes that close the MoE performance gap:

1. _custom_ops.py: Add ix_bridge as Priority 0 for topk_softmax
   - Before: tries our .cu kernel (fails) → PyTorch fallback (1-3 TPS)
   - After: tries ix_bridge → ixformer::infer::topk_softmax() → FAST
   - Call chain: _custom_ops.topk_softmax() → ix_bridge.topk_softmax()
     → ix_moe_bridge.so → ixformer::infer::topk_softmax()

2. Dockerfile: Add ix_moe_bridge.cpp precompile step
   - This was the missing link: code existed but was never compiled
   - Uses torch.utils.cpp_extension.load() to link against libixformer.so

3. upstream_ref sync from GitHub (cloned, not rewritten):
   - xLLM-AI/xllm: ILU kernels + CUDA MoE + GDN fp32 state mgmt
   - Deep-Spark/vllm: latest MoE kernel sources
 2026-08-11 01:27:33 +00:00
project6-dev
2aedf7377b ref(upstream): add Deep-Spark/vllm latest + xllm ILU kernel sources 
Cloned from GitHub:
  - Deep-Spark/vllm (latest): qwen3_5.py with multimodal support,
    transformers configs, multimodal registry, model registry
  - jd-opensource/xllm (latest): ILU kernel implementations
    (attention, fused_moe, group_gemm, activation, norm, rope, matmul)
    + GatedDeltaNet layer for Qwen3.5

These are the REAL upstream implementations that the base Docker image
is compiled from. Our dlopen modules should match these interfaces:
  - ilu_ops_api.h: 14 functions in xllm::kernel::ilu namespace
  - ixformer.h: 15 functions in ixformer::infer namespace

Key interface signatures for dlopen targets:
  batch_prefill()  → ixinfer_flash_attn_unpad_with_block_tables
  batch_decode()   → xllm_paged_attention
  moe_active_topk()→ topk_softmax
  moe_gen_idx()    → moe_compute_token_index_api
  group_gemm()     → moe_w16a16_group_gemm
  silu_and_mul()   → silu_and_mul
  rms_norm()       → rms_norm + residual_rms_norm
 2026-08-10 09:44:11 +00:00
EX Engine
002f9879b2 ref(upstream): FULL TREE — Deep-Spark xllm (1470) + ds_vllm csrc/models (703) 
Replaces cherry-picked upstream_ref with complete source trees.

xllm/ — Iluvatar official C++ inference engine (15MB, 1470 files)
  Complete: kernels → layers → models → runtime → scheduler → api
  Excluded: .git, binary images, third_party submodule checkouts

ds_vllm/ — Iluvatar official vllm fork (8MB, 703 files)
  Included: csrc/ (ALL CUDA kernels), fused_moe/, qwen3_5 model, _custom_ops
  Excluded: tests, benchmarks, docs, examples (not needed for reference)

Critical call chains now fully traceable:
  MoE: moe_topk_softmax_kernels.cuh → ixformer.h → fused_moe.cpp → layer
  GDN: qwen3_gated_delta_net_base.cpp → qwen3_5_gated_delta_net.cpp
  Attention: ixformer.h → xllm_paged_attention → attention.cpp
 2026-08-10 02:54:03 +00:00
EX Engine
ea82b00e54 ref(upstream): add Deep-Spark xllm + vllm MoE/GDN reference code 
Sources (Apache 2.0, cloned 2026-08-09):
- Deep-Spark/xllm: Iluvatar's official C++ inference engine
- Deep-Spark/vllm: Iluvatar's vllm fork

Key files for our EX Engine development:

MoE topk_softmax (fixes 2304 calls/token PyTorch fallback):
- xllm/kernels/cuda/moe/moe_topk_softmax_kernels.cuh
  CUB-based fused softmax+topk, power-of-2 expert count optimized
  For 64 experts: topk_gating_softmax<T,VPT=2,64,WARPS=4,BYTES=4>
- xllm/kernels/ilu/ixformer.h
  Official ixformer C++ API: topk_softmax(), paged_attention(), etc.
- xllm/kernels/ilu/fused_moe.cpp
  How xllm calls ixformer::infer::topk_softmax()
- ds_vllm/csrc/moe/topk_softmax_kernels.cu
  vllm-native topk_softmax (TensorRT-LLM derived, 874 lines)

GatedDeltaNet (fixes NaN in 4 GDN layers):
- xllm/layers/npu_torch/qwen3_gated_delta_net_base.cpp
  fp32 state accumulation, proper recurrent update

Complete FusedMoE pipeline reference:
- xllm/layers/ilu/fused_moe.cpp
  gate -> topk -> expand -> gemm1 -> act -> gemm2 -> combine
 2026-08-10 02:48:23 +00:00