ref(upstream): FULL TREE — Deep-Spark xllm (1470) + ds_vllm csrc/models (703)
Replaces cherry-picked upstream_ref with complete source trees. xllm/ — Iluvatar official C++ inference engine (15MB, 1470 files) Complete: kernels → layers → models → runtime → scheduler → api Excluded: .git, binary images, third_party submodule checkouts ds_vllm/ — Iluvatar official vllm fork (8MB, 703 files) Included: csrc/ (ALL CUDA kernels), fused_moe/, qwen3_5 model, _custom_ops Excluded: tests, benchmarks, docs, examples (not needed for reference) Critical call chains now fully traceable: MoE: moe_topk_softmax_kernels.cuh → ixformer.h → fused_moe.cpp → layer GDN: qwen3_gated_delta_net_base.cpp → qwen3_5_gated_delta_net.cpp Attention: ixformer.h → xllm_paged_attention → attention.cpp
This commit is contained in:
EX Engine
@@ -1,84 +1,28 @@
|
||||
# Upstream Reference: Deep-Spark xllm + vllm
|
||||
# Upstream Reference: Deep-Spark xllm + vllm (FULL TREE)
|
||||
|
||||
Source repos (cloned 2026-08-09):
|
||||
- `Deep-Spark/xllm` — Iluvatar's C++ inference engine (Apache 2.0)
|
||||
- `Deep-Spark/vllm` — Iluvatar's vllm fork (Apache 2.0)
|
||||
Source repos (cloned 2026-08-09, Apache 2.0):
|
||||
- `Deep-Spark/xllm` — Iluvatar official C++ LLM inference engine (1470 files)
|
||||
- `Deep-Spark/vllm` — Iluvatar official vllm fork (703 files, csrc + model layer)
|
||||
|
||||
## Call Chain: MoE topk_softmax on BI-V100
|
||||
## What's here
|
||||
|
||||
```
|
||||
Our code xllm reference Iluvatar SDK
|
||||
───────────────────────────────── ────────────────────────────── ──────────────
|
||||
qwen3_5.py
|
||||
Qwen3_5MoeSparseBlock.forward()
|
||||
_custom_ops.py:topk_softmax()
|
||||
ixf_F.vllm_moe_topk_softmax ← MISSING in base image
|
||||
│
|
||||
├── xllm path (C++ native):
|
||||
│ kernels/ilu/fused_moe.cpp
|
||||
│ → ixformer::infer::topk_softmax() ← ixformer.h
|
||||
│ → CUDA kernel (moe_topk_softmax_kernels.cuh)
|
||||
│ topk_gating_softmax<T,VPT,64,4,BYTES_PER_LDG>()
|
||||
│
|
||||
├── ds_vllm path (Python torch extension):
|
||||
│ csrc/moe/topk_softmax_kernels.cu
|
||||
│ → torch.ops._moe_C.topk_softmax()
|
||||
│ → topk_gating_softmax_kernel_launcher<T>()
|
||||
│
|
||||
└── Our EX Engine path (dlopen .so):
|
||||
ex_engine/csrc/factor_moe_topk_softmax.cu
|
||||
→ ex_factor_0.so via ctypes
|
||||
→ moe_topk_softmax_kernel()
|
||||
```
|
||||
### xllm/ (complete source minus git/binaries/submodules)
|
||||
天数智芯官方下一代推理引擎，C++ 原生，多平台(CUDA/ILU/MLU/NPU)。
|
||||
包含 kernels → layers → models → runtime → scheduler → api_service 完整栈。
|
||||
|
||||
## Call Chain: GatedDeltaNet (GDN) on BI-V100
|
||||
Key subtrees:
|
||||
- `xllm/core/kernels/ilu/` — ixformer API wrappers (ixformer.h是金矿)
|
||||
- `xllm/core/kernels/cuda/moe/` — MoE CUDA kernels (topk_softmax, fused_topk)
|
||||
- `xllm/core/kernels/cuda/` — activation, norm, rope, attention CUDA kernels
|
||||
- `xllm/core/layers/ilu/` — Iluvatar FusedMoE完整pipeline
|
||||
- `xllm/core/layers/npu_torch/` — GatedDeltaNet C++ implementation
|
||||
- `xllm/models/llm/qwen3_5.h` — Qwen3.5 model definition
|
||||
- `xllm/compiler/tilelang/` — GDN kernel code generation
|
||||
|
||||
```
|
||||
Our code xllm reference
|
||||
───────────────────────────────── ──────────────────────────────
|
||||
qwen3_5.py
|
||||
GatedDeltaNet.forward()
|
||||
prefill path:
|
||||
_torch_chunk_gated_delta_rule ← produces NaN (fp16 overflow)
|
||||
│
|
||||
├── xllm path:
|
||||
│ layers/npu_torch/qwen3_gated_delta_net_base.cpp
|
||||
│ → process_mixed_qkv() + recurrent state update
|
||||
│ → full fp32 accumulation
|
||||
│
|
||||
└── Our EX Engine path:
|
||||
ex_engine/csrc/factor_gdn_chunk_fwd.cu
|
||||
→ fp32 state accumulation, tile-based
|
||||
```
|
||||
|
||||
## File Index
|
||||
|
||||
### xllm/kernels/cuda/moe/ — CUDA kernels (the actual GPU code)
|
||||
- `moe_topk_softmax_kernels.cuh` — **KEY**: fused softmax+topk, CUB-based, power-of-2 expert count optimized
|
||||
- `moe_fused_topk.cu` — sigmoid/softmax topk dispatcher
|
||||
- `moe_topk.cuh` — topk helper functions
|
||||
- `moe_topk_sigmoid_kernels.cuh` — sigmoid variant for DeepSeek-style routing
|
||||
|
||||
### xllm/kernels/ilu/ — Iluvatar ixformer API wrappers
|
||||
- `ixformer.h` — **KEY**: official ixformer C++ API declarations (topk_softmax, paged_attention, etc.)
|
||||
- `fused_moe.cpp` — how xllm calls ixformer::infer::topk_softmax()
|
||||
- `activation.cpp` — silu_and_mul, gelu_and_mul wrappers
|
||||
- `attention.cpp` — paged_attention wrappers
|
||||
- `norm.cpp` — rms_norm, fused_add_rms_norm wrappers
|
||||
- `rope.cpp` — rotary embedding wrappers
|
||||
|
||||
### xllm/layers/ilu/ — Complete FusedMoE layer for Iluvatar
|
||||
- `fused_moe.cpp` — **KEY**: full MoE pipeline: gate → topk → expand → gemm1 → act → gemm2 → combine
|
||||
- `fused_moe.h` — layer interface
|
||||
|
||||
### xllm/layers/npu_torch/ — GatedDeltaNet implementation
|
||||
- `qwen3_gated_delta_net_base.cpp` — base GDN with fp32 state management
|
||||
- `qwen3_5_gated_delta_net.cpp` — Qwen3.5 specific GDN
|
||||
|
||||
### ds_vllm/csrc/moe/ — vllm-native MoE CUDA kernels
|
||||
- `topk_softmax_kernels.cu` — vllm's topk_softmax (TensorRT-LLM derived)
|
||||
- `moeTopKFuncs.cuh` — shared topk reduction primitives
|
||||
- `moe_align_sum_kernels.cu` — block alignment for scatter
|
||||
|
||||
### ds_vllm/vllm/ — Python layer
|
||||
- `_custom_ops.py` — how vllm calls torch.ops._moe_C.topk_softmax
|
||||
### ds_vllm/ (csrc + model layers + fused_moe)
|
||||
天数智芯官方vllm fork，Python + CUDA torch extension。
|
||||
- `csrc/` — ALL CUDA source (attention, moe, quantization, cache)
|
||||
- `csrc/libtorch_stable/moe/topk_softmax_kernels.cu` — vllm topk_softmax
|
||||
- `vllm/_custom_ops.py` — Python → torch.ops._moe_C bridge
|
||||
- `vllm/model_executor/models/qwen3_5.py` — ds_vllm的qwen3_5实现
|
||||
- `vllm/model_executor/layers/fused_moe/` — vllm FusedMoE Python layer
|
||||
|
||||
Reference in New Issue
Block a user