87a19d2d00
来源:
1. Chranos/ixformer (GitHub) → ixformer_sdk/ (230 files, 70K lines)
- inference/functions/vllm.py: vllm_moe_topk_softmax 完整实现 (2033 lines)
- inference/functions/moe.py: MoE ops 完整实现 (1380 lines)
- contrib/vllm_flash_attn/: FA2 Python 接口 (1018 lines)
- contrib/tgi/fused_moe.py: TGI fused MoE (429 lines)
- csrc/include/ixformer/: C++ kernel headers + cmake
2. Deep-Spark/xllm (GitHub) → upstream_ref/xllm_latest/ (+15 files)
- npu_torch/qwen3_5_decoder_layer_impl.cpp/.h
- npu_torch/qwen3_5_gated_delta_net.cpp/.h
- npu_torch/qwen3_next_*.cpp/.h (6 files)
- npu_torch/attention.cpp/.h + fused_moe.cpp/.h + CMakeLists.txt
- models/llm/qwen3_5.h + qwen3_5_mtp.h + qwen3_next.h
- models/vlm/qwen3_5.h
调用链完整性:
ixformer_sdk/inference/functions/vllm.py
→ ops.infer.moe_topk_softmax() (C++ 层)
→ 这就是 base 镜像 libixformer.so 里的实现
upstream_ref/xllm_latest/core/layers/ilu/fused_moe.cpp
→ ixformer::infer::topk_softmax() (直接 C++ 调用)
→ ixformer::infer::group_gemm() → 完整 7-step MoE pipeline
34 lines
1.4 KiB
Python
34 lines
1.4 KiB
Python
from typing import Union
|
|
|
|
import ixformer._C as ops
|
|
import torch
|
|
from torch.autograd.function import Function, FunctionCtx
|
|
|
|
__all__ = ["softmax", "ref_softmax"]
|
|
|
|
|
|
def ref_softmax(input: torch.Tensor, dim: int = None, _stacklevel: int = 3, dtype=None):
|
|
out = torch.nn.functional.softmax(
|
|
input, dim=dim, _stacklevel=_stacklevel, dtype=dtype
|
|
)
|
|
return out
|
|
|
|
|
|
def softmax(input: torch.Tensor, dim=None, _stacklevel=3, dtype=None):
|
|
|
|
"""
|
|
Args:
|
|
input: (...) torch.float16
|
|
dim: int
|
|
要进行softmax的维度,目前只支持最后一维， dim==-1 or dim == input.dim()-1
|
|
_stacklevel: int
|
|
这个参数只是为了与pytorch中对齐。 stacklevel is used in python to indicate warning mechanism how far up the stack it has to go to find the line that called the function which issued the warning.
|
|
dtype: torch.float16
|
|
Returns:
|
|
Tensor: (...) torch.float16
|
|
"""
|
|
output = torch.empty_like(input)
|
|
ops.infer.softmax(input, output, dim)
|
|
output = output.to(dtype)
|
|
return output
|