EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

support speculative decoding kernel in sgl-kernel (#3373)

Browse Source 
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
This commit is contained in:
Yineng Zhang
2025-02-07 20:29:51 +08:00
committed by GitHub
parent 45c87e083f
commit f9905d59a8
13 changed files with 1298 additions and 132 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
sgl-kernel/setup.py
View File

@@ -99,6 +99,8 @@ sources = [
"src/sgl-kernel/csrc/fp8_gemm_kernel.cu",
"src/sgl-kernel/csrc/lightning_attention_decode_kernel.cu",
"src/sgl-kernel/csrc/fused_add_rms_norm_kernel.cu",
"src/sgl-kernel/csrc/eagle_utils.cu",
"src/sgl-kernel/csrc/speculative_sampling.cu",
"3rdparty/flashinfer/csrc/activation.cu",
"3rdparty/flashinfer/csrc/bmm_fp8.cu",
"3rdparty/flashinfer/csrc/norm.cu",