EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,563 Commits 8 Branches 0 Tags
c4ffbeca19232f7cd4502ebd5ff37a800bcd0506
Commit Graph

82 Commits

Author SHA1 Message Date
Yuan Luo
43baba649e [EP] Add cuda kernel for moe_ep_post_reorder (#6837) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-06-05 00:33:47 -07:00
zyksir
8e3797be1c support 1 shot allreduce in 1-node and 2-node using mscclpp (#6277) 2025-06-04 22:11:24 -07:00
Cheng Wan
81964328b7 Set num_fused_shared_experts as num_shared_experts when shared_experts fusion is not disabled (#6736) 2025-06-04 15:53:22 -07:00
Xiaoyu Zhang
bd75690f4e fix ep_moe_reorder kernel bugs (#6858) 
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
 2025-06-04 19:13:59 +08:00
Cheng Wan
8a5480528d [Refactor] Rename n_share_experts_fusion as num_fused_shared_experts (#6735) 2025-06-03 17:48:24 -07:00
jianan-gu
ff00895c46 Add CPU optimized kernels for topk and rope fusions (#6456) 2025-06-02 17:37:34 -07:00
Pavani Majety
eb38c7d1ca [1/2] Add Kernel support for Cutlass based Fused FP4 MoE (#6093) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-06-02 13:48:03 -07:00
Yuan Luo
55444ed667 [EP] Add cuda kernel for moe_ep_pre_reorder (#6699) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-06-01 20:49:01 -07:00
Chunyuan WU
3ded6235c9 Add fp8 fused_experts kernel for CPU in sgl-kernel and add UT (#6404) 2025-05-23 02:01:55 -07:00
blzheng
4ba1eea83f Add fp8 qkv_proj_with_rope kernel for CPU in sgl-kernel and add UT (#6493) 2025-05-23 00:14:46 -07:00
HandH1998
4d643f6c7a [1/2] Support Qserve (#6457) 
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
 2025-05-21 19:48:59 -07:00
blzheng
cfe48c5902 [CPU] Fix build issue (#6419) 2025-05-21 11:17:10 -07:00
YanbingJiang
32cc66efa5 Update extend/decode attention kernel for CPU in sgl-kernel and add UTs (#6405) 
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-19 21:23:17 -07:00
Chunyuan WU
5dd62c3a6f Add fp8 shared_expert kernel for CPU in sgl-kernel and add UT (#6339) 
Co-authored-by: Jiang, Yanbing <yanbing.jiang@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-18 12:42:15 -07:00
Elfie Guo
6fc9357503 [2/2] Add python wrapper for CUTLASS FP8 Blockscale MoE Kernel. (#5694) 2025-05-16 13:14:07 -07:00
Elfie Guo
c23a7072b6 Upgrade CUTLASS 4.0 (#6336) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-05-15 17:42:23 -07:00
Chunyuan WU
fb4959b2c5 Add fp8 gemm kernel for CPU in sgl-kernel and add gemm UT (#6216) 
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-15 09:10:40 -07:00
blzheng
0f75b907c6 [CPU] Add CMakeLists.txt for sgl-kernel (#6115) 2025-05-13 15:30:37 -07:00
Lianmin Zheng
e8e18dcdcc Revert "fix some typos" (#6244) 2025-05-12 12:53:26 -07:00
applesaucethebun
d738ab52f8 fix some typos (#6209) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-13 01:42:38 +08:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Trevor Morris
0ab3f437ab Cutlass MLA: Disable split kv due to https://github.com/NVIDIA/cutlass/issues/2274 (#6101) 2025-05-08 18:44:30 -07:00
PGFLMG
f6f96b0521 [sgl-kernel] fix: fix cu118 compile error (#6123) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-05-08 14:26:51 -07:00
Xiaoyu Zhang
d25398cbc8 fix custom_allreduce namespace (#6039) 2025-05-06 19:13:06 -07:00
Yineng Zhang
6f56614734 chore: upgrade cutlass 3.9.2 (#6004) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-05-06 13:34:08 -07:00
Xiaoyu Zhang
5bb0accbcf cutlass 3.9 supported to improve fp8_blockwise_gemm (#5820) 2025-04-28 21:52:36 -07:00
PGFLMG
ee71ed8a41 [Feat] QWen-1M context support[1/2]: Update block sparse attention backend utils kernel (#5847) 
Co-authored-by: sighingnow <sighingnow@gmail.com>
 2025-04-28 11:03:17 -07:00
Yineng Zhang
ce4ecba477 fix: only compile ApplyTokenBitmaskInplace cu124+ (#5686) 2025-04-23 14:17:42 -07:00
Yineng Zhang
15fabcc07f fix sgl-kernel unit tests (#5666) 2025-04-23 01:18:30 -07:00
Elfie Guo
e62c49557d [1/2] Add FP8 Blockscale MoE CUTLASS kernel for Blackwell (#5281) 2025-04-22 22:28:20 -07:00
Xiaoyu Zhang
8e09b37077 Sgl kernel fused_moe_gate support n_shared_experts (#5440) 2025-04-17 23:05:15 -07:00
PGFLMG
c08a717c77 [Feat] Update sgl-kernel flashinfer to latest main version (#5500) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-04-17 12:43:23 -07:00
DefTruth
12ef7e3bc3 bugfix: fix merge_state_v2 cuda graph (#5419) 2025-04-15 10:18:47 -07:00
DefTruth
388e15c0db kernel: support slightly faster merge_state_v2 cuda kernel (#5381) 2025-04-14 21:28:23 -07:00
Yineng Zhang
b62e7e99b8 feat: adapt merge_state (#5337) 2025-04-12 21:14:04 -07:00
Yineng Zhang
812e82f35e fix: solve cu118 issue for cutlass mla (#5331) 2025-04-12 12:51:09 -07:00
PGFLMG
4879e50c6d [Feat] Add sparse attn to sgl-kernel (#5327) 2025-04-12 11:36:36 -07:00
Zhaoyi Li
3c9740d200 update variable naming and comments for rocm (#5299) 2025-04-11 23:15:05 -07:00
Trevor Morris
f65b8d5c89 Blackwell Cutlass MLA kernel (#5142) 2025-04-11 22:16:51 -07:00
Yineng Zhang
136b8e6afb fix: remove cublas_grouped_gemm (#5307) 2025-04-11 16:22:37 -07:00
Richard Zou
76f44c2a8d Fix deepseek-v3 with torch.compile in PyTorch 2.6. (#5213) 2025-04-10 09:14:38 -07:00
Xiaoyu Zhang
f730362ee2 reduce moe_align_block_size_kernel small batch mode overhead (#5086) 2025-04-09 17:59:35 -07:00
Yi Zhang
ebf495f013 sgl-kernel use cutlass latest version for fp8 blockwise gemm (#5207) 2025-04-09 11:47:04 -07:00
Ma Mingfei
a73c4df438 Add optimized native kernels in sgl-kernel (#5150) 
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
Co-authored-by: blzheng <beilei.zheng@intel.com>
 2025-04-08 09:37:46 -07:00
Yi Zhang
bcbbf519f9 sgl-kernel transfer custom allreduce from trt kernel to vllm kernel (#5079) 2025-04-05 14:23:20 -07:00
yinfan98
b8b6008f47 [Fix] fix fa3 build at cu118 (#5036) 2025-04-03 11:52:35 -07:00
Xiaoyu Zhang
2c8fd99363 [sgl-kernel] per token group quant support COLUMN MAJOR (#4817) 2025-04-02 18:29:59 -07:00
Yuhong Guo
ee47a6c1c3 [Build] Fix cuda12.8 build error in nvfp4_scaled_mm_kernels.cu (#4953) 2025-03-31 12:00:34 -07:00
yinfan98
c7457191a0 [Fix] revert clean m.def for cudagraph (#4944) 2025-03-31 02:08:55 -07:00
yinfan98
37c66ec856 [feat] add fa3 in sgl-kernel (#4902) 
Co-authored-by: Sleepcoo <Sleepcoo@gmail.com>
 2025-03-30 12:57:10 -07:00