Commit Graph

438 Commits

Author SHA1 Message Date
Yineng Zhang
dd949ace23 Revert "[1/2][resubmit] sgl-kernel: Fuse routed scaling factor into m… (#9035) 2025-08-10 17:34:54 -07:00
Lianmin Zheng
2c7f01bc89 Reorganize CI and test files (#9027) 2025-08-10 12:30:06 -07:00
huangtingwei
86497d99f2 fix page first per layer pf2lf kernel (#8915)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2025-08-09 17:16:11 -07:00
cctry
5c31b35db2 [hicache] Optimization for DMA copy (#8245) 2025-08-09 17:16:07 -07:00
ishandhanani
de8b8b6e5c chore(deps): update minimum python to 3.10 (#8984) 2025-08-09 00:30:23 -07:00
Trevor Morris
591c232f7c [1/2][resubmit] sgl-kernel: Fuse routed scaling factor into moe_fused_gate (select_experts) (#8770) 2025-08-08 17:55:06 -07:00
ishandhanani
4e7f025219 chore(gb200): update to CUDA 12.9 and improve build process (#8772) 2025-08-08 13:42:47 -07:00
Yineng Zhang
54ea57f245 chore: bump sgl-kernel v0.3.3 (#8957) 2025-08-08 01:35:37 -07:00
triple-mu
444013585d Fix typos and unify size(s)/stride(s) API calls (#8799) 2025-08-08 00:18:08 -07:00
Hongbo Xu
39fd178831 refactor: Move scalar_types.py to sgl-kernel to avoid circular import (#8720) 2025-08-07 19:22:16 -07:00
Yineng Zhang
cbbd685a46 chore: use torch 2.8 stable (#8880) 2025-08-06 15:51:40 -07:00
Yuan Luo
1bd5316873 fix benchmark fp8 blockwise group gemm (#8815) 2025-08-06 21:02:21 +08:00
Yineng Zhang
8e8545caf6 fix: update cmake (#8817) 2025-08-05 09:38:30 -07:00
Yineng Zhang
75df31b60e chore: bump sgl-kernel v0.3.2 (#8802) 2025-08-05 02:35:20 -07:00
Yineng Zhang
194561f27a feat: support sgl-kernel cu129 (#8800) 2025-08-05 02:33:47 -07:00
Chunyuan WU
08f8f49016 [CPU][sgl-kernel] biased_grouped_topk: fix correction_bias dtype to float32 (#8212)
Co-authored-by: jianan-gu <jianan.gu@intel.com>
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
2025-08-04 18:28:31 -07:00
Yineng Zhang
02bc1c7d80 chore: bump sgl-kernel v0.3.1 (#8771) 2025-08-04 13:18:54 -07:00
Qiaolin Yu
fc8c8e5041 Integrate triton_kernels in sgl-kernel (#8762) 2025-08-04 12:12:14 -07:00
Xiaoyu Zhang
f57d2dc162 [sgl-kernel] avoid per_token_quant_fp8.cu hardcode sm_count (#8738) 2025-08-04 12:55:57 +08:00
Yineng Zhang
5ce5093b97 chore: bump sgl-kernel 0.3.0 with torch 2.8.0 (#8718) 2025-08-03 02:31:50 -07:00
Qi Yuhang
d9def43dcd [Perf]Use Cooperative Schedule for H100 & H200 & H800 in fp8_blockwise_scaled_grouped_mm (#8722) 2025-08-02 21:13:47 -07:00
Yineng Zhang
0a56b721d5 chore: bump sgl-kernel v0.2.9 (#8713) 2025-08-02 16:21:56 -07:00
Liangsheng Yin
603f5ce020 [Bug] fix green context's incompatibility with cuda < 12.4 (#8701) 2025-08-02 15:23:11 -07:00
Liangsheng Yin
f9f0138f80 Revert "[1/2] sgl-kernel: Fuse routed scaling factor into select_experts" (#8706) 2025-08-02 20:14:30 +08:00
PGFLMG
ac6962ccd6 [Doc] Polish sgl-kernel readme for cu126 build error (#8704) 2025-08-02 17:03:07 +08:00
Trevor Morris
f642524fd9 [1/2] sgl-kernel: Fuse routed scaling factor into select_experts (#8364) 2025-08-01 18:14:24 -07:00
YanbingJiang
1fe691a429 Fix FP8 block quantization when N or K is not multiples of 128 (#8648) 2025-08-01 15:57:19 -07:00
Stefan He
db7343c992 fix per token cuda kernel hidden dim cannot divide by 16 (#8543) 2025-08-01 09:27:18 -07:00
Hongbo Xu
533cb5b274 [DOC]Update sgl-kernel README (#8665) 2025-08-01 22:59:27 +08:00
Peter Pan
6bdd27861b [Kimi K2] dsv3_router_gemm supports NUM_EXPERTS == 384 (#8013) 2025-08-01 22:01:24 +08:00
Tao He
5d15fb8c9d [bugifx] QWen-1M context support[2/3] using current cuda stream in the DCA's kernel for bugfix. (#8611)
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: sa-buc <linzhu.ht@w32d09270.cloud.sqa.na131>
2025-07-31 22:41:39 +08:00
Yineng Zhang
43118f5f2a chore: bump sgl-kernel v0.2.8 (#8599) 2025-07-30 22:23:52 -07:00
Cheng Wan
a5f5ab4030 update sgl-kernel for EP: kernel part (#8514)
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2025-07-30 22:19:55 -07:00
Qi Yuhang
9b9e82539b [Fix]Fix index oob in get_group_gemm_starts kernel. (#8564) 2025-07-30 19:49:35 -07:00
Yuan Luo
3bdcdd134b [Hot-Fix] moe_aligned_block_size CI failed in AMD (#8461)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
2025-07-31 00:28:32 +08:00
Ke Bao
5973675bc3 Fix moe align kernel test (#8531) 2025-07-29 11:03:02 -07:00
Xiaoyu Zhang
7a4309cc8a [sgl-kernel performace] fix fp8 quant kernels dispatch __nv_fp8_e4m3 bug to improve performance 10%-20% (#8499)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2025-07-29 23:31:54 +08:00
Xiaoyu Zhang
2262369905 Revert "[kernel] opt moe align block kernel by block/warp scan algorithm" (#8457) 2025-07-28 01:35:43 -07:00
strgrb
fb4ce17de6 Fix per_token_group_quant_8bit when hidden_dim // group_size is not divided by 4. (#8449)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
2025-07-28 01:32:46 -07:00
Elfie Guo
5c9c275bc8 Use FlashInfer FP4 gemm. (#8241) 2025-07-27 01:05:22 -07:00
Baizhou Zhang
91e3d1542e Update Cutlass in sgl-kernel to v4.1 (#8392) 2025-07-27 00:36:15 -07:00
fzyzcjy
e34cf6ad75 Fix bench script making input data on L2 cache (#7739) 2025-07-27 00:30:24 -07:00
Qi Yuhang
426b74936a Add nvfp4 scaled mm benchmark. (#8401) 2025-07-26 23:18:04 -07:00
Ke Bao
8af145b7dc Fix test_moe_fused_gate_combined sgl-kernel ci test (#8374) 2025-07-26 09:30:12 +08:00
Hubert Lu
af4b9bae95 [AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs (#7135)
Co-authored-by: yiakwy-xpu-ml-framework-team <961186938@qq.com>
Co-authored-by: HAI <hixiao@gmail.com>
2025-07-24 23:44:28 -07:00
li haoyang
28d4d47280 [Feature] Integrate quick allreduce and select the best allreduce implementation (#6619)
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
2025-07-24 20:48:42 -07:00
Zhiqiang Xie
d40846d456 breakdown kernel update (#8334) 2025-07-25 08:33:17 +08:00
Yineng Zhang
4c605235aa fix: workaround for deepgemm warmup issue (#8302) 2025-07-23 12:01:51 -07:00
Yuan Luo
0c8dab9e67 [sgl-kernel] Opt per_token_quant_fp8 with warp reduce (#8130)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2025-07-23 21:22:59 +08:00
Zhiqiang Xie
b43263307f Hicache IO kernel refactoring (#8264) 2025-07-23 16:49:03 +08:00