Commit Graph

258 Commits

Author SHA1 Message Date
Elfie Guo
85ec0440a5 Update cutlass dependency. (#5447) 2025-04-15 23:28:04 -07:00
Trevor Morris
e8f62b20ca BLackwell cutlass mla: Add check for bad page size/block num combinations (#5431) 2025-04-15 14:07:42 -07:00
Yineng Zhang
88defc4d89 fix: solve release issue (#5434) 2025-04-15 12:58:11 -07:00
Yineng Zhang
6f509d5503 chore: bump sgl-kernel v0.0.9.post1 (#5430) 2025-04-15 11:00:21 -07:00
DefTruth
12ef7e3bc3 bugfix: fix merge_state_v2 cuda graph (#5419) 2025-04-15 10:18:47 -07:00
Lianmin Zheng
838fa0f218 [minor] cleanup cmakelists.txt (#5420) 2025-04-15 07:07:07 -07:00
Yineng Zhang
e940dc4f06 chore: bump sgl-kernel 0.0.9 (#5400) 2025-04-14 21:34:04 -07:00
DefTruth
388e15c0db kernel: support slightly faster merge_state_v2 cuda kernel (#5381) 2025-04-14 21:28:23 -07:00
Yineng Zhang
6c41fcf0e4 chore: upgrade DeepGEMM (#5395) 2025-04-14 20:32:46 -07:00
Lianmin Zheng
dae7944440 minor clean up of sgl-kernel/CMakeLists.txt (#5393) 2025-04-14 18:38:44 -07:00
Yineng Zhang
b62e7e99b8 feat: adapt merge_state (#5337) 2025-04-12 21:14:04 -07:00
Yineng Zhang
b371f7cd36 chore: bump sgl-kernel v0.0.8.post3 (#5332) 2025-04-12 12:53:37 -07:00
Yineng Zhang
812e82f35e fix: solve cu118 issue for cutlass mla (#5331) 2025-04-12 12:51:09 -07:00
PGFLMG
4879e50c6d [Feat] Add sparse attn to sgl-kernel (#5327) 2025-04-12 11:36:36 -07:00
Yineng Zhang
115ae2e728 chore: bump sgl-kernel v0.0.8.post2 (#5317) 2025-04-11 23:42:03 -07:00
Baizhou Zhang
e4155e96d0 Add flash_attn_varlen_func to sgl-kernel (#5315) 2025-04-11 23:36:36 -07:00
Zhaoyi Li
3c9740d200 update variable naming and comments for rocm (#5299) 2025-04-11 23:15:05 -07:00
Yineng Zhang
2eb55770f9 misc: cleanup 3rdparty (#5311) 2025-04-11 22:53:50 -07:00
Trevor Morris
f65b8d5c89 Blackwell Cutlass MLA kernel (#5142) 2025-04-11 22:16:51 -07:00
Yineng Zhang
4f288113ce fix: update flash attn (#5308) 2025-04-11 16:23:09 -07:00
Yineng Zhang
136b8e6afb fix: remove cublas_grouped_gemm (#5307) 2025-04-11 16:22:37 -07:00
Yineng Zhang
c1dd773c19 fix: use fa3 unit test on hopper only (#5304) 2025-04-11 15:10:49 -07:00
Yineng Zhang
c163bf4ff1 chore: bump sgl-kernel v0.0.8.post1 (#5289) 2025-04-11 02:11:53 -07:00
Yineng Zhang
5598634326 chore: relax the torch version restriction for sgl-kernel compilation (#5288) 2025-04-11 02:05:53 -07:00
Yineng Zhang
b75275b6f2 feat: add cu128 identifier for sgl-kernel (#5287) 2025-04-11 01:58:46 -07:00
Yineng Zhang
7074e9ca20 fix: enable fp4 compilation on cu128 (#5286) 2025-04-11 01:43:44 -07:00
Elfie Guo
a222945df2 Update Makefile / build script to avoid installing incompatible torch dependency (#5245) 2025-04-10 22:21:02 +00:00
PGFLMG
ed01b4515e [Misc] Clean sgl-kernel test (#5216) 2025-04-10 11:28:41 -07:00
HAI
d050df368c ROCm sgl-kernel: compatible to later torch (#5167) 2025-04-10 09:18:36 -07:00
Richard Zou
76f44c2a8d Fix deepseek-v3 with torch.compile in PyTorch 2.6. (#5213) 2025-04-10 09:14:38 -07:00
Xiaoyu Zhang
f730362ee2 reduce moe_align_block_size_kernel small batch mode overhead (#5086) 2025-04-09 17:59:35 -07:00
Yi Zhang
ebf495f013 sgl-kernel use cutlass latest version for fp8 blockwise gemm (#5207) 2025-04-09 11:47:04 -07:00
yinfan98
d2e507df3c [Misc] clean up vllm in sgl-kernel test (#5189) 2025-04-09 01:22:13 -07:00
Trevor Morris
11d760d56a FP4 weight loading and inference (2/2) (#3972) 2025-04-08 17:26:21 -07:00
Ma Mingfei
a73c4df438 Add optimized native kernels in sgl-kernel (#5150)
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
Co-authored-by: blzheng <beilei.zheng@intel.com>
2025-04-08 09:37:46 -07:00
yinfan98
9798e72baa [Misc] Use pytest.mark.skipif in sgl-kernel test (#5137) 2025-04-07 21:35:14 -07:00
Yineng Zhang
496dde8491 bump sgl-kernel 0.0.8 (#5089) 2025-04-05 14:28:04 -07:00
Yi Zhang
bcbbf519f9 sgl-kernel transfer custom allreduce from trt kernel to vllm kernel (#5079) 2025-04-05 14:23:20 -07:00
Yineng Zhang
3f287b8579 support sgl-kernel on blackwell (#5074) 2025-04-04 16:59:32 -07:00
Xiaoyu Zhang
924ca7c92c Add DeepSeek V3/R1 shared experts fusion (#4918) 2025-04-04 01:59:29 -07:00
Yineng Zhang
d7954b7682 bump sgl-kernel v0.0.7 (#5046) 2025-04-03 13:38:13 -07:00
yinfan98
b8b6008f47 [Fix] fix fa3 build at cu118 (#5036) 2025-04-03 11:52:35 -07:00
Zhiqiang Xie
9d0b36c47a fix deepgemm as well (#5030) 2025-04-03 02:41:37 -07:00
Yuhong Guo
7d8c0ce7ce [Build] Support build sgl-kernel with ccache (#5020) 2025-04-03 00:22:37 -07:00
Zhiqiang Xie
a2aea59b6e update cutlass tag (#5011) 2025-04-02 18:30:30 -07:00
Xiaoyu Zhang
2c8fd99363 [sgl-kernel] per token group quant support COLUMN MAJOR (#4817) 2025-04-02 18:29:59 -07:00
Yuhong Guo
ee47a6c1c3 [Build] Fix cuda12.8 build error in nvfp4_scaled_mm_kernels.cu (#4953) 2025-03-31 12:00:34 -07:00
Yineng Zhang
6384d31776 bump sgl-kernel v0.0.6 (#4950) 2025-03-31 11:24:09 -07:00
yinfan98
c7457191a0 [Fix] revert clean m.def for cudagraph (#4944) 2025-03-31 02:08:55 -07:00
Yineng Zhang
4814ecaff9 cleanup sgl-kernel (#4933) 2025-03-30 14:12:30 -07:00