Commit Graph

Select branches

Hide Pull Requests

0.5.3rc0

v0.5.2

v0.5.2rc1

v0.5.3_dev

v0.5.4

v0.5.4_dev

v0.5.4_dev_liucong

v0.5.4_dev_maxiao

Mono Color

• ac5b78baf6 fix: update test config (#5392) Yineng Zhang 2025-04-14 17:39:47 -07:00
• 38076dea84 apply fused moe gate in ds v3/r1 (#5371) Xiaoyu Zhang 2025-04-15 07:24:26 +08:00
• 5e0a9b0981 Apply deepseek cuda rope (#5385) Ke Bao 2025-04-15 06:22:43 +08:00
• bdde237562 [perf] experimental enhance fp8 per-tensor quant (#5370) JieXin Liang 2025-04-15 03:35:43 +08:00
• e9fc2ac7b6 [PD Bug] fix MLA get_contiguous_buf_infos error (#5384) ybyang 2025-04-14 22:56:39 +08:00
• 44afde82d7 Fix PD disaggregation bugs (#5326) Liangsheng Yin 2025-04-14 19:27:30 +08:00
• 072df75354 Support for Qwen2.5-VL Model in bitsandbytes Format (#5003) yhyang201 2025-04-14 17:03:40 +08:00
• defede5073 Fix DeepSeek DP Attention + torch compile (#5367) fzyzcjy 2025-04-14 16:07:58 +08:00
• fc72871975 Free metadata_buffer_index after transfer finished (#5364) Yongtong Wu 2025-04-14 16:06:14 +08:00
• 14e8bd889f Free metadata_buffer_index after transfer finished (#5364) Yongtong Wu 2025-04-14 16:04:46 +08:00
• adca585bfb [DeepEP] Reduce routed scaling overhead (#5277) yulei 2025-04-14 07:03:09 +08:00
• 39d90449f3 feat: update experiment_runner (#5360) Yineng Zhang 2025-04-13 15:37:05 -07:00
• 39e411385c fix #5322 (#5359) Yineng Zhang 2025-04-13 13:57:36 -07:00
• 5fbafbb8f8 fix MLATokenToKVPoolHost get_size_per_token bug (#5161) huangtingwei 2025-04-14 03:37:26 +08:00
• a9499885e9 [PD] Add transfer backend abstraction (#5328) Byron Hsu 2025-04-13 10:39:39 -07:00
• f765579046 Fix typo: infight -> inflight (#5357) Liangsheng Yin 2025-04-14 01:25:30 +08:00
• f58b929a51 chore: upgrade sgl-kernel 0.0.8.post3 (#5342) Yineng Zhang 2025-04-13 00:45:59 -07:00
• c1270aabc5 docs: update adoption and sponsorship list with Oracle (#5343) Yineng Zhang 2025-04-12 22:55:25 -07:00
• 8311b07fb9 Fix: Ensure tensors for dist.broadcast match NCCL backend device (#5322) mlmz 2025-04-13 13:50:37 +08:00
• c138025731 misc: update sagemaker Dockerfile (#5341) Yineng Zhang 2025-04-12 22:39:49 -07:00
• b62e7e99b8 feat: adapt merge_state (#5337) Yineng Zhang 2025-04-12 21:14:04 -07:00
• 7d3b7c87f5 fix: determine if flashinfer is installed (#5336) Yineng Zhang 2025-04-12 19:59:13 -07:00
• 75015bb688 ci: update release node (#5333) Yineng Zhang 2025-04-12 14:22:45 -07:00
• b371f7cd36 chore: bump sgl-kernel v0.0.8.post3 (#5332) Yineng Zhang 2025-04-12 12:53:37 -07:00
• 812e82f35e fix: solve cu118 issue for cutlass mla (#5331) Yineng Zhang 2025-04-12 12:51:09 -07:00
• 4879e50c6d [Feat] Add sparse attn to sgl-kernel (#5327) PGFLMG 2025-04-13 02:36:36 +08:00
• bc92107b03 Support server based rollout in Verlengine (#4848) tianlian yi 2025-04-13 01:07:52 +08:00
• 3e4794aad8 refine fused_moe tuning docs (#5294) Xiaoyu Zhang 2025-04-13 01:01:13 +08:00
• 690ec20587 Delete python/sglang/srt/layers/moe/fused_moe_triton/configs/E=257,N=… (#5321) Xiaoyu Zhang 2025-04-13 01:00:03 +08:00
• 2074a2e6b6 Fix: docs/backend/structured_outputs.ipynb (#4884) thyecust 2025-04-12 17:18:55 +08:00
• 57de7c6b5f feat: use fa3 mla by default on hopper (#5210) Yineng Zhang 2025-04-12 01:09:25 -07:00
• 115ae2e728 chore: bump sgl-kernel v0.0.8.post2 (#5317) Yineng Zhang 2025-04-11 23:42:03 -07:00
• aea98512a8 Fix fa3 window size setup (#5316) Qingquan Song 2025-04-11 23:37:52 -07:00
• e4155e96d0 Add flash_attn_varlen_func to sgl-kernel (#5315) Baizhou Zhang 2025-04-11 23:36:36 -07:00
• 1b1b47a949 Fix w8a8_int8 model shared experts fusion load weights error (#5120) lambert0312 2025-04-12 14:33:51 +08:00
• 3c9740d200 update variable naming and comments for rocm (#5299) Zhaoyi Li 2025-04-12 01:15:05 -05:00
• 2eb55770f9 misc: cleanup 3rdparty (#5311) Yineng Zhang 2025-04-11 22:53:50 -07:00
• f65b8d5c89 Blackwell Cutlass MLA kernel (#5142) Trevor Morris 2025-04-11 22:16:51 -07:00
• 5ad0571903 Adjust ci test threshold (#5271) Ke Bao 2025-04-12 13:03:37 +08:00
• 34ef6c8135 [VLM] Adopt fast image processor by default (#5065) Mick 2025-04-12 12:46:58 +08:00
• 611720919d fix: use deepgemm only on hopper (#5310) Yineng Zhang 2025-04-11 20:48:24 -07:00
• 4f288113ce fix: update flash attn (#5308) Yineng Zhang 2025-04-11 16:23:09 -07:00
• 136b8e6afb fix: remove cublas_grouped_gemm (#5307) Yineng Zhang 2025-04-11 16:22:37 -07:00
• 034c5256cc misc: update blackwell Dockerfile (#5306) Yineng Zhang 2025-04-11 15:58:04 -07:00
• c1dd773c19 fix: use fa3 unit test on hopper only (#5304) Yineng Zhang 2025-04-11 15:10:49 -07:00
• 6f8593799b feat: add blackwell workflow (#5303) Yineng Zhang 2025-04-11 13:42:00 -07:00
• f774a0d275 feat: add blackwell Dockerfile (#5302) Yineng Zhang 2025-04-11 13:08:53 -07:00
• 60bcbf2a35 remove moe_align_block_size torch.zeros in small batch/expert mode (#5298) Xiaoyu Zhang 2025-04-12 03:13:55 +08:00
• a0a9f6d64f [Docs] Remove the older supported docs section (#5301) Adarsh Shirawalmath 2025-04-12 00:00:18 +05:30
• 80aa8ca84e fix: update update_wheel_index for cu128 (#5300) Yineng Zhang 2025-04-11 09:31:03 -07:00
• 4aa6bab0b0 [Docs] Supported Model Docs - Major restructuring (#5290) Adarsh Shirawalmath 2025-04-11 21:47:47 +05:30
• c35dcfdb30 [PD] fix: skip warmup request in disaggregation mode to prevent crash on timeout (#5292) Yusong Gao 2025-04-11 23:03:07 +08:00
• c163bf4ff1 chore: bump sgl-kernel v0.0.8.post1 (#5289) Yineng Zhang 2025-04-11 02:11:53 -07:00
• 5598634326 chore: relax the torch version restriction for sgl-kernel compilation (#5288) Yineng Zhang 2025-04-11 02:05:53 -07:00
• b75275b6f2 feat: add cu128 identifier for sgl-kernel (#5287) Yineng Zhang 2025-04-11 01:58:46 -07:00
• 7074e9ca20 fix: enable fp4 compilation on cu128 (#5286) Yineng Zhang 2025-04-11 01:43:44 -07:00
• fc14cca088 Fix a 404 link in send_request.ipynb (#5280) Michael Yao 2025-04-11 16:38:45 +08:00
• e7beff8a13 fix: examples for token_in_token_out_vlm (#5193) XinyuanTong 2025-04-11 01:38:23 -07:00
• 4d2e305149 doc: nested loop code for offline engine (#5244) mlmz 2025-04-11 16:36:30 +08:00
• e53a0b3d5b [fix] fix mrope positions not picked up (#5265) Mick 2025-04-11 16:29:45 +08:00
• 038bc5d521 Support --enable-llama4-multimodal (#5254) Cheng Wan 2025-04-11 04:24:14 -04:00
• aee62d744b Optimize GPU memory usage in FlashAttentionBackend's strided indexing (#5262) Chang Su 2025-04-11 00:34:17 -07:00
• cd7e32e2cb Optimize attention in llama4 (#5127) fzyzcjy 2025-04-11 15:32:41 +08:00
• 8879944800 ROCm/AITER CK_MoE: update 2-stage kernels & support both Activations (#5228) HAI 2025-04-10 18:19:57 -07:00
• a879811c4b Fix torch.compile cacheing (#5259) Richard Zou 2025-04-10 21:08:45 -04:00
• a222945df2 Update Makefile / build script to avoid installing incompatible torch dependency (#5245) Elfie Guo 2025-04-10 15:21:02 -07:00
• ed01b4515e [Misc] Clean sgl-kernel test (#5216) PGFLMG 2025-04-11 02:28:41 +08:00
• d050df368c ROCm sgl-kernel: compatible to later torch (#5167) HAI 2025-04-10 09:18:36 -07:00
• 76f44c2a8d Fix deepseek-v3 with torch.compile in PyTorch 2.6. (#5213) Richard Zou 2025-04-10 12:14:38 -04:00
• 1078396f47 Update deps for mllama4 (#5215) Ke Bao 2025-04-11 00:12:44 +08:00
• 7e4f72dd8c [PD] Add get_contiguous_buf_infos interface for MLATokenToKVPool (#5204) Teng Ma 2025-04-10 20:05:34 +08:00
• 4c31ae9f6d [PD] Support KV transfer with mooncake (#4880) Teng Ma 2025-04-10 14:23:23 +08:00
• f730362ee2 reduce moe_align_block_size_kernel small batch mode overhead (#5086) Xiaoyu Zhang 2025-04-10 08:59:35 +08:00
• e3c4bd3153 Fix DeepSeek error when using DeepEP mode (#5190) fzyzcjy 2025-04-10 08:43:22 +08:00
• 5db37c8626 [metrics] Add in queue metrics (#4444) Stefan He 2025-04-09 17:19:27 -07:00
• 4cb53ecd0c fix: log warning when disable cuda graph (#5209) Yineng Zhang 2025-04-09 14:16:13 -07:00
• 456b008bd8 Add H20 dtype fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 (#5196) Zhaoyang Hao 2025-04-10 02:54:36 +08:00
• ebf495f013 sgl-kernel use cutlass latest version for fp8 blockwise gemm (#5207) Yi Zhang 2025-04-10 02:47:04 +08:00
• 7f875f1293 update grok test (#5171) saienduri 2025-04-09 11:09:47 -07:00
• fbebcb7aa4 model: support mllama4 (#5144) Mick 2025-04-10 00:28:44 +08:00
• 87eddedfa2 [ci] fix ci test fused_moe op (#5102) Xiaoyu Zhang 2025-04-09 23:52:46 +08:00
• 4065248214 Support Llama4 fp8 inference (#5194) HandH1998 2025-04-09 20:14:34 +08:00
• 86a876d883 Optimize topk operation in llama4 (#5128) fzyzcjy 2025-04-09 17:50:22 +08:00
• 92823069c4 Fix ci test "test_eval_fp8_accuracy" failed (#5185) kk 2025-04-09 17:44:05 +08:00
• d2e507df3c [Misc] clean up vllm in sgl-kernel test (#5189) yinfan98 2025-04-09 16:22:13 +08:00
• 61970b08d8 Let bench_one_batch support enable_dp_attention (#4058) fzyzcjy 2025-04-09 14:44:25 +08:00
• 76c48a0913 [DeepEP] fix: import buffer error (#5179) Cheng Wan 2025-04-09 01:12:14 -04:00
• 90caf06c00 fix: use DeepEPDispatcher on CUDA (#5180) Yineng Zhang 2025-04-08 21:56:53 -07:00
• 6669d12707 feat: add DeepGEMM build warning (#5176) Yineng Zhang 2025-04-08 21:16:23 -07:00
• f2b70afde0 docs: remove the use of Downward API for LWS_WORKER_INDEX (#5110) Kay Yan 2025-04-09 11:46:11 +08:00
• bc3f6db2dd [Fix] DeepEP Compatibility with Low Latency (#5068) Jinyan Chen 2025-04-09 11:31:31 +08:00
• aac531c53b [Bugfix] Fix index out of bounds in local attention with large sequences (#5173) Chang Su 2025-04-08 18:43:13 -07:00
• 39efad4fbc Tiny disable model that does not work (#5175) fzyzcjy 2025-04-09 09:42:37 +08:00
• 466899e69c Fix multimodal hashing error (#5174) fzyzcjy 2025-04-09 09:42:26 +08:00
• 11d760d56a FP4 weight loading and inference (2/2) (#3972) Trevor Morris 2025-04-08 17:26:21 -07:00
• 5039d54772 Support 2x8xH100 for Llama 4 (#5159) fzyzcjy 2025-04-09 05:55:14 +08:00
• d09a51f1f6 [feat&refactor] Enhance multimodal input support with refactor io_struct (#4938) XinyuanTong 2025-04-08 14:48:07 -07:00
• f8194b267c Small improvement of native api docs (#5139) simveit 2025-04-08 21:09:26 +02:00
• 6d3b35fae9 [PD] Simplify mini LB (#4911) Byron Hsu 2025-04-08 09:42:34 -07:00
• a73c4df438 Add optimized native kernels in sgl-kernel (#5150) Ma Mingfei 2025-04-09 00:37:46 +08:00