EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
4,190 Commits 8 Branches 0 Tags
f8260f25391fcd68e2fe82324d5b4b970aa76b60
Commit Graph

2850 Commits

Author SHA1 Message Date
Chang Su
f8260f2539 [Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357) 2025-07-25 12:03:16 -07:00
Xu Wenqing
12cb760a37 Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#8344) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-07-25 10:58:12 -07:00
Stepan Kargaltsev
1b9cea5ade [P/D] Support ipv6 in P/D scenario (#7858) 
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-07-25 08:53:30 -07:00
Xiaoyu Zhang
9045cc1eb8 [torch.compile bug] avoid biased_grouped_topk_impl func repeatedly triggering torch.compile in forward pass (#8353) 2025-07-25 21:17:47 +08:00
Hubert Lu
af4b9bae95 [AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs (#7135) 
Co-authored-by: yiakwy-xpu-ml-framework-team <961186938@qq.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2025-07-24 23:44:28 -07:00
Ying Wang
7ad6b766c5 fix: Fix failed functional tests https://github.com/meta-llama/llama-stack-evals (#8266) 2025-07-24 23:11:32 -07:00
Cheng Wan
c0fb25e949 DP Enhancement (#8280) 2025-07-24 21:36:21 -07:00
li haoyang
28d4d47280 [Feature] Integrate quick allreduce and select the best allreduce implementation (#6619) 
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-07-24 20:48:42 -07:00
ZhichenJiang
f4674df646 support idle batch for TBO (#8233) 2025-07-24 20:43:52 -07:00
Zhiqiang Xie
145482f422 HiCache Storage TP Refinement (#8307) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
 2025-07-25 08:31:47 +08:00
Swipe4057
8d1c5b948e chore: upgrade flashinfer v0.2.9rc1 (#8301) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-07-24 14:29:56 -07:00
Xiaoyu Zhang
a167fd0bcb [code style] Clean dead triton kernel code in fused_moe and useless vllm_ops import (#8310) 2025-07-24 14:38:30 +08:00
Minho Ryu
bfb118c01e fix bug when eos_ids==0 (#8315) 2025-07-23 23:18:47 -07:00
Simo Lin
5dd0f870ab [bug] fix pd completion protocol for batching support (#8317) 2025-07-23 23:18:17 -07:00
Haohui Mai
f7e102d56a Pin the version of petit kernel to fix the APIs (#8235) 2025-07-23 17:57:20 -07:00
xianzhiT
624a3b8d1f Fix incomplete tool call capture issue in streaming response of DeepSeek-V3 when enable MTP (#7562) 2025-07-23 17:40:23 -07:00
Chang Su
01079e174f feat(function call): complete utility method for KimiK2Detector and enhance documentation (#8043) 2025-07-23 17:37:31 -07:00
J
0e7a5b2694 fix: prevent crashes due to logit bias dimension mismatch (#7685) 2025-07-23 15:30:55 -07:00
Yineng Zhang
4953f4ca9a chore: upgrade sgl-kernel 0.2.7 (#8304) 2025-07-23 15:07:27 -07:00
Xinyuan Tong
38000a5f44 Fix gemma3n with hybrid swa (#8240) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-07-23 13:29:18 -07:00
Xinyuan Tong
70251e935e fix: match chat-template for internvl3 (#8262) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-07-23 13:29:03 -07:00
xianzhiT
c87d4fec99 Fix the issue of incorrect finish reason in final stream response chunk returned during tool call (#7708) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-23 13:28:53 -07:00
YiXR
a99801e075 [Performance][PD Disaggregation] optimize TokenToKVPoolAllocator by sorting free pages (#8133) 
Signed-off-by: Xingrui Yi <yixingrui@linux.alibaba.com>
Co-authored-by: Xingrui Yi <yixingrui@linux.alibaba.com>
 2025-07-23 13:28:12 -07:00
Zhiqiang Xie
f39037fffb HiCache Fix (#8288) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
 2025-07-23 16:51:32 +08:00
Lifu Huang
8abd3e77fe Introduce Stable LoRA ID System for Overlapped Updates and Prefix Caching (#8261) 2025-07-23 00:32:16 -07:00
Ke Bao
e2d66f60c8 Skip llama4 vision module loading when multimodal disabled (#8272) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-07-23 12:41:25 +08:00
Yineng Zhang
01c000043c chore: bump v0.4.9.post3 (#8265) 2025-07-22 15:55:48 -07:00
yhyang201
0dfe2491ac Preliminary Support for Qwen3XMLDetector (#8260) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-07-23 06:49:38 +08:00
Hubert Lu
e50109f2ed [AMD] Remove vllm's scaled_fp8_quant and moe_sum when SGLANG_USE_AITER=1 (#7484) 2025-07-21 17:33:19 -07:00
Xinyuan Tong
69adc4f81c fix: retrieve mm token by modality, raise error if none (#8221) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-07-22 08:06:35 +08:00
Yineng Zhang
74f59ae555 chore: upgrade sgl-kernel 0.2.6.post1 (#8202) 2025-07-21 02:10:24 -07:00
Ke Bao
6936be3221 Remve router gemm output dtype conversion (#8204) 2025-07-21 15:37:00 +08:00
Xinyuan Tong
8430bfe3e9 [Refactor] simplify multimodal data processing (#8107) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-07-20 21:43:09 -07:00
Ke Bao
c9e8613c97 Apply fused sorted token ids padding (#8193) 2025-07-21 11:19:48 +08:00
JieXin Liang
7eebd44047 [fix] fix modelopt fp4 on b200 (#8195) 2025-07-20 17:39:57 -07:00
Ke Bao
465968b2e3 Fix dtype error in CI (#8197) 2025-07-21 00:27:55 +08:00
GuoYipin
750838adc4 fix: fix the bug of loading Internvl3 (#8067) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-07-20 22:22:54 +08:00
Jay Zhou
99aefa037e Fix eagle3 cuda graph (#8163) 2025-07-20 15:28:06 +08:00
Qiaolin Yu
bbcfbc1a02 feat: add h200 tp 16 kimi k2 moe config (#8183) 2025-07-19 23:30:08 -07:00
Praneth Paruchuri
83c104b188 Feat: Support for Persimmon Model (#7983) 2025-07-19 23:07:47 -07:00
Lianmin Zheng
55381a46ac Revert "[Feature] Simple Improve Health Check Mechanism for Production-Grade Stability" (#8181) 2025-07-19 22:41:30 -07:00
Atream
a589a07167 fix moe gate dtype, fix tbo, fix fake dispatch (#7825) 2025-07-19 22:13:46 -07:00
Yineng Zhang
f62d75b6a1 feat: add b200 tp 16 kimi k2 moe config (#8178) 2025-07-19 20:04:12 -07:00
Yineng Zhang
0f9b11e310 feat: add h200 tp 16 kimi k2 moe config (#8176) 2025-07-19 20:04:02 -07:00
Pavel Logachev
877e35d775 Add get_hidden_dim to qwen3.py for correct lora (#7312) 2025-07-19 19:31:16 -07:00
Clay
cbdfb77123 Enable FlashInfer support encoder models and add head_dim padding workaround (#6230) 2025-07-19 19:30:16 -07:00
ybyang
4540a4666a [Feature] Simple Improve Health Check Mechanism for Production-Grade Stability (#8115) 
Signed-off-by: ybyang <ybyang7@iflytek.com>
 2025-07-19 18:10:00 -07:00
Baizhou Zhang
8cddfa56a1 Clean warning logs for gate_proj loading in Lora (#8172) 2025-07-19 15:56:50 -07:00
Lifu Huang
4e3defe5a7 Support start up LoRA server without initial adapters (#8019) 2025-07-19 15:38:09 -07:00
Garry Fang
60468da4e2 bugfix: fix sglang crash in NVIDIA MIG container (#8167) 
Signed-off-by: Garrybest <garrybest@foxmail.com>
 2025-07-19 14:41:27 -07:00