EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
4,233 Commits 8 Branches 0 Tags
e983d66680e4cd24fb382f8cf4183b64e228d995
Commit Graph

2881 Commits

Author SHA1 Message Date
Binyao Jiang
e983d66680 Fix: Improve test_openai_function_calling unit test and fix reasoning_parser.py think_start_token logic (#8316) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-07-27 13:12:59 -07:00
fzyzcjy
b58c3c285e Support ue8m0 for triton quant kernel (#7603) 2025-07-27 13:04:35 -07:00
Lifu Huang
df90645525 Support overlapped lora updates (#8213) 2025-07-27 13:00:44 -07:00
Shangming Cai
22e00eeb4a [Bugfix] Prevent PD server crash from invalid grammar (#8062) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-07-28 00:17:51 +08:00
Yuan Luo
b3eac168e7 Support triton kernels v3.4.0 for fused_moe (#8258) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-07-27 02:28:49 -07:00
Yineng Zhang
10ee89559e chore: upgrade flashinfer v0.2.9rc2 (#8406) 2025-07-27 01:41:22 -07:00
Cheng Wan
4d921f2b79 [hotfix] fix merge conflicts in FlashInferEPMoE (#8405) 2025-07-27 01:24:10 -07:00
Kevin Xiang Li
44d600cd67 Support precomputed_embeddings for Llama 4 (#8156) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-07-27 01:14:49 -07:00
Elfie Guo
5c9c275bc8 Use FlashInfer FP4 gemm. (#8241) 2025-07-27 01:05:22 -07:00
Cheng Wan
bf0f448fe5 [2/N] MoE Refactor: Unify weight loader and quant methods (#8397) 2025-07-27 01:00:21 -07:00
Yingchun Lai
36d6f0ba5b fix: fix the missing metrics on non-rank0 nodes (#7720) 2025-07-27 00:55:25 -07:00
Li Hui
2a1936de96 Add A800 fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#8351) 2025-07-27 00:46:07 -07:00
Mick
0bcc195f4e fix: minor fix TransportProxyTensor under tp (#8382) 2025-07-27 00:38:49 -07:00
Kaixi Hou
85486b6f6f [NVIDIA] Add Flashinfer MoE blockscale fp8 backend (#8036) 2025-07-27 00:34:41 -07:00
fzyzcjy
62222bd27e Minor tool for comparison of benchmark results (#7974) 2025-07-27 00:27:50 -07:00
fzyzcjy
ed0fdbf35b Tool to dump and compare internal activation tensors (#7976) 2025-07-27 00:27:21 -07:00
Xinyuan Tong
b602f42354 Urgent Fix: intern-s1 chat-template matching (#8403) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-07-27 00:22:31 -07:00
Zhiqiang Xie
528bd1ed85 HiCache, check before terminate prefetching (#8372) 2025-07-26 23:13:16 -07:00
Lifu Huang
761546315c Remove slot usage in code to be backward-compatible with python 3.9 (#8396) 2025-07-26 21:24:22 -07:00
Lifu Huang
5c705b1dce Add perf tests for LoRA (#8314) 2025-07-26 14:55:22 -07:00
RunningLeon
b7094a5ef1 model: support intern-s1 (#8350) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zxy <zhou0493@e.ntu.edu.sg>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-26 13:48:51 -07:00
fzyzcjy
da0c026084 Tiny assert EPLB is used together with expert parallel (#8381) 2025-07-26 03:20:39 -07:00
Mick
3212c2ad3f vlm: optimize tensor transport (#6003) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-26 17:41:01 +08:00
Mick
534756749a chore: improvements on mm_utils (#7737) 2025-07-26 17:38:56 +08:00
Stefan He
ce32bc2ba9 Extract update_weights from RL Engine to SGLang to keep simplicity and fix torch reduce (#8267) 
Co-authored-by: CuiBo 82354186+SuperCB@users.noreply.github.com
Co-authored-by: GeLee 865038696@qq.com
Co-authored-by: 杨睿 yangruipis@163.com
 2025-07-26 02:00:59 -07:00
Cheng Wan
e236d8fee8 Save peak memory in logits processor (#8343) 2025-07-26 01:46:42 -07:00
Yineng Zhang
2272c2a5b5 chore: bump v0.4.9.post4 (#8305) 2025-07-25 17:12:47 -07:00
Trevor Morris
58c468f404 Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333) 2025-07-25 16:40:23 -07:00
Yineng Zhang
f8ca2368b2 fix: kimi k2 xgrammar crash (#8367) 
Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com>
Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>
 2025-07-25 15:44:01 -07:00
Chang Su
d8ee15643b [Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 (#8363) 2025-07-25 14:59:42 -07:00
Lianmin Zheng
ed2e313eb6 Clean up server_args, triton cache manager (#8332) 2025-07-25 14:14:51 -07:00
Chang Su
f8260f2539 [Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357) 2025-07-25 12:03:16 -07:00
Xu Wenqing
12cb760a37 Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#8344) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-07-25 10:58:12 -07:00
Stepan Kargaltsev
1b9cea5ade [P/D] Support ipv6 in P/D scenario (#7858) 
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-07-25 08:53:30 -07:00
Xiaoyu Zhang
9045cc1eb8 [torch.compile bug] avoid biased_grouped_topk_impl func repeatedly triggering torch.compile in forward pass (#8353) 2025-07-25 21:17:47 +08:00
Hubert Lu
af4b9bae95 [AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs (#7135) 
Co-authored-by: yiakwy-xpu-ml-framework-team <961186938@qq.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2025-07-24 23:44:28 -07:00
Ying Wang
7ad6b766c5 fix: Fix failed functional tests https://github.com/meta-llama/llama-stack-evals (#8266) 2025-07-24 23:11:32 -07:00
Cheng Wan
c0fb25e949 DP Enhancement (#8280) 2025-07-24 21:36:21 -07:00
li haoyang
28d4d47280 [Feature] Integrate quick allreduce and select the best allreduce implementation (#6619) 
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-07-24 20:48:42 -07:00
ZhichenJiang
f4674df646 support idle batch for TBO (#8233) 2025-07-24 20:43:52 -07:00
Zhiqiang Xie
145482f422 HiCache Storage TP Refinement (#8307) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
 2025-07-25 08:31:47 +08:00
Swipe4057
8d1c5b948e chore: upgrade flashinfer v0.2.9rc1 (#8301) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-07-24 14:29:56 -07:00
Xiaoyu Zhang
a167fd0bcb [code style] Clean dead triton kernel code in fused_moe and useless vllm_ops import (#8310) 2025-07-24 14:38:30 +08:00
Minho Ryu
bfb118c01e fix bug when eos_ids==0 (#8315) 2025-07-23 23:18:47 -07:00
Simo Lin
5dd0f870ab [bug] fix pd completion protocol for batching support (#8317) 2025-07-23 23:18:17 -07:00
Haohui Mai
f7e102d56a Pin the version of petit kernel to fix the APIs (#8235) 2025-07-23 17:57:20 -07:00
xianzhiT
624a3b8d1f Fix incomplete tool call capture issue in streaming response of DeepSeek-V3 when enable MTP (#7562) 2025-07-23 17:40:23 -07:00
Chang Su
01079e174f feat(function call): complete utility method for KimiK2Detector and enhance documentation (#8043) 2025-07-23 17:37:31 -07:00
J
0e7a5b2694 fix: prevent crashes due to logit bias dimension mismatch (#7685) 2025-07-23 15:30:55 -07:00
Yineng Zhang
4953f4ca9a chore: upgrade sgl-kernel 0.2.7 (#8304) 2025-07-23 15:07:27 -07:00