EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
4,244 Commits 8 Branches 0 Tags
484d0e021dcf825e692c1912365ed2a6cd3d947a
Commit Graph

4244 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Qiaolin Yu
484d0e021d doc: add bench_one_batch_server in the benchmark doc (#8441) 2025-07-27 23:07:54 -07:00
kyleliang-nv
5922c0cbf6 Remove zstd compression for building Dockerfile.gb200 (#8442) 2025-07-27 22:58:53 -07:00
Yuxuan Zhang
6d6a8bc278 GLM-4.5 Model Support (#8224) 
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-07-27 22:54:07 -07:00
Shangming Cai
2fd5c7049f [PD] Fix abort_request for PD disaggregation (#8352) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-07-27 21:48:27 -07:00
Stefan He
4ad9737045 chore: bump transformer to 4.54.0 (#8416) 
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-07-27 21:27:25 -07:00
Qiaolin Yu
2810338401 [feat] Support different attention backends for prefill and decode (#6338) 
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-07-28 11:42:29 +08:00
Simo Lin
fe6a445d1e [router] improve router logs and request id header (#8415) 2025-07-27 19:30:19 -07:00
Chang Su
dd487e5553 bugfix: Fix XGrammar backend to use model's EOS tokens for constrained generation (#8422) 2025-07-28 10:01:02 +08:00
kyleliang-nv
bb81daefb8 Fix docker buildx push error (#8425) 2025-07-27 17:59:38 -07:00
Chang Su
58dd95fbc8 Fix test_openai_server (#8419) 2025-07-27 13:36:01 -07:00
Chang Su
b47eda3316 bugfix: Fix multiple finish_reason chunks and tool_calls finish reason check (#8417) 2025-07-27 13:31:06 -07:00
Binyao Jiang
e983d66680 Fix: Improve test_openai_function_calling unit test and fix reasoning_parser.py think_start_token logic (#8316) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-07-27 13:12:59 -07:00
fzyzcjy
b58c3c285e Support ue8m0 for triton quant kernel (#7603) 2025-07-27 13:04:35 -07:00
Lifu Huang
df90645525 Support overlapped lora updates (#8213) 2025-07-27 13:00:44 -07:00
kyleliang-nv
95217a9b4d Change to use native arm runner (#8414) 2025-07-27 12:48:12 -07:00
Shangming Cai
22e00eeb4a [Bugfix] Prevent PD server crash from invalid grammar (#8062) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-07-28 00:17:51 +08:00
Yuan Luo
b3eac168e7 Support triton kernels v3.4.0 for fused_moe (#8258) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-07-27 02:28:49 -07:00
Yineng Zhang
10ee89559e chore: upgrade flashinfer v0.2.9rc2 (#8406) 2025-07-27 01:41:22 -07:00
Yineng Zhang
bf3352c559 chore: update CODEOWNERS (#8407) 2025-07-27 01:39:36 -07:00
Cheng Wan
4d921f2b79 [hotfix] fix merge conflicts in FlashInferEPMoE (#8405) 2025-07-27 01:24:10 -07:00
Kevin Xiang Li
44d600cd67 Support precomputed_embeddings for Llama 4 (#8156) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-07-27 01:14:49 -07:00
Elfie Guo
5c9c275bc8 Use FlashInfer FP4 gemm. (#8241) 2025-07-27 01:05:22 -07:00
Cheng Wan
bf0f448fe5 [2/N] MoE Refactor: Unify weight loader and quant methods (#8397) 2025-07-27 01:00:21 -07:00
Yingchun Lai
36d6f0ba5b fix: fix the missing metrics on non-rank0 nodes (#7720) 2025-07-27 00:55:25 -07:00
Li Hui
2a1936de96 Add A800 fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#8351) 2025-07-27 00:46:07 -07:00
Simo Lin
2ab97023e3 [router] add different policies for p node and d node (#8395) 2025-07-27 00:39:20 -07:00
Mick
0bcc195f4e fix: minor fix TransportProxyTensor under tp (#8382) 2025-07-27 00:38:49 -07:00
Baizhou Zhang
91e3d1542e Update Cutlass in sgl-kernel to v4.1 (#8392) 2025-07-27 00:36:15 -07:00
Kaixi Hou
85486b6f6f [NVIDIA] Add Flashinfer MoE blockscale fp8 backend (#8036) 2025-07-27 00:34:41 -07:00
fzyzcjy
e34cf6ad75 Fix bench script making input data on L2 cache (#7739) 2025-07-27 00:30:24 -07:00
fzyzcjy
62222bd27e Minor tool for comparison of benchmark results (#7974) 2025-07-27 00:27:50 -07:00
fzyzcjy
ed0fdbf35b Tool to dump and compare internal activation tensors (#7976) 2025-07-27 00:27:21 -07:00
Xinyuan Tong
b602f42354 Urgent Fix: intern-s1 chat-template matching (#8403) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-07-27 00:22:31 -07:00
Qi Yuhang
426b74936a Add nvfp4 scaled mm benchmark. (#8401) 2025-07-26 23:18:04 -07:00
Zhiqiang Xie
528bd1ed85 HiCache, check before terminate prefetching (#8372) 2025-07-26 23:13:16 -07:00
kyleliang-nv
62a6b7c773 Add docker release flow for gb200 (#8394) 2025-07-26 21:25:07 -07:00
Lifu Huang
761546315c Remove slot usage in code to be backward-compatible with python 3.9 (#8396) 2025-07-26 21:24:22 -07:00
Lifu Huang
5c705b1dce Add perf tests for LoRA (#8314) 2025-07-26 14:55:22 -07:00
RunningLeon
b7094a5ef1 model: support intern-s1 (#8350) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zxy <zhou0493@e.ntu.edu.sg>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-26 13:48:51 -07:00
fzyzcjy
da0c026084 Tiny assert EPLB is used together with expert parallel (#8381) 2025-07-26 03:20:39 -07:00
Mick
3212c2ad3f vlm: optimize tensor transport (#6003) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-07-26 17:41:01 +08:00
Mick
534756749a chore: improvements on mm_utils (#7737) 2025-07-26 17:38:56 +08:00
Stefan He
ce32bc2ba9 Extract update_weights from RL Engine to SGLang to keep simplicity and fix torch reduce (#8267) 
Co-authored-by: CuiBo 82354186+SuperCB@users.noreply.github.com
Co-authored-by: GeLee 865038696@qq.com
Co-authored-by: 杨睿 yangruipis@163.com
 2025-07-26 02:00:59 -07:00
Cheng Wan
e236d8fee8 Save peak memory in logits processor (#8343) 2025-07-26 01:46:42 -07:00
Mick
4fa44d63c6 chore: improve mmmu benchmark (#7000) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-07-26 16:19:45 +08:00
kyleliang-nv
e6312d271d Uodate Dockerfile.gb200 to latest sglang (#8356) 2025-07-26 00:22:06 -07:00
Ke Bao
8af145b7dc Fix test_moe_fused_gate_combined sgl-kernel ci test (#8374) 2025-07-26 09:30:12 +08:00
Yineng Zhang
2272c2a5b5 chore: bump v0.4.9.post4 (#8305) 2025-07-25 17:12:47 -07:00
Lianmin Zheng
3ec0b21229 [CI] Fix flaky threshold (#8370) 2025-07-25 16:41:56 -07:00
Trevor Morris
58c468f404 Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333) 2025-07-25 16:40:23 -07:00