EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,241 Commits 8 Branches 0 Tags
d18c6b3358185cee49db931727c9f99018da7f5d
Commit Graph

2156 Commits

Author SHA1 Message Date
Lianmin Zheng
d18c6b3358 Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 14:33:38 -07:00
shangmingc
f1c896007a [PD] Add support for different TP sizes per DP rank (#5922) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-12 13:55:42 -07:00
Yineng Zhang
f94543d22b chore: add hf_xet dep (#6243) 2025-05-12 13:08:40 -07:00
Lianmin Zheng
e8e18dcdcc Revert "fix some typos" (#6244) 2025-05-12 12:53:26 -07:00
Ying Sheng
bad7c26fdc [PP] Fix init_memory_pool desync & add PP for mixtral (#6223) 2025-05-12 12:38:09 -07:00
applesaucethebun
d738ab52f8 fix some typos (#6209) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-13 01:42:38 +08:00
Lianmin Zheng
fba8eccd7e Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 00:17:33 -07:00
Cheng Wan
25c83fff6a Performing Vocabulary Parallelism for LM Head across Attention TP Groups (#5558) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
 2025-05-11 23:36:29 -07:00
Xiaoyu Zhang
9f2c9568f0 [doc] add a note for --n-share-experts-fusion args (#6154) 2025-05-11 23:18:38 -07:00
fzyzcjy
3f2702ae51 Fix start_profile does not support with_stack and record_shapes (#6043) 2025-05-11 23:11:32 -07:00
Lifu Huang
6e2da51561 Replace time.time() to time.perf_counter() for benchmarking. (#6178) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-11 14:32:49 -07:00
Lianmin Zheng
03227c5fa6 [CI] Reorganize the 8 gpu tests (#6192) 2025-05-11 10:55:06 -07:00
Lianmin Zheng
01bdbf7f80 Improve structured outputs: fix race condition, server crash, metrics and style (#6188) 2025-05-11 08:36:16 -07:00
Ravi Theja
41a645f556 Handle empty input string for embedding models (#5621) 
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
 2025-05-11 23:17:15 +08:00
Yineng Zhang
230106304d chore: upgrade sgl-kernel v0.1.2.post1 (#6196) 
Co-authored-by: alcanderian <alcanderian@gmail.com>
 2025-05-11 22:41:37 +08:00
Yusong Gao
41273fd71f fix: handle None multimodal_inputs during merging and filtering batches in disaggregation decode mode (#6169) 2025-05-11 00:28:21 -07:00
JieXin Liang
e9bebafb19 [fix] remove mixtral from is_fa3_default_architecture (#6191) 2025-05-11 00:15:54 -07:00
Yineng Zhang
4d1c9db66c feat: support loogle eval (#6190) 2025-05-10 23:52:44 -07:00
shangmingc
31d1f6e7f4 [PD] Add simple unit test for disaggregation feature (#5654) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-11 13:35:27 +08:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Lianmin Zheng
de167cf5fa Fix request abortion (#6184) 2025-05-10 21:54:46 -07:00
Lianmin Zheng
4319978c73 Fix data parallel perf regression (#6183) 2025-05-10 19:18:35 -07:00
Steven Shimizu
03dd785cd0 Added async_encode method to Engine (#4701) 2025-05-10 18:58:40 -07:00
Yineng Zhang
66fc63d6b1 Revert "feat: add thinking_budget (#6089)" (#6181) 2025-05-10 16:07:45 -07:00
XinyuanTong
9d8ec2e67e Fix and Clean up chat-template requirement for VLM (#6114) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-11 00:14:09 +08:00
JieXin Liang
c178abdabc [fix] fix determine_n_share_experts_fusion (#6118) 2025-05-10 01:19:09 -07:00
Simon (Jiyou) Li
b29a026e14 KV‑Cache (MHA, MLA): add missing start_layer / end_layer fields to MHATokenToKVPoolHost and MLATokenToKVPoolHost (#6016) 
Co-authored-by: 继优 <jiyou.ljy@alibaba-inc.com>
Co-authored-by: chus-chus <chus-chus@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-05-09 15:50:06 -07:00
Yineng Zhang
678d8cc987 chore: bump v0.4.6.post3 (#6165) 2025-05-09 15:38:47 -07:00
huangtingwei
d2cb3024f2 fix bug that gpu0 occupies more memory when hicache is turned on (#5778) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-05-09 15:36:08 -07:00
Chang Su
1940cdec61 [Bugfix] Fix Llama4 gibberish output with long context and CUDA graph (#6162) 2025-05-09 15:33:02 -07:00
thyecust
63484f9fd6 feat: add thinking_budget (#6089) 2025-05-09 08:22:09 -07:00
xu-yfei
e30c273bc9 opt flashinfer mla cat (#5822) 
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
 2025-05-08 23:17:14 -07:00
yhyang201
cec98f1034 [Fix] Incorrect Memory Allocation on CUDA:0 by Non-Zero CUDA Processes in TP/DP (#5745) 2025-05-08 17:52:26 -07:00
Yixin Dong
911f3ba6f4 upgrade xgrammar to 0.1.19 (#6129) 2025-05-08 14:42:02 -07:00
Hubert Lu
2a936a841e [AMD] switch to custom allreduce regardless of MSCCL setting on ROCm (#6097) 2025-05-08 13:46:58 -07:00
JieXin Liang
5e02330137 [perf] dsv3 bmm fallback to bf16 (#5662) 2025-05-08 11:43:39 -07:00
Zhu Chen
fa7d7fd9e5 [Feature] Add FlashAttention3 as a backend for VisionAttention (#5764) 
Co-authored-by: othame <chenzhu_912@zju.edu.cn>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2025-05-08 10:01:19 -07:00
JieXin Liang
f1ff736d68 [fix] fix pyproject.toml dependencies (#6119) 2025-05-08 02:14:36 -07:00
lukec
acc816d8a2 DeepEP normal support deepgemm-contiguous (#5626) 
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
Co-authored-by: ZhengHSI <zhenghsi@qq.com>
 2025-05-08 01:20:32 -07:00
fzyzcjy
cef91b1ed7 [PD] Add control to slow down a server (#5572) 2025-05-08 01:03:08 -07:00
fzyzcjy
6450c1228c Tiny refactor weight loading logic (#5232) 2025-05-08 01:02:56 -07:00
fzyzcjy
b6cf3532b5 Tiny refactor ModelConfig.from_server_args (#5219) 2025-05-08 01:02:43 -07:00
fzyzcjy
3b2680a44d Overlap shared expert and routed expert computations (#5121) 2025-05-08 01:02:32 -07:00
Minglei Zhu
79961afa82 optimize pad operations in fa3 to accelarate 100+us (#6077) 2025-05-07 23:40:08 -07:00
Baizhou Zhang
cfca4e0ed2 adding Triton configs for DeepSeekV3 FusedMoE kernel on Blackwell (#6111) 2025-05-07 23:39:10 -07:00
XinyuanTong
e88dd482ed [CI]Add performance CI for VLM (#6038) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-07 19:20:03 -07:00
Baizhou Zhang
73600673bb Clean logs for DeepSeek-V3 launching (#6079) 2025-05-07 18:54:50 -07:00
JieXin Liang
b70957fcf8 [refactor] slightly tidy fp8 module (#5993) 2025-05-07 17:28:24 -07:00
ishandhanani
e444c13fb4 feat(engine): add bootstrap parameters to generate methods (dynamo) (#6075) 2025-05-07 10:33:58 -07:00
fzyzcjy
c68de47915 Super tiny fix doc (#5233) 2025-05-07 22:41:50 +08:00