EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,201 Commits 8 Branches 0 Tags
c178abdabc563564664c935613a99c368f2a7d50
Commit Graph

2131 Commits

Author SHA1 Message Date
JieXin Liang
c178abdabc [fix] fix determine_n_share_experts_fusion (#6118) 2025-05-10 01:19:09 -07:00
Simon (Jiyou) Li
b29a026e14 KV‑Cache (MHA, MLA): add missing start_layer / end_layer fields to MHATokenToKVPoolHost and MLATokenToKVPoolHost (#6016) 
Co-authored-by: 继优 <jiyou.ljy@alibaba-inc.com>
Co-authored-by: chus-chus <chus-chus@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-05-09 15:50:06 -07:00
Yineng Zhang
678d8cc987 chore: bump v0.4.6.post3 (#6165) 2025-05-09 15:38:47 -07:00
huangtingwei
d2cb3024f2 fix bug that gpu0 occupies more memory when hicache is turned on (#5778) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-05-09 15:36:08 -07:00
Chang Su
1940cdec61 [Bugfix] Fix Llama4 gibberish output with long context and CUDA graph (#6162) 2025-05-09 15:33:02 -07:00
thyecust
63484f9fd6 feat: add thinking_budget (#6089) 2025-05-09 08:22:09 -07:00
xu-yfei
e30c273bc9 opt flashinfer mla cat (#5822) 
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
 2025-05-08 23:17:14 -07:00
yhyang201
cec98f1034 [Fix] Incorrect Memory Allocation on CUDA:0 by Non-Zero CUDA Processes in TP/DP (#5745) 2025-05-08 17:52:26 -07:00
Yixin Dong
911f3ba6f4 upgrade xgrammar to 0.1.19 (#6129) 2025-05-08 14:42:02 -07:00
Hubert Lu
2a936a841e [AMD] switch to custom allreduce regardless of MSCCL setting on ROCm (#6097) 2025-05-08 13:46:58 -07:00
JieXin Liang
5e02330137 [perf] dsv3 bmm fallback to bf16 (#5662) 2025-05-08 11:43:39 -07:00
Zhu Chen
fa7d7fd9e5 [Feature] Add FlashAttention3 as a backend for VisionAttention (#5764) 
Co-authored-by: othame <chenzhu_912@zju.edu.cn>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2025-05-08 10:01:19 -07:00
JieXin Liang
f1ff736d68 [fix] fix pyproject.toml dependencies (#6119) 2025-05-08 02:14:36 -07:00
lukec
acc816d8a2 DeepEP normal support deepgemm-contiguous (#5626) 
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
Co-authored-by: ZhengHSI <zhenghsi@qq.com>
 2025-05-08 01:20:32 -07:00
fzyzcjy
cef91b1ed7 [PD] Add control to slow down a server (#5572) 2025-05-08 01:03:08 -07:00
fzyzcjy
6450c1228c Tiny refactor weight loading logic (#5232) 2025-05-08 01:02:56 -07:00
fzyzcjy
b6cf3532b5 Tiny refactor ModelConfig.from_server_args (#5219) 2025-05-08 01:02:43 -07:00
fzyzcjy
3b2680a44d Overlap shared expert and routed expert computations (#5121) 2025-05-08 01:02:32 -07:00
Minglei Zhu
79961afa82 optimize pad operations in fa3 to accelarate 100+us (#6077) 2025-05-07 23:40:08 -07:00
Baizhou Zhang
cfca4e0ed2 adding Triton configs for DeepSeekV3 FusedMoE kernel on Blackwell (#6111) 2025-05-07 23:39:10 -07:00
XinyuanTong
e88dd482ed [CI]Add performance CI for VLM (#6038) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-07 19:20:03 -07:00
Baizhou Zhang
73600673bb Clean logs for DeepSeek-V3 launching (#6079) 2025-05-07 18:54:50 -07:00
JieXin Liang
b70957fcf8 [refactor] slightly tidy fp8 module (#5993) 2025-05-07 17:28:24 -07:00
ishandhanani
e444c13fb4 feat(engine): add bootstrap parameters to generate methods (dynamo) (#6075) 2025-05-07 10:33:58 -07:00
fzyzcjy
c68de47915 Super tiny fix doc (#5233) 2025-05-07 22:41:50 +08:00
fzyzcjy
4c7b42424c Hint users DeepEP normal mode is incompatible with CUDA Graph (#5014) 2025-05-07 22:40:59 +08:00
Song Zhang
00c2c1f08b [Feature] Support for Ascend NPU backend (#3853) 
Signed-off-by: Song Zhang <gepin.zs@antgroup.com>
Co-authored-by: 22dimensions <waitingwind@foxmail.com>
 2025-05-06 20:32:53 -07:00
Jinyan Chen
8a828666a3 Add DeepEP to CI PR Test (#5655) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-05-06 17:36:03 -07:00
Liangsheng Yin
a3e4e9bf9e Better PD initialization (#5751) 2025-05-07 01:12:57 +08:00
Liangsheng Yin
6d4d3bc81d Fix not "import os" (#6057) 2025-05-06 22:06:41 +08:00
Zhiqiang Xie
b26cb1c55a Fix problem of large page size with chunked prefill (#6046) 2025-05-06 15:19:47 +08:00
Zhiqiang Xie
f8e460930a Fix prefill OOM error in the case of large page size (#5081) 2025-05-05 16:02:55 -07:00
Adarsh Shirawalmath
683707c314 [Security][Bug] Prevent binding to all TCP interfaces (#5752) 2025-05-06 03:21:45 +08:00
mlmz
a68ed76682 feat: append more comprehensive fields in messages instead of merely role and content (#5996) 2025-05-05 11:43:34 -07:00
DefTruth
82653f6622 feat: Add a unified merge_state API (#5428) 2025-05-05 10:32:33 -07:00
Wenxuan Tan
22da3d978f Fix "Avoid computing lse in Ragged Prefill when there's no prefix match" (#5555) 2025-05-05 10:32:17 -07:00
shangmingc
56f6589ecb [PD] Optimize disaggregation ib device help info (#5781) 2025-05-05 13:47:37 +08:00
fzyzcjy
3008db9c1a [PD] Allow customizing reserved tokens to avoid KV cache waste (#6002) 2025-05-05 11:23:15 +08:00
Junrong Lin
357fb2dba5 fix: fix broadcast_pyobj breaking VerlEngine (#5997) 2025-05-04 13:15:53 -07:00
Qiaolin Yu
3042f1da61 Fix flaky issues of lora and add multi batch tests (#5957) 2025-05-04 13:11:40 -07:00
Lifu Huang
2b63798c7d [Minor] Fix duplicate method definitions in conversation.py (#6012) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-04 13:02:53 -07:00
Baizhou Zhang
bf203cb7a2 [Fix] Suppress dynamo logging when using flashinfer backend with torch compile (#5992) 2025-05-04 09:49:13 -07:00
JieXin Liang
8ebde73f7d [perf] H100 DeepSeek-V3 fused moe tuned config (#5998) 2025-05-03 14:02:26 -07:00
Stefan He
6b0fae797a Fix Phi3 serving which was broke by earlier change (#5991) 
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-03 00:28:47 -07:00
Ke Bao
d8ab60117f Overlap qk norm with two streams (#5977) 2025-05-02 09:26:30 -07:00
Ke Bao
6579cd7daf Fix set kv cache multi-stream (#5975) 2025-05-02 09:26:00 -07:00
Yongtong Wu
97ac42b634 [PD] NIXL backend Prefill TP & Decode TP+DP (#5681) 2025-05-02 22:14:03 +08:00
Lifu Huang
1acca3a2c6 FA3 speed up: skip len operation and get batch size directly from forward batch (#5969) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-02 00:26:12 -07:00
xm:D
3409aaab32 Support InternVL3 (#5350) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-05-01 22:38:59 -07:00
Chang Su
170d1f218a feat: Refactor DeepSeekV3 function call (#5908) 2025-05-01 21:28:57 -07:00