EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,824 Commits 8 Branches 0 Tags
112b496a6cf8c26ce63b03c01f5dfb887c295f3b
Commit Graph

2594 Commits

Author SHA1 Message Date
Chang Su
112b496a6c misc: Improvement to serving_chat.py and add more ut (#7489) 2025-06-24 17:19:51 -07:00
Chang Su
3562256bb2 fix: Add --model as an alias for --model-path in server_args (#7505) 2025-06-24 12:08:08 -07:00
Trevor Morris
5f527834a8 [PD] NIXL: Register kv args in advance and cleanup finished requests (#6717) 2025-06-24 11:26:09 -07:00
xianzhiT
9f1787fa60 Support multi-thread model weight loading (#7277) 2025-06-24 10:39:10 -07:00
Ke Bao
7151194bb2 Remove cumsum_buffer initilization (#7439) 2025-06-24 02:22:16 -07:00
Francis
2ed68d7a6c [PD Disaggregation] replace transfer with batch transfer for better performance (#7236) 2025-06-24 02:12:04 -07:00
valarLip
e984d5073b enable aiter_biased_grouped_topk kernel (#7423) 2025-06-24 02:09:42 -07:00
Alex Sun
755f314785 [AMD] add aiter fused moe in DeepEP path (#7268) 2025-06-24 09:05:47 +00:00
Yineng Zhang
7c3a12c000 chore: bump v0.4.8 (#7493) 2025-06-23 23:14:22 -07:00
Charles Chen
15f3401343 Fix MTP with Deepseek R1 Fp4 (#7376) 2025-06-23 21:38:07 -07:00
Chang Su
d04163b3fa Fix RequestValidationError response format (#7487) 2025-06-23 20:35:11 -07:00
huangtingwei
7732bbe458 bugfix: Prevent global mutation of conv.stop_str across requests (#7347) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-06-23 19:36:23 -07:00
u4lr451
ed0a0b692c Perormance: Enable cuda graph for dp idle batch (#7269) 
Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>
 2025-06-23 17:34:13 -07:00
Yuhong Guo
e5afb88b1c Support weight loading without mmap (#7469) 2025-06-23 15:13:59 -07:00
Charles Chen
e5ddeb04d5 Quick fix for DeepGemm requant to also cover MTP. (#7378) 2025-06-23 12:08:54 -07:00
JieXin Liang
bdbb8d009a [perf] slightly imporve DeepSeek-R1-FP4 TP8 (#7481) 2025-06-23 12:05:30 -07:00
Cheng Wan
f8d48fd311 Fix dtype for idle input in spec decoding (#7456) 2025-06-23 11:23:25 -07:00
Chang Su
34b6b8426f feat(func_call): Add more check in BaseFormatDetector.parse_streaming_increment (#7479) 2025-06-23 11:15:47 -07:00
Liangsheng Yin
25549433e8 Fix prefill OOM due to wrong token calculation when page > 1 (#7397) 2025-06-24 02:12:29 +08:00
Lianmin Zheng
55e03b10c4 Fix a bug in BatchTokenIDOut & Misc style and dependency updates (#7457) 2025-06-23 06:20:39 -07:00
xutizhou
506c4928f5 feat: integrate deepgemm into EPMoE (#6821) 
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-06-23 01:38:58 -07:00
Cheng Wan
ac5010e0ba Fix CUDA Graph Check under Deepep with DP FFN (#7451) 2025-06-22 20:35:58 -07:00
Yi Zhang
3cee035e99 add fused moe config for qwen3 in triton3.3.1 (#7445) 2025-06-22 18:29:25 -07:00
kk
bd4f581896 Fix torch compile run (#7391) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
 2025-06-22 15:33:09 -07:00
Ke Bao
50f1b6d6b1 Remove copy after bmm (#7441) 2025-06-22 15:07:44 -07:00
Trevor Morris
5962e70d8d FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327) 
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>
 2025-06-22 13:38:47 -07:00
fzyzcjy
edc21cc8ae Tiny add logging for GC (#7406) 2025-06-22 12:40:02 +08:00
Liangsheng Yin
05c9bc8956 [minor] simplify the TokenToKVPoolAllocator (#7414) 2025-06-22 12:37:18 +08:00
Lifu Huang
1998ce4046 Refactor LoRAManager and LoRAMemoryPool state management logic for dynamic LoRA loading support (#7412) 2025-06-21 16:09:19 -07:00
Chang Su
72676cd6c0 feat(oai refactor): Replace openai_api with entrypoints/openai (#7351) 
Co-authored-by: Jin Pan <jpan236@wisc.edu>
 2025-06-21 13:21:06 -07:00
Atream
02bf31ef29 [fix] PD disaggregation when enable mtp and tp!=dp (#7420) 2025-06-21 12:03:11 -07:00
Liangsheng Yin
5ea5d22170 Fix CPU offloading for MLA memory pool (#7409) 2025-06-22 02:39:05 +08:00
Chang Su
fdfd5224bf fix: Fix CI test_function_call_parser.py (#7425) 2025-06-22 00:25:08 +08:00
pansicheng
7f3ee861ae fix overlap pagecount (#6984) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-06-21 15:34:45 +08:00
ehuaa
bec5891083 [BugFix]fix qwen25 invoke function call streaming responses with curly braces as the starting indicator (#7394) 2025-06-20 22:50:08 -07:00
woodx
9edf6608c9 [BugFix]: fix EmbeddingReqInput single input error (#7396) 2025-06-20 20:35:21 -07:00
Keyang Ru
5e7fdc79fa [OAI Server Refactor] [ChatCompletions & Completions] Support Return Hidden State (#7329) 
Signed-off-by: keru <rukeyang@gmail.com>
 2025-06-20 19:18:53 -07:00
Cheng Wan
5041df2d01 Fix 7285 Merge Conflicts (#7403) 2025-06-20 16:02:50 -07:00
Cheng Wan
256801e973 Update usage_processor.py (#7402) 2025-06-20 15:55:38 -07:00
Cheng Wan
73b13e69b4 Optimize DP attn scheduling for speculative decoding (#7285) 2025-06-20 15:06:41 -07:00
Cheng Wan
8609e637a9 Fix All-Gather under world size one (#7219) 2025-06-20 14:57:34 -07:00
yhyang201
dea2b84bc3 [OAI Server Refactor] [ChatCompletions & Completions] Implement UsageInfo Processor (#7360) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-06-20 14:51:21 -07:00
woodx
cfb2fb5afc [OAI refactor] Add rerank and score serving (#7399) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-06-20 14:51:10 -07:00
Cheng Wan
22bfed7509 [DeepSeekNextN] fix: residual of head norm can be None (#7398) 2025-06-20 14:45:16 -07:00
Cheng Wan
e879d8b7a8 [Feature] Comprehensive Hybrid Parallelism Support (#6389) 2025-06-20 14:43:11 -07:00
Xinyuan Tong
0998808009 Refine OpenAI serving entrypoint to remove batch requests (#7372) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
 2025-06-20 14:33:43 -07:00
Wenbo Yang
794be55af2 Support THUDM/GLM-4-0414 (GLM-Z1) Glm4ForCausalLM architecture. (#5485) 2025-06-20 10:07:43 -07:00
Shangming Cai
187b85b7f3 [PD] Optimize custom mem pool usage and bump mooncake version (#7393) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-06-20 09:50:39 -07:00
strgrb
ceba0ce4f6 support return logprobs for pipeline (#7356) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-06-19 23:50:45 -07:00
Ata Fatahi
1ab6be1b26 Purge VerlEngine (#7326) 
Signed-off-by: Ata Fatahi <immrata@gmail.com>
 2025-06-19 23:47:21 -07:00