EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,650 Commits 8 Branches 0 Tags
b65db0287b55f4fbeaf318f03e4ab76ff6a2e7d8
Commit Graph

3792 Commits

Author SHA1 Message Date
fzyzcjy
b65db0287b Tiny cleanup deepseek_v2.py (#11163) 2025-10-02 21:54:52 +08:00
b8zhong
948278f173 fix cpp JIT compilation issue of ngram speculative decoding (#10837) 2025-10-02 21:05:01 +08:00
Liangsheng Yin
7d00479950 Clean up ascend allocator (#11152) 2025-10-02 20:34:26 +08:00
ilyasch2
083629c235 [model] Add mamba2 and Falcon-H1 support. (#10988) 
Co-authored-by: Younes Belkada <younes.belkada@tii.ae>
Co-authored-by: Younes B <49240599+younesbelkada@users.noreply.github.com>
 2025-10-02 19:15:36 +08:00
fzyzcjy
5e786cca3a Support single batch overlap (#10422) 2025-10-02 18:04:36 +08:00
fzyzcjy
0b9dfba787 Support dispatch low latency (#10263) 
Co-authored-by: Kaixi Hou <4001424+kaixih@users.noreply.github.com>
 2025-10-02 18:02:19 +08:00
Liangsheng Yin
6a29003410 Remove unused pack .item() in paged allocator. (#11156) 2025-10-02 18:01:21 +08:00
fzyzcjy
2ac453b07f Tiny detect slow ranks (#10508) 2025-10-02 18:00:33 +08:00
fzyzcjy
f35def8652 Fuse quantize and rope in trtllm_mla MTP (#10779) 2025-10-02 17:59:37 +08:00
fzyzcjy
d61615fe93 Tiny fix missing alt stream in nextn layer (#10768) 2025-10-02 17:58:23 +08:00
fzyzcjy
b1ccaf01cd Tiny improve dumper (#11132) 2025-10-02 17:55:01 +08:00
Lianmin Zheng
097725bb66 Clean up parallel_state.py (#11148) 2025-10-02 01:09:13 -07:00
fzyzcjy
44b1fbe258 Fix DeepSeek chunked prefill memory issue (#11149) 2025-10-01 23:56:59 -07:00
sogalin
c0dbbdd12b [ROCm] To reduce the compiling time when using torch compile. (#10559) 2025-10-01 23:53:14 -07:00
Liangsheng Yin
25e7dbe8af Fix ngram spec with page size > 1 (#11135) 2025-10-02 12:34:23 +08:00
Zhang Junda
0b2aa8a70c Intoduce cpu tensor as metadata to avoid blocking gpu kernel launch (#10720) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2025-10-02 10:51:25 +08:00
Lianmin Zheng
609f65ba23 Remove debug print statement from scheduler output (#11145) 2025-10-01 13:37:05 -07:00
Lianmin Zheng
2d62af6be5 Fix metrics and request tracing (TimeStats) (#11123) 2025-10-01 13:03:07 -07:00
Simo Lin
96fe2d0f15 [router] add pd service in grpc router for pd (#11120) 2025-10-01 08:09:21 -07:00
ykwd
bfa274380b [HiCache] Configurable and Dynamic Prefetch Timeout (#10512) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-01 08:44:10 -05:00
Shisong Ma
86cb4db058 [Feature] Add EIC as sglang HiCache Storage backend (#10271) 
Co-authored-by: mashisong <mashisong@bytedance.com>
 2025-10-01 08:43:34 -05:00
zhangzuo21
2e130b7618 [HiCache]bug fix: fixed blank item in host_mem_release_queue (#11005) 
Co-authored-by: Zuoyuan Zhang <zuoyuanzhang@ZuoyuandeMacBook-Pro.local>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-01 08:42:37 -05:00
eigen
ac1f2928ae feat: add fast_decode_plan from flashinfer, flashinfer to 0.4.0rc3 (#10760) 
Co-authored-by: Zihao Ye <yezihhhao@gmail.com>
Co-authored-by: Sleepcoo <Sleepcoo@gmail.com>
 2025-10-01 02:56:13 -07:00
yhyang201
6130529143 Quick Fix: fix Qwen3-VL launch failure caused by MRotaryEmbedding arg (#10985) 2025-09-30 22:17:05 -07:00
Lianmin Zheng
5dddb331c4 [Auto Sync] Update base_grammar_backend.py, xgrammar_backen... (20250930) (#11115) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-09-30 21:50:43 -07:00
Liangsheng Yin
73d4a5f879 Organize spec-related data structures (#10735) 2025-10-01 09:45:30 +08:00
Xinyuan Tong
1193f13181 fix: KimiK2Detector Improve tool call ID parsing with regex (#10972) 2025-09-30 17:44:44 -07:00
Chenxi Li
84a9f5d660 Feature/make PEFT adapter module format compatibile (#11080) 2025-09-30 16:32:02 -07:00
Chang Su
8ce830a8b0 [router][bugfix] Fix input_logprobs handling with None value and logprob_start_len = -1 (#11113) 2025-09-30 16:09:40 -07:00
qrskannbara
fb367acfcb Support Dots.ocr model (#11071) 2025-09-30 12:18:39 -07:00
Trevor Morris
a6cc86df9d Fix DSR1 accuracy for flashinfer_trtllm MoE with FP8 quantization (#11081) 2025-09-30 10:33:12 -07:00
Zaili Wang
229d2b95f1 [CPU] Adding Memory Capacity Acquisition Functionality (#11102) 2025-09-30 08:41:20 -07:00
Liangsheng Yin
9710f718fb [Eval] Add --repeat in run_eval (#11101) 2025-09-30 23:35:54 +08:00
Ke Bao
91847e382a Fix eagle radix cache (#10846) 2025-09-30 22:59:20 +08:00
Simo Lin
5a290a5644 [router][grpc-server] Fix gRPC server shutdown (#11094) 2025-09-30 07:12:12 -04:00
Lianmin Zheng
1237aa19ce [Auto Sync] Update fused_moe_triton_config.py (20250930) (#11099) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-30 00:57:16 -07:00
jacky.cheng
5991119541 [Fix] Resolve performance drop in speculative decoding aiter backend (#11087) 2025-09-29 23:51:30 -07:00
Ke Bao
424591d53d Fix spec filter batch when target extend (#10991) 2025-09-30 14:44:02 +08:00
narutolhy
d17986f8c6 Enable optional FP32 compute for LM Head (#10729) 
Thanks to MiniMax Team and Chenyang Zhao's support.
 2025-09-29 20:45:17 -07:00
DevashishLal-CB
8831c55c3d [model] added support for w8a8int8 used by neuralmagic/Qwen2-0.5B-Ins… (#9642) 2025-09-30 11:26:17 +08:00
li-kesen
2bc61dd194 Remove hybrid_linear_attn attention backend and refactor attention registry (#10816) 
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2025-09-30 10:16:16 +08:00
Cheng Wan
6535fda127 [Profile] dump memory trace when cuda graph profile is enabled (#11083) 2025-09-29 17:36:48 -07:00
Jimmy
3713eb6135 feat(reasoning): improve enable thinking from request (#10875) 2025-09-29 16:59:08 -07:00
Chang Su
5937a56d47 [router][grpc] Add logprobs support to router (#11082) 2025-09-29 15:55:06 -07:00
Mick
9de1320b63 fix: fp8 mllama4 without vision modules being quantized (#10611) 2025-09-29 21:17:12 +00:00
Lianmin Zheng
dda34c2f93 Fix mem fraction static for nightly tests (#11076) 2025-09-29 12:57:41 -07:00
Lianmin Zheng
a17e70f5cc Use more general heuristics to set the default value of --mem-fraction-static (#10975) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-29 10:11:03 -07:00
Yineng Zhang
3a641d9085 chore: upgrade sgl-kernel 0.3.13 (#11056) 2025-09-29 02:22:25 -07:00
Yineng Zhang
5942fdb480 chore: upgrade cutedsl 4.2.1 (#11054) 2025-09-29 00:24:17 -07:00
Xiaoyu Zhang
11965b0daf Fix sgl-kernel benchmark dead code (#11022) 2025-09-29 15:06:40 +08:00