EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,621 Commits 8 Branches 0 Tags
5dddb331c48039c2649ccdba3a81a7fbe43a7fe5
Commit Graph

3768 Commits

Author SHA1 Message Date
Lianmin Zheng
5dddb331c4 [Auto Sync] Update base_grammar_backend.py, xgrammar_backen... (20250930) (#11115) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-09-30 21:50:43 -07:00
Liangsheng Yin
73d4a5f879 Organize spec-related data structures (#10735) 2025-10-01 09:45:30 +08:00
Xinyuan Tong
1193f13181 fix: KimiK2Detector Improve tool call ID parsing with regex (#10972) 2025-09-30 17:44:44 -07:00
Chenxi Li
84a9f5d660 Feature/make PEFT adapter module format compatibile (#11080) 2025-09-30 16:32:02 -07:00
Chang Su
8ce830a8b0 [router][bugfix] Fix input_logprobs handling with None value and logprob_start_len = -1 (#11113) 2025-09-30 16:09:40 -07:00
qrskannbara
fb367acfcb Support Dots.ocr model (#11071) 2025-09-30 12:18:39 -07:00
Trevor Morris
a6cc86df9d Fix DSR1 accuracy for flashinfer_trtllm MoE with FP8 quantization (#11081) 2025-09-30 10:33:12 -07:00
Zaili Wang
229d2b95f1 [CPU] Adding Memory Capacity Acquisition Functionality (#11102) 2025-09-30 08:41:20 -07:00
Liangsheng Yin
9710f718fb [Eval] Add --repeat in run_eval (#11101) 2025-09-30 23:35:54 +08:00
Ke Bao
91847e382a Fix eagle radix cache (#10846) 2025-09-30 22:59:20 +08:00
Simo Lin
5a290a5644 [router][grpc-server] Fix gRPC server shutdown (#11094) 2025-09-30 07:12:12 -04:00
Lianmin Zheng
1237aa19ce [Auto Sync] Update fused_moe_triton_config.py (20250930) (#11099) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-30 00:57:16 -07:00
jacky.cheng
5991119541 [Fix] Resolve performance drop in speculative decoding aiter backend (#11087) 2025-09-29 23:51:30 -07:00
Ke Bao
424591d53d Fix spec filter batch when target extend (#10991) 2025-09-30 14:44:02 +08:00
narutolhy
d17986f8c6 Enable optional FP32 compute for LM Head (#10729) 
Thanks to MiniMax Team and Chenyang Zhao's support.
 2025-09-29 20:45:17 -07:00
DevashishLal-CB
8831c55c3d [model] added support for w8a8int8 used by neuralmagic/Qwen2-0.5B-Ins… (#9642) 2025-09-30 11:26:17 +08:00
li-kesen
2bc61dd194 Remove hybrid_linear_attn attention backend and refactor attention registry (#10816) 
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2025-09-30 10:16:16 +08:00
Cheng Wan
6535fda127 [Profile] dump memory trace when cuda graph profile is enabled (#11083) 2025-09-29 17:36:48 -07:00
Jimmy
3713eb6135 feat(reasoning): improve enable thinking from request (#10875) 2025-09-29 16:59:08 -07:00
Chang Su
5937a56d47 [router][grpc] Add logprobs support to router (#11082) 2025-09-29 15:55:06 -07:00
Mick
9de1320b63 fix: fp8 mllama4 without vision modules being quantized (#10611) 2025-09-29 21:17:12 +00:00
Lianmin Zheng
dda34c2f93 Fix mem fraction static for nightly tests (#11076) 2025-09-29 12:57:41 -07:00
Lianmin Zheng
a17e70f5cc Use more general heuristics to set the default value of --mem-fraction-static (#10975) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-29 10:11:03 -07:00
Yineng Zhang
3a641d9085 chore: upgrade sgl-kernel 0.3.13 (#11056) 2025-09-29 02:22:25 -07:00
Yineng Zhang
5942fdb480 chore: upgrade cutedsl 4.2.1 (#11054) 2025-09-29 00:24:17 -07:00
Xiaoyu Zhang
11965b0daf Fix sgl-kernel benchmark dead code (#11022) 2025-09-29 15:06:40 +08:00
Vincent Zhong
71959545df Fix gemma 3 launch with transformers: the error: AttributeError: 'TransformersForCausalLM' object has no attribute 'tp_size' (#9614) 2025-09-29 06:18:29 +00:00
Zhihao Zhang
24f7cb1ece [speculative decoding] rename lookahead to ngram (#11010) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-09-28 21:06:59 -07:00
huangtingwei
e05555fad8 [HiCacheStorage] mooncake store support page_first_direct layout (#10591) 2025-09-28 20:45:48 -07:00
Mick
43fa9f22bd fix: check if weights are already local before downloading (#11015) 2025-09-28 20:11:33 -07:00
Mick
2e7633982c fix: show failed models in nightly ci (#10986) 2025-09-28 12:38:29 -07:00
Yuxuan Zhang
abb6781573 Update GLM-4.5 Model Doc (#11017) 2025-09-28 11:21:27 -07:00
Lianmin Zheng
07440f5f34 Fix FusedSetKVBufferArg in RotaryEmbedding (#11003) 2025-09-28 11:17:27 -07:00
Teng Ma
9816989bff [HiCache] bug: fix mooncake store batch set v1 (#11013) 2025-09-28 23:18:48 +08:00
lbk-sys
2a9d995c09 prepare for sglang+verl (#10555) 
Signed-off-by: lbk-sys <hello_lbk@163.com>
Signed-off-by: libaokui <libaokui@hauwei.com>
Co-authored-by: libaokui <libaokui@hauwei.com>
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-09-28 01:39:17 -07:00
wejoncy
a9050b5c39 [bugfix]Add empty_context import to two_batch_overlap.py (#10964) 2025-09-28 01:26:20 -07:00
YanbingJiang
66face3598 Fix CI failure of TypeError: RotaryEmbedding.forward_cpu() got an unexpected keyword argument 'fused_set_kv_buffer_arg' (#11009) 2025-09-28 00:31:08 -07:00
Simo Lin
5519766a4d [router] fix chat template loading and tokenizer path (#10999) 2025-09-27 23:54:12 -04:00
Lianmin Zheng
f6bc3f529b Fix profiler (#10997) 2025-09-27 14:56:18 -07:00
Tejesh Anand
8cc27fdc46 Use jsonschema to constrain required or specific tool choice (#10550) 2025-09-27 13:18:50 -04:00
Shangming Cai
9c339d6b47 [PD] Extract the PP transfer layer calculate logic from Mooncake to Common backend (#10565) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:10:41 +08:00
Shangming Cai
e23e280e16 Add support for topk metadata transferring for PD (#10616) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:09:38 +08:00
Muqi Li
51f7c6bd3c Add auth to get server info (#10751) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-27 02:54:39 -07:00
Xinyuan Tong
62e2e99db6 fix: make inference deterministic for large TP (#10930) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-27 02:46:45 -07:00
kk
8ebf72fef3 [Fix] RuntimeError: get_cfg Unsupported input_type:Float4_e2m1fn_x2 in using aiter-mxfp4-moe (#10981) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-09-26 22:12:22 -07:00
Yueyang Pan
8260574729 fix: fp8 quantization failure of qwen 2.5 VL 7B model (#10112) 
Signed-off-by: PanJason <pyyjason@gmail.com>
 2025-09-27 05:05:23 +00:00
Chang Su
37f3325b06 [router][grpc] Support E2E non-stream chat completions (#10980) 2025-09-26 22:02:06 -07:00
Muqi Li
bd95944cf6 [Bugfix][Minor][Benchmark] Fix some bugs due to PR #10495 (#10982) 2025-09-26 22:01:05 -07:00
hzh0425
c8a5d12abe [HiCache]: Support dynamic loading backends for hicache (#10551) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-26 18:34:11 -07:00
Chang Su
0c3db88978 [router][grpc] Add helpfer functions for decoder in router.rs and fix specs (#10971) 2025-09-26 20:10:45 -04:00