EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,598 Commits 8 Branches 0 Tags
6535fda1279d300bce88e6d18190ef0d8d51f0cd
Commit Graph

3751 Commits

Author SHA1 Message Date
Cheng Wan
6535fda127 [Profile] dump memory trace when cuda graph profile is enabled (#11083) 2025-09-29 17:36:48 -07:00
Jimmy
3713eb6135 feat(reasoning): improve enable thinking from request (#10875) 2025-09-29 16:59:08 -07:00
Chang Su
5937a56d47 [router][grpc] Add logprobs support to router (#11082) 2025-09-29 15:55:06 -07:00
Mick
9de1320b63 fix: fp8 mllama4 without vision modules being quantized (#10611) 2025-09-29 21:17:12 +00:00
Lianmin Zheng
dda34c2f93 Fix mem fraction static for nightly tests (#11076) 2025-09-29 12:57:41 -07:00
Lianmin Zheng
a17e70f5cc Use more general heuristics to set the default value of --mem-fraction-static (#10975) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-29 10:11:03 -07:00
Yineng Zhang
3a641d9085 chore: upgrade sgl-kernel 0.3.13 (#11056) 2025-09-29 02:22:25 -07:00
Yineng Zhang
5942fdb480 chore: upgrade cutedsl 4.2.1 (#11054) 2025-09-29 00:24:17 -07:00
Xiaoyu Zhang
11965b0daf Fix sgl-kernel benchmark dead code (#11022) 2025-09-29 15:06:40 +08:00
Vincent Zhong
71959545df Fix gemma 3 launch with transformers: the error: AttributeError: 'TransformersForCausalLM' object has no attribute 'tp_size' (#9614) 2025-09-29 06:18:29 +00:00
Zhihao Zhang
24f7cb1ece [speculative decoding] rename lookahead to ngram (#11010) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-09-28 21:06:59 -07:00
huangtingwei
e05555fad8 [HiCacheStorage] mooncake store support page_first_direct layout (#10591) 2025-09-28 20:45:48 -07:00
Mick
43fa9f22bd fix: check if weights are already local before downloading (#11015) 2025-09-28 20:11:33 -07:00
Mick
2e7633982c fix: show failed models in nightly ci (#10986) 2025-09-28 12:38:29 -07:00
Yuxuan Zhang
abb6781573 Update GLM-4.5 Model Doc (#11017) 2025-09-28 11:21:27 -07:00
Lianmin Zheng
07440f5f34 Fix FusedSetKVBufferArg in RotaryEmbedding (#11003) 2025-09-28 11:17:27 -07:00
Teng Ma
9816989bff [HiCache] bug: fix mooncake store batch set v1 (#11013) 2025-09-28 23:18:48 +08:00
lbk-sys
2a9d995c09 prepare for sglang+verl (#10555) 
Signed-off-by: lbk-sys <hello_lbk@163.com>
Signed-off-by: libaokui <libaokui@hauwei.com>
Co-authored-by: libaokui <libaokui@hauwei.com>
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-09-28 01:39:17 -07:00
wejoncy
a9050b5c39 [bugfix]Add empty_context import to two_batch_overlap.py (#10964) 2025-09-28 01:26:20 -07:00
YanbingJiang
66face3598 Fix CI failure of TypeError: RotaryEmbedding.forward_cpu() got an unexpected keyword argument 'fused_set_kv_buffer_arg' (#11009) 2025-09-28 00:31:08 -07:00
Simo Lin
5519766a4d [router] fix chat template loading and tokenizer path (#10999) 2025-09-27 23:54:12 -04:00
Lianmin Zheng
f6bc3f529b Fix profiler (#10997) 2025-09-27 14:56:18 -07:00
Tejesh Anand
8cc27fdc46 Use jsonschema to constrain required or specific tool choice (#10550) 2025-09-27 13:18:50 -04:00
Shangming Cai
9c339d6b47 [PD] Extract the PP transfer layer calculate logic from Mooncake to Common backend (#10565) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:10:41 +08:00
Shangming Cai
e23e280e16 Add support for topk metadata transferring for PD (#10616) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:09:38 +08:00
Muqi Li
51f7c6bd3c Add auth to get server info (#10751) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-27 02:54:39 -07:00
Xinyuan Tong
62e2e99db6 fix: make inference deterministic for large TP (#10930) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-27 02:46:45 -07:00
kk
8ebf72fef3 [Fix] RuntimeError: get_cfg Unsupported input_type:Float4_e2m1fn_x2 in using aiter-mxfp4-moe (#10981) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-09-26 22:12:22 -07:00
Yueyang Pan
8260574729 fix: fp8 quantization failure of qwen 2.5 VL 7B model (#10112) 
Signed-off-by: PanJason <pyyjason@gmail.com>
 2025-09-27 05:05:23 +00:00
Chang Su
37f3325b06 [router][grpc] Support E2E non-stream chat completions (#10980) 2025-09-26 22:02:06 -07:00
Muqi Li
bd95944cf6 [Bugfix][Minor][Benchmark] Fix some bugs due to PR #10495 (#10982) 2025-09-26 22:01:05 -07:00
hzh0425
c8a5d12abe [HiCache]: Support dynamic loading backends for hicache (#10551) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-26 18:34:11 -07:00
Chang Su
0c3db88978 [router][grpc] Add helpfer functions for decoder in router.rs and fix specs (#10971) 2025-09-26 20:10:45 -04:00
amysaq2023
2bdaf482f9 refactor loading weights from remote instance coding format (#10941) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-09-26 15:25:39 -07:00
Mick
777eb53897 ci: refactor nightly test (#10495) 2025-09-26 15:24:30 -07:00
Xiaoyu Zhang
05a3526654 Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs (#10372) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-26 15:10:49 -07:00
Lianmin Zheng
e56c64bfaf Update label field comment to indicate deprecation (#10970) 2025-09-26 12:59:59 -07:00
Yuan Luo
a5095d6262 Fuse write kv buffer into rope for qwen3 moe & bailing moe (#10749) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-26 15:18:41 +08:00
hzh0425
7ec5b4e89c [PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-25 23:20:49 -07:00
Yineng Zhang
172bcf0152 Revert "Refactor kv_cache_scheme handling for quantization (#10132)" (#10935) 2025-09-25 20:14:15 -07:00
Chang Su
37158f2018 router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929) 2025-09-25 20:03:35 -07:00
eraser00
0ac6114694 Replace the Kimi-K2 generated tool call idx with history tool call count (#10612) 
Co-authored-by: eraser00 <eraser00@github.com>
 2025-09-25 18:47:40 -07:00
Chang Su
7dcd689b47 [router][refactor] Clean up protobuf fields (#10923) 2025-09-25 17:48:47 -07:00
Lianmin Zheng
f68dd998b9 Rename customer label -> custom label (#10899) 
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-25 16:19:53 -07:00
Lianmin Zheng
35ec2a45a8 [minor] Remove deprecated function get_ip (#10883) 2025-09-25 16:18:04 -07:00
Swipe4057
0035f1cefa fix env flashinfer (#10910) 2025-09-25 15:44:48 -07:00
Mohammad Miadh Angkad
cd4da1f19b Refactor kv_cache_scheme handling for quantization (#10132) 2025-09-25 10:32:15 -07:00
Chang Su
916784746b router: Fix constraint proto and build_constraint in grpc router (#10881) 2025-09-25 11:12:06 -04:00
yi wang
fce170480a integrate AIBrix KVcache (#10376) 2025-09-25 14:47:09 +08:00
Zhiqiang Xie
3d40794fcf [HiCache] Cleaning the deprecated host memory state (#10778) 2025-09-25 14:43:53 +08:00