EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,610 Commits 8 Branches 0 Tags
443a1b4ab32a87a7c6a503f212374beeb4209282
Commit Graph

3760 Commits

Author SHA1 Message Date
maxiao1
443a1b4ab3 Update pyproject_other.toml 2025-09-30 10:47:20 +00:00
maxiao
852a49c5cc adapt to dsv32 on dcu 2025-09-30 18:37:31 +08:00
maxiao
8f7453e3af adapt to ds3.2 2025-09-30 17:44:54 +08:00
Lianmin Zheng
1237aa19ce [Auto Sync] Update fused_moe_triton_config.py (20250930) (#11099) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-30 00:57:16 -07:00
jacky.cheng
5991119541 [Fix] Resolve performance drop in speculative decoding aiter backend (#11087) 2025-09-29 23:51:30 -07:00
Ke Bao
424591d53d Fix spec filter batch when target extend (#10991) 2025-09-30 14:44:02 +08:00
narutolhy
d17986f8c6 Enable optional FP32 compute for LM Head (#10729) 
Thanks to MiniMax Team and Chenyang Zhao's support.
 2025-09-29 20:45:17 -07:00
DevashishLal-CB
8831c55c3d [model] added support for w8a8int8 used by neuralmagic/Qwen2-0.5B-Ins… (#9642) 2025-09-30 11:26:17 +08:00
li-kesen
2bc61dd194 Remove hybrid_linear_attn attention backend and refactor attention registry (#10816) 
Co-authored-by: Yi Zhang <1109276519@qq.com>
 2025-09-30 10:16:16 +08:00
Cheng Wan
6535fda127 [Profile] dump memory trace when cuda graph profile is enabled (#11083) 2025-09-29 17:36:48 -07:00
Jimmy
3713eb6135 feat(reasoning): improve enable thinking from request (#10875) 2025-09-29 16:59:08 -07:00
Chang Su
5937a56d47 [router][grpc] Add logprobs support to router (#11082) 2025-09-29 15:55:06 -07:00
Mick
9de1320b63 fix: fp8 mllama4 without vision modules being quantized (#10611) 2025-09-29 21:17:12 +00:00
Lianmin Zheng
dda34c2f93 Fix mem fraction static for nightly tests (#11076) 2025-09-29 12:57:41 -07:00
Lianmin Zheng
a17e70f5cc Use more general heuristics to set the default value of --mem-fraction-static (#10975) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-29 10:11:03 -07:00
Yineng Zhang
3a641d9085 chore: upgrade sgl-kernel 0.3.13 (#11056) 2025-09-29 02:22:25 -07:00
Yineng Zhang
5942fdb480 chore: upgrade cutedsl 4.2.1 (#11054) 2025-09-29 00:24:17 -07:00
Xiaoyu Zhang
11965b0daf Fix sgl-kernel benchmark dead code (#11022) 2025-09-29 15:06:40 +08:00
Vincent Zhong
71959545df Fix gemma 3 launch with transformers: the error: AttributeError: 'TransformersForCausalLM' object has no attribute 'tp_size' (#9614) 2025-09-29 06:18:29 +00:00
Zhihao Zhang
24f7cb1ece [speculative decoding] rename lookahead to ngram (#11010) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
 2025-09-28 21:06:59 -07:00
huangtingwei
e05555fad8 [HiCacheStorage] mooncake store support page_first_direct layout (#10591) 2025-09-28 20:45:48 -07:00
Mick
43fa9f22bd fix: check if weights are already local before downloading (#11015) 2025-09-28 20:11:33 -07:00
Mick
2e7633982c fix: show failed models in nightly ci (#10986) 2025-09-28 12:38:29 -07:00
Yuxuan Zhang
abb6781573 Update GLM-4.5 Model Doc (#11017) 2025-09-28 11:21:27 -07:00
Lianmin Zheng
07440f5f34 Fix FusedSetKVBufferArg in RotaryEmbedding (#11003) 2025-09-28 11:17:27 -07:00
Teng Ma
9816989bff [HiCache] bug: fix mooncake store batch set v1 (#11013) 2025-09-28 23:18:48 +08:00
lbk-sys
2a9d995c09 prepare for sglang+verl (#10555) 
Signed-off-by: lbk-sys <hello_lbk@163.com>
Signed-off-by: libaokui <libaokui@hauwei.com>
Co-authored-by: libaokui <libaokui@hauwei.com>
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
 2025-09-28 01:39:17 -07:00
wejoncy
a9050b5c39 [bugfix]Add empty_context import to two_batch_overlap.py (#10964) 2025-09-28 01:26:20 -07:00
YanbingJiang
66face3598 Fix CI failure of TypeError: RotaryEmbedding.forward_cpu() got an unexpected keyword argument 'fused_set_kv_buffer_arg' (#11009) 2025-09-28 00:31:08 -07:00
Simo Lin
5519766a4d [router] fix chat template loading and tokenizer path (#10999) 2025-09-27 23:54:12 -04:00
Lianmin Zheng
f6bc3f529b Fix profiler (#10997) 2025-09-27 14:56:18 -07:00
Tejesh Anand
8cc27fdc46 Use jsonschema to constrain required or specific tool choice (#10550) 2025-09-27 13:18:50 -04:00
Shangming Cai
9c339d6b47 [PD] Extract the PP transfer layer calculate logic from Mooncake to Common backend (#10565) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:10:41 +08:00
Shangming Cai
e23e280e16 Add support for topk metadata transferring for PD (#10616) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:09:38 +08:00
Muqi Li
51f7c6bd3c Add auth to get server info (#10751) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-27 02:54:39 -07:00
Xinyuan Tong
62e2e99db6 fix: make inference deterministic for large TP (#10930) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-27 02:46:45 -07:00
kk
8ebf72fef3 [Fix] RuntimeError: get_cfg Unsupported input_type:Float4_e2m1fn_x2 in using aiter-mxfp4-moe (#10981) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-09-26 22:12:22 -07:00
Yueyang Pan
8260574729 fix: fp8 quantization failure of qwen 2.5 VL 7B model (#10112) 
Signed-off-by: PanJason <pyyjason@gmail.com>
 2025-09-27 05:05:23 +00:00
Chang Su
37f3325b06 [router][grpc] Support E2E non-stream chat completions (#10980) 2025-09-26 22:02:06 -07:00
Muqi Li
bd95944cf6 [Bugfix][Minor][Benchmark] Fix some bugs due to PR #10495 (#10982) 2025-09-26 22:01:05 -07:00
hzh0425
c8a5d12abe [HiCache]: Support dynamic loading backends for hicache (#10551) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-26 18:34:11 -07:00
Chang Su
0c3db88978 [router][grpc] Add helpfer functions for decoder in router.rs and fix specs (#10971) 2025-09-26 20:10:45 -04:00
amysaq2023
2bdaf482f9 refactor loading weights from remote instance coding format (#10941) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-09-26 15:25:39 -07:00
Mick
777eb53897 ci: refactor nightly test (#10495) 2025-09-26 15:24:30 -07:00
Xiaoyu Zhang
05a3526654 Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs (#10372) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-26 15:10:49 -07:00
Lianmin Zheng
e56c64bfaf Update label field comment to indicate deprecation (#10970) 2025-09-26 12:59:59 -07:00
Yuan Luo
a5095d6262 Fuse write kv buffer into rope for qwen3 moe & bailing moe (#10749) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-26 15:18:41 +08:00
hzh0425
7ec5b4e89c [PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-25 23:20:49 -07:00
Yineng Zhang
172bcf0152 Revert "Refactor kv_cache_scheme handling for quantization (#10132)" (#10935) 2025-09-25 20:14:15 -07:00
Chang Su
37158f2018 router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929) 2025-09-25 20:03:35 -07:00