EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,562 Commits 8 Branches 0 Tags
9c339d6b473b236018aa1ca87fbc0df0d1e43ec0
Commit Graph

3728 Commits

Author SHA1 Message Date
Shangming Cai
9c339d6b47 [PD] Extract the PP transfer layer calculate logic from Mooncake to Common backend (#10565) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:10:41 +08:00
Shangming Cai
e23e280e16 Add support for topk metadata transferring for PD (#10616) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-28 00:09:38 +08:00
Muqi Li
51f7c6bd3c Add auth to get server info (#10751) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-27 02:54:39 -07:00
Xinyuan Tong
62e2e99db6 fix: make inference deterministic for large TP (#10930) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-27 02:46:45 -07:00
kk
8ebf72fef3 [Fix] RuntimeError: get_cfg Unsupported input_type:Float4_e2m1fn_x2 in using aiter-mxfp4-moe (#10981) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2025-09-26 22:12:22 -07:00
Yueyang Pan
8260574729 fix: fp8 quantization failure of qwen 2.5 VL 7B model (#10112) 
Signed-off-by: PanJason <pyyjason@gmail.com>
 2025-09-27 05:05:23 +00:00
Chang Su
37f3325b06 [router][grpc] Support E2E non-stream chat completions (#10980) 2025-09-26 22:02:06 -07:00
Muqi Li
bd95944cf6 [Bugfix][Minor][Benchmark] Fix some bugs due to PR #10495 (#10982) 2025-09-26 22:01:05 -07:00
hzh0425
c8a5d12abe [HiCache]: Support dynamic loading backends for hicache (#10551) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-26 18:34:11 -07:00
Chang Su
0c3db88978 [router][grpc] Add helpfer functions for decoder in router.rs and fix specs (#10971) 2025-09-26 20:10:45 -04:00
amysaq2023
2bdaf482f9 refactor loading weights from remote instance coding format (#10941) 
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
 2025-09-26 15:25:39 -07:00
Mick
777eb53897 ci: refactor nightly test (#10495) 2025-09-26 15:24:30 -07:00
Xiaoyu Zhang
05a3526654 Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs (#10372) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-26 15:10:49 -07:00
Lianmin Zheng
e56c64bfaf Update label field comment to indicate deprecation (#10970) 2025-09-26 12:59:59 -07:00
Yuan Luo
a5095d6262 Fuse write kv buffer into rope for qwen3 moe & bailing moe (#10749) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-26 15:18:41 +08:00
hzh0425
7ec5b4e89c [PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-25 23:20:49 -07:00
Yineng Zhang
172bcf0152 Revert "Refactor kv_cache_scheme handling for quantization (#10132)" (#10935) 2025-09-25 20:14:15 -07:00
Chang Su
37158f2018 router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929) 2025-09-25 20:03:35 -07:00
eraser00
0ac6114694 Replace the Kimi-K2 generated tool call idx with history tool call count (#10612) 
Co-authored-by: eraser00 <eraser00@github.com>
 2025-09-25 18:47:40 -07:00
Chang Su
7dcd689b47 [router][refactor] Clean up protobuf fields (#10923) 2025-09-25 17:48:47 -07:00
Lianmin Zheng
f68dd998b9 Rename customer label -> custom label (#10899) 
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-25 16:19:53 -07:00
Lianmin Zheng
35ec2a45a8 [minor] Remove deprecated function get_ip (#10883) 2025-09-25 16:18:04 -07:00
Swipe4057
0035f1cefa fix env flashinfer (#10910) 2025-09-25 15:44:48 -07:00
Mohammad Miadh Angkad
cd4da1f19b Refactor kv_cache_scheme handling for quantization (#10132) 2025-09-25 10:32:15 -07:00
Chang Su
916784746b router: Fix constraint proto and build_constraint in grpc router (#10881) 2025-09-25 11:12:06 -04:00
yi wang
fce170480a integrate AIBrix KVcache (#10376) 2025-09-25 14:47:09 +08:00
Zhiqiang Xie
3d40794fcf [HiCache] Cleaning the deprecated host memory state (#10778) 2025-09-25 14:43:53 +08:00
Lianmin Zheng
3e43eb137b [Auto Sync] Update model_config.py (20250925) (#10885) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-09-24 22:59:16 -07:00
Xiaoyu Zhang
c4e314f986 Restruct sgl-kernel benchmark (#10861) 2025-09-25 07:45:25 +08:00
kushanam
d7b20dd65d chore: Initial support for input config files (#10534) 
Co-authored-by: root <root@umbriel-b200-017.ipp4a1.colossus.nvidia.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-24 14:45:52 -07:00
GuoweiWangU
cd641a995c fix bailing_moe with enable_dp_attention (#10860) 2025-09-24 02:29:32 -07:00
Xinyuan Tong
71f24ef8f6 feat: add cache_salt support to request (#10718) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-23 23:30:25 -07:00
Lianmin Zheng
32d893730f Revert "[fix][pd-disag]no need set next batch sampling info done in prefill" (#10828) 2025-09-23 17:02:01 -07:00
Lianmin Zheng
f47a2c67e6 [Auto Sync] Update load_config.py, model_config.py, configu... (20250923) (#10825) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-23 16:48:12 -07:00
Lianmin Zheng
312bfc4c95 [Auto Sync] Update simple_eval_common.py (20250923) (#10824) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-23 13:50:47 -07:00
Lianmin Zheng
e290303ea1 [Auto Sync] Update elementwise.py (20250923) (#10823) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-23 13:50:22 -07:00
Xinyuan Tong
aab35bccb4 fix: draft model IMA by overide max_positional_embeddings (#10787) 
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
 2025-09-23 12:56:16 -07:00
Yineng Zhang
42aedb02af [Auto Sync] Update protocol.py (20250923) (#10820) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com>
 2025-09-23 12:49:56 -07:00
Yiakwy
984730b732 add tunning files for QWEN-3-NEXT (#10794) 2025-09-23 12:46:30 -07:00
Lzhang-hub
d42975c641 Remove duplicate code in qwen2 model (#10540) 2025-09-24 02:40:51 +08:00
Jimmy
4b5ef3002c [fix][pd-disag]no need set next batch sampling info done in prefill (#10259) 2025-09-24 01:24:36 +08:00
Zheng Li
4f564b9e83 model: support qwen3-vl series (#10323) 
Co-authored-by: ocss884 <ocss.lin@gmail.com>
Co-authored-by: cao1zhg <653506626@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-23 10:15:52 -07:00
Liangsheng Yin
4a762041d7 move environ into sglang.srt to avoid break SRT auto sync. (#10791) 2025-09-23 02:04:20 -07:00
Lianmin Zheng
ea338676b5 Clean up server args (#10770) 2025-09-23 00:22:32 -07:00
Yineng Zhang
8c1ef0f914 chore: upgrade sgl-kernel 0.3.12 (#10782) 2025-09-23 00:18:54 -07:00
Liangsheng Yin
f5a2faf2b8 Introduce FutureMap (#10715) 2025-09-22 23:27:30 -07:00
Lifu Huang
9241f4fd20 Move cached kernel to srt.utils (#10776) 2025-09-22 23:00:36 -07:00
fzyzcjy
063c3791fe Fix trtllm_mla slow concat kernel in MTP (#10777) 2025-09-22 22:47:49 -07:00
Liangsheng Yin
632b7d8cc9 Use simulate acc len from sglang.environ (#10771) 2025-09-23 12:59:50 +08:00
Lianmin Zheng
113f8f65a2 [Auto Sync] Update configurer.py (20250923) (#10765) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-22 17:34:34 -07:00