EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,548 Commits 8 Branches 0 Tags
05a352665479ef8ea308d96b9b614c76c80740c0
Commit Graph

3716 Commits

Author SHA1 Message Date
Xiaoyu Zhang
05a3526654 Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs (#10372) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-26 15:10:49 -07:00
Lianmin Zheng
e56c64bfaf Update label field comment to indicate deprecation (#10970) 2025-09-26 12:59:59 -07:00
Yuan Luo
a5095d6262 Fuse write kv buffer into rope for qwen3 moe & bailing moe (#10749) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-26 15:18:41 +08:00
hzh0425
7ec5b4e89c [PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-25 23:20:49 -07:00
Yineng Zhang
172bcf0152 Revert "Refactor kv_cache_scheme handling for quantization (#10132)" (#10935) 2025-09-25 20:14:15 -07:00
Chang Su
37158f2018 router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929) 2025-09-25 20:03:35 -07:00
eraser00
0ac6114694 Replace the Kimi-K2 generated tool call idx with history tool call count (#10612) 
Co-authored-by: eraser00 <eraser00@github.com>
 2025-09-25 18:47:40 -07:00
Chang Su
7dcd689b47 [router][refactor] Clean up protobuf fields (#10923) 2025-09-25 17:48:47 -07:00
Lianmin Zheng
f68dd998b9 Rename customer label -> custom label (#10899) 
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-25 16:19:53 -07:00
Lianmin Zheng
35ec2a45a8 [minor] Remove deprecated function get_ip (#10883) 2025-09-25 16:18:04 -07:00
Swipe4057
0035f1cefa fix env flashinfer (#10910) 2025-09-25 15:44:48 -07:00
Mohammad Miadh Angkad
cd4da1f19b Refactor kv_cache_scheme handling for quantization (#10132) 2025-09-25 10:32:15 -07:00
Chang Su
916784746b router: Fix constraint proto and build_constraint in grpc router (#10881) 2025-09-25 11:12:06 -04:00
yi wang
fce170480a integrate AIBrix KVcache (#10376) 2025-09-25 14:47:09 +08:00
Zhiqiang Xie
3d40794fcf [HiCache] Cleaning the deprecated host memory state (#10778) 2025-09-25 14:43:53 +08:00
Lianmin Zheng
3e43eb137b [Auto Sync] Update model_config.py (20250925) (#10885) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-09-24 22:59:16 -07:00
Xiaoyu Zhang
c4e314f986 Restruct sgl-kernel benchmark (#10861) 2025-09-25 07:45:25 +08:00
kushanam
d7b20dd65d chore: Initial support for input config files (#10534) 
Co-authored-by: root <root@umbriel-b200-017.ipp4a1.colossus.nvidia.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-24 14:45:52 -07:00
GuoweiWangU
cd641a995c fix bailing_moe with enable_dp_attention (#10860) 2025-09-24 02:29:32 -07:00
Xinyuan Tong
71f24ef8f6 feat: add cache_salt support to request (#10718) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-23 23:30:25 -07:00
Lianmin Zheng
32d893730f Revert "[fix][pd-disag]no need set next batch sampling info done in prefill" (#10828) 2025-09-23 17:02:01 -07:00
Lianmin Zheng
f47a2c67e6 [Auto Sync] Update load_config.py, model_config.py, configu... (20250923) (#10825) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-23 16:48:12 -07:00
Lianmin Zheng
312bfc4c95 [Auto Sync] Update simple_eval_common.py (20250923) (#10824) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-23 13:50:47 -07:00
Lianmin Zheng
e290303ea1 [Auto Sync] Update elementwise.py (20250923) (#10823) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-23 13:50:22 -07:00
Xinyuan Tong
aab35bccb4 fix: draft model IMA by overide max_positional_embeddings (#10787) 
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
 2025-09-23 12:56:16 -07:00
Yineng Zhang
42aedb02af [Auto Sync] Update protocol.py (20250923) (#10820) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com>
 2025-09-23 12:49:56 -07:00
Yiakwy
984730b732 add tunning files for QWEN-3-NEXT (#10794) 2025-09-23 12:46:30 -07:00
Lzhang-hub
d42975c641 Remove duplicate code in qwen2 model (#10540) 2025-09-24 02:40:51 +08:00
Jimmy
4b5ef3002c [fix][pd-disag]no need set next batch sampling info done in prefill (#10259) 2025-09-24 01:24:36 +08:00
Zheng Li
4f564b9e83 model: support qwen3-vl series (#10323) 
Co-authored-by: ocss884 <ocss.lin@gmail.com>
Co-authored-by: cao1zhg <653506626@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-23 10:15:52 -07:00
Liangsheng Yin
4a762041d7 move environ into sglang.srt to avoid break SRT auto sync. (#10791) 2025-09-23 02:04:20 -07:00
Lianmin Zheng
ea338676b5 Clean up server args (#10770) 2025-09-23 00:22:32 -07:00
Yineng Zhang
8c1ef0f914 chore: upgrade sgl-kernel 0.3.12 (#10782) 2025-09-23 00:18:54 -07:00
Liangsheng Yin
f5a2faf2b8 Introduce FutureMap (#10715) 2025-09-22 23:27:30 -07:00
Lifu Huang
9241f4fd20 Move cached kernel to srt.utils (#10776) 2025-09-22 23:00:36 -07:00
fzyzcjy
063c3791fe Fix trtllm_mla slow concat kernel in MTP (#10777) 2025-09-22 22:47:49 -07:00
Liangsheng Yin
632b7d8cc9 Use simulate acc len from sglang.environ (#10771) 2025-09-23 12:59:50 +08:00
Lianmin Zheng
113f8f65a2 [Auto Sync] Update configurer.py (20250923) (#10765) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-22 17:34:34 -07:00
ronnie_zheng
e22f3a5ec9 [Ascend]optimize Qwen3 on Ascend (#10574) 
Co-authored-by: c30031083 <chenxu140@huawei.com>
 2025-09-22 17:18:36 -07:00
ronnie_zheng
095093ee5a [Ascend] optimize Qwen-vl on Ascend (#10556) 
Co-authored-by: wangqihui01 <wangqh10@163.com>
 2025-09-22 17:18:16 -07:00
Even Zhou
d27a6f7092 [Feature] Add MLAProcess for DeepSeek MLA on NPU (#10130) 2025-09-22 17:17:48 -07:00
Yineng Zhang
0753ef831e [Auto Sync] Update flashattention_backend.py (20250922) (#10762) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gordon Gustafson <ggustafson@together.ai>
 2025-09-22 16:41:42 -07:00
ishandhanani
662393f27d fix: kv events with tp > 1 (#10541) 2025-09-22 15:55:44 -07:00
pranavm-nvidia
b1bb8e7490 Enables TRT-LLM backend to be used for target_verify (#10281) 
Co-authored-by: Pranav Marathe <pranavm@ipp1-3309.ipp1a1.colossus.nvidia.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-09-22 15:54:00 -07:00
Lianmin Zheng
38c00ed7a1 Fix multimodal registry and code sync scripts (#10759) 
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-22 15:36:01 -07:00
pansicheng
d4041a5eeb refactor zero copy (#10300) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-22 15:17:31 -07:00
Jue WANG
9c53dad809 Fix MTP MoE weight loading with NVFP4 target model. (#10758) 2025-09-22 14:21:57 -07:00
Baizhou Zhang
aa1c5cf5bd Add warnings and remove dependency for deterministic inference (#10724) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-22 10:56:02 -07:00
Lifu Huang
2101d93b4f Fix CI TestChunkedSGMV (#10737) 2025-09-22 16:09:58 +08:00
feng397
944f1ea0ec fix capture_bs when speculative decoding enabled (#10730) 2025-09-22 00:43:12 -07:00