EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,507 Commits 8 Branches 0 Tags
d7b20dd65d310f68d79d585fae4b268283d1f93a
Commit Graph

3699 Commits

Author SHA1 Message Date
kushanam
d7b20dd65d chore: Initial support for input config files (#10534) 
Co-authored-by: root <root@umbriel-b200-017.ipp4a1.colossus.nvidia.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-24 14:45:52 -07:00
GuoweiWangU
cd641a995c fix bailing_moe with enable_dp_attention (#10860) 2025-09-24 02:29:32 -07:00
Xinyuan Tong
71f24ef8f6 feat: add cache_salt support to request (#10718) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-23 23:30:25 -07:00
Lianmin Zheng
32d893730f Revert "[fix][pd-disag]no need set next batch sampling info done in prefill" (#10828) 2025-09-23 17:02:01 -07:00
Lianmin Zheng
f47a2c67e6 [Auto Sync] Update load_config.py, model_config.py, configu... (20250923) (#10825) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-23 16:48:12 -07:00
Lianmin Zheng
312bfc4c95 [Auto Sync] Update simple_eval_common.py (20250923) (#10824) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-23 13:50:47 -07:00
Lianmin Zheng
e290303ea1 [Auto Sync] Update elementwise.py (20250923) (#10823) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-23 13:50:22 -07:00
Xinyuan Tong
aab35bccb4 fix: draft model IMA by overide max_positional_embeddings (#10787) 
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
 2025-09-23 12:56:16 -07:00
Yineng Zhang
42aedb02af [Auto Sync] Update protocol.py (20250923) (#10820) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com>
 2025-09-23 12:49:56 -07:00
Yiakwy
984730b732 add tunning files for QWEN-3-NEXT (#10794) 2025-09-23 12:46:30 -07:00
Lzhang-hub
d42975c641 Remove duplicate code in qwen2 model (#10540) 2025-09-24 02:40:51 +08:00
Jimmy
4b5ef3002c [fix][pd-disag]no need set next batch sampling info done in prefill (#10259) 2025-09-24 01:24:36 +08:00
Zheng Li
4f564b9e83 model: support qwen3-vl series (#10323) 
Co-authored-by: ocss884 <ocss.lin@gmail.com>
Co-authored-by: cao1zhg <653506626@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: 瑀澈 <yuche.lz@alibaba-inc.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-23 10:15:52 -07:00
Liangsheng Yin
4a762041d7 move environ into sglang.srt to avoid break SRT auto sync. (#10791) 2025-09-23 02:04:20 -07:00
Lianmin Zheng
ea338676b5 Clean up server args (#10770) 2025-09-23 00:22:32 -07:00
Yineng Zhang
8c1ef0f914 chore: upgrade sgl-kernel 0.3.12 (#10782) 2025-09-23 00:18:54 -07:00
Liangsheng Yin
f5a2faf2b8 Introduce FutureMap (#10715) 2025-09-22 23:27:30 -07:00
Lifu Huang
9241f4fd20 Move cached kernel to srt.utils (#10776) 2025-09-22 23:00:36 -07:00
fzyzcjy
063c3791fe Fix trtllm_mla slow concat kernel in MTP (#10777) 2025-09-22 22:47:49 -07:00
Liangsheng Yin
632b7d8cc9 Use simulate acc len from sglang.environ (#10771) 2025-09-23 12:59:50 +08:00
Lianmin Zheng
113f8f65a2 [Auto Sync] Update configurer.py (20250923) (#10765) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-22 17:34:34 -07:00
ronnie_zheng
e22f3a5ec9 [Ascend]optimize Qwen3 on Ascend (#10574) 
Co-authored-by: c30031083 <chenxu140@huawei.com>
 2025-09-22 17:18:36 -07:00
ronnie_zheng
095093ee5a [Ascend] optimize Qwen-vl on Ascend (#10556) 
Co-authored-by: wangqihui01 <wangqh10@163.com>
 2025-09-22 17:18:16 -07:00
Even Zhou
d27a6f7092 [Feature] Add MLAProcess for DeepSeek MLA on NPU (#10130) 2025-09-22 17:17:48 -07:00
Yineng Zhang
0753ef831e [Auto Sync] Update flashattention_backend.py (20250922) (#10762) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gordon Gustafson <ggustafson@together.ai>
 2025-09-22 16:41:42 -07:00
ishandhanani
662393f27d fix: kv events with tp > 1 (#10541) 2025-09-22 15:55:44 -07:00
pranavm-nvidia
b1bb8e7490 Enables TRT-LLM backend to be used for target_verify (#10281) 
Co-authored-by: Pranav Marathe <pranavm@ipp1-3309.ipp1a1.colossus.nvidia.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-09-22 15:54:00 -07:00
Lianmin Zheng
38c00ed7a1 Fix multimodal registry and code sync scripts (#10759) 
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-22 15:36:01 -07:00
pansicheng
d4041a5eeb refactor zero copy (#10300) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-22 15:17:31 -07:00
Jue WANG
9c53dad809 Fix MTP MoE weight loading with NVFP4 target model. (#10758) 2025-09-22 14:21:57 -07:00
Baizhou Zhang
aa1c5cf5bd Add warnings and remove dependency for deterministic inference (#10724) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-22 10:56:02 -07:00
Lifu Huang
2101d93b4f Fix CI TestChunkedSGMV (#10737) 2025-09-22 16:09:58 +08:00
feng397
944f1ea0ec fix capture_bs when speculative decoding enabled (#10730) 2025-09-22 00:43:12 -07:00
Yingchun Lai
9d7e82a0ab EPLB: prefer to use physical experts in the same node (#9849) 2025-09-22 00:34:30 -07:00
Yang Yu
f05805515b Convert FLASHINFER_WORKSPACE_SIZE to integer (#10731) 2025-09-22 00:26:42 -07:00
Lifu Huang
635ccda673 [4/4] Introduce CachedKernel to reduce CSGMV kernel launch overheads by 60% (#10709) 2025-09-21 22:26:42 -07:00
Qiaolin Yu
e2ac7888b8 [2/2] Support deterministic inference for temperature > 0 (#10678) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
 2025-09-21 19:36:08 -07:00
Stefan He
86527a4799 [deterministic inference] Move batch invariant pkg to sglang (#10695) 2025-09-21 19:35:14 -07:00
Ethan (Yusheng) Su
134b4f7ec2 Support deterministic inference with triton backend (#10694) 2025-09-22 09:20:40 +08:00
Yineng Zhang
f67d1f45bc [Auto Sync] Update deepseek_v2.py (20250922) (#10717) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Granado <mgranado@together.ai>
 2025-09-21 17:43:50 -07:00
Xinyuan Tong
12d6cf18f0 Refactors radix cache for extra key support (#10317) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-22 02:16:16 +08:00
Lifu Huang
08ecd0aa2a [3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592) 2025-09-20 22:47:48 -07:00
fzyzcjy
720c1c8ca3 Super tiny fix extra logs (#10697) 2025-09-20 21:30:54 -07:00
Yineng Zhang
d403c143e3 feat: update server args (#10696) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-20 18:52:19 -07:00
Stefan He
cba0d8c309 [Feature] Support deterministic inference with FA3 backend (#10651) 2025-09-20 17:50:21 -07:00
Yineng Zhang
f1d7892318 [Auto Sync] Update modelopt_quant.py (20250920) (#10688) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-20 02:37:49 -07:00
Yineng Zhang
7c876de7f5 fix: remove awq_dequantize deps (#10686) 2025-09-20 01:47:01 -07:00
dmitrygx
2b7417bf6a fix(disagg): fix sending KV cache in case of MLA for NIXL backend (#10673) 2025-09-20 00:30:10 -07:00
Baizhou Zhang
f111649580 Replace os.environ in layernorm.py (#10684) 2025-09-20 00:20:33 -07:00
Binyao Jiang
bd7eb0205a [Performance] Qwen3-Next: optimize causal_conv1d_fn triton kernel - up to 9% faster (#10680) 2025-09-20 00:12:43 -07:00