EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,465 Commits 8 Branches 0 Tags
095093ee5a3ef7a946a1cced2cfbd6e5164a522f
Commit Graph

3677 Commits

Author SHA1 Message Date
ronnie_zheng
095093ee5a [Ascend] optimize Qwen-vl on Ascend (#10556) 
Co-authored-by: wangqihui01 <wangqh10@163.com>
 2025-09-22 17:18:16 -07:00
Even Zhou
d27a6f7092 [Feature] Add MLAProcess for DeepSeek MLA on NPU (#10130) 2025-09-22 17:17:48 -07:00
Yineng Zhang
0753ef831e [Auto Sync] Update flashattention_backend.py (20250922) (#10762) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gordon Gustafson <ggustafson@together.ai>
 2025-09-22 16:41:42 -07:00
ishandhanani
662393f27d fix: kv events with tp > 1 (#10541) 2025-09-22 15:55:44 -07:00
pranavm-nvidia
b1bb8e7490 Enables TRT-LLM backend to be used for target_verify (#10281) 
Co-authored-by: Pranav Marathe <pranavm@ipp1-3309.ipp1a1.colossus.nvidia.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
 2025-09-22 15:54:00 -07:00
Lianmin Zheng
38c00ed7a1 Fix multimodal registry and code sync scripts (#10759) 
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-22 15:36:01 -07:00
pansicheng
d4041a5eeb refactor zero copy (#10300) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2025-09-22 15:17:31 -07:00
Jue WANG
9c53dad809 Fix MTP MoE weight loading with NVFP4 target model. (#10758) 2025-09-22 14:21:57 -07:00
Baizhou Zhang
aa1c5cf5bd Add warnings and remove dependency for deterministic inference (#10724) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-09-22 10:56:02 -07:00
Lifu Huang
2101d93b4f Fix CI TestChunkedSGMV (#10737) 2025-09-22 16:09:58 +08:00
feng397
944f1ea0ec fix capture_bs when speculative decoding enabled (#10730) 2025-09-22 00:43:12 -07:00
Yingchun Lai
9d7e82a0ab EPLB: prefer to use physical experts in the same node (#9849) 2025-09-22 00:34:30 -07:00
Yang Yu
f05805515b Convert FLASHINFER_WORKSPACE_SIZE to integer (#10731) 2025-09-22 00:26:42 -07:00
Lifu Huang
635ccda673 [4/4] Introduce CachedKernel to reduce CSGMV kernel launch overheads by 60% (#10709) 2025-09-21 22:26:42 -07:00
Qiaolin Yu
e2ac7888b8 [2/2] Support deterministic inference for temperature > 0 (#10678) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
 2025-09-21 19:36:08 -07:00
Stefan He
86527a4799 [deterministic inference] Move batch invariant pkg to sglang (#10695) 2025-09-21 19:35:14 -07:00
Ethan (Yusheng) Su
134b4f7ec2 Support deterministic inference with triton backend (#10694) 2025-09-22 09:20:40 +08:00
Yineng Zhang
f67d1f45bc [Auto Sync] Update deepseek_v2.py (20250922) (#10717) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Granado <mgranado@together.ai>
 2025-09-21 17:43:50 -07:00
Xinyuan Tong
12d6cf18f0 Refactors radix cache for extra key support (#10317) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-22 02:16:16 +08:00
Lifu Huang
08ecd0aa2a [3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592) 2025-09-20 22:47:48 -07:00
fzyzcjy
720c1c8ca3 Super tiny fix extra logs (#10697) 2025-09-20 21:30:54 -07:00
Yineng Zhang
d403c143e3 feat: update server args (#10696) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-20 18:52:19 -07:00
Stefan He
cba0d8c309 [Feature] Support deterministic inference with FA3 backend (#10651) 2025-09-20 17:50:21 -07:00
Yineng Zhang
f1d7892318 [Auto Sync] Update modelopt_quant.py (20250920) (#10688) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-20 02:37:49 -07:00
Yineng Zhang
7c876de7f5 fix: remove awq_dequantize deps (#10686) 2025-09-20 01:47:01 -07:00
dmitrygx
2b7417bf6a fix(disagg): fix sending KV cache in case of MLA for NIXL backend (#10673) 2025-09-20 00:30:10 -07:00
Baizhou Zhang
f111649580 Replace os.environ in layernorm.py (#10684) 2025-09-20 00:20:33 -07:00
Binyao Jiang
bd7eb0205a [Performance] Qwen3-Next: optimize causal_conv1d_fn triton kernel - up to 9% faster (#10680) 2025-09-20 00:12:43 -07:00
Yineng Zhang
b17e67df36 [Auto Sync] Update deepseek_v2.py (20250920) (#10683) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 23:43:31 -07:00
Baizhou Zhang
8ecef73f12 [1/2] Support deterministic inference with flashinfer attention backend (#10645) 
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-19 23:34:29 -07:00
Yineng Zhang
60e2a7cead [Auto Sync] Update model_runner.py (20250920) (#10679) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 18:26:54 -07:00
Jinyan Chen
d88ef4a388 limit sgl-kernel causal conv1d to cuda only (#10648) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-09-19 16:59:37 -07:00
Yineng Zhang
6f993e8b9e chore: cleanup docker image (#10671) 2025-09-19 16:56:49 -07:00
Yineng Zhang
dab4663b4e [Auto Sync] Update .clang-format (20250919) (#10670) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 12:31:44 -07:00
Yineng Zhang
610a6d6e86 fix: resolve sync issue (#10668) 2025-09-19 12:05:39 -07:00
huangtingwei
7f399e4bce [HiCacheStorage]support page_first_direct layout for generic set&get (#10522) 2025-09-19 05:47:16 -07:00
Baizhou Zhang
3fa3c22ae2 Fix fast decode plan for flashinfer v0.4.0rc1 and upgrade sgl-kernel 0.3.11 (#10634) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-09-19 01:25:29 -07:00
Jimmy
56b991b12d [Feature]feat(get_ip): unify get_ip_xxx (#10081) 2025-09-18 22:35:26 -07:00
FlyPanda
8b713c7248 Hicache L3 backend mooncake optimization configuration reading method (#10319) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: shicang <shicang@shicang>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-19 12:25:01 +08:00
zixuanzhang226
8c52de6fab feat: add fused moe config for Qwen3-Next-80B-A3B-Instruct on B200 (#10631) 2025-09-18 17:31:58 -07:00
Chang Su
c1815a99b7 model support: Sarashina2VisionForCausalLM (#10632) 2025-09-18 17:30:38 -07:00
Binyao Jiang
4e6c4923a0 [Performance] Qwen3-Next: speed up update_mamba_state_after_mtp_verify by 10x; e2e up to 3.54% faster (#10586) 2025-09-18 17:13:59 -07:00
Binyao Jiang
b91cb67e7a [Performance] Qwen3-Next: replace arange to cached query_start_loc_li… (#10553) 2025-09-18 17:02:42 -07:00
Zhihao Zhang
e7bc600304 [Feature] Speculative decoding support lookahead (#9873) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-18 16:42:41 -07:00
Yineng Zhang
2a2ff9a840 refactor: use registry for _get_attention_backend_from_str (#10629) 2025-09-18 16:27:59 -07:00
brayden-hai
67073dde85 Garbage collector regression in the online server (#10621) 2025-09-18 11:52:35 -07:00
yuk.igalaxy
9a5c42f9ad feat: Add FlexAttention Backend for Efficient Sparse Attention (#9947) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-09-18 11:49:17 -07:00
yhyang201
388c05d544 Fix bias handling in TritonMoeQuantInfo within quantization/mxfp4.py (#10579) 2025-09-18 11:44:43 -07:00
Jinyan Chen
fc809665fd [Performance] qwen3-next improve causal conv1d in prefill phase (#10595) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-09-18 11:42:49 -07:00
Yi Zhang
1344ebc833 support qwen3-next-fp8 deepep (#10622) 2025-09-18 11:36:22 -07:00