EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,422 Commits 8 Branches 0 Tags
8ecef73f12ea7e5fba1cee95fac557eebe2389be
Commit Graph

5422 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Baizhou Zhang
8ecef73f12 [1/2] Support deterministic inference with flashinfer attention backend (#10645) 
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-19 23:34:29 -07:00
Simo Lin
1d1ce62495 [router] refactor router and worker management 2.5/n (#10677) 2025-09-19 20:54:40 -07:00
Yineng Zhang
60e2a7cead [Auto Sync] Update model_runner.py (20250920) (#10679) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 18:26:54 -07:00
Jinyan Chen
d88ef4a388 limit sgl-kernel causal conv1d to cuda only (#10648) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-09-19 16:59:37 -07:00
Yineng Zhang
6f993e8b9e chore: cleanup docker image (#10671) 2025-09-19 16:56:49 -07:00
Chang Su
03ce92e594 router-spec: Reorder ChatCompletionRequest and fix validation logic (#10675) 2025-09-19 16:41:21 -07:00
Simo Lin
00eb5eb721 [router] refactor router and worker management 2/n (#10666) 2025-09-19 12:37:57 -07:00
Yineng Zhang
dab4663b4e [Auto Sync] Update .clang-format (20250919) (#10670) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 12:31:44 -07:00
Yineng Zhang
610a6d6e86 fix: resolve sync issue (#10668) 2025-09-19 12:05:39 -07:00
Simo Lin
36efd5be8a [router] refactor router and worker management 1/n (#10664) 2025-09-19 06:19:57 -07:00
Fabian Gebhart
68cdc1893d [router] preserve order of json params using preserve_order feature (#10661) 2025-09-19 06:15:22 -07:00
huangtingwei
7f399e4bce [HiCacheStorage]support page_first_direct layout for generic set&get (#10522) 2025-09-19 05:47:16 -07:00
Simo Lin
873d858b28 [router] refactor worker to builder pattern 5/n (#10653) 2025-09-19 05:43:23 -04:00
Baizhou Zhang
3fa3c22ae2 Fix fast decode plan for flashinfer v0.4.0rc1 and upgrade sgl-kernel 0.3.11 (#10634) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-09-19 01:25:29 -07:00
Simo Lin
4f2055ad56 [router] refactor worker to builder pattern 4/n (#10650) 2025-09-18 23:49:10 -07:00
Yuan Luo
616a3e20df [sgl-kernel] Support moe_sum_reduce cuda kernel (#10321) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-19 14:12:09 +08:00
Simo Lin
ac2a723bb3 [router] refactor worker to builder pattern 3/n (#10647) 2025-09-18 22:52:57 -07:00
Jimmy
56b991b12d [Feature]feat(get_ip): unify get_ip_xxx (#10081) 2025-09-18 22:35:26 -07:00
Simo Lin
780d6a22cd [router] refactor worker to builder pattern 2/n (#10633) 2025-09-18 21:47:56 -07:00
FlyPanda
8b713c7248 Hicache L3 backend mooncake optimization configuration reading method (#10319) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: shicang <shicang@shicang>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-19 12:25:01 +08:00
Yineng Zhang
5bfafdfcb4 chore: bump sgl-kernel 0.3.11 (#10630) 2025-09-18 18:43:20 -07:00
zixuanzhang226
8c52de6fab feat: add fused moe config for Qwen3-Next-80B-A3B-Instruct on B200 (#10631) 2025-09-18 17:31:58 -07:00
Chang Su
c1815a99b7 model support: Sarashina2VisionForCausalLM (#10632) 2025-09-18 17:30:38 -07:00
Binyao Jiang
4e6c4923a0 [Performance] Qwen3-Next: speed up update_mamba_state_after_mtp_verify by 10x; e2e up to 3.54% faster (#10586) 2025-09-18 17:13:59 -07:00
Binyao Jiang
b91cb67e7a [Performance] Qwen3-Next: replace arange to cached query_start_loc_li… (#10553) 2025-09-18 17:02:42 -07:00
Zhihao Zhang
e7bc600304 [Feature] Speculative decoding support lookahead (#9873) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-18 16:42:41 -07:00
Yineng Zhang
2a2ff9a840 refactor: use registry for _get_attention_backend_from_str (#10629) 2025-09-18 16:27:59 -07:00
Simo Lin
5291f32d75 [router] refactor worker to builder pattern 1/n (#10628) 2025-09-18 13:25:40 -07:00
brayden-hai
67073dde85 Garbage collector regression in the online server (#10621) 2025-09-18 11:52:35 -07:00
yuk.igalaxy
9a5c42f9ad feat: Add FlexAttention Backend for Efficient Sparse Attention (#9947) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-09-18 11:49:17 -07:00
yhyang201
388c05d544 Fix bias handling in TritonMoeQuantInfo within quantization/mxfp4.py (#10579) 2025-09-18 11:44:43 -07:00
Jinyan Chen
fc809665fd [Performance] qwen3-next improve causal conv1d in prefill phase (#10595) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-09-18 11:42:49 -07:00
Zaili Wang
6fd4816d9f Fix sgl_kernel import failure on devices other than CUDA (#10610) 2025-09-18 11:38:02 -07:00
Yi Zhang
1344ebc833 support qwen3-next-fp8 deepep (#10622) 2025-09-18 11:36:22 -07:00
Yi Zhang
e07b21ceaf update deepep version for qwen3-next deepep moe (#10624) 2025-09-18 11:35:22 -07:00
shaharmor98
52f248cd31 Feat/add heartbeat mechanism for nixl conn (#10222) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
 2025-09-18 10:20:42 -07:00
penguin_wwy
93f75778be [RL] Add destroy process group api (#9979) 2025-09-19 00:31:56 +08:00
AlphaBaby
4039c626e2 fix deepep assert when PD disaggregation == null (#8274) 
Co-authored-by: fujianhao.fjh <fujianhao.fjh@antgroup.com>
 2025-09-19 00:31:01 +08:00
Yi Zhang
db71c38fcd Scale kkt after reduction (#10604) 2025-09-18 20:51:40 +08:00
zhannngchen
7a68b4225a [improvement] add average input/output token length for hicache benchmark stats output (#10525) 2025-09-18 00:38:03 -07:00
Shangming Cai
60fc5b51f6 chore: upgrade mooncake 0.3.6 (#10596) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-18 00:19:30 -07:00
Shangming Cai
a13dd1e492 [PD] Improve disaggregation common backend and refactor mooncake backend (#10273) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-17 22:58:07 -07:00
HAI
d500eb9173 aiter v0.1.5.post2 (#10563) 2025-09-17 22:10:45 -07:00
Xuchun Shang
1ccd59c715 [HICache] introduce evict policy (#10190) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-18 11:10:20 +08:00
sogalin
c32fb7a24d [ROCm] Fix fp8 quantization accuracy issue. (#10558) 2025-09-17 17:44:59 -07:00
Shu Wang
1ba137e98f Enable trtllm mla prefix extend (#10526) 2025-09-17 16:44:11 -07:00
Kevin Xiang Li
de28f8e741 vlm: remove redundant d2h movement of mm feature tensors (#9987) 
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
 2025-09-17 15:00:39 -07:00
Yineng Zhang
564050766d fix: update dsv3 fp4 ut (#10584) 2025-09-17 14:34:58 -07:00
Ziming Huang
b73ac629cd [BugFix] Fix incorrect hidden_states_tensor in pd disaggregation + eagle (#9976) 2025-09-17 10:37:14 -07:00
Teng Ma
77098aea7b [HiCache] Add tests for hicache storage mooncake backend (#10171) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-18 01:07:16 +08:00