EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,435 Commits 8 Branches 0 Tags
fc3e54200932b653d359b206d7dcceffa0d76718
Commit Graph

3658 Commits

Author SHA1 Message Date
Lifu Huang
08ecd0aa2a [3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592) 2025-09-20 22:47:48 -07:00
fzyzcjy
720c1c8ca3 Super tiny fix extra logs (#10697) 2025-09-20 21:30:54 -07:00
Yineng Zhang
d403c143e3 feat: update server args (#10696) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-20 18:52:19 -07:00
Stefan He
cba0d8c309 [Feature] Support deterministic inference with FA3 backend (#10651) 2025-09-20 17:50:21 -07:00
Yineng Zhang
f1d7892318 [Auto Sync] Update modelopt_quant.py (20250920) (#10688) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-20 02:37:49 -07:00
Yineng Zhang
7c876de7f5 fix: remove awq_dequantize deps (#10686) 2025-09-20 01:47:01 -07:00
dmitrygx
2b7417bf6a fix(disagg): fix sending KV cache in case of MLA for NIXL backend (#10673) 2025-09-20 00:30:10 -07:00
Baizhou Zhang
f111649580 Replace os.environ in layernorm.py (#10684) 2025-09-20 00:20:33 -07:00
Binyao Jiang
bd7eb0205a [Performance] Qwen3-Next: optimize causal_conv1d_fn triton kernel - up to 9% faster (#10680) 2025-09-20 00:12:43 -07:00
Yineng Zhang
b17e67df36 [Auto Sync] Update deepseek_v2.py (20250920) (#10683) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 23:43:31 -07:00
Baizhou Zhang
8ecef73f12 [1/2] Support deterministic inference with flashinfer attention backend (#10645) 
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-19 23:34:29 -07:00
Yineng Zhang
60e2a7cead [Auto Sync] Update model_runner.py (20250920) (#10679) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 18:26:54 -07:00
Jinyan Chen
d88ef4a388 limit sgl-kernel causal conv1d to cuda only (#10648) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-09-19 16:59:37 -07:00
Yineng Zhang
6f993e8b9e chore: cleanup docker image (#10671) 2025-09-19 16:56:49 -07:00
Yineng Zhang
dab4663b4e [Auto Sync] Update .clang-format (20250919) (#10670) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-19 12:31:44 -07:00
Yineng Zhang
610a6d6e86 fix: resolve sync issue (#10668) 2025-09-19 12:05:39 -07:00
huangtingwei
7f399e4bce [HiCacheStorage]support page_first_direct layout for generic set&get (#10522) 2025-09-19 05:47:16 -07:00
Baizhou Zhang
3fa3c22ae2 Fix fast decode plan for flashinfer v0.4.0rc1 and upgrade sgl-kernel 0.3.11 (#10634) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-09-19 01:25:29 -07:00
Jimmy
56b991b12d [Feature]feat(get_ip): unify get_ip_xxx (#10081) 2025-09-18 22:35:26 -07:00
FlyPanda
8b713c7248 Hicache L3 backend mooncake optimization configuration reading method (#10319) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: shicang <shicang@shicang>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-19 12:25:01 +08:00
zixuanzhang226
8c52de6fab feat: add fused moe config for Qwen3-Next-80B-A3B-Instruct on B200 (#10631) 2025-09-18 17:31:58 -07:00
Chang Su
c1815a99b7 model support: Sarashina2VisionForCausalLM (#10632) 2025-09-18 17:30:38 -07:00
Binyao Jiang
4e6c4923a0 [Performance] Qwen3-Next: speed up update_mamba_state_after_mtp_verify by 10x; e2e up to 3.54% faster (#10586) 2025-09-18 17:13:59 -07:00
Binyao Jiang
b91cb67e7a [Performance] Qwen3-Next: replace arange to cached query_start_loc_li… (#10553) 2025-09-18 17:02:42 -07:00
Zhihao Zhang
e7bc600304 [Feature] Speculative decoding support lookahead (#9873) 
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2025-09-18 16:42:41 -07:00
Yineng Zhang
2a2ff9a840 refactor: use registry for _get_attention_backend_from_str (#10629) 2025-09-18 16:27:59 -07:00
brayden-hai
67073dde85 Garbage collector regression in the online server (#10621) 2025-09-18 11:52:35 -07:00
yuk.igalaxy
9a5c42f9ad feat: Add FlexAttention Backend for Efficient Sparse Attention (#9947) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-09-18 11:49:17 -07:00
yhyang201
388c05d544 Fix bias handling in TritonMoeQuantInfo within quantization/mxfp4.py (#10579) 2025-09-18 11:44:43 -07:00
Jinyan Chen
fc809665fd [Performance] qwen3-next improve causal conv1d in prefill phase (#10595) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-09-18 11:42:49 -07:00
Yi Zhang
1344ebc833 support qwen3-next-fp8 deepep (#10622) 2025-09-18 11:36:22 -07:00
shaharmor98
52f248cd31 Feat/add heartbeat mechanism for nixl conn (#10222) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
 2025-09-18 10:20:42 -07:00
penguin_wwy
93f75778be [RL] Add destroy process group api (#9979) 2025-09-19 00:31:56 +08:00
AlphaBaby
4039c626e2 fix deepep assert when PD disaggregation == null (#8274) 
Co-authored-by: fujianhao.fjh <fujianhao.fjh@antgroup.com>
 2025-09-19 00:31:01 +08:00
Yi Zhang
db71c38fcd Scale kkt after reduction (#10604) 2025-09-18 20:51:40 +08:00
Shangming Cai
a13dd1e492 [PD] Improve disaggregation common backend and refactor mooncake backend (#10273) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-17 22:58:07 -07:00
Xuchun Shang
1ccd59c715 [HICache] introduce evict policy (#10190) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-18 11:10:20 +08:00
sogalin
c32fb7a24d [ROCm] Fix fp8 quantization accuracy issue. (#10558) 2025-09-17 17:44:59 -07:00
Shu Wang
1ba137e98f Enable trtllm mla prefix extend (#10526) 2025-09-17 16:44:11 -07:00
Kevin Xiang Li
de28f8e741 vlm: remove redundant d2h movement of mm feature tensors (#9987) 
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
 2025-09-17 15:00:39 -07:00
Ziming Huang
b73ac629cd [BugFix] Fix incorrect hidden_states_tensor in pd disaggregation + eagle (#9976) 2025-09-17 10:37:14 -07:00
Liangsheng Yin
5ccf0b03bd [bench] Fix random seed in bench_one_batch_server (#10548) 2025-09-17 19:30:32 +08:00
Yichen Yan
4f9e71df3c Remove duplicated code (#10545) 
Co-authored-by: jiapingW <56055330+jiapingW@users.noreply.github.com>
 2025-09-16 20:48:22 -07:00
Shu Wang
124097fc5b enable prefix cache with dp (#10459) 2025-09-16 18:26:58 -07:00
kyleliang-nv
e1d45bc280 Fix decord dependency for aarch64 docker build (#10529) 2025-09-16 17:34:37 -07:00
harrisonlimh
14fdd52740 feat: add priority based scheduling with priority based request acceptance and preemption (#8746) 2025-09-16 17:10:10 -07:00
Lianmin Zheng
f949ad5794 [Auto Sync] Update activation.py, chunk_cache.py, utils.py (20250917) (#10538) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-16 17:06:43 -07:00
Lianmin Zheng
c49484a658 [Auto Sync] Update scheduler_profiler_mixin.py, rpd_utils.p... (20250916) (#10494) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-16 17:02:20 -07:00
cicirori
a2f7218a2e support using fa4 on deepseek on blackwell (#9928) 2025-09-16 16:16:06 -07:00
fzyzcjy
311de47bb7 [2/2] Speed up trtllm_mla attention backend (#10474) 2025-09-16 15:49:22 -07:00