EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
5,383 Commits 8 Branches 0 Tags
7a68b4225a5c4c608e52dd6feead05f7512c3aa2
Commit Graph

5383 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
zhannngchen
7a68b4225a [improvement] add average input/output token length for hicache benchmark stats output (#10525) 2025-09-18 00:38:03 -07:00
Shangming Cai
60fc5b51f6 chore: upgrade mooncake 0.3.6 (#10596) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-18 00:19:30 -07:00
Shangming Cai
a13dd1e492 [PD] Improve disaggregation common backend and refactor mooncake backend (#10273) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-17 22:58:07 -07:00
HAI
d500eb9173 aiter v0.1.5.post2 (#10563) 2025-09-17 22:10:45 -07:00
Xuchun Shang
1ccd59c715 [HICache] introduce evict policy (#10190) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-18 11:10:20 +08:00
sogalin
c32fb7a24d [ROCm] Fix fp8 quantization accuracy issue. (#10558) 2025-09-17 17:44:59 -07:00
Shu Wang
1ba137e98f Enable trtllm mla prefix extend (#10526) 2025-09-17 16:44:11 -07:00
Kevin Xiang Li
de28f8e741 vlm: remove redundant d2h movement of mm feature tensors (#9987) 
Co-authored-by: Xiang (Kevin) Li <lik@nvidia.com>
 2025-09-17 15:00:39 -07:00
Yineng Zhang
564050766d fix: update dsv3 fp4 ut (#10584) 2025-09-17 14:34:58 -07:00
Ziming Huang
b73ac629cd [BugFix] Fix incorrect hidden_states_tensor in pd disaggregation + eagle (#9976) 2025-09-17 10:37:14 -07:00
Teng Ma
77098aea7b [HiCache] Add tests for hicache storage mooncake backend (#10171) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-18 01:07:16 +08:00
Liangsheng Yin
5ccf0b03bd [bench] Fix random seed in bench_one_batch_server (#10548) 2025-09-17 19:30:32 +08:00
EduardDurech
a77564e0fb CUDA Arch Independent (#8813) 2025-09-16 23:01:45 -07:00
Yichen Yan
4f9e71df3c Remove duplicated code (#10545) 
Co-authored-by: jiapingW <56055330+jiapingW@users.noreply.github.com>
 2025-09-16 20:48:22 -07:00
zhannngchen
541551cefe [bugfix]hicache bench_long_context.py run failed (#10523) 2025-09-17 11:27:06 +08:00
Shu Wang
124097fc5b enable prefix cache with dp (#10459) 2025-09-16 18:26:58 -07:00
kyleliang-nv
e1d45bc280 Fix decord dependency for aarch64 docker build (#10529) 2025-09-16 17:34:37 -07:00
harrisonlimh
14fdd52740 feat: add priority based scheduling with priority based request acceptance and preemption (#8746) 2025-09-16 17:10:10 -07:00
Lianmin Zheng
f949ad5794 [Auto Sync] Update activation.py, chunk_cache.py, utils.py (20250917) (#10538) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-16 17:06:43 -07:00
Lianmin Zheng
c49484a658 [Auto Sync] Update scheduler_profiler_mixin.py, rpd_utils.p... (20250916) (#10494) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-16 17:02:20 -07:00
cicirori
a2f7218a2e support using fa4 on deepseek on blackwell (#9928) 2025-09-16 16:16:06 -07:00
fzyzcjy
311de47bb7 [2/2] Speed up trtllm_mla attention backend (#10474) 2025-09-16 15:49:22 -07:00
gongwei-130
373080ea6c skip vision_model for lora (#10530) 2025-09-16 12:34:42 -07:00
Philip Kiely - Baseten
7f028b07c4 Fix formatting in long code blocks (#10528) 2025-09-16 12:02:05 -07:00
ybyang
0abb41c70d adjust import setuptools_rust (#10524) 2025-09-16 11:01:58 -04:00
Zaili Wang
925dbb3218 [CPU] fix CPU backend sel. issue for Llama4 (#10511) 2025-09-16 02:57:45 -07:00
fzyzcjy
8df7353af3 Support sgl-router parallel_batch in bench_one_batch_server (#10506) 2025-09-16 02:52:57 -07:00
fzyzcjy
ae4be601c2 Fix CI when sgl-kernel is changed but srt is not changed (#10515) 2025-09-16 02:49:54 -07:00
Qi Yuhang
9b876889b7 Update CUTLASS. Refine KernelSchedule for fp8 (grouped) gemm. (#10491) 2025-09-16 02:47:37 -07:00
Yineng Zhang
c0c6f543e4 chore: upgrade sgl-kernel 0.3.10 (#10500) 2025-09-16 02:00:53 -07:00
Shangming Cai
edd6a07bc0 Minor fix lint introduced by #10466 (#10507) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-09-16 01:38:25 -07:00
cao1zhg
b6dd4bcb81 feat: update support for qwen3next model (#10466) 2025-09-16 16:09:56 +08:00
b8zhong
b2435be682 Cache the result of is_blackwell platform check (#10498) 2025-09-15 22:30:28 -07:00
Chang Su
5fe39e85a2 [router] fix router manager and router init in server (#10499) 2025-09-15 22:23:26 -07:00
Liangsheng Yin
fa5d0bf6a5 Remove wrong imports from sglang.python (#10493) 2025-09-15 22:12:21 -07:00
Simo Lin
16e9335998 [router] add router db connector for responses api (#10487) 2025-09-15 22:04:56 -07:00
Night
f1c692f6f8 Add Logprobs unit test with a loose threshold (#10230) 
Co-authored-by: Yusheng Su <yushengsu.thu@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Ryan <ryan@ryanmini.mynetworksettings.com>
 2025-09-16 13:04:40 +08:00
brayden-hai
80572c8345 [ModelOpt] Respect kv_cache_quant_algo in ModelOpt checkpoints (#10336) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-09-15 20:16:49 -07:00
ykwd
4bb08f6e07 [Hicache] Evaluate Per-Round Metrics in Multiturn Bench (#10203) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-15 19:34:40 -07:00
kk
ec272dda9c Temporay work-around for rocm 7.0.0 alpha with enabling data-parallel issue (#10434) 
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
 2025-09-15 19:08:04 -07:00
scut-cbq
a220c14f81 fix crash of DeepSeek-V3 update_weights_from_disk (#8863) 
Co-authored-by: parkeychen <parkeychen@tencent.com>
 2025-09-16 09:45:15 +08:00
Chang Su
35ef3f2902 [router] fix worker registration in multi model mode (#10486) 2025-09-15 21:05:00 -04:00
Lianmin Zheng
31fb19a0a2 [Auto Sync] Update registry.py (20250915) (#10484) 
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-15 17:34:28 -07:00
Lifu Huang
3f41b48c40 [2/2] Introduce Chunked-SGMV kernels and corresponding LoRA backend for improved performance (#10286) 2025-09-15 16:04:03 -07:00
Chang Su
2689f0bf02 [router] multi model registration fix (#10481) 2025-09-15 15:22:21 -07:00
Yineng Zhang
5207424014 chore: bump v0.3.10 sgl-kernel (#10478) 2025-09-15 15:20:09 -07:00
Liangsheng Yin
c3c26f76b3 [Env] minimal version for organizing envs (#10479) 2025-09-16 03:51:25 +08:00
Liangsheng Yin
2cf811a9da Fix --dataset-path in bench_one_batch_server (#10475) 2025-09-16 02:55:02 +08:00
fzyzcjy
3b25dc127a [1/2] Speed up trtllm_mla attention backend (>10% e2e) (#10473) 2025-09-15 11:53:21 -07:00
Yineng Zhang
5c08d7d21d fix: resolve sgl-kernel ut (#10476) 2025-09-15 11:42:48 -07:00