EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
4,221 Commits 8 Branches 0 Tags
36d6f0ba5bf8792d146deef766f544c725d9df3d
Commit Graph

258 Commits

Author SHA1 Message Date
fzyzcjy
62222bd27e Minor tool for comparison of benchmark results (#7974) 2025-07-27 00:27:50 -07:00
Lifu Huang
5c705b1dce Add perf tests for LoRA (#8314) 2025-07-26 14:55:22 -07:00
Hubert Lu
af4b9bae95 [AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs (#7135) 
Co-authored-by: yiakwy-xpu-ml-framework-team <961186938@qq.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2025-07-24 23:44:28 -07:00
Hubert Lu
e50109f2ed [AMD] Remove vllm's scaled_fp8_quant and moe_sum when SGLANG_USE_AITER=1 (#7484) 2025-07-21 17:33:19 -07:00
Pavel Logachev
877e35d775 Add get_hidden_dim to qwen3.py for correct lora (#7312) 2025-07-19 19:31:16 -07:00
Lifu Huang
4e3defe5a7 Support start up LoRA server without initial adapters (#8019) 2025-07-19 15:38:09 -07:00
Lianmin Zheng
bb0e8a32b5 Clean up server args (#8161) 2025-07-19 11:32:52 -07:00
Cheng Wan
15ad6c9086 [1/N] MoE Refactor: refactor select_experts (#7966) 2025-07-19 00:51:15 -07:00
Hongbo Xu
1f76fc8747 [3/n] chore: decouple AWQ implementation from vLLM dependency (#8113) 
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
 2025-07-18 11:45:22 -07:00
Hank Han
2117f82def [ci] CI supports use cached models (#7874) 2025-07-14 11:42:21 +00:00
Lifu Huang
e2ed9d049a Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes (#7844) 2025-07-13 18:36:01 -07:00
SijiaYang
cb9d91ea8a feat: support DeepSeek-R1-W4AFP8 model with ep-moe mode (#7762) 
Signed-off-by: yangsijia.614 <yangsijia.614@bytedance.com>
 2025-07-07 14:47:21 -07:00
YanbingJiang
4de0395343 Add V2-lite model test (#7390) 
Co-authored-by: DiweiSun <105627594+DiweiSun@users.noreply.github.com>
 2025-07-03 22:25:50 -07:00
Lifu Huang
49538d111b Support dynamic LoRA loading / unloading in engine/server API (#7446) 2025-06-27 21:00:27 -07:00
Lifu Huang
2373faa317 Fix flakiness in LoRA batch test. (#7552) 2025-06-27 19:51:43 -07:00
xutizhou
506c4928f5 feat: integrate deepgemm into EPMoE (#6821) 
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: TianQiLin666666 <1834987979@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-06-23 01:38:58 -07:00
Stefan He
3774f07825 Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099) 2025-06-19 00:56:37 -07:00
JieXin Liang
5ca07eed90 [fix] fix DeepGEMM blackwell input quant & ut & fix style and log (#7247) 2025-06-16 11:45:54 -07:00
woodx
e30ef368ab Feat/support rerank (#6058) 2025-06-16 10:50:01 -07:00
Lianmin Zheng
f47a1b1d0f Increase timeout in test/srt/test_disaggregation.py (#7175) 2025-06-13 23:12:14 -07:00
Quanfeng Li
ef32677444 Fix positional argument (#7093) 2025-06-11 18:31:13 -07:00
Baizhou Zhang
25a6a9aa22 Fix circular import in test_prefix_chunk_info.py (#7097) 2025-06-11 10:57:45 -07:00
Yineng Zhang
56ccd3c22c chore: upgrade flashinfer v0.2.6.post1 jit (#6958) 
Co-authored-by: alcanderian <alcanderian@gmail.com>
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2025-06-09 09:22:39 -07:00
Sai Enduri
77e928d00e Update server timeout time in AMD CI. (#6953) 2025-06-07 15:10:27 -07:00
Zaili Wang
562f279a2d [CPU] enable CI for PRs, add Dockerfile and auto build task (#6458) 
Co-authored-by: diwei sun <diwei.sun@intel.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-06-05 13:43:54 -07:00
Pavani Majety
0df6765c83 [CUTLASS-FP4-MOE] Introduce CutlassMoEParams class for easy initialization of Cutlass Grouped Gems Metadata (#6887) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-06-05 13:13:14 -07:00
Marc Sun
37f1547587 [FEAT] Add transformers backend support (#5929) 2025-06-03 21:05:29 -07:00
Pavani Majety
eb38c7d1ca [1/2] Add Kernel support for Cutlass based Fused FP4 MoE (#6093) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-06-02 13:48:03 -07:00
Lianmin Zheng
20fd53b8f6 Correctly abort the failed grammar requests & Improve the handling of abort (#6803) 2025-06-01 19:00:07 -07:00
Lianmin Zheng
2d72fc47cf Improve profiler and integrate profiler in bench_one_batch_server (#6787) 2025-05-31 15:53:55 -07:00
Yineng Zhang
7eb9d8e594 chore: upgrade transformers 4.52.3 (#6575) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-05-25 22:49:58 -07:00
Byron Hsu
8233cc10fd [PD] Support logprob & Add failure test (#6558) 2025-05-23 14:29:20 -07:00
Lifu Huang
3cf1473a09 Use monotonic clock for interval measurement (#6211) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-17 16:49:18 -07:00
Elfie Guo
6fc9357503 [2/2] Add python wrapper for CUTLASS FP8 Blockscale MoE Kernel. (#5694) 2025-05-16 13:14:07 -07:00
Kiv Chen
5380cd7ea3 model(vlm): pixtral (#5084) 2025-05-13 00:16:10 -07:00
Lianmin Zheng
e8e18dcdcc Revert "fix some typos" (#6244) 2025-05-12 12:53:26 -07:00
applesaucethebun
d738ab52f8 fix some typos (#6209) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-13 01:42:38 +08:00
Lianmin Zheng
fba8eccd7e Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 00:17:33 -07:00
Lifu Huang
6e2da51561 Replace time.time() to time.perf_counter() for benchmarking. (#6178) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-11 14:32:49 -07:00
shangmingc
31d1f6e7f4 [PD] Add simple unit test for disaggregation feature (#5654) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-11 13:35:27 +08:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Lianmin Zheng
de167cf5fa Fix request abortion (#6184) 2025-05-10 21:54:46 -07:00
XinyuanTong
e88dd482ed [CI]Add performance CI for VLM (#6038) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-07 19:20:03 -07:00
JieXin Liang
b70957fcf8 [refactor] slightly tidy fp8 module (#5993) 2025-05-07 17:28:24 -07:00
Jinyan Chen
8a828666a3 Add DeepEP to CI PR Test (#5655) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-05-06 17:36:03 -07:00
mlmz
256c4c2519 fix: correct stream response when enable_thinking is set to false (#5881) 2025-04-30 19:44:37 -07:00
Qiaolin Yu
7bcd8b1cb2 Fix lora batch processing when input lora_path contains None (#5930) 2025-04-30 19:42:42 -07:00
Ying Sheng
11383cec3c [PP] Add pipeline parallelism (#5724) 2025-04-30 18:18:07 -07:00
Qiaolin Yu
58195dd588 [Fix] Unload lora in HF_Runner if needed (#5899) 2025-04-29 20:17:42 -07:00
Lianmin Zheng
849c83a0c0 [CI] test chunked prefill more (#5798) 2025-04-28 10:57:17 -07:00