EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,344 Commits 8 Branches 0 Tags
d6e1d28c8aff0ba6cca1d33985d1e1663a734f6f
Commit Graph

639 Commits

Author SHA1 Message Date
Jiajun Li
4024e1d2a8 Implement Siglip Vision model, and support BNB quantization for gemma3-mm (#5339) 2025-05-20 23:53:46 -07:00
HAI
5c0b38f369 aiter attention-backend (default enabled on AMD/ROCm) (#6381) 2025-05-20 22:52:41 -07:00
YanbingJiang
32cc66efa5 Update extend/decode attention kernel for CPU in sgl-kernel and add UTs (#6405) 
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-19 21:23:17 -07:00
fzyzcjy
f0653886a5 Expert distribution recording without overhead for EPLB (#4957) 2025-05-19 20:07:43 -07:00
Yineng Zhang
b146555749 Revert "Implement return_hidden_states for the OpenAI API (#6137)" (#6440) 2025-05-19 18:21:29 -07:00
Trevor Morris
7adf245ba2 [Metrics] Add KV events publishing (#6098) 2025-05-19 14:19:54 -07:00
Baizhou Zhang
299fd22f9e Fix throughput threshold for amd ci test (#6414) 2025-05-19 14:17:41 -07:00
kyle-pena-kuzco
4f39bcf7ab Implement return_hidden_states for the OpenAI API (#6137) 2025-05-18 22:30:25 -07:00
Chunyuan WU
5dd62c3a6f Add fp8 shared_expert kernel for CPU in sgl-kernel and add UT (#6339) 
Co-authored-by: Jiang, Yanbing <yanbing.jiang@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-18 12:42:15 -07:00
fzyzcjy
f11481b921 Add 4-GPU runner tests and split existing tests (#6383) 2025-05-18 11:56:51 -07:00
libra
11553c1a37 Add pipeline parallelism for Qwen2 and Qwen3 Model (#6250) 2025-05-18 00:42:55 -07:00
Mick
01dd39bac1 refactor: minor refactors regarding multimodal processing (#6187) 2025-05-17 22:53:20 -07:00
fzyzcjy
fd08c04821 Support custom DeepEP tuning config (#6257) 2025-05-17 17:09:42 -07:00
Lifu Huang
3cf1473a09 Use monotonic clock for interval measurement (#6211) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-17 16:49:18 -07:00
Kiv Chen
64825b8395 model(vlm): mistral 3.1 (#5099) 
Co-authored-by: KivenChen <sleigh-queue-0y@icloud.com>
 2025-05-16 18:36:18 -07:00
Lianmin Zheng
c2b7ddca49 [Minor] cleanup unused imports (#6358) 2025-05-16 14:52:38 -07:00
Yury Sulsky
f19a9204cd Support precomputed multimodal features for Qwen-VL and Gemma3 models. (#6136) 
Co-authored-by: Yury Sulsky <ysulsky@tesla.com>
 2025-05-16 12:26:15 -07:00
Chunyuan WU
fb4959b2c5 Add fp8 gemm kernel for CPU in sgl-kernel and add gemm UT (#6216) 
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
 2025-05-15 09:10:40 -07:00
quinnrong94
2e4babdb0a [Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) 
Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>
 2025-05-15 00:48:09 -07:00
Zilin Zhu
44a3783d13 [fix][RL] Remove the incorrect barrier in init_weights_update_group (#5914) 2025-05-14 19:15:21 -07:00
Sai Enduri
73eb67c087 Enable unit tests for AMD CI. (#6283) 2025-05-14 12:55:36 -07:00
Kiv Chen
5380cd7ea3 model(vlm): pixtral (#5084) 2025-05-13 00:16:10 -07:00
Lianmin Zheng
ac2324c177 Skip the flaky test_stateful_custom_logit_processor (#6251) 2025-05-12 18:29:41 -07:00
shangmingc
f1c896007a [PD] Add support for different TP sizes per DP rank (#5922) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-12 13:55:42 -07:00
Sai Enduri
983c663de6 Update AMD nightly deps. (#6241) 2025-05-12 13:39:20 -07:00
Lianmin Zheng
e8e18dcdcc Revert "fix some typos" (#6244) 2025-05-12 12:53:26 -07:00
Ying Sheng
bad7c26fdc [PP] Fix init_memory_pool desync & add PP for mixtral (#6223) 2025-05-12 12:38:09 -07:00
applesaucethebun
d738ab52f8 fix some typos (#6209) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-13 01:42:38 +08:00
shangmingc
3ee40ff919 [CI] Re-enable pd disaggregation test (#6231) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-12 10:09:12 -07:00
Lianmin Zheng
fba8eccd7e Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 00:17:33 -07:00
Lifu Huang
6e2da51561 Replace time.time() to time.perf_counter() for benchmarking. (#6178) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-11 14:32:49 -07:00
Lianmin Zheng
03227c5fa6 [CI] Reorganize the 8 gpu tests (#6192) 2025-05-11 10:55:06 -07:00
Lianmin Zheng
01bdbf7f80 Improve structured outputs: fix race condition, server crash, metrics and style (#6188) 2025-05-11 08:36:16 -07:00
Ravi Theja
41a645f556 Handle empty input string for embedding models (#5621) 
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
 2025-05-11 23:17:15 +08:00
Lianmin Zheng
17c36c5511 [CI] Disabled deepep tests temporarily because it takes too much time. (#6186) 2025-05-10 23:40:50 -07:00
shangmingc
31d1f6e7f4 [PD] Add simple unit test for disaggregation feature (#5654) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-11 13:35:27 +08:00
Emmanuel Ferdman
a823c6e834 Remove duplicate IO Struct test (#6180) 
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
 2025-05-11 12:57:26 +08:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Lianmin Zheng
de167cf5fa Fix request abortion (#6184) 2025-05-10 21:54:46 -07:00
Steven Shimizu
03dd785cd0 Added async_encode method to Engine (#4701) 2025-05-10 18:58:40 -07:00
Yineng Zhang
66fc63d6b1 Revert "feat: add thinking_budget (#6089)" (#6181) 2025-05-10 16:07:45 -07:00
XinyuanTong
9d8ec2e67e Fix and Clean up chat-template requirement for VLM (#6114) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-11 00:14:09 +08:00
thyecust
63484f9fd6 feat: add thinking_budget (#6089) 2025-05-09 08:22:09 -07:00
fzyzcjy
a05bd83a94 Change AMD test threshold (#6091) 2025-05-08 01:05:52 -07:00
fzyzcjy
b6cf3532b5 Tiny refactor ModelConfig.from_server_args (#5219) 2025-05-08 01:02:43 -07:00
XinyuanTong
e88dd482ed [CI]Add performance CI for VLM (#6038) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-07 19:20:03 -07:00
Cheng Wan
9bddf1c82d Deferring 8 GPU test (#6102) 2025-05-07 18:49:58 -07:00
Stefan He
24c13ca950 Clean up fa3 test from 8 gpus (#6105) 2025-05-07 18:38:40 -07:00
Jinyan Chen
8a828666a3 Add DeepEP to CI PR Test (#5655) 
Co-authored-by: Jinyan Chen <jinyanc@nvidia.com>
 2025-05-06 17:36:03 -07:00
Baizhou Zhang
bdd17998e6 [Fix] Fix and rename flashmla CI test (#6045) 2025-05-06 13:25:15 -07:00