EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,272 Commits 8 Branches 0 Tags
6fc93575033676ab754ce579065ae642bf1f98a8
Commit Graph

2178 Commits

Author SHA1 Message Date
Elfie Guo
6fc9357503 [2/2] Add python wrapper for CUTLASS FP8 Blockscale MoE Kernel. (#5694) 2025-05-16 13:14:07 -07:00
Baizhou Zhang
839fb31e5f [Fix] Improve dependencies for Blackwell image (#6334) 2025-05-16 12:38:22 -07:00
Yury Sulsky
f19a9204cd Support precomputed multimodal features for Qwen-VL and Gemma3 models. (#6136) 
Co-authored-by: Yury Sulsky <ysulsky@tesla.com>
 2025-05-16 12:26:15 -07:00
Lianmin Zheng
e07a6977e7 Minor improvements of TokenizerManager / health check (#6327) 2025-05-15 15:29:25 -07:00
Qiaolin Yu
cd8d4b9dfc Fix lora bench (#6302) 2025-05-15 10:09:55 -07:00
fzyzcjy
f194e14fb7 Reduce MoE memory usage (#6147) 2025-05-15 09:38:28 -07:00
Yi Liu
cfc9f9ab8d Fix gpu mem check on CPU (#6317) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
 2025-05-15 09:37:45 -07:00
JieXin Liang
9a405274e2 [misc] remove redundant platform codes (#6298) 2025-05-15 00:51:30 -07:00
quinnrong94
2e4babdb0a [Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) 
Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>
 2025-05-15 00:48:09 -07:00
Zilin Zhu
44a3783d13 [fix][RL] Remove the incorrect barrier in init_weights_update_group (#5914) 2025-05-14 19:15:21 -07:00
Junrong Lin
f3bf611054 feat: add flush cache to EngineBase and HttpServerEngineAdapter (#6009) 2025-05-14 19:15:02 -07:00
Hubert Lu
198b9056d1 [AMD] Fix Llama 4 Scout and Maverick accuracy issues on MI300X (#6274) 2025-05-14 22:07:29 +00:00
Lifu Huang
3e350a931e [Bug] Fix accidental logger override caused by internVL. (#6282) 2025-05-13 23:29:25 -07:00
Ying Sheng
fb71725c98 Fix a bug in schedule_policy (#6276) 2025-05-13 18:04:00 -07:00
Chang Su
912788c095 perf: optimize local_block_table memory allocation (#6273) 2025-05-13 17:18:38 -07:00
Yineng Zhang
16267d4fa7 chore: bump v0.4.6.post4 (#6245) 2025-05-13 01:57:51 -07:00
JieXin Liang
17299f088a [misc] deep_gemm fallback to NVRTC when NVCC not found (#6252) 2025-05-13 01:41:35 -07:00
Kiv Chen
5380cd7ea3 model(vlm): pixtral (#5084) 2025-05-13 00:16:10 -07:00
Cheng Wan
b2e95f62b4 Fix two issues related to --moe-dense-tp-size=1 (#5657) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: 颉沆 <xiehang.lsy@alibaba-inc.com>
 2025-05-12 23:51:39 -07:00
Stefan He
1ab14c4c5c [VERL Use Case] Add torch_memory_saver into deps (#6247) 2025-05-12 19:09:03 -07:00
Lianmin Zheng
ac2324c177 Skip the flaky test_stateful_custom_logit_processor (#6251) 2025-05-12 18:29:41 -07:00
Yineng Zhang
f24fc5b86d fix typo (#6248) 2025-05-12 15:45:12 -07:00
Lianmin Zheng
d18c6b3358 Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 14:33:38 -07:00
shangmingc
f1c896007a [PD] Add support for different TP sizes per DP rank (#5922) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-12 13:55:42 -07:00
Yineng Zhang
f94543d22b chore: add hf_xet dep (#6243) 2025-05-12 13:08:40 -07:00
Lianmin Zheng
e8e18dcdcc Revert "fix some typos" (#6244) 2025-05-12 12:53:26 -07:00
Ying Sheng
bad7c26fdc [PP] Fix init_memory_pool desync & add PP for mixtral (#6223) 2025-05-12 12:38:09 -07:00
applesaucethebun
d738ab52f8 fix some typos (#6209) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-13 01:42:38 +08:00
Lianmin Zheng
fba8eccd7e Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-05-12 00:17:33 -07:00
Cheng Wan
25c83fff6a Performing Vocabulary Parallelism for LM Head across Attention TP Groups (#5558) 
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
 2025-05-11 23:36:29 -07:00
Xiaoyu Zhang
9f2c9568f0 [doc] add a note for --n-share-experts-fusion args (#6154) 2025-05-11 23:18:38 -07:00
fzyzcjy
3f2702ae51 Fix start_profile does not support with_stack and record_shapes (#6043) 2025-05-11 23:11:32 -07:00
Lifu Huang
6e2da51561 Replace time.time() to time.perf_counter() for benchmarking. (#6178) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-11 14:32:49 -07:00
Lianmin Zheng
03227c5fa6 [CI] Reorganize the 8 gpu tests (#6192) 2025-05-11 10:55:06 -07:00
Lianmin Zheng
01bdbf7f80 Improve structured outputs: fix race condition, server crash, metrics and style (#6188) 2025-05-11 08:36:16 -07:00
Ravi Theja
41a645f556 Handle empty input string for embedding models (#5621) 
Co-authored-by: Ravi Theja Desetty <ravitheja@Ravis-MacBook-Pro.local>
 2025-05-11 23:17:15 +08:00
Yineng Zhang
230106304d chore: upgrade sgl-kernel v0.1.2.post1 (#6196) 
Co-authored-by: alcanderian <alcanderian@gmail.com>
 2025-05-11 22:41:37 +08:00
Yusong Gao
41273fd71f fix: handle None multimodal_inputs during merging and filtering batches in disaggregation decode mode (#6169) 2025-05-11 00:28:21 -07:00
JieXin Liang
e9bebafb19 [fix] remove mixtral from is_fa3_default_architecture (#6191) 2025-05-11 00:15:54 -07:00
Yineng Zhang
4d1c9db66c feat: support loogle eval (#6190) 2025-05-10 23:52:44 -07:00
shangmingc
31d1f6e7f4 [PD] Add simple unit test for disaggregation feature (#5654) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-05-11 13:35:27 +08:00
applesaucethebun
2ce8793519 Add typo checker in pre-commit (#6179) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-05-11 12:55:00 +08:00
Lianmin Zheng
de167cf5fa Fix request abortion (#6184) 2025-05-10 21:54:46 -07:00
Lianmin Zheng
4319978c73 Fix data parallel perf regression (#6183) 2025-05-10 19:18:35 -07:00
Steven Shimizu
03dd785cd0 Added async_encode method to Engine (#4701) 2025-05-10 18:58:40 -07:00
Yineng Zhang
66fc63d6b1 Revert "feat: add thinking_budget (#6089)" (#6181) 2025-05-10 16:07:45 -07:00
XinyuanTong
9d8ec2e67e Fix and Clean up chat-template requirement for VLM (#6114) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-05-11 00:14:09 +08:00
JieXin Liang
c178abdabc [fix] fix determine_n_share_experts_fusion (#6118) 2025-05-10 01:19:09 -07:00
Simon (Jiyou) Li
b29a026e14 KV‑Cache (MHA, MLA): add missing start_layer / end_layer fields to MHATokenToKVPoolHost and MLATokenToKVPoolHost (#6016) 
Co-authored-by: 继优 <jiyou.ljy@alibaba-inc.com>
Co-authored-by: chus-chus <chus-chus@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-05-09 15:50:06 -07:00
Yineng Zhang
678d8cc987 chore: bump v0.4.6.post3 (#6165) 2025-05-09 15:38:47 -07:00