EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,912 Commits 8 Branches 0 Tags
40e5cb7a9c0e3ff5ef711e1cee0bdbe9714bcf97
Commit Graph

2656 Commits

Author SHA1 Message Date
Chunyuan WU
40e5cb7a9c [CPU] Bind threads and numa node for each TP rank (#6549) 
Co-authored-by: srinarayan-srikanthan <srinarayan.srikanthan@intel.com>
 2025-07-02 19:57:59 -07:00
Xiaoyu Zhang
8e64140e35 [b200] support trt-llm allreduce fuse rms_norm_add kernel (#7621) 2025-07-02 19:36:20 -07:00
Zilin Zhu
0626f678de [RL] support update_weights_from_distributed with different group and multiple weights (#7292) 2025-07-02 19:29:11 -07:00
Zilin Zhu
09e699bba4 [RL] add --skip-warmup (#7416) 2025-07-02 18:50:43 -07:00
Baizhou Zhang
88f484ce4c Apply dsv3 router gemm kernel for deepseek-r1 fp4 (#7677) 2025-07-02 12:30:18 -07:00
AniZpZ
8e03b641ba [1/n] apply wna16marlin kernel in moe weight only quantization (#7683) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: 弋云 <yiyun.wyt@antgroup.com>
Co-authored-by: walker-ai <2398833647@qq.com>
 2025-07-01 23:21:25 -07:00
Kyungmin Lee
b3fa5dc3c8 Fix GPTQMarlinMoE (#7697) 2025-07-01 22:34:43 -07:00
Ke Bao
00aec6ad6c Apply dsv3_fused_a_gemm kernel (#7635) 2025-07-01 22:32:05 -07:00
Lifu Huang
1a08358aed Improve error handling for requests with unloaded LoRA path(s) (#7642) 2025-07-01 20:05:34 -07:00
Yineng Zhang
f18a8fddd4 chore: upgrade flashinfer v0.2.7.post1 (#7698) 2025-07-01 14:05:57 -07:00
Simon_CQK
a7efbb2757 fix(model loader): use safe_open to prevent file handle leaks. (#7684) 2025-07-01 13:18:35 -07:00
Zhiqiang Xie
f9eb04ddb2 upgrade sgl kernel to 0.2.1 for main (#7676) 2025-07-01 00:00:13 -07:00
Xinyuan Tong
3a911b854d Refactor mm processors and Enable mixed modality processing (#7629) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-30 23:14:48 -07:00
lukec
886d344964 support llama4 eagle3 (#6985) 
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
Co-authored-by: Shenggui Li <somerlee.9@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-06-30 22:34:10 -07:00
narutolhy
3e34e9004f Fix: sync prepare_fp8_layer_for_marlin with latest vllm changes (#7648) 2025-06-30 21:51:01 -07:00
Yineng Zhang
392e441ad1 chore: upgrade flashinfer v0.2.7 jit (#7663) 2025-06-30 13:26:26 -07:00
Lianmin Zheng
22352d47a9 Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-06-29 23:16:19 -07:00
Chunyuan WU
c5131f7a2f [CPU] add c++ kernel to bind CPU cores and memory node (#7524) 2025-06-29 19:45:25 -07:00
Lianmin Zheng
78700893ee [EAGLE] remove a wrong adjustment for page_size > 1 & topk > 1 in server_args.py (#7643) 2025-06-29 19:25:28 -07:00
JieXin Liang
b691dcc490 [misc] reduce weird rope_scaling_factor warning (#7176) 2025-06-29 15:42:45 -07:00
fzyzcjy
0c9c6c75a8 Move files related to EPLB (#7580) 2025-06-29 15:39:38 -07:00
Xinyuan Tong
8f335b5bd6 Fix stream reasoning parser and Adds Kimi reasoning parser (#7432) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-29 14:39:05 -07:00
Lianmin Zheng
071a1f51ae [Minor] clean up multimodal processor and tokenizer manager (#7624) 2025-06-29 02:50:14 -07:00
Xinyuan Tong
c45e49d817 oai: Adds support for OpenAI chat completions API in bench_serving (#7036) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-06-28 22:59:20 +00:00
fzyzcjy
00c7b1ad07 Let EP prefill support new DeepGEMM (#7310) 2025-06-28 01:45:30 -07:00
fzyzcjy
82eccae44e Let ep_scatter support arbitrary strides / ue8m0 format (#7309) 2025-06-28 01:38:33 -07:00
Shangming Cai
eb429b88a4 [PD] Respect sampling_params.max_new_tokens when PD disaggregation is activated (#7598) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-06-27 22:22:01 -07:00
Lifu Huang
49538d111b Support dynamic LoRA loading / unloading in engine/server API (#7446) 2025-06-27 21:00:27 -07:00
Sheng Qi
cfe2edac38 [BUG] fix local_rank in initialize_dp_attention (#7584) 2025-06-27 20:01:01 -07:00
Lifu Huang
2373faa317 Fix flakiness in LoRA batch test. (#7552) 2025-06-27 19:51:43 -07:00
fzyzcjy
9efb2993da Tiny add logs for expert location updater (#7308) 2025-06-27 19:12:33 -07:00
Chunyuan WU
a5317b2fd3 [CPU] add optimizations for INT8 and FP8 DeepSeek (#6769) 
Co-authored-by: Zheng, Beilei <beilei.zheng@intel.com>
 2025-06-27 19:04:29 -07:00
tarinkk
eb6c2c1663 Hybrid kv cache for LLaMA4 (#6563) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-06-27 18:58:55 -07:00
Xinyuan Tong
357921aa51 Fix: Minicpm (#7612) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-27 17:32:29 -07:00
Lianmin Zheng
ce3a3e8783 Move multimodal processors into a separate folder (#7581) 2025-06-27 11:58:24 -07:00
Qiaolin Yu
41650b0d70 feat: support compatibility between MTP and two-batch-overlap (#7225) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-06-27 01:10:27 -07:00
Xinyuan Tong
1b95162008 Updates transformers and timm dependencies (#7577) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-27 00:30:17 -07:00
Keyang Ru
29bd4c8135 [CI] Add CI Testing for Prefill-Decode Disaggregation with Router (#7540) 2025-06-27 00:18:56 -07:00
fzyzcjy
3d7cdb2ebd Fix MTP error when enabling two-batch overlap (#7569) 2025-06-26 15:40:54 -07:00
Xinyuan Tong
604efe07e1 Updates Gemma3n MLP layer to adapt latest transformers version (#7573) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-26 15:07:22 -07:00
Cheng Wan
1b8cf77b01 [Fix] incorrect assert in EPLB (#7575) 2025-06-26 14:59:20 -07:00
Trevor Morris
bb9b608c86 [PD][NIXL] Set is_sorted=False to fix NIXL_ERR_NOT_FOUND (#7330) 2025-06-26 10:39:39 -07:00
Yineng Zhang
69183f8808 chore: bump v0.4.8.post1 (#7559) 2025-06-26 02:21:12 -07:00
Xinyuan Tong
9b00990bea chore: remove vlm unnecessary import (#7541) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-06-26 01:38:15 -07:00
Meng, Peng
2390a2bc8d Add Tencent HunYuanMoEV1 model support (#7549) 2025-06-25 20:59:53 -07:00
Shangming Cai
5c2142579a [PD] Raise error for incompatible mooncake version and some minor fixes (#7527) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-06-25 18:55:24 -07:00
Qiaolin Yu
b8df43ab9c Fix gathered_buffer issues in tbo (#7531) 2025-06-25 14:42:21 -07:00
Yuhong Guo
a1c1ebe935 Fix FP8 KV Cache Support in FA3 Backend (#7148) 2025-06-25 02:14:40 -07:00
mlmz
fe2a0f962f minor: 'role' must be system/assistant/tool, but case insensitive for now (#7499) 2025-06-25 02:11:03 -07:00
eigen
20beb3702b feat: add return hidden_states at async generation (#7507) 2025-06-25 02:10:09 -07:00