EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
3,929 Commits 8 Branches 0 Tags
c01a1df5888c63a010a26ddb297e052d96da7ad7
Commit Graph

769 Commits

Author SHA1 Message Date
ronnie_zheng
1e0e549766 Ascend attention backend(PA&MLA) (#7722) 
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: VDV1985 <vladdv85@mail.ru>
 2025-07-03 09:23:19 -07:00
Albert
d3c275b117 Support updating weights at once by stopping all requests (#6698) 
Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
Co-authored-by: Zilin Zhu <zhuzilinallen@gmail.com>
 2025-07-02 22:26:06 -07:00
YanbingJiang
b044400dd3 Support non-contiguous query input for extend/decode attention (#7462) 2025-07-02 19:59:45 -07:00
Zilin Zhu
0626f678de [RL] support update_weights_from_distributed with different group and multiple weights (#7292) 2025-07-02 19:29:11 -07:00
Hubert Lu
b116b21a46 [AMD] Temporarily disable test_no_overlap_scheduler and test_vision_chunked_prefill (#7717) 2025-07-02 12:39:18 -07:00
AniZpZ
8e03b641ba [1/n] apply wna16marlin kernel in moe weight only quantization (#7683) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: HandH1998 <1335248067@qq.com>
Co-authored-by: 弋云 <yiyun.wyt@antgroup.com>
Co-authored-by: walker-ai <2398833647@qq.com>
 2025-07-01 23:21:25 -07:00
Lifu Huang
1a08358aed Improve error handling for requests with unloaded LoRA path(s) (#7642) 2025-07-01 20:05:34 -07:00
Lianmin Zheng
22352d47a9 Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-06-29 23:16:19 -07:00
Chunyuan WU
c5131f7a2f [CPU] add c++ kernel to bind CPU cores and memory node (#7524) 2025-06-29 19:45:25 -07:00
Hubert Lu
3b3f1e3aeb [AMD] Add unit-test-sgl-kernel-amd to AMD CI (#7539) 2025-06-29 15:50:09 -07:00
fzyzcjy
0c9c6c75a8 Move files related to EPLB (#7580) 2025-06-29 15:39:38 -07:00
Xinyuan Tong
8f335b5bd6 Fix stream reasoning parser and Adds Kimi reasoning parser (#7432) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-29 14:39:05 -07:00
Yineng Zhang
a8c10aeeee fix unit tests (#7618) 2025-06-28 00:32:41 -07:00
Lifu Huang
49538d111b Support dynamic LoRA loading / unloading in engine/server API (#7446) 2025-06-27 21:00:27 -07:00
Lifu Huang
2373faa317 Fix flakiness in LoRA batch test. (#7552) 2025-06-27 19:51:43 -07:00
Lifu Huang
d7374d7467 Fix broken CI TestVILAServer (#7610) 2025-06-27 15:01:03 -07:00
Lianmin Zheng
ce3a3e8783 Move multimodal processors into a separate folder (#7581) 2025-06-27 11:58:24 -07:00
Qiaolin Yu
41650b0d70 feat: support compatibility between MTP and two-batch-overlap (#7225) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-06-27 01:10:27 -07:00
Ata Fatahi
031f64aa1b Add e2e test for multi instance multi stage memory release/resume occupuation (#7208) 
Signed-off-by: Ata Fatahi <immrata@gmail.com>
 2025-06-26 17:40:38 -07:00
YanbingJiang
0e05fe8cf4 Update seed in CPU UTs to avoid flaky failure with single test (#7544) 2025-06-25 21:25:50 -07:00
Yuhong Guo
a1c1ebe935 Fix FP8 KV Cache Support in FA3 Backend (#7148) 2025-06-25 02:14:40 -07:00
zixuanzhang226
f3cbd24541 feat: send kvmetrics from sglang scheduler (#6721) 2025-06-25 01:57:49 -07:00
Chunyuan WU
7eb47b0f3d [CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641) 
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-06-25 01:43:33 -07:00
Chang Su
112b496a6c misc: Improvement to serving_chat.py and add more ut (#7489) 2025-06-24 17:19:51 -07:00
Chang Su
fa42e41962 ci: Revert openai_server related tests in AMD suites (#7449) 2025-06-23 15:28:22 -07:00
Chang Su
34b6b8426f feat(func_call): Add more check in BaseFormatDetector.parse_streaming_increment (#7479) 2025-06-23 11:15:47 -07:00
Chang Su
b7a2df0a44 refactor(test): reorganize OpenAI test file structure (#7408) 2025-06-21 19:37:48 -07:00
Chang Su
72676cd6c0 feat(oai refactor): Replace openai_api with entrypoints/openai (#7351) 
Co-authored-by: Jin Pan <jpan236@wisc.edu>
 2025-06-21 13:21:06 -07:00
Keyang Ru
5e7fdc79fa [OAI Server Refactor] [ChatCompletions & Completions] Support Return Hidden State (#7329) 
Signed-off-by: keru <rukeyang@gmail.com>
 2025-06-20 19:18:53 -07:00
Cheng Wan
e879d8b7a8 [Feature] Comprehensive Hybrid Parallelism Support (#6389) 2025-06-20 14:43:11 -07:00
Xinyuan Tong
0998808009 Refine OpenAI serving entrypoint to remove batch requests (#7372) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
 2025-06-20 14:33:43 -07:00
Ata Fatahi
1ab6be1b26 Purge VerlEngine (#7326) 
Signed-off-by: Ata Fatahi <immrata@gmail.com>
 2025-06-19 23:47:21 -07:00
woodx
4df5fc2156 Feat/refactor embedding server (#7322) 2025-06-19 23:46:01 -07:00
Stefan He
3774f07825 Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099) 2025-06-19 00:56:37 -07:00
Chunyuan WU
9179ea1595 add seed in CPU UTs to avoid flaky failure (#7333) 2025-06-18 19:12:14 -07:00
Jinn
ffd1a26e09 Add more refactored openai test & in CI (#7284) 2025-06-18 13:52:55 -07:00
YanbingJiang
094c116f7d Update python API of activation, topk, norm and rope and remove vllm dependency (#6614) 
Co-authored-by: Wu, Chunyuan <chunyuan.wu@intel.com>
Co-authored-by: jianan-gu <jianan.gu@intel.com>
Co-authored-by: sdp <sdp@gnr799219.jf.intel.com>
 2025-06-17 22:11:50 -07:00
Chang Su
fc554105f6 ci: Fix test_ebnf_generate_all_optional_function_params (#7288) 2025-06-17 16:39:42 -07:00
Chang Su
e726131523 bugfix(tool call ebnf): Fix EBNF generation for optional function parameters (#7283) 2025-06-17 13:36:07 -07:00
u4lr451
10d60cd41b feat: mtp support dp-attention (#6081) 
Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>
 2025-06-17 00:33:28 -07:00
Xinyuan Tong
70c471a868 [Refactor] OAI Server components (#7167) 
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
 2025-06-16 20:45:20 -07:00
KavioYu
873ae12cee support custom weight loader for model runner (#7122) 
Co-authored-by: kavioyu <kavioyu@tencent.com>
 2025-06-16 16:28:15 -07:00
Sai Enduri
62a7aa2efc Update CI flakes. (#7244) 2025-06-16 15:19:32 -07:00
woodx
e30ef368ab Feat/support rerank (#6058) 2025-06-16 10:50:01 -07:00
Lianmin Zheng
53a525bf33 [Eagle] Fix kernel call after updating speculative sampling kernels (#7231) 2025-06-16 07:25:59 -07:00
Lianmin Zheng
b1286a116a [EAGLE] Refactor code for page size > 1 & more simplifications (#7213) 2025-06-16 03:04:29 -07:00
Lianmin Zheng
fff10809bf Revert "[EAGLE] Refactor code for page size > 1 & more simplifications" (#7210) 2025-06-15 02:48:00 -07:00
Lianmin Zheng
5f1ab32717 [EAGLE] Refactor code for page size > 1 & more simplifications (#7163) 2025-06-14 23:16:23 -07:00
Lianmin Zheng
a023856b12 Move host memory pools into a separate file (#7200) 2025-06-14 21:31:42 -07:00
Byron Hsu
db0cc57e75 [PD] Support decode retract and update decode.py (#7196) 2025-06-14 19:48:05 -07:00