EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
2,860 Commits 8 Branches 0 Tags
8f783c1943af25e5bbccff628ba4385579b044e1
Commit Graph

39 Commits

Author SHA1 Message Date
Baizhou Zhang
a42736bbb8 Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113) 2025-04-15 22:01:22 -07:00
Mick
34ef6c8135 [VLM] Adopt fast image processor by default (#5065) 2025-04-11 21:46:58 -07:00
Baizhou Zhang
efbae697b3 [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) 2025-04-05 01:23:02 -07:00
Lianmin Zheng
74885a848b Revert "Replace enable_flashinfer_mla argument with attention_backend" (#5048) 2025-04-03 13:30:56 -07:00
Baizhou Zhang
e8999b13b7 Replace enable_flashinfer_mla argument with attention_backend (#5005) 2025-04-03 02:53:58 -07:00
Jinyan Chen
23c764b18a [Feature] Support DeepEP Low Latency (#4767) 
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: ch-wan <cwan39@gatech.edu>
 2025-04-01 09:23:25 -07:00
tarinkk
7f19e083c1 Support (1 <= dp < tp) in the dp attention in DeepEP (#4770) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
 2025-03-27 17:09:35 -07:00
Jiří Suchomel
f60f293195 [k8s] Clarified the usage of shared memory. (#4341) 2025-03-27 08:53:19 -07:00
Jinyan Chen
f44db16c8e [Feature] Integrate DeepEP into SGLang (#4232) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
 2025-03-19 08:16:31 -07:00
Chayenne
e1a5e7e47d docs: hot fix torch compile cache (#4442) 2025-03-14 19:05:59 -07:00
Jun Liu
14344caa38 [docs] Update outdated description about torch.compile (#3844) 2025-03-12 22:09:38 -07:00
Peter Pan
016033188c docs: add parameter --log-requests-level (#4335) 2025-03-12 21:19:37 -07:00
Chayenne
ebddb65aed Docs: add torch compile cache (#4151) 
Co-authored-by: ybyang <ybyang7@iflytek.com>
 2025-03-06 14:27:09 -08:00
samzong
d2d0d061d9 fix cross-reference error and spelling mistakes (#4101) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-03-05 16:39:02 -08:00
Qiaolin Yu
357671e216 Add examples for server token-in-token-out (#4103) 
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
 2025-03-05 13:16:31 -08:00
Baizhou Zhang
fc91d08a8f [Revision] Add fast decode plan for flashinfer mla (#4012) 2025-03-05 11:20:41 -08:00
Mick
583d6af71b example: add vlm to token in & out example (#3941) 
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
 2025-03-04 22:18:26 -08:00
Chayenne
146ac8df07 Add examples in sampling parameters (#4039) 2025-03-03 13:04:32 -08:00
Lianmin Zheng
935cda944b Misc clean up; Remove the support of jump forward (#4032) 2025-03-03 07:02:14 -08:00
Lianmin Zheng
ac2387279e Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
 2025-03-03 00:12:04 -08:00
Chayenne
728e175fc4 Add examples to token-in-token-out for LLM (#4010) 2025-03-02 21:03:49 -08:00
Lianmin Zheng
9e1014cf99 Revert "Add fast decode plan for flashinfer mla" (#4008) 2025-03-02 19:29:10 -08:00
Baizhou Zhang
fa56106731 Add fast decode plan for flashinfer mla (#3987) 2025-03-02 19:16:37 -08:00
Zhousx
7fbab730bd [feat] add small vocab table for eagle's draft model[1]. (#3822) 
Co-authored-by: Achazwl <323163497@qq.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-03-02 18:58:45 -08:00
Baizhou Zhang
90a4b7d98a [Feature]Support ragged prefill in flashinfer mla backend (#3967) 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
 2025-02-28 18:13:56 -08:00
Baizhou Zhang
3e02526b1f [Doc] Add experimental tag for flashinfer mla (#3925) 2025-02-27 01:55:36 -08:00
Baizhou Zhang
71ed01833d [doc] Update document for flashinfer mla (#3907) 2025-02-26 20:40:45 -08:00
simveit
44a2c4bd56 Docs: improve link to docs (#3860) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-02-26 10:29:25 -08:00
Yuanheng Zhao
3758d209a0 [Doc] Fix typo in server-argument description (#3641) 2025-02-24 16:57:13 -08:00
Baizhou Zhang
ac05310098 [Docs] Modify ep related server args and remove cublas part of deepseek (#3732) 2025-02-21 03:37:56 +08:00
Mick
7711ac6ed0 doc: emphasize and notify the usage of chat_template (#3589) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-02-15 00:10:32 -08:00
Didier Durand
1e2cf2b541 fix server_arguments typo (#3499) 2025-02-12 02:59:53 +08:00
Baizhou Zhang
70817a7eae [Feature] Define backends and add Triton backend for Lora (#3161) 
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
 2025-02-03 22:09:13 -08:00
simveit
7b5a374114 Update server args doc (#3273) 
Co-authored-by: Shi Shuai <126407087+shuaills@users.noreply.github.com>
 2025-02-03 23:39:41 +00:00
simveit
1c4e0d2445 Docs: Update doc for server arguments (#2742) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
 2025-01-23 11:32:05 -08:00
Lianmin Zheng
8b6ce52e92 Support multi-node DP attention (#2925) 
Co-authored-by: dhou-xai <dhou@x.ai>
 2025-01-16 11:15:00 -08:00
Chayenne
2e6346fc2e Docs：Update the style of llma 3.1 405B docs (#2789) 2025-01-08 01:07:54 -08:00
mlmz
977f785dad Docs: Rewrite docs for LLama 405B and ModelSpace (#2773) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-01-08 00:02:59 -08:00
Shi Shuai
062c48d2bd [Docs] Add Support for Pydantic Structured Output Format (#2697) 2025-01-01 15:08:43 -08:00