EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
2,553 Commits 8 Branches 0 Tags
26f07294f1559d093e322c713478e785f8b684bf
Commit Graph

1697 Commits

Author SHA1 Message Date
fzyzcjy
26f07294f1 Warn users when release_memory_occupation is called without memory saver enabled (#4566) 2025-03-26 00:18:14 -07:00
Baizhou Zhang
34e07a65f1 [Fix] Fix unexpected idx bug of Phi-3-small (#4728) 2025-03-25 21:33:48 -07:00
fzyzcjy
15ddd84322 Add retry for flaky tests in CI (#4755) 2025-03-25 16:53:12 -07:00
fzyzcjy
52029bd1e3 Fix warmup error when dp=1 (#4753) 2025-03-25 02:01:21 -07:00
fzyzcjy
eb934bdf3b Fix test_expert_distribution failure (#4752) 2025-03-25 01:17:03 -07:00
DarkSharpness
ac3fae8445 [Feature] Support "strict" in function calling (#4310) 2025-03-24 22:15:25 -07:00
yuhsaun-t
199bb01d00 Add endpoints to dump selected expert ids (#4435) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-03-24 21:34:19 -07:00
fzyzcjy
6b7038babd Speedup warmup when DP > 1 (#4695) 2025-03-24 21:08:05 -07:00
lukec
57eec0bfbc fix FlashMLA cudagraph config (#4691) 
Co-authored-by: yinfan98 <1106310035@qq.com>
 2025-03-24 21:06:58 -07:00
fzyzcjy
f01b092519 Super tiny fix typo (#4738) 2025-03-24 21:05:45 -07:00
Mick
1e86457c90 model: Minicpmo (#3023) 2025-03-24 20:08:40 -07:00
Ximingwang-09
22c3702e1e [Model] Support Qwen2ForSequenceClassification (#4609) 
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
 2025-03-24 19:13:44 -07:00
Stefan He
4c584fc632 Fix circular imports in gptq.py and unblock test explorer (#4736) 2025-03-24 18:07:08 -07:00
Ke Bao
77cf771ebe Fix EAGLE3 for llama3.3 70b (#4716) 2025-03-24 17:31:19 -07:00
Xuchun Shang
8154de5a32 [PD] Remove invalid parameter (#4721) 
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
 2025-03-24 13:14:16 -07:00
Yineng Zhang
c11cfda07b update pyproject (#4731) 2025-03-24 09:50:28 -07:00
Yuhong Guo
64edeb798f Support dynamic version name in sglang's pyproject.toml (#4720) 2025-03-24 08:56:31 -07:00
Stefan He
5d7edc8e55 Support FA3 as Attention backend by using --attention-backend fa3 (#4680) 
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
 2025-03-23 23:28:11 -07:00
Alex Sun
af6535e7aa [ROCm] Enable MTP (NextN) on AMD GPU (#4631) 2025-03-23 22:58:05 -07:00
c1lovez1
93cf7fc5cd Unify variable naming: replace is_in_free_group with is_not_in_free_group (#4698) 2025-03-23 21:51:08 -07:00
Kyungmin Lee
2a206b22ed Fix RotaryEmbedding when using Triton backend for EXAONE-3.5-2.4B (#4064) 2025-03-23 17:58:12 -07:00
Zhiqiang Xie
4d25305700 Move mem_state update into debug mode (#4525) 2025-03-23 00:52:27 -07:00
Mick
11577cedb7 refactor: bug fixes and refactor for vlm (#4661) 2025-03-22 22:48:49 -07:00
fzyzcjy
ca75741e86 Support async in DeepEP (#4610) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
 2025-03-22 22:39:56 -07:00
fzyzcjy
c6d549e773 Multiple tiny code cleanups (#4608) 2025-03-22 22:39:11 -07:00
xutizhou
c2bd094d6e Optimize Permute Kernel in DeepEP (#4643) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-03-22 14:30:34 -07:00
Adarsh Shirawalmath
f8f9244a61 [Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984) 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
 2025-03-22 14:27:39 -07:00
Zhiqiang Xie
ecbfe58bb0 Bug fix for metrics counter (#4660) 2025-03-22 13:39:21 -07:00
Yun Dai
8cd4250401 [quantization] fix channelwise conversion with scalar weight scale (#4596) 2025-03-22 00:47:52 -07:00
mlmz
f6ab4ca6bc fix: fix ipython running error for Engine due to outlines nest_asyncio (#4582) 
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
 2025-03-21 19:11:15 -07:00
Byron Hsu
c7c7dbebbe [PD] Release initial code (#4654) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai
 2025-03-21 14:47:47 -07:00
Yuhong Guo
417fc72f6f Align completion and chat_completion response to OpenAI API (#4637) 2025-03-20 22:59:04 -07:00
lukec
4c56e5dbee Set deepgemm to the default value in the hopper architecture. (#4613) 2025-03-20 22:03:00 -07:00
Cheng Wan
7b5fc71972 fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640) 2025-03-20 21:45:07 -07:00
Yuhong Guo
b36ab493b3 Enable setting sglang logger from Env Variable SGLANG_LOGGING_CONFIG_PATH (#4592) 
Signed-off-by: Yuhong Guo <yuhong.gyh@antgroup.com>
 2025-03-20 02:10:32 -07:00
JieXin Liang
9e93ef3f8e [fix] fix illegal mem access and clean up triton attention backend (#4571) 2025-03-20 02:01:52 -07:00
Chuyue Sun
fad86a6863 Support n in OpenAI API completions (#3446) 
Co-authored-by: Shan Yu <shanyu1@g.ucla.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: chuyue sun <chuyue@lmsys.us-northcentral1-a.compute.internal>
 2025-03-20 13:46:46 +08:00
strgrb
df7014a8d2 avoid cudaStreamSynchronize in DeepSeekV2AttentionMLA (#4577) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-03-19 10:02:26 -07:00
JieXin Liang
4942074174 [fix] fix initialization of _ENABLE_TORCH_INFERENCE_MODE (#4549) 2025-03-19 09:57:59 -07:00
lukec
b6944f97a6 Support FlashMLA backend cuda graph (#4514) 
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
Co-authored-by: ispobock <ispobaoke@163.com>
 2025-03-19 08:25:34 -07:00
Jinyan Chen
f44db16c8e [Feature] Integrate DeepEP into SGLang (#4232) 
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
 2025-03-19 08:16:31 -07:00
strgrb
f9c53cbb42 Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515) 
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
 2025-03-19 00:02:43 -07:00
Baizhou Zhang
90532b7627 [Fix] Fix raw_bs bug when using flashinfer mla and eagle (#4557) 2025-03-18 21:26:53 -07:00
JieXin Liang
c0e9a36c5f Optimize Triton decoding kernel for dynamic workload (#4553) 2025-03-18 21:25:38 -07:00
aoshen524
588865f0e0 [Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274) 
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-03-18 20:33:07 -07:00
Cheng Wan
3196999f63 Reduce computation and communication in DP attention (#4521) 2025-03-18 13:41:36 -07:00
James Liu
9e0186f352 [Feature] Support EAGLE 3 (#4247) 2025-03-18 07:35:23 -07:00
Wei Wu
8baf9a0c18 [Fix] Type annotation correction for UpdateWeightsFromTensorReqInput (#4532) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-03-18 00:52:47 -07:00
Yineng Zhang
c787298547 use sgl custom all reduce (#4441) 2025-03-18 00:46:41 -07:00
Yineng Zhang
c16b33ccac cleanup deps 3/n (#4541) 2025-03-18 00:11:36 -07:00