EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
2,368 Commits 8 Branches 0 Tags
91b19949d78880f83eea4a82fd3aa8fcc7504986
Commit Graph

1574 Commits

Author SHA1 Message Date
Wang Ran (汪然)
91b19949d7 typo: Update http_server.py (#4350) 2025-03-12 15:05:30 -07:00
Zhiqiang Xie
10b544ae9b Hierarchical Caching Refactoring and Fixing TP issue (#4082) 2025-03-12 11:22:35 -07:00
Mick
01090e8ac3 model: Support Janus-pro (#3203) 2025-03-12 11:02:11 -07:00
yych0745
6f43a9b9f4 remove the unused readline dependency from the Qwen2 model implementa… (#4340) 2025-03-12 02:47:27 -07:00
JieXin Liang
0540fef7a1 [Fix] fix _yarn_linear_ramp_mask with device parameter (#4337) 2025-03-12 02:28:19 -07:00
lambert0312
481f608b8e Add INT8 support MTP NextN function (#3911) 2025-03-12 01:37:16 -07:00
Yineng Zhang
ed91561f79 upgrade sgl-kernel 0.0.4.post3 (#4334) 2025-03-12 01:36:41 -07:00
Stefan He
e0917e6bd0 Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215) 
Co-authored-by: Stefan He <bhe@linkedin.com>
 2025-03-12 00:08:03 -07:00
lambert0312
7140ba3573 Add A800 tuning configs for DeepSeek R1/V3 channel-wise INT8 (#4323) 2025-03-11 18:25:56 -07:00
Yineng Zhang
d1da58e275 unify is_cuda and is_hip (#4321) 2025-03-11 18:12:56 -07:00
Yineng Zhang
1cf63485c1 upgrade flashinfer 0.2.3 (#4317) 
Co-authored-by:  qingquansong <qsong@linkedin.com>
 2025-03-11 15:37:17 -07:00
Mick
ff2ce0b86f refactor: move image processors to separate files (#4229) 2025-03-11 12:35:35 -07:00
Ximingwang-09
0f2a2e3c19 Add H20 tuning configs support DeepSeek V3/R1 INT8(block-wise) (#4220) 
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
 2025-03-11 12:32:33 -07:00
yigex
690e1f2371 [AMD] Fix rocm sgl-kernel missing modules error (#4311) 
Co-authored-by: yiakwy-xpu-ml-framework-team <leiwang2@amd.com>
 2025-03-11 10:35:28 -07:00
yych0745
6a02b32d07 Add A100 tuning configs for DeepSeek R1/V3 channel-wise INT8 (#4287) 
Co-authored-by: HandH1998 <1335248067@qq.com>
 2025-03-11 00:49:06 -07:00
lukec
dce303e279 linear support deepgemm (#4199) 
Co-authored-by: yinfan98 <1106310035@qq.com>
 2025-03-11 00:38:37 -07:00
Yineng Zhang
4d27eb9ad1 update sgl-kernel 0.0.4.post2 (#4291) 2025-03-11 00:34:33 -07:00
lambert0312
d3ecd63204 Add A800 tuning configs support DeepSeek V3/R1 BF16 and INT8(block-wise) (#4136) 2025-03-11 00:32:25 -07:00
Yineng Zhang
e187a3d595 upgrade xgrammar 0.1.15 (#4275) 2025-03-10 14:53:24 -07:00
HandH1998
2ac189edc8 Amd test fp8 (#4261) 2025-03-10 10:12:09 -07:00
Lianmin Zheng
5a6400eec5 Test no vllm custom allreduce (#4256) 2025-03-10 10:08:25 -07:00
Lianmin Zheng
00d25a7f5e Fix quantization and nightly tests (#4258) 2025-03-10 03:06:21 -07:00
shimin
ac69885056 fix the input_ids is None error (#4144) 2025-03-10 01:38:37 -07:00
Lianmin Zheng
aa957102a9 Simplify tests & Fix trtllm custom allreduce registration (#4252) 2025-03-10 01:24:22 -07:00
DavidChan
4455b26e76 [Bug fixed] fixed the crash when enable the dp-attention on the single card (#3958) 2025-03-10 00:50:34 -07:00
Lianmin Zheng
e8a69e4d0c Clean up fp8 support (#4230) 2025-03-09 21:46:35 -07:00
Lianmin Zheng
fbd560028a Auto balance CI tests (#4238) 2025-03-09 21:05:55 -07:00
Lianmin Zheng
730d084f2a Minor style fix for sgl-kernel (#4243) 2025-03-09 20:15:13 -07:00
Lianmin Zheng
4a05bdfa86 Revert "Check eagle server args" (#4242) 2025-03-09 18:53:33 -07:00
Ying Sheng
34c8898755 Check eagle server args (#4217) 2025-03-09 01:10:43 -08:00
HandH1998
0dd6cda288 Apply sgl w8a8 fp8 kernel (#3148) 2025-03-09 00:03:32 -08:00
Baizhou Zhang
9fb48f951f Support nextn for flashinfer mla attention backend (#4218) 2025-03-09 00:01:54 -08:00
Yineng Zhang
89ccb533ad use sgl-kernel 0.0.4 (#4224) 2025-03-08 23:43:09 -08:00
Lianmin Zheng
1361ab9e03 Lazily import lora backends (#4225) 2025-03-08 23:39:26 -08:00
Lianmin Zheng
8abf74e3c9 Rename files in sgl kernel to avoid nested folder structure (#4213) 
Co-authored-by: zhyncs <me@zhyncs.com>
 2025-03-08 22:54:51 -08:00
Mingshan
0fe7c13be1 Fix bench_serving flush cache not recognizing OPENAI_API_KEY (#4181) 
Signed-off-by: Mingshan <git@brighill.com>
 2025-03-08 01:03:38 -08:00
Lianmin Zheng
08c4d764a5 lazy import attn backends (#4200) 2025-03-08 00:41:35 -08:00
Lianmin Zheng
d4017a6b63 [EAGLE] many fixes for eagle (#4195) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-03-07 22:12:13 -08:00
Lianmin Zheng
d052f4c8a9 New clang format for sgl kernel (#4194) 2025-03-07 20:21:08 -08:00
Ke Bao
20c8119915 Fix eagle hang issue for max_new_tokens=1 (#4185) 2025-03-07 12:11:18 -08:00
Yineng Zhang
eb61f5c9af Revert "ROCm: Flex Attention Enablement with custom backends (#4178)" (#4186) 2025-03-07 10:27:52 -08:00
HAI
0beea4503f ROCm: Flex Attention Enablement with custom backends (#4178) 
Co-authored-by: linsun12 <linsun12@amd.com>
 2025-03-07 04:38:53 -08:00
Zhiqiang Xie
9376ac361d Memory pool fix for upstream change about eagle (#4170) 2025-03-07 00:58:20 -08:00
HandH1998
c7f254468f [Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888) 
Co-authored-by: yych0745 <1398089567@qq.com>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: b0urnee <2769086541@qq.com>
 2025-03-06 20:54:52 -08:00
Pan Lyu
361971b859 Add Support for Qwen2-VL Multi-modal Embedding Models (#3694) 2025-03-06 16:46:20 -08:00
HAI
13bc39c5d6 ROCm: enable trillion-parameter MoE models with INT4-FP8 single node (#4152) 2025-03-06 15:33:02 -08:00
Lianmin Zheng
9c58e68b4c Release v0.4.3.post4 (#4140) 2025-03-06 12:50:28 -08:00
Oliver Stanley
d03b3467b8 Fix constrained generation errors by adding datasets dependency (#4142) 2025-03-06 12:07:51 -08:00
yinfan98
ab7fba0ece Fix nightly ci Gsm8k & Fix flashinfer backend kvcache quant (#4147) 2025-03-06 11:50:07 -08:00
Lianmin Zheng
bc1534ff32 Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134) 
Co-authored-by: Sehoon Kim <kssteven418@gmail.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-03-06 06:13:59 -08:00