JieXin Liang
|
51ac297ace
|
[feat] interface for platforms abstraction (#4928)
|
2025-03-31 00:04:21 -07:00 |
|
Zhiqiang Xie
|
a169b9f813
|
Fix oom error for large page size (#4913)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-03-30 21:34:21 -07:00 |
|
Baizhou Zhang
|
4a63bc32b7
|
[Fix] Add torch compile for torch.clamp back (#4936)
|
2025-03-30 20:46:07 -07:00 |
|
fzyzcjy
|
a303325fdb
|
Fix DeepSeek bug causing 2.2% MMLU drop when TP!=DP (#4883)
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-03-30 20:10:21 -07:00 |
|
Baizhou Zhang
|
e62d60fe6d
|
[Fix] avoid stream sync and torch compile in prefill for fa3 backend (#4932)
|
2025-03-30 13:53:44 -07:00 |
|
SEPLOS
|
032f8faaab
|
Fix sglang frontend's incorrect dependency on torch (#4931)
|
2025-03-30 13:00:24 -07:00 |
|
Lianmin Zheng
|
9adf178cc2
|
Fix 2-gpu CI test and suppress some warnings (#4930)
|
2025-03-30 12:51:44 -07:00 |
|
Lianmin Zheng
|
4ede6770cd
|
Fix retract for page size > 1 (#4914)
|
2025-03-30 02:57:15 -07:00 |
|
Lianmin Zheng
|
b26bc86b36
|
Support page size > 1 + eagle (#4908)
|
2025-03-30 00:46:23 -07:00 |
|
fzyzcjy
|
8690c40bb0
|
Improve stack trace of retry errors (#4845)
|
2025-03-29 08:21:31 -07:00 |
|
fzyzcjy
|
b1cfb4e972
|
Fix BadRequestError wrong arguments and remove openai dependency (#4882)
|
2025-03-29 08:16:21 -07:00 |
|
Yineng Zhang
|
19e96e5923
|
bump v0.4.4.post3 (#4878)
|
2025-03-28 23:21:24 -07:00 |
|
Yineng Zhang
|
d8a136a113
|
upgrade sgl-kernel 0.0.5.post4 (#4873)
|
2025-03-28 19:48:56 -07:00 |
|
Baizhou Zhang
|
20c90be23d
|
[Feature] Support FA3 backend for MLA (#4831)
|
2025-03-28 18:30:14 -07:00 |
|
Qingquan Song
|
044c315970
|
Make torch compile configurable for biased_grouped_topk (#4749)
|
2025-03-28 10:57:52 -07:00 |
|
Fr4nk1in
|
c483377ed7
|
Fix wrong variable name when stopping memory profile (#4772)
|
2025-03-28 10:35:02 -07:00 |
|
Lianmin Zheng
|
74e0ac1dbd
|
Clean up import vllm in quantization/__init__.py (#4834)
|
2025-03-28 10:34:10 -07:00 |
|
chaobo jia
|
ef9a378a20
|
[Feature] add multi-rank support for Lora (#4492)
Co-authored-by: rudy152 <czh1137892874@gmail.com>
|
2025-03-28 09:38:44 -07:00 |
|
Qingquan Song
|
6ffb6bd47a
|
Fix fa3 cuda graph page_size > 1 precision and page_size=1 speed (#4855)
|
2025-03-28 01:35:59 -07:00 |
|
Lianmin Zheng
|
47e6628aae
|
Fix CI tests (#4853)
|
2025-03-28 00:28:35 -07:00 |
|
fzyzcjy
|
8c04f0f2e1
|
Support with_stack and record_shapes in profiler (#4740)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-03-27 23:01:42 -07:00 |
|
fzyzcjy
|
265e756494
|
Super tiny remove unused code (#4750)
|
2025-03-27 22:32:14 -07:00 |
|
fzyzcjy
|
d3f71f5e19
|
Fix torch.cuda.MemPool() internal assertion failure (#4687)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-03-27 22:29:36 -07:00 |
|
Kebe
|
e0166f8ab4
|
Remove empty tool function name (#4704)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-27 22:23:30 -07:00 |
|
fzyzcjy
|
53a2c3b466
|
Support controlling nsys start and end range programmatically (#4688)
|
2025-03-27 22:21:13 -07:00 |
|
BroadbentJim
|
550586ef42
|
fix: Inappropriate lack of Optional type on OpenAI ChatCompletionRequest (#4681)
|
2025-03-27 22:19:05 -07:00 |
|
fzyzcjy
|
cf29fe9e78
|
Fix Engine error when enabling DP attention (#4648)
|
2025-03-27 22:17:30 -07:00 |
|
Stefan He
|
26c0f13126
|
Support Page Size > 1 for FA3 (#4832)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-03-27 22:07:14 -07:00 |
|
rongfu.leng
|
f9970bd1af
|
fix: when use SGLANG_PORT this env,port is str (#4528)
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
|
2025-03-27 21:46:06 -07:00 |
|
lambert0312
|
2e0f94ab79
|
[Fix] fix output_top_logprobs is not exist (#4597)
|
2025-03-27 21:45:57 -07:00 |
|
Vincent
|
e2e2ab70e0
|
IPv6 support (#3949)
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-03-27 21:42:13 -07:00 |
|
Jiaqi
|
72031173e4
|
fix: fix typo of comments in w8a8_fp8.py (#4843)
|
2025-03-27 21:06:47 -07:00 |
|
Qiaolin Yu
|
9fdc6d6abc
|
Fix the lora adapter when lora path is none (#4799)
Co-authored-by: Beichen Ma <mabeichen12@gmail.com>
|
2025-03-27 21:03:08 -07:00 |
|
XinyuanTong
|
42a45df043
|
[Fix] self.worker assignment in TpModelWorker and refactor references (#4788)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-03-27 20:28:38 -07:00 |
|
Jon Durbin
|
04eb6062e4
|
Include context length in /v1/models response. (#4809)
|
2025-03-27 20:23:18 -07:00 |
|
Brayden Zhong
|
e84f4ba0ab
|
[Misc] Fix issues reported by torchfix (#4837)
|
2025-03-27 20:10:32 -07:00 |
|
Kebe
|
10a9ab7b07
|
Fix error due to CustomAllreduce setup failure (#4815)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-27 18:52:10 -07:00 |
|
Junrong Lin
|
bb0fd749a6
|
[Fix] Add compressed_tensors as deps (#4819)
|
2025-03-27 18:08:24 -07:00 |
|
tarinkk
|
7f19e083c1
|
Support (1 <= dp < tp) in the dp attention in DeepEP (#4770)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-27 17:09:35 -07:00 |
|
Juwan Yoo
|
188105a21b
|
deps: lazy import optional dependencies gguf and torchvision (#4826)
|
2025-03-27 14:35:36 -07:00 |
|
Yineng Zhang
|
bbab97a6a8
|
add partial_json_parser and einops (#4827)
|
2025-03-27 13:24:54 -07:00 |
|
Juwan Yoo
|
0bc0bf5734
|
gemma3: impl get_attention_sliding_window_size for attn init (#4823)
|
2025-03-27 10:43:58 -07:00 |
|
AinL
|
17000d2b3a
|
Remove Unintended Capture Batch Sizes in AMD HIP Graph Runner (#4638)
|
2025-03-27 08:41:33 -07:00 |
|
strgrb
|
668ecc6c5b
|
Fix ut mla-test-1-gpu-amd (#4813)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 08:27:51 -07:00 |
|
strgrb
|
886fcbdd09
|
Use apply_rope_with_cos_sin_cache_inplace for DeepSeek (#4764)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 01:45:37 -07:00 |
|
Stefan He
|
1b9175cb23
|
[FA3 Attn Backend] Remove Unnecessary Device Sync for FA3 (#4745)
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
|
2025-03-27 00:45:11 -07:00 |
|
fzyzcjy
|
92bb49a7f9
|
Patch PyTorch's bug that cross-process tensor transfer will lead to wrong device (#4565)
|
2025-03-27 00:22:33 -07:00 |
|
Yineng Zhang
|
6f5cc5eb05
|
update xgrammar 0.1.17 (#4804)
|
2025-03-27 00:21:59 -07:00 |
|
Pan Lyu
|
c913ed4046
|
support clip embedding model (#4506)
|
2025-03-27 00:18:15 -07:00 |
|
Xihuai Wang
|
1afe3d0798
|
Align finish reason and stream mode in openai api (#4388)
|
2025-03-27 00:16:52 -07:00 |
|