Qiaolin Yu
|
9fdc6d6abc
|
Fix the lora adapter when lora path is none (#4799)
Co-authored-by: Beichen Ma <mabeichen12@gmail.com>
|
2025-03-27 21:03:08 -07:00 |
|
XinyuanTong
|
42a45df043
|
[Fix] self.worker assignment in TpModelWorker and refactor references (#4788)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-03-27 20:28:38 -07:00 |
|
Jon Durbin
|
04eb6062e4
|
Include context length in /v1/models response. (#4809)
|
2025-03-27 20:23:18 -07:00 |
|
Brayden Zhong
|
e84f4ba0ab
|
[Misc] Fix issues reported by torchfix (#4837)
|
2025-03-27 20:10:32 -07:00 |
|
Kebe
|
10a9ab7b07
|
Fix error due to CustomAllreduce setup failure (#4815)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-27 18:52:10 -07:00 |
|
Junrong Lin
|
bb0fd749a6
|
[Fix] Add compressed_tensors as deps (#4819)
|
2025-03-27 18:08:24 -07:00 |
|
tarinkk
|
7f19e083c1
|
Support (1 <= dp < tp) in the dp attention in DeepEP (#4770)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-27 17:09:35 -07:00 |
|
Juwan Yoo
|
188105a21b
|
deps: lazy import optional dependencies gguf and torchvision (#4826)
|
2025-03-27 14:35:36 -07:00 |
|
Yineng Zhang
|
bbab97a6a8
|
add partial_json_parser and einops (#4827)
|
2025-03-27 13:24:54 -07:00 |
|
Juwan Yoo
|
0bc0bf5734
|
gemma3: impl get_attention_sliding_window_size for attn init (#4823)
|
2025-03-27 10:43:58 -07:00 |
|
AinL
|
17000d2b3a
|
Remove Unintended Capture Batch Sizes in AMD HIP Graph Runner (#4638)
|
2025-03-27 08:41:33 -07:00 |
|
strgrb
|
668ecc6c5b
|
Fix ut mla-test-1-gpu-amd (#4813)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 08:27:51 -07:00 |
|
strgrb
|
886fcbdd09
|
Use apply_rope_with_cos_sin_cache_inplace for DeepSeek (#4764)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 01:45:37 -07:00 |
|
Stefan He
|
1b9175cb23
|
[FA3 Attn Backend] Remove Unnecessary Device Sync for FA3 (#4745)
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
|
2025-03-27 00:45:11 -07:00 |
|
fzyzcjy
|
92bb49a7f9
|
Patch PyTorch's bug that cross-process tensor transfer will lead to wrong device (#4565)
|
2025-03-27 00:22:33 -07:00 |
|
Yineng Zhang
|
6f5cc5eb05
|
update xgrammar 0.1.17 (#4804)
|
2025-03-27 00:21:59 -07:00 |
|
Pan Lyu
|
c913ed4046
|
support clip embedding model (#4506)
|
2025-03-27 00:18:15 -07:00 |
|
Xihuai Wang
|
1afe3d0798
|
Align finish reason and stream mode in openai api (#4388)
|
2025-03-27 00:16:52 -07:00 |
|
laixin
|
ae25d36dc6
|
[3/3] fix dsv3 awq issue (#4719)
Co-authored-by: AniZpZ <aniz1905@gmail.com>
|
2025-03-26 23:13:43 -07:00 |
|
Yineng Zhang
|
1099f6c974
|
bump v0.4.4.post2 (#4669)
|
2025-03-26 19:58:00 -07:00 |
|
Xiaoyu Zhang
|
04e3ff6975
|
Support compressed tensors fp8w8a8 (#4743)
|
2025-03-26 13:21:25 -07:00 |
|
Yi Pan
|
45fdf1f7f3
|
Fix shared memory OOM on sm86 GPUs. (#4797)
|
2025-03-26 10:41:53 -07:00 |
|
Kebe
|
d89c0e4b7e
|
Use metadata to detect version of package (#4782)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-26 00:41:43 -07:00 |
|
fzyzcjy
|
fa3c9e0668
|
Fix popen_launch_server wait for 20 minutes when child process exits (#4777)
|
2025-03-26 00:32:19 -07:00 |
|
Cheng Wan
|
0d658ac3df
|
Support recording experts workload in QWen2-MoE (#4775)
|
2025-03-26 00:29:57 -07:00 |
|
Thysrael
|
ced35a0649
|
fix(typo): fix reply to replay in base_attn_backend.py (#4784)
|
2025-03-26 00:19:12 -07:00 |
|
fzyzcjy
|
26f07294f1
|
Warn users when release_memory_occupation is called without memory saver enabled (#4566)
|
2025-03-26 00:18:14 -07:00 |
|
Baizhou Zhang
|
34e07a65f1
|
[Fix] Fix unexpected idx bug of Phi-3-small (#4728)
|
2025-03-25 21:33:48 -07:00 |
|
fzyzcjy
|
15ddd84322
|
Add retry for flaky tests in CI (#4755)
|
2025-03-25 16:53:12 -07:00 |
|
fzyzcjy
|
52029bd1e3
|
Fix warmup error when dp=1 (#4753)
|
2025-03-25 02:01:21 -07:00 |
|
fzyzcjy
|
eb934bdf3b
|
Fix test_expert_distribution failure (#4752)
|
2025-03-25 01:17:03 -07:00 |
|
DarkSharpness
|
ac3fae8445
|
[Feature] Support "strict" in function calling (#4310)
|
2025-03-24 22:15:25 -07:00 |
|
yuhsaun-t
|
199bb01d00
|
Add endpoints to dump selected expert ids (#4435)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-24 21:34:19 -07:00 |
|
fzyzcjy
|
6b7038babd
|
Speedup warmup when DP > 1 (#4695)
|
2025-03-24 21:08:05 -07:00 |
|
lukec
|
57eec0bfbc
|
fix FlashMLA cudagraph config (#4691)
Co-authored-by: yinfan98 <1106310035@qq.com>
|
2025-03-24 21:06:58 -07:00 |
|
fzyzcjy
|
f01b092519
|
Super tiny fix typo (#4738)
|
2025-03-24 21:05:45 -07:00 |
|
Mick
|
1e86457c90
|
model: Minicpmo (#3023)
|
2025-03-24 20:08:40 -07:00 |
|
Ximingwang-09
|
22c3702e1e
|
[Model] Support Qwen2ForSequenceClassification (#4609)
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
|
2025-03-24 19:13:44 -07:00 |
|
Stefan He
|
4c584fc632
|
Fix circular imports in gptq.py and unblock test explorer (#4736)
|
2025-03-24 18:07:08 -07:00 |
|
Ke Bao
|
77cf771ebe
|
Fix EAGLE3 for llama3.3 70b (#4716)
|
2025-03-24 17:31:19 -07:00 |
|
Xuchun Shang
|
8154de5a32
|
[PD] Remove invalid parameter (#4721)
Signed-off-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
|
2025-03-24 13:14:16 -07:00 |
|
Yineng Zhang
|
c11cfda07b
|
update pyproject (#4731)
|
2025-03-24 09:50:28 -07:00 |
|
Yuhong Guo
|
64edeb798f
|
Support dynamic version name in sglang's pyproject.toml (#4720)
|
2025-03-24 08:56:31 -07:00 |
|
Stefan He
|
5d7edc8e55
|
Support FA3 as Attention backend by using --attention-backend fa3 (#4680)
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
|
2025-03-23 23:28:11 -07:00 |
|
Alex Sun
|
af6535e7aa
|
[ROCm] Enable MTP (NextN) on AMD GPU (#4631)
|
2025-03-23 22:58:05 -07:00 |
|
c1lovez1
|
93cf7fc5cd
|
Unify variable naming: replace is_in_free_group with is_not_in_free_group (#4698)
|
2025-03-23 21:51:08 -07:00 |
|
Kyungmin Lee
|
2a206b22ed
|
Fix RotaryEmbedding when using Triton backend for EXAONE-3.5-2.4B (#4064)
|
2025-03-23 17:58:12 -07:00 |
|
Zhiqiang Xie
|
4d25305700
|
Move mem_state update into debug mode (#4525)
|
2025-03-23 00:52:27 -07:00 |
|
Mick
|
11577cedb7
|
refactor: bug fixes and refactor for vlm (#4661)
|
2025-03-22 22:48:49 -07:00 |
|
fzyzcjy
|
ca75741e86
|
Support async in DeepEP (#4610)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-22 22:39:56 -07:00 |
|