fzyzcjy
|
8c04f0f2e1
|
Support with_stack and record_shapes in profiler (#4740)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-03-27 23:01:42 -07:00 |
|
fzyzcjy
|
265e756494
|
Super tiny remove unused code (#4750)
|
2025-03-27 22:32:14 -07:00 |
|
fzyzcjy
|
d3f71f5e19
|
Fix torch.cuda.MemPool() internal assertion failure (#4687)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-03-27 22:29:36 -07:00 |
|
Kebe
|
e0166f8ab4
|
Remove empty tool function name (#4704)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-27 22:23:30 -07:00 |
|
fzyzcjy
|
53a2c3b466
|
Support controlling nsys start and end range programmatically (#4688)
|
2025-03-27 22:21:13 -07:00 |
|
BroadbentJim
|
550586ef42
|
fix: Inappropriate lack of Optional type on OpenAI ChatCompletionRequest (#4681)
|
2025-03-27 22:19:05 -07:00 |
|
fzyzcjy
|
cf29fe9e78
|
Fix Engine error when enabling DP attention (#4648)
|
2025-03-27 22:17:30 -07:00 |
|
Stefan He
|
26c0f13126
|
Support Page Size > 1 for FA3 (#4832)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-03-27 22:07:14 -07:00 |
|
rongfu.leng
|
f9970bd1af
|
fix: when use SGLANG_PORT this env,port is str (#4528)
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
|
2025-03-27 21:46:06 -07:00 |
|
lambert0312
|
2e0f94ab79
|
[Fix] fix output_top_logprobs is not exist (#4597)
|
2025-03-27 21:45:57 -07:00 |
|
Vincent
|
e2e2ab70e0
|
IPv6 support (#3949)
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2025-03-27 21:42:13 -07:00 |
|
Jiaqi
|
72031173e4
|
fix: fix typo of comments in w8a8_fp8.py (#4843)
|
2025-03-27 21:06:47 -07:00 |
|
Qiaolin Yu
|
9fdc6d6abc
|
Fix the lora adapter when lora path is none (#4799)
Co-authored-by: Beichen Ma <mabeichen12@gmail.com>
|
2025-03-27 21:03:08 -07:00 |
|
XinyuanTong
|
42a45df043
|
[Fix] self.worker assignment in TpModelWorker and refactor references (#4788)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-03-27 20:28:38 -07:00 |
|
Jon Durbin
|
04eb6062e4
|
Include context length in /v1/models response. (#4809)
|
2025-03-27 20:23:18 -07:00 |
|
Brayden Zhong
|
e84f4ba0ab
|
[Misc] Fix issues reported by torchfix (#4837)
|
2025-03-27 20:10:32 -07:00 |
|
Kebe
|
10a9ab7b07
|
Fix error due to CustomAllreduce setup failure (#4815)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-27 18:52:10 -07:00 |
|
Junrong Lin
|
bb0fd749a6
|
[Fix] Add compressed_tensors as deps (#4819)
|
2025-03-27 18:08:24 -07:00 |
|
tarinkk
|
7f19e083c1
|
Support (1 <= dp < tp) in the dp attention in DeepEP (#4770)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
|
2025-03-27 17:09:35 -07:00 |
|
Juwan Yoo
|
188105a21b
|
deps: lazy import optional dependencies gguf and torchvision (#4826)
|
2025-03-27 14:35:36 -07:00 |
|
Yineng Zhang
|
bbab97a6a8
|
add partial_json_parser and einops (#4827)
|
2025-03-27 13:24:54 -07:00 |
|
Juwan Yoo
|
0bc0bf5734
|
gemma3: impl get_attention_sliding_window_size for attn init (#4823)
|
2025-03-27 10:43:58 -07:00 |
|
AinL
|
17000d2b3a
|
Remove Unintended Capture Batch Sizes in AMD HIP Graph Runner (#4638)
|
2025-03-27 08:41:33 -07:00 |
|
strgrb
|
668ecc6c5b
|
Fix ut mla-test-1-gpu-amd (#4813)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 08:27:51 -07:00 |
|
strgrb
|
886fcbdd09
|
Use apply_rope_with_cos_sin_cache_inplace for DeepSeek (#4764)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 01:45:37 -07:00 |
|
Stefan He
|
1b9175cb23
|
[FA3 Attn Backend] Remove Unnecessary Device Sync for FA3 (#4745)
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
|
2025-03-27 00:45:11 -07:00 |
|
fzyzcjy
|
92bb49a7f9
|
Patch PyTorch's bug that cross-process tensor transfer will lead to wrong device (#4565)
|
2025-03-27 00:22:33 -07:00 |
|
Yineng Zhang
|
6f5cc5eb05
|
update xgrammar 0.1.17 (#4804)
|
2025-03-27 00:21:59 -07:00 |
|
Pan Lyu
|
c913ed4046
|
support clip embedding model (#4506)
|
2025-03-27 00:18:15 -07:00 |
|
Xihuai Wang
|
1afe3d0798
|
Align finish reason and stream mode in openai api (#4388)
|
2025-03-27 00:16:52 -07:00 |
|
laixin
|
ae25d36dc6
|
[3/3] fix dsv3 awq issue (#4719)
Co-authored-by: AniZpZ <aniz1905@gmail.com>
|
2025-03-26 23:13:43 -07:00 |
|
Yineng Zhang
|
1099f6c974
|
bump v0.4.4.post2 (#4669)
|
2025-03-26 19:58:00 -07:00 |
|
Xiaoyu Zhang
|
04e3ff6975
|
Support compressed tensors fp8w8a8 (#4743)
|
2025-03-26 13:21:25 -07:00 |
|
Yi Pan
|
45fdf1f7f3
|
Fix shared memory OOM on sm86 GPUs. (#4797)
|
2025-03-26 10:41:53 -07:00 |
|
Kebe
|
d89c0e4b7e
|
Use metadata to detect version of package (#4782)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2025-03-26 00:41:43 -07:00 |
|
fzyzcjy
|
fa3c9e0668
|
Fix popen_launch_server wait for 20 minutes when child process exits (#4777)
|
2025-03-26 00:32:19 -07:00 |
|
Cheng Wan
|
0d658ac3df
|
Support recording experts workload in QWen2-MoE (#4775)
|
2025-03-26 00:29:57 -07:00 |
|
Thysrael
|
ced35a0649
|
fix(typo): fix reply to replay in base_attn_backend.py (#4784)
|
2025-03-26 00:19:12 -07:00 |
|
fzyzcjy
|
26f07294f1
|
Warn users when release_memory_occupation is called without memory saver enabled (#4566)
|
2025-03-26 00:18:14 -07:00 |
|
Baizhou Zhang
|
34e07a65f1
|
[Fix] Fix unexpected idx bug of Phi-3-small (#4728)
|
2025-03-25 21:33:48 -07:00 |
|
fzyzcjy
|
15ddd84322
|
Add retry for flaky tests in CI (#4755)
|
2025-03-25 16:53:12 -07:00 |
|
fzyzcjy
|
52029bd1e3
|
Fix warmup error when dp=1 (#4753)
|
2025-03-25 02:01:21 -07:00 |
|
fzyzcjy
|
eb934bdf3b
|
Fix test_expert_distribution failure (#4752)
|
2025-03-25 01:17:03 -07:00 |
|
DarkSharpness
|
ac3fae8445
|
[Feature] Support "strict" in function calling (#4310)
|
2025-03-24 22:15:25 -07:00 |
|
yuhsaun-t
|
199bb01d00
|
Add endpoints to dump selected expert ids (#4435)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-03-24 21:34:19 -07:00 |
|
fzyzcjy
|
6b7038babd
|
Speedup warmup when DP > 1 (#4695)
|
2025-03-24 21:08:05 -07:00 |
|
lukec
|
57eec0bfbc
|
fix FlashMLA cudagraph config (#4691)
Co-authored-by: yinfan98 <1106310035@qq.com>
|
2025-03-24 21:06:58 -07:00 |
|
fzyzcjy
|
f01b092519
|
Super tiny fix typo (#4738)
|
2025-03-24 21:05:45 -07:00 |
|
Mick
|
1e86457c90
|
model: Minicpmo (#3023)
|
2025-03-24 20:08:40 -07:00 |
|
Ximingwang-09
|
22c3702e1e
|
[Model] Support Qwen2ForSequenceClassification (#4609)
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
|
2025-03-24 19:13:44 -07:00 |
|