huiwq1990
|
5cbd709ea1
|
Fix: modelscope env comment (#4474)
Signed-off-by: huiwq1990 <huiwq1990@163.com>
|
2025-03-16 18:11:33 -07:00 |
|
Yinghai Lu
|
2e4a1e2d05
|
Initialize image processor for skip-tokenizer-init codepath (#4479)
Co-authored-by: Alex Kirillov <alex@iterationlab.org>
|
2025-03-16 18:10:09 -07:00 |
|
Mick
|
9d02bb3e2a
|
Urgent model support: support gemma-3-it (#4424)
|
2025-03-16 17:37:32 -07:00 |
|
ZelinTan
|
402db5c58c
|
Benchmark: Statistical Analysis of the Output Stability of the Deepseek Model (#4202)
Co-authored-by: Chayenne <zhaochen20@outlook.com>
|
2025-03-16 17:32:57 -07:00 |
|
Lianmin Zheng
|
754a0e8278
|
Update CODEOWNERS (#4484)
|
2025-03-16 17:10:15 -07:00 |
|
Yinghai Lu
|
799fb5f455
|
400 on empty input_ids (#4481)
|
2025-03-16 14:01:23 -07:00 |
|
Yi Zhang
|
25e1816eff
|
fix custom allreduce performance/accuracy problem (#4477)
|
2025-03-16 12:16:30 -07:00 |
|
lukec
|
a53fe428f9
|
Support FlashMLA backend (#4472)
Co-authored-by: yinfan98 <1106310035@qq.com>
|
2025-03-16 09:07:06 -07:00 |
|
Ying Sheng
|
1b859295f4
|
[Eagle] Remove the greedy branch and some redundant code (#4363)
Co-authored-by: Sehoon Kim <sehoon@x.ai>
|
2025-03-16 02:48:55 -07:00 |
|
Yineng Zhang
|
9971dc2283
|
Revert "feat: Add FlashMLA submodule (#4449)" (#4470)
|
2025-03-16 01:30:05 -07:00 |
|
Lianmin Zheng
|
3db35c1af4
|
Release sgl-kernel v0.0.5.post2 (#4469)
|
2025-03-16 01:01:53 -07:00 |
|
Ying Sheng
|
52a34d7448
|
Add greedy verification kernel (#4383)
|
2025-03-16 00:58:26 -07:00 |
|
Lianmin Zheng
|
06d12b39d3
|
Remove filter for pr-tests (#4468)
|
2025-03-16 00:57:26 -07:00 |
|
Lianmin Zheng
|
c30976fb41
|
Fix finish step for pr tests and notebook tests (#4467)
|
2025-03-16 00:52:06 -07:00 |
|
JieXin Liang
|
1a3fa75f2f
|
[Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466)
|
2025-03-16 00:02:47 -07:00 |
|
Shi Shuai
|
81f431eded
|
feat: Add FlashMLA submodule (#4449)
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
|
2025-03-15 23:30:25 -07:00 |
|
Yineng Zhang
|
65b7c9b78f
|
cleanup deps 2/n (#4464)
|
2025-03-15 23:06:17 -07:00 |
|
Lianmin Zheng
|
2c4f5ccac1
|
Fix minor style (#4460)
|
2025-03-15 21:51:12 -07:00 |
|
Wang Ran (汪然)
|
158430473e
|
Fix typos (#4368)
|
2025-03-15 21:27:58 -07:00 |
|
Mick
|
8ec2ce0726
|
perf: update fused moe config (#4459)
|
2025-03-15 21:23:57 -07:00 |
|
Michael Feil
|
1fd0cf8a7b
|
Update comment in qwen2.py (#4447)
|
2025-03-15 21:14:29 -07:00 |
|
vikram singh shekhawat
|
bf63ee54ed
|
Auto-detect device if not specified in server arguments. (#4423)
|
2025-03-15 21:13:51 -07:00 |
|
Wang Ran (汪然)
|
22c96f78a6
|
typos: Update sampling_params.md (#4391)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-03-15 16:40:18 -07:00 |
|
Wang Ran (汪然)
|
2892b9bb97
|
bugfix: Update sampling_params.py (#4413)
|
2025-03-15 16:39:19 -07:00 |
|
Xu Song
|
470b474075
|
Update bench_serving.py (#4454)
|
2025-03-15 16:33:58 -07:00 |
|
江家瑋
|
26c372c13c
|
docs: Add Llama 3.3 to supported models (#4453)
Signed-off-by: JiangJiaWei1103 <waynechuang97@gmail.com>
|
2025-03-15 16:33:43 -07:00 |
|
Chen Shengzhi
|
86d9baedc2
|
[Fix] Fix errors when using the device except cuda. (#4455)
|
2025-03-15 16:33:00 -07:00 |
|
lukec
|
21d485f835
|
Fix test_create_kvindices unit test (#4452)
|
2025-03-15 16:01:04 -07:00 |
|
Mick
|
035ac2ab74
|
ci: update transformers==4.48.3 (#4451)
|
2025-03-15 13:27:26 -07:00 |
|
Chayenne
|
e1a5e7e47d
|
docs: hot fix torch compile cache (#4442)
|
2025-03-14 19:05:59 -07:00 |
|
Yineng Zhang
|
ad1ae7f7cd
|
use topk_softmax with sgl-kernel (#4439)
|
2025-03-14 15:59:06 -07:00 |
|
Lianmin Zheng
|
e73167ade3
|
Fix maximum recursion depth triggered on exception exit (#4438)
|
2025-03-14 15:12:26 -07:00 |
|
Yineng Zhang
|
862fe52241
|
bump v0.0.5.post1 (#4437)
|
2025-03-14 15:00:26 -07:00 |
|
Qingquan Song
|
61e4433caf
|
Add moe topk softmax templated from vllm (#4302)
|
2025-03-14 12:03:33 -07:00 |
|
Zhan Lu
|
660305c38a
|
[Doc] fix wrong flag in deepseek documentation (#4427)
|
2025-03-14 11:30:55 -07:00 |
|
Baoyuan Qi
|
642ab418f3
|
[bug] fix duplicate variable MAX_PIXELS in qwen_vl.py (#4419)
|
2025-03-14 01:28:25 -07:00 |
|
wangyu
|
1ce4878d31
|
feat(remote_model): support variable remote backend for model loader (#3964)
Signed-off-by: wangyu <wangyu.steph@bytedance.com>
|
2025-03-14 00:40:44 -07:00 |
|
Yineng Zhang
|
977d7cd26a
|
cleanup deps 1/n (#4400)
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
|
2025-03-14 00:00:33 -07:00 |
|
Lu Changqi
|
0e0ec70200
|
Hierarchical Caching supports MLA (#4009)
Signed-off-by: Changqi Lu <luchangqi.123@bytedance.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2025-03-13 20:42:14 -07:00 |
|
Lianmin Zheng
|
bb37855653
|
Update CODEOWNERS (#4403)
|
2025-03-13 17:54:40 -07:00 |
|
Yineng Zhang
|
ba80c102f9
|
bump v0.4.4.post1 (#4402)
|
2025-03-13 17:53:46 -07:00 |
|
Zhiqiang Xie
|
fbdb50501f
|
Hot fix for hicache with new page aligned radixtree (#4397)
|
2025-03-13 15:50:49 -07:00 |
|
Lianmin Zheng
|
f0afaf5289
|
Add a dummy grok test case (#4399)
|
2025-03-13 15:29:48 -07:00 |
|
Qiaolin Yu
|
85d2365d33
|
Fix the output of hidden states after HTTP requests (#4269)
|
2025-03-13 14:54:06 -07:00 |
|
Chang Su
|
5fe79605a8
|
Fix Llama3.3 tool call support (#4320)
|
2025-03-13 14:01:41 -07:00 |
|
Lianmin Zheng
|
c6d7f8d370
|
Add some fused elementwise kernels for grok-1 (#4398)
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-03-13 13:39:10 -07:00 |
|
Lianmin Zheng
|
a5a892ffd3
|
Fix auto merge & add back get_flat_data_by_layer (#4393)
|
2025-03-13 08:46:25 -07:00 |
|
Lianmin Zheng
|
8e66fbecee
|
Improve DP attention (#4390)
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
|
2025-03-13 08:23:56 -07:00 |
|
Lianmin Zheng
|
f141298a3c
|
Update ci_install_dependency.sh to use accelerate 1.4.0 (#4392)
Co-authored-by: wangyu <wangyu.steph@bytedance.com>
Co-authored-by: wangyu <yuwangauto@foxmail.com>
|
2025-03-13 07:16:11 -07:00 |
|
Lianmin Zheng
|
4fea040ca1
|
Fix a regression introduced by overlapping KV cache writing (#4375)
|
2025-03-13 03:49:05 -07:00 |
|