Commit Graph
204 Commits
Author SHA1 Message Date
CaranLic 168ad600b5 [main] add pd transfer for ascend scheduler (#2753)
### What this PR does / why we need it?
For offline scenarios, adjust the scheduling process to prioritize the
prefill phase of all requests, then process the decode phase of all
requests.

### How was this patch tested?

```
max_num_seqs=24,
additional_config={
    "ascend_scheduler_config":{
        "enabled": True,
        "enable_pd_transfer": True,
        "decode_max_num_seqs": 24,
        "enable_chunked_prefill": False
    }
},
```
| input | output | num prompts | max_num_seqs | dp | tp | scheduler |
tps |
| ------ | ------ | ---------- | ---------------- | ---- | ---- |
---------------- | --------------- |
| dapo-math-17K | 2K | 384 | 24 | 2 | 1 | v1 | 234.06 |
| dapo-math-17K | 2K | 384 | 24 | 2 | 1 | pd transfer | 239.59(+2.4%) |
| dapo-math-17K| 2K | 384 | 24 | 4 | 1 | v1 | 222.85 |
| dapo-math-17K| 2K | 384 | 24 | 4 | 1 | pd transfer | 225.81(+1.3%) |


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/6fb27881634d89c2e70e9e5fbad1b918c0d916cf

---------

Signed-off-by: CaranLic <740821011@qq.com>
2025-09-10 08:46:39 +08:00
Mengqing Cao edf1f600ad [CI] Remove compatibility maintenance for vllm v0.10.1 and v0.10.1.1 (#2840)
### What this PR does / why we need it?
Remove compatibility maintenance for vllm v0.10.1 and v0.10.1.1

### Does this PR introduce _any_ user-facing change?
branch main of vllm-ascend will not be compatible with vllm v0.10.1 and
v0.10.1.1

### How was this patch tested?
CI passed with existing test.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/6fb27881634d89c2e70e9e5fbad1b918c0d916cf

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-09-10 08:43:10 +08:00
yupeng a746f8274f [DOC] Qwen3 PD disaggregation user guide (#2751)
### What this PR does / why we need it?
The PR is for the document of the prefiller&decoder disaggregation
deloyment guide.

The scenario of the guide is:
- Use 3 nodes totally and 2 NPUs on each node
- Qwen3-30B-A3B
- 1P2D
- Expert Parallel

The deployment can be used to verify PD Disggregation / Expert Parallel
features with a slightly less resources.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
No.


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/e599e2c65ee32abcc986733ab0a55becea158bb4

---------

Signed-off-by: paulyu12 <507435917@qq.com>
2025-09-07 10:35:37 +08:00
Yikun Jiang 752e272a55 Add note for Ascend HDK version (#2765)
### What this PR does / why we need it?
Add note for Ascend HDK version

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/e599e2c65ee32abcc986733ab0a55becea158bb4

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-09-07 10:33:41 +08:00
lidenghui1110andzzh 5a7181569c [feat]: oproj tensor parallelism in pure DP and graph-mode scenarios. (#2167)
### What this PR does / why we need it?
This PR introduces Oproj matrix tensor model parallel to achieve
decreasing of memory consumption. It only support graph mode in pure DP
scenario.

In deepseek r1 w8a8 PD disagregated Decode instance, using pure DP, with
oproj_tensor_parallel_size = 8, we have 1 ms TPOT increasing, saved 5.8
GB NPU memory per RANK. We got best performance when
oproj_tensor_parallel_size=4 without TPOT increasing.

performance data:
<img width="1442" height="442" alt="image"
src="https://github.com/user-attachments/assets/83270fc5-868a-4387-b0a9-fac29b4a376d"
/>

### Does this PR introduce _any_ user-facing change?
This PR introduces one new config in `additional_config`.
| Name | Effect | Required | Type | Constraints |
| :---------------------------- |
:--------------------------------------- | :------- | :--- |
:----------------- |
| oproj_tensor_parallel_size | Split the o_proj matrix along the row
dimension (head num * head dim) into oproj_tensor_parallel_size pieces.
| No | int | default value is None, once this value is set, the feature
will be enabled, head num * head dim must be divisible by this value. |

example

`--additional_config={"oproj_tensor_parallel_size": 8}`

### How was this patch tested?


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/eddaafc1c77b0690194cbd1b73747d572793838c

---------

Signed-off-by: zzhx1 <zzh_201018@outlook.com>
Co-authored-by: zzh <zzh_201018@outlook.com>
2025-09-07 10:31:32 +08:00
vllm-ascend-ciandvllm-ascend-ci 3a2a7d88db [Doc] Update accuracy reports for v0.10.1rc1 (#2755)
The accuracy results running on NPU Altlas A2 have changed, updating
reports for: All models (Qwen3-30B-A3B, Qwen2.5-VL-7B-Instruct,
Qwen3-8B-Base, DeepSeek-V2-Lite)

  - [Workflow run][1]
  
[1]:
https://github.com/vllm-project/vllm-ascend/actions/runs/17459225764
- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/2b30afa4420cbada6dd9084de3ee7eb19142b7ff

Signed-off-by: vllm-ascend-ci <vllm-ascend-ci@users.noreply.github.com>
Co-authored-by: vllm-ascend-ci <vllm-ascend-ci@users.noreply.github.com>
2025-09-04 22:17:17 +08:00
Mengqing Cao 7e16b4a7cd [ReleaseNote] Add Release Note for v0.10.1rc1 (#2635)
Add Release Note for v0.10.1rc1

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/b5ee1e3261d9edf94d76ba8b437ebdef7ac599ea

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-09-04 11:26:47 +08:00
wangxiyuan 41b028aa5f [Doc] add v0.9.1 release note (#2646)
Add release note for 0.9.1

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/8bd5844989373d6914f6dea94a56822e6d7380d4

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-09-03 18:04:27 +08:00
panchao-hubandzhangdepeng ea53f9076e support torchair mode (#2641)
### What this PR does / why we need it?
support torchair mode
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/5438967fbc7a10ae6eee7a98182f4de94101e858

Signed-off-by: zhangdepeng <zhangdepeng2@huawei.com>
Signed-off-by: p00465316 <panchao13@huawei.com>
Co-authored-by: zhangdepeng <zhangdepeng2@huawei.com>
2025-09-01 15:49:07 +08:00
lidenghui1110andzhangzihang 600b08f754 [Feat]: Add custom lmhead tensor model parallel (#2309)
### What this PR does / why we need it?
This PR introduces LMhead tensor model parallel to achieve decreasing of
memory consumption, and TPOT performance improvement. It support both
eager mode and graph mode.

In deepseek r1 w8a8 PD disagregated Decode instance, using pure DP, with
lmhead_tensor_parallel_size = 8, we have 1 ms TPOT optimization, saved
1.48 GB NPU memory per RANK.

performance data:
<img width="1444" height="438" alt="image"
src="https://github.com/user-attachments/assets/3c5ef0d3-a7c7-46fd-9797-4de728eb0cb0"
/>

### Does this PR introduce _any_ user-facing change?
This PR introduces one new config in `additional_config`.
| Name | Effect | Required | Type | Constraints |
| :---------------------------- |
:--------------------------------------- | :------- | :--- |
:----------------- |
| lmhead_tensor_parallel_size | Split the lm_head matrix along the
column dimension (vocab_size) into lmhead_tensor_parallel_size pieces |
No | int | default value is None, once this value is set, the feature
will be enabled, vocab_size must be divisible by this value. |

example

`--additional_config={"lmhead_tensor_parallel_size": 8}`

### How was this patch tested?


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/de533ab2a14192e461900a4950e2b426d99a6862

---------

Signed-off-by: zzhx1 <zzh_201018@outlook.com>
Co-authored-by: zhangzihang <zzh_201018@outlook.com>
2025-08-29 11:41:21 +08:00
c8d1df3a3f [Refactor][WIP] Refactor mla_v1 by moving all MLA preprocessing ops into mla_v1 attention impl (#2465)
### What this PR does / why we need it?
In order to support fused kernels, multi-stream, communication
optimization etc, it's better to aggregate all opreations in Attention
layer togather. This PR tries to refactor mla_v1 by moving all MLA
preprocessing ops into mla_v1 attention impl.
Note that new mla_v1 doesn't take torchair into consideration. So this
PR can only be merged after torchair related mla_v1 is isolated into a
new file.
### Does this PR introduce _any_ user-facing change?
NO
### How was this patch tested?

### Features Test

<img width="506" height="141" alt="image"
src="https://github.com/user-attachments/assets/f1ab2906-a1ac-4450-8433-94811cd89466"
/>

### Performance After Refact
<img width="648" height="486" alt="image"
src="https://github.com/user-attachments/assets/e33e038c-c5d9-4ba7-a8e9-1ac22f9833eb"
/>

### Performance Before Refact
<img width="618" height="494" alt="image"
src="https://github.com/user-attachments/assets/83861dc2-dc51-4af3-9310-90ab10c43bb1"
/>


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/e03940762b43812fccd3c214bda60201cff9d16a

---------

Signed-off-by: lwq <liwenquan5@huawei.com>
Signed-off-by: whx-sjtu <2952154980@qq.com>
Signed-off-by: SunnyLee219 <3294305115@qq.com>
Co-authored-by: lwq <liwenquan5@huawei.com>
Co-authored-by: whx-sjtu <2952154980@qq.com>
2025-08-28 10:35:57 +08:00
Li Wang 516e14ae6a [Doc] Upgrade to multi-node tutorial model to deepseek-v3.1-w8a8 (#2553)
### What this PR does / why we need it?
Upgrade to multi-node tutorial model to deepseek-v3.1-w8a8
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/de02b07db4741cc9ed40b8262d7a67e6bce30211

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-08-27 14:16:44 +08:00
Li Wang 042605f4b2 [Doc] Add stable modelslim branch (#2545)
### What this PR does / why we need it?
The branch `br_release_MindStudio_8.1.RC2_TR5_20260624` is commercial
delivery version of modelslim in Q3, and has been verified available
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/7d67a9d9f93f86b74066c64c373405aa088e4a16

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-08-27 09:05:46 +08:00
Shanshan Shen 334c44613a [Doc] Update release version info (#2518)
### What this PR does / why we need it?
Update release version info.

### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/712d0f88d89940291b6a6a4db635c60187398acd

Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2025-08-25 15:39:10 +08:00
Shanshan Shen 98c68220c1 [Doc] Update v0.9.1rc3 doc (#2512)
### What this PR does / why we need it?
Update `v0.9.1rc3` doc, which are supplements to
https://github.com/vllm-project/vllm-ascend/pull/2488.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/170e8ea9ea95294d1bdc4af4bea73741ea759e22

Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2025-08-25 11:39:29 +08:00
Mengqing Cao 4c4ffeebe5 [Doc] update vllm version in ci (#2513)
### What this PR does / why we need it?
update vllm version in ci

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/170e8ea9ea95294d1bdc4af4bea73741ea759e22

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-25 11:35:37 +08:00
Shanshan Shen f0be3eed84 [Doc] Add release note for v0.9.1rc3 (#2488)
### What this PR does / why we need it?

Add release note for `v0.9.1rc3`.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71

---------

Signed-off-by: shen-shanshan <467638484@qq.com>
2025-08-22 16:06:29 +08:00
LookAround0301 e9fb895b10 [Doc] Add feature branch long_seq_optimization (#2477)
### What this PR does / why we need it?
Add cp/sp feature branch

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/0c6e40bbaa4707528286a1e7bf17c90c88a1d920

Signed-off-by: LookAround <lixushi@huawei.com>
2025-08-22 08:53:12 +08:00
Yikun Jiang 67a222c383 [Doc] Add feature branch policy (#2432)
### What this PR does / why we need it?

This patch add the feature branch policy.

After this patch: maintainers are allowed to create a feature branch.
Feature branches are used for collaboration and must include an RFC
link, merge plan and mentor info.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

CI passed

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/7be5d113d8784536b79f27f24cfa91958dc291b0

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-08-21 10:37:21 +08:00
yupeng 973a7cfdf0 [DOC] update doc: LoRA with ACLGraph (#2430)
### What this PR does / why we need it?
Update DOC. Guide users to run LoRA with ACLGraph.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
No.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/de7b67a0232e35ae8e8ecd944aeddfc8cbc02631

---------

Signed-off-by: paulyu12 <507435917@qq.com>
2025-08-21 08:55:55 +08:00
Wang Kunpeng 1de16ead8e [main][bugfix] Modify the default value of the enable_shared_pert_dp to false (#2457)
### What this PR does / why we need it?
enable_shared_pert_dp is currently on by default. This optimization is
currently only valid for deepseek series models. The default opening
affects the accuracy of the qwen series models.
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?
use parameter --additional_config='{"enable_shared_expert_dp": true}'

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/d983769c41db224e0897fac2e9aefc5f57ad1122

Signed-off-by: Wang Kunpeng <1289706727@qq.com>
2025-08-20 20:25:53 +08:00
Jade Zheng 955411611c Nominate Mengqing Cao as vllm-ascend maintainer (#2433)
I would like to nominate Mengqing Cao (@MengqingCao
https://github.com/MengqingCao) as a maintainer, starting with my +1.

## Reason

Review Quality‌: She has completed [120+
reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao)
since Feb. 2025, include
[#review-3077842852](https://github.com/vllm-project/vllm-ascend/pull/2088#pullrequestreview-3077842852),
[comment-2990074116](https://github.com/vllm-project/vllm-ascend/pull/1032#issuecomment-2990074116),
[comment-2921063723](https://github.com/vllm-project/vllm-ascend/pull/1013#issuecomment-2921063723)
high quality review.

Sustained and Quality Contributions: She has Deep understanding of
‌vLLM‌ and ‌vLLM Ascend‌ codebases and solid contributions include The
vLLM contributions and help vLLM Ascend release is the main reason I
nominated her:

- vLLM: Things worth mentioning that she completed [28+ PR
contributions](https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+)
in vllm-project/vllm, especially for vLLM platform module to improve
vLLM mult hardware support. She is one of the important co-authors of
[vllm#8054](https://github.com/vllm-project/vllm/pull/8054) and hardware
plugin RFC, this makes vllm-ascend plugin possible.
Community Involvement: She is also very active and involved in [60+
issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao).

So I think she's a great addition to the vLLM Ascend Maintainer team.

- ✅**Review Quality‌:**

She has completed 120+ reviews since Feb. 2025.

https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao,
include
https://github.com/vllm-project/vllm-ascend/pull/2088#pullrequestreview-3077842852,
https://github.com/vllm-project/vllm-ascend/pull/1446#issuecomment-3015166908,
https://github.com/vllm-project/vllm-ascend/pull/1032#issuecomment-2990074116,
https://github.com/vllm-project/vllm-ascend/pull/1013#issuecomment-2921063723
quality review.

- ✅**Sustained Contributions:**

99+ PR merged in vllm-project/vllm-ascend

https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged

- ✅**Quality Contribution‌:**

She is one of the important co-authors of
https://github.com/vllm-project/vllm/pull/8054 , this makes vllm-ascend
plugin possible.

Things worth mentioning that she complete 28+ PR contributions in
vllm-project/vllm, especially for vLLM platform module to improve vLLM
mult hardware support:

https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+.

At 2025 Q2, She also lead the [[RFC]: E2E CI test for key
features](https://github.com/vllm-project/vllm-ascend/issues/413) and
[[RFC]: Unit test coverage
improvement](https://github.com/vllm-project/vllm-ascend/issues/1298) to
help vllm ascend improve the coverage.

Her main contributions focus on the adaptation of parallel strategies
and communicator, such as
https://github.com/vllm-project/vllm-ascend/pull/1800,
https://github.com/vllm-project/vllm-ascend/pull/1856.

These contributions are sufficient to prove she has “Deep understanding
of ‌vLLM‌ and ‌vLLM Ascend‌ codebases”

- ✅**Community Involvement‌:**

Involved in 63+ issue reviewer
https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao

She led the v0.10.1 release as release manager


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/78dba404adaa2b9814a4480249e2d9c0512364c1

Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
2025-08-19 14:13:54 +08:00
wangxiyuan 6335fe39ea Nominate ApsarasX as vllm-ascend maintainer (#2419)
I would like to nominate Wengang Chen (@ApsarasX
https://github.com/ApsarasX) as a maintainer, starting with my +1.

## Reason
Review Quality‌: He focuses on the vLLM Ascend Core module review with
100+ high quality review, such as [#2326
(comment)](https://github.com/vllm-project/vllm-ascend/pull/2326#discussion_r2268509365),
[#768
(comment)](https://github.com/vllm-project/vllm-ascend/pull/768#discussion_r2075278516),
[#2312
(comment)](https://github.com/vllm-project/vllm-ascend/pull/2312#issuecomment-3174677159),
[#2268
(comment)](https://github.com/vllm-project/vllm-ascend/pull/2268#discussion_r2260920578),
[#2192
(comment)](https://github.com/vllm-project/vllm-ascend/pull/2192#issuecomment-3149414586),
[#2156
(comment)](https://github.com/vllm-project/vllm-ascend/pull/2156#discussion_r2249096673).
This helped vLLM Ascend v0.9.x and v0.10.x to be released with high
quality.

Sustained and Quality Contributions: He has a very good habit of sharing
his design ideas, development process, performance test results, such as
[#966](https://github.com/vllm-project/vllm-ascend/pull/966), he
contributed [many
PRs](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Amerged+),
valuable bugfixes and also perf improvements.

Community Involvement: Active involved in community discussion, he is
collaborative and helps the users solve problems, involved in [120+ PR
and
issues](https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX).
He is also the speaker of [vLLM Beijing
Meetup](https://mp.weixin.qq.com/s/7n8OYNrCC_I9SJaybHA_-Q).

So I think he's a great addition to the vLLM Ascend Maintainer team.

- ✅Review Quality‌:
108+ PR with valuable review
https://github.com/vllm-project/vllm-ascend/pulls?q=commenter%3AApsarasX
with many valuable review, like 

https://github.com/vllm-project/vllm-ascend/pull/2326#discussion_r2268509365

https://github.com/vllm-project/vllm-ascend/pull/768#discussion_r2075278516

https://github.com/vllm-project/vllm-ascend/pull/2312#issuecomment-3174677159

https://github.com/vllm-project/vllm-ascend/pull/2268#discussion_r2260920578

https://github.com/vllm-project/vllm-ascend/pull/2192#issuecomment-3149414586

https://github.com/vllm-project/vllm-ascend/pull/2156#discussion_r2249096673

- ✅ Sustained and Major Contributions
https://github.com/vllm-project/vllm-ascend/pulls/ApsarasX

- ✅ Quality Contribution‌:

https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Aclosed
Good quality with well documents
[Perf] Refactor tensor disposal logic to reduce memory usage
https://github.com/vllm-project/vllm-ascend/pull/966

- ✅Community Involvement‌: 
7 issue:

https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20author%3AApsarasX

- 120+ PR and issue:

https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-08-19 10:44:35 +08:00
TaoYu Chen 9e7c168d99 Add ModelRunner_prepare_inputs doc (#1493)
### What this PR does / why we need it?
To help more developers quickly get started with vLLM, we need to write
clear and easy-to-understand code documentation and technical
interpretations. This will effectively lower the learning curve, attract
more excellent contributors, and collectively build a better developer
community.

Add ModelRunner_prepare_inputs doc

### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
Pass CI


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/4be02a37767f05a3fd27d66435d5cebea7a9bfe8

---------

Signed-off-by: ChenTaoyu-SJTU <ctynb@qq.com>
2025-08-18 15:41:24 +08:00
Li Wang 2ad7e1251e [Doc] Fix quant documentation to make it reproducible (#2277)
### What this PR does / why we need it?
Fixed the expression of msit for code clone

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/afa5b7ca0b417abadfa85e32f28969b72e58a885

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-08-14 17:19:47 +08:00
jack 8bfd16a145 [Doc] Add container image save/load FAQ for offline environments (#2347)
### What this PR does / why we need it?

Add Docker export/import guide for air-gapped environments

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

NA

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/d16aa3dae446d93f870a2e51b240e18a01cac294

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
2025-08-13 16:00:43 +08:00
Mengqing Cao 49ec6c98b7 [Doc] Update faq (#2334)
### What this PR does / why we need it?
  - update determinitic calculation
  - update support device

### Does this PR introduce _any_ user-facing change?
- Users should update ray and protobuf when using ray as distributed
backend
- Users should change to use `export HCCL_DETERMINISTIC=true` when
enabling determinitic calculation

### How was this patch tested?
N/A

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ea1292ad3ee724e44b3dfec2a26778cd614729f9

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-12 14:12:53 +08:00
Wang KunpengandSlightwindSec dc585f148a [main][prefill optimization] Optimize parallel strategies to reduce communication overhead (#2198)
### What this PR does / why we need it?
1.Shared Expert Sharding Strategy Update: Switched from TP-aligned to
pure DP for shared experts, enabling more efficient execution.
2.O_Proj AllReduce → ReduceScatter: Reduced communication overhead by
using ReduceScatter, made possible by pure DP sharding.
3.AllGather Postponed: Delayed to after QKV down projection to reduce
synchronization impact during prefill.

### How was this patch tested?
Adding ut case in `tests/ut/attention/test_mla_v1.py`

#### How to run

use parameter `--additional_config='{"enable_shared_expert_dp": true}'`

##### a.How to run eager mode

eg:
python -m vllm.entrypoints.openai.api_server --model=/model_path
--trust-remote-code -tp 8 -dp 2 --enable_expert_parallel --port 8002
--max-model-len 5120 --max-num-batched-tokens 16384 --enforce-eager
--disable-log-requests
--additional_config='{"ascend_scheduler_config":{"enabled":true},"enable_shared_expert_dp":
true,"chunked_prefill_for_mla":true}'

##### b.How to run graph mode

eg:
python -m vllm.entrypoints.openai.api_server --model=/model_path
--trust-remote-code -tp 8 -dp 2 --enable_expert_parallel --port 8002
--max-model-len 5120 --max-num-batched-tokens 16384
--disable-log-requests
--additional_config='{"ascend_scheduler_config":{"enabled":true},"enable_shared_expert_dp":
true,"chunked_prefill_for_mla":true,"torchair_graph_config":{"enabled":true}}'


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/9edd1db02bc6dce6da503503a373657f3466a78b

---------

Signed-off-by: Wang Kunpeng <1289706727@qq.com>
Signed-off-by: SlightwindSec <slightwindsec@gmail.com>
Co-authored-by: SlightwindSec <slightwindsec@gmail.com>
2025-08-12 14:12:12 +08:00
Mengqing Cao 4604882a3e [ReleaseNote] Release note of v0.10.0rc1 (#2225)
### What this PR does / why we need it?
Release note of v0.10.0rc1

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/8e8e0b6af189d262bcfdaef6c0cfb94772e86b0b

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-07 14:46:49 +08:00
zhangxinyuehfad 92eebc0c9b [Doc] Update user guide for suported models (#2263)
### What this PR does / why we need it?
 Update user guide for suported models 

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/4be02a37767f05a3fd27d66435d5cebea7a9bfe8

---------

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
2025-08-07 14:39:51 +08:00
22dimensions 440d28a138 [Tutorial] Add qwen3 8b w4a8 tutorial (#2249)
### What this PR does / why we need it?

Add a new single npu quantization tutorial, and using the latest qwen3
model.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/8e8e0b6af189d262bcfdaef6c0cfb94772e86b0b

Signed-off-by: 22dimensions <waitingwind@foxmail.com>
2025-08-07 14:39:38 +08:00
zhangxinyuehfad bcd0b532f5 [Doc] Update user guide for using lm-eval (#1325)
### What this PR does / why we need it?
Update user guide for using lm-eval
1. add using lm-eval on online server
2. add using offline datasets

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/9edd1db02bc6dce6da503503a373657f3466a78b

---------

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
2025-08-07 14:15:49 +08:00
zhangxinyuehfad dbba3cabb0 [Doc] Update tutorials for single_npu_audio and single_npu_multimodal (#2252)
### What this PR does / why we need it?
Update tutorials for single_npu_audio and single_npu_multimodal

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/6b47ef24de3d3b4f551aca0bc21b9f16f3d21b6a

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
2025-08-07 14:08:14 +08:00
Li Wang bf84f2dbfa [Doc] Support kimi-k2-w8a8 (#2162)
### What this PR does / why we need it?
In fact, the kimi-k2 model is similar to the deepseek model, and we only
need to make a few changes to support it. what does this pr do:
1. Add kimi-k2-w8a8 deployment doc
2. Update quantization doc
3. Upgrade torchair support list
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/9edd1db02bc6dce6da503503a373657f3466a78b

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-08-06 19:28:47 +08:00
Yikun Jiang 54ace9e12b Add release note for v0.9.1rc2 (#2188)
### What this PR does / why we need it?
Add release note for v0.9.1rc2

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/c494f96fbcf5e9f19f59e3dea6c2780aeb6c567f

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-08-06 09:04:46 +08:00
leo-ponyandYikun Jiang 807f0895b2 Bump torch version to 2.7.1 (#1562)
### What this PR does / why we need it?
Bump torch version to 2.7.1, and cleanup infer schema patch
https://github.com/vllm-project/vllm-ascend/commit/857f489
(https://github.com/vllm-project/vllm-ascend/pull/837), this patch
depends on also: https://github.com/vllm-project/vllm-ascend/pull/1974

### Does this PR introduce any user-facing change?
No

#### How was this patch tested?
CI passed

torch-npu 2.7.1rc1 install guide:
https://gitee.com/ascend/pytorch/tree/v2.7.1/
install depending:
```
pip3 install pyyaml
pip3 install setuptools
```
install torch-npu:

Closes: https://github.com/vllm-project/vllm-ascend/issues/1866
Closes: https://github.com/vllm-project/vllm-ascend/issues/1390


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/9af654cc38c74cd51b00c609eaa290e495f225e1

---------

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
Signed-off-by: leo-pony <nengjunma@outlook.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
2025-08-05 08:43:24 +08:00
leo-pony f0c1f0c828 [Doc] Add qwen vl example in tutorials for 310I series (#2160)
### What this PR does / why we need it?
Add qwen vl example in tutorials for 310I series. 

Model: Qwen2.5-VL-3B-Instruct
Accuracy test result, dataset MMM-val:
| | 910B3 | 310P3 |
| --- | --- | --- |
|Summary|0.455 | 0.46 |
|--art_and_design| 0.558 | 0.566 |
|--business| 0.373 | 0.366 |
|--health_and_medicine|0.513 | 0.52 |
|--science|0.333 | 0.333 |
|--tech_and_engineering|0.362 | 0.380 |
|--humanities_and_social_science|0.691 | 0.691 |

Function test result:

1. On line:
![image](https://github.com/user-attachments/assets/d81bba61-df28-4676-a246-c5d094815ac7)
![image](https://github.com/user-attachments/assets/0be81628-9999-4ef2-93c1-898b3043e09e)

2. Offline:
![image](https://github.com/user-attachments/assets/603275c1-6ed6-4cfc-a6e2-7726156de087)

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad57f23f6a528ab01066998b41796a44340fd43d

---------

Signed-off-by: leo-pony <nengjunma@outlook.com>
2025-08-02 08:58:56 +08:00
Mengqing Cao 936df1cb9b [Doc] Fix cann related urls (#2106)
### What this PR does / why we need it?
Fix cann related urls in installation doc.

### Does this PR introduce _any_ user-facing change?
The users install cann manually could use the correct url after this pr

### How was this patch tested?
N/A

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/5bbaf492a6238ff517249e73151ae9989f7bea9e

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-07-30 22:31:30 +08:00
Shanshan Shen 61fc35184b [Doc] Add performance tuning doc to main (#1392)
### What this PR does / why we need it?
Add performance tuning doc to main.

Closes: https://github.com/vllm-project/vllm-ascend/issues/1387


- vLLM version: v0.9.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/923147b5e8551887fd64a0fc242c361d5216e1d7

---------

Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2025-07-29 19:36:34 +08:00
Wang Kunpeng e3a2443c3a [main][Doc] add mla pertoken quantization FAQ (#2018)
### What this PR does / why we need it?
When using deepseek series models generated by the --dynamic parameter,
if torchair graph mode is enabled, we should modify the configuration
file in the CANN package to prevent incorrect inference results.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/7728dd77bb802e1876012eb264df4d2fa2fc6f3c

---------

Signed-off-by: Wang Kunpeng <1289706727@qq.com>
2025-07-27 08:47:51 +08:00
Mengqing Cao ed2ab8a197 [CI/Build] Upgrade CANN to 8.2.RC1 (#1653)
### What this PR does / why we need it?
Upgrade CANN to 8.2.rc1

Backport: https://github.com/vllm-project/vllm-ascend/pull/1653

### Does this PR introduce _any_ user-facing change?
Yes, docker image will use 8.2.RC1

### How was this patch tested?
CI passed

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/7728dd77bb802e1876012eb264df4d2fa2fc6f3c

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-07-26 22:37:46 +08:00
zhangxinyuehfad d1c640841b [Bugfix] Fix num_hidden_layers when Qwen2-Audio 7B (#1803)
### What this PR does / why we need it?
Fix num_hidden_layers when Qwen2-Audio 7B and #1760 :
```
INFO 07-15 04:38:53 [platform.py:174] PIECEWISE compilation enabled on NPU. use_inductor not supported - using only ACL Graph mode
Traceback (most recent call last):
  File "/workspace/test1.py", line 58, in <module>
    main(audio_count)
  File "/workspace/test1.py", line 38, in main
    llm = LLM(model="Qwen/Qwen2-Audio-7B-Instruct",
  File "/vllm-workspace/vllm/vllm/entrypoints/llm.py", line 271, in __init__
    self.llm_engine = LLMEngine.from_engine_args(
  File "/vllm-workspace/vllm/vllm/engine/llm_engine.py", line 494, in from_engine_args
    vllm_config = engine_args.create_engine_config(usage_context)
  File "/vllm-workspace/vllm/vllm/engine/arg_utils.py", line 1286, in create_engine_config
    config = VllmConfig(
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/pydantic/_internal/_dataclasses.py", line 123, in __init__
    s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
  File "/vllm-workspace/vllm/vllm/config.py", line 4624, in __post_init__
    current_platform.check_and_update_config(self)
  File "/vllm-workspace/vllm-ascend/vllm_ascend/platform.py", line 180, in check_and_update_config
    update_aclgraph_sizes(vllm_config)
  File "/vllm-workspace/vllm-ascend/vllm_ascend/utils.py", line 307, in update_aclgraph_sizes
    num_hidden_layers = vllm_config.model_config.hf_config.num_hidden_layers
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/transformers/configuration_utils.py", line 211, in __getattribute__
    return super().__getattribute__(key)
AttributeError: 'Qwen2AudioConfig' object has no attribute 'num_hidden_layers'
```

### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

Closes: https://github.com/vllm-project/vllm-ascend/issues/1780
https://github.com/vllm-project/vllm-ascend/issues/1760
https://github.com/vllm-project/vllm-ascend/issues/1276
https://github.com/vllm-project/vllm-ascend/issues/359

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/7728dd77bb802e1876012eb264df4d2fa2fc6f3c

Signed-off-by: hfadzxy <starmoon_zhang@163.com>
2025-07-26 20:13:00 +08:00
Yikun Jiang 17a430f7b8 Upgrade vLLM to v0.10.0 (#1927)
### What this PR does / why we need it?
- Upgrade to v0.10.0
- Drop v0.9.2 version compatibility
- Add patch for
`vllm_ascend/patch/worker/patch_common/patch_sampler_gather_logprobs.py`
as workaround of
https://github.com/vllm-project/vllm/commit/f3a683b7c9df8b251092e48e53d58220bb920f2c
for v0.10.0 and also add e2e test `test_models_prompt_logprobs`
- Pin transformers<4.54.0 as workaround of
https://github.com/vllm-project/vllm-ascend/issues/2034

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Test locally:
`VLLM_USE_MODELSCOPE=true pytest -sv
tests/e2e/singlecard/test_offline_inference.py::test_models_prompt_logprobs`
- CI passed

- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/7728dd77bb802e1876012eb264df4d2fa2fc6f3c

---------

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-07-26 15:43:29 +08:00
Li Wang bdfb065b5d [1/2/N] Enable pymarkdown and python __init__ for lint system (#2011)
### What this PR does / why we need it?
1. Enable pymarkdown check
2. Enable python `__init__.py` check for vllm and vllm-ascend
3. Make clean code

### How was this patch tested?


- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/29c6fbe58cfa705c26ed1b38f262d5ade0b4f9ba

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-07-25 22:16:10 +08:00
wangxiyuan 326dcf2576 [Doc] Update support feature (#1828)
The feature support matrix is out of date. This PR refresh the content.

- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/107111a85991cbe9c8d16b697e048d91aa4135f6

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-07-23 15:19:15 +08:00
aidoczhandYikun Jiang c32eea96b7 [Doc]Add Chinese translation for documentation (#1870)
### What this PR does / why we need it?

This PR adds a complete Chinese translation for the documentation using
PO files and the gettext toolchain. The goal is to make the
documentation more accessible to Chinese-speaking users and help the
community grow.

### Does this PR introduce any user-facing change?

Yes. This PR introduces Chinese documentation, which users can access
alongside the original English documentation. No changes to the core
code or APIs.

### How was this patch tested?

The translated documentation was built locally using the standard
documentation build process (`make html` or `sphinx-build`). I checked
the generated HTML pages to ensure the Chinese content displays
correctly and matches the original structure. No code changes were made,
so no additional code tests are required.

vLLM version: v0.9.2  
vLLM main: vllm-project/vllm@5780121

---

Please review the translation and let me know if any improvements are
needed. I am happy to update the translation based on feedback.

- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/7ba34b1241ada58f8212f350a8b17382cb412cf2

---------

Signed-off-by: aidoczh <aidoczh@163.com>
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
2025-07-21 11:26:27 +08:00
Mengqing Cao 8cfd257992 [Dist][EP] Remove ETP/EP maintained in vllm-ascend (#1681)
### What this PR does / why we need it?
Remove ETP/EP maintained in branch main. We drop this as there is no
relevant scenarios to use ETP now, and we may subsequently advocate
implementing expert tensor parallelism in vLLM to support scenarios
where the expert is needed to be sliced

This is a part of #1422 backport.

Fixes https://github.com/vllm-project/vllm-ascend/issues/1396
https://github.com/vllm-project/vllm-ascend/issues/1154

### Does this PR introduce _any_ user-facing change?
We'll not maintain etp/ep in vllm-ascend anymore, and use the tp/ep in
vllm instead.

### How was this patch tested?
CI passed with new added and existing test.


- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/fe8a2c544ad97119f4dafd316e5d9664521b73f9

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-07-21 09:08:04 +08:00
JohnJan 54f2b31184 [Doc] Add a doc for qwen omni (#1867)
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>

### What this PR does / why we need it?
Add FAQ note for qwen omni
Fixes https://github.com/vllm-project/vllm-ascend/issues/1760 issue1



- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/b9a21e9173508e38ac693a8781c48ee24c8873ec
2025-07-20 09:05:41 +08:00
Zhu Yi Lin 538dd357e6 Add graph mode and improve on multi_npu_moge.md (#1849)
### What this PR does / why we need it?
Add graph mode and improve on multi_npu_moge.md

### Does this PR introduce _any_ user-facing change?
yes

### How was this patch tested?
CI passed with new existing test.


- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/5a7fb3ab9ee4bd8174a5100595326f6f40024cdc

Signed-off-by: GDzhu01 <809721801@qq.com>
2025-07-17 17:53:37 +08:00
wangxiyuan eb921d2b6f [Doc] Fix 404 error (#1797)
Fix url 404 error in doc
- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/9ad0a4588ba4e9c979cda0d178dec4fcdb89fd0c

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-07-15 11:52:38 +08:00