Commit Graph
769 Commits
Author SHA1 Message Date
c8d1df3a3f [Refactor][WIP] Refactor mla_v1 by moving all MLA preprocessing ops into mla_v1 attention impl (#2465)
### What this PR does / why we need it?
In order to support fused kernels, multi-stream, communication
optimization etc, it's better to aggregate all opreations in Attention
layer togather. This PR tries to refactor mla_v1 by moving all MLA
preprocessing ops into mla_v1 attention impl.
Note that new mla_v1 doesn't take torchair into consideration. So this
PR can only be merged after torchair related mla_v1 is isolated into a
new file.
### Does this PR introduce _any_ user-facing change?
NO
### How was this patch tested?

### Features Test

<img width="506" height="141" alt="image"
src="https://github.com/user-attachments/assets/f1ab2906-a1ac-4450-8433-94811cd89466"
/>

### Performance After Refact
<img width="648" height="486" alt="image"
src="https://github.com/user-attachments/assets/e33e038c-c5d9-4ba7-a8e9-1ac22f9833eb"
/>

### Performance Before Refact
<img width="618" height="494" alt="image"
src="https://github.com/user-attachments/assets/83861dc2-dc51-4af3-9310-90ab10c43bb1"
/>


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/e03940762b43812fccd3c214bda60201cff9d16a

---------

Signed-off-by: lwq <liwenquan5@huawei.com>
Signed-off-by: whx-sjtu <2952154980@qq.com>
Signed-off-by: SunnyLee219 <3294305115@qq.com>
Co-authored-by: lwq <liwenquan5@huawei.com>
Co-authored-by: whx-sjtu <2952154980@qq.com>
2025-08-28 10:35:57 +08:00
320edde2df [main] [refactor] refactor fused_moe.py to enable token_dispatchers (#2570)
### What this PR does / why we need it?
Enable token_dispatcher to replace fused_experts_with_xxx in eager mode
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
e2e & ut


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/704432af3c129b7a57fca9b059eefe214159f836

Signed-off-by: Pr0Wh1teGivee <calvin_zhu0210@outlook.com>
Co-authored-by: sherie <963372609@qq.com>
Co-authored-by: weijinqian0 <12153182+weijinqian0@users.noreply.github.com>
Co-authored-by: shiyuan680 <72335504+shiyuan680@users.noreply.github.com>
2025-08-28 10:13:35 +08:00
Wang Yixuan 936c102105 [bugfix][refactor]fix torchair_w8a8 (#2569)
### What this PR does / why we need it?
torchair w8a8 and w4a8 Separate from fused_moe due to the refactor and
change for fused_moe

### Does this PR introduce _any_ user-facing change?
NO

### How was this patch tested?
vLLM version: main
vLLM main:
https://github.com/vllm-project/vllm/commit/ab9f2cfd1942f7ddfee658ce86ea96b4789862af


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/69244e67e6822f1c15816f887659e1ccc18c2632

Signed-off-by: hust17yixuan <303660421@qq.com>
2025-08-28 09:10:31 +08:00
Wang Yixuan a955e5d404 [4/N][refactor]delete torchair from quantization (#2535)
### What this PR does / why we need it?
After moved torchair related quantization section into
torchair_quantization, split the torchair from the origin quantization

### Does this PR introduce _any_ user-facing change?
NO

### How was this patch tested?
vLLM version: main
vLLM main:
https://github.com/vllm-project/vllm/commit/ab9f2cfd1942f7ddfee658ce86ea96b4789862af


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/69244e67e6822f1c15816f887659e1ccc18c2632

Signed-off-by: hust17yixuan <303660421@qq.com>
2025-08-28 09:10:03 +08:00
Icey c578f817ca [CustomOp] Register VocabParallelEmbedding instead of overwrite forward (#2515)
### What this PR does / why we need it?
Register VocabParallelEmbedding instead of overwrite forward

### Does this PR introduce _any_ user-facing change?
N/A

### How was this patch tested?
CI passed with new added/existing test.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/644d57d53191b94d9e50a4765891c498790d924b

---------

Signed-off-by: Icey <1790571317@qq.com>
2025-08-28 08:57:34 +08:00
Li Wang 516e14ae6a [Doc] Upgrade to multi-node tutorial model to deepseek-v3.1-w8a8 (#2553)
### What this PR does / why we need it?
Upgrade to multi-node tutorial model to deepseek-v3.1-w8a8
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/de02b07db4741cc9ed40b8262d7a67e6bce30211

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-08-27 14:16:44 +08:00
rjg-lyh 2bfbf9b9b3 [main][bugfix] Fix bugs and refactor cached mask generation logic (#2442)
### What this PR does / why we need it?
This PR fix bugs and refactor cached mask generation logic. Now just
pre-construct and use the cached mask on cpu instead of device on npu.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
CI passed with new added/existing test.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/9b5f64238fbd0f98928587b3426cbf69eea96ae7

Signed-off-by: rjg-lyh <1318825571@qq.com>
2025-08-27 12:07:29 +08:00
huangxialu 6881c19458 [main] convert the format of gmm to nz (#2474)
### What this PR does / why we need it?
convert the format of gmm to nz

### Does this PR introduce _any_ user-facing change?
not involved

### How was this patch tested?
ut: test_fused_ops.py and e2e: test_fused_moe.py

**performance**:
(qwen3 30B, 2k->20k)

base:
Total Token throughput (tok/s):          719.93

gmm nz:
Total Token throughput (tok/s):          728.52


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/bfc1edc9f5bde581e0eec5c830a5a4a7b710fe6a

Signed-off-by: huangxialu <huangxialu1@huawei.com>
2025-08-27 11:25:02 +08:00
wangxiyuan c0e12143a3 [CI] Fix UT failure (#2563)
UT is broken by vLLM commit
https://github.com/vllm-project/vllm/pull/23664

This PR mock the related config to recover the CI

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/6dab89b8ece7e022bd3df5774c9ddf309e2eb2d9

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-08-27 11:24:35 +08:00
Wang Yixuan 20a7bc4b71 [3/N][refactor] refactoer quantization (#2504)
### What this PR does / why we need it?
Move torchair related qunatization section into torchair dir to make the
code clear. Next step we'll remove all torchair related code outside of
torchair quantization.

### Does this PR introduce _any_ user-facing change?
NO

### How was this patch tested?
vLLM version: main
vLLM main:
https://github.com/vllm-project/vllm/commit/ab9f2cfd1942f7ddfee658ce86ea96b4789862af


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/959783fb996d0d15598f45ca12ffcbee4b681424

Signed-off-by: hust17yixuan <303660421@qq.com>
2025-08-27 10:45:50 +08:00
weiguihua2 acdc53c2f6 [Bugfix] Fix the bug of cos invalid shape when dp (#2558)
### What this PR does / why we need it?
 Fix the bug of cos invalid shape when dp

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/1fdc732419d9b9eb00e003f38d6e02c480131ac8

Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
2025-08-27 10:36:23 +08:00
Mengqing Cao a9e78a3299 [Aclgraph] Update compilation config in check_and_update_config (#2540)
### What this PR does / why we need it?
This pr updates compilation config in `check_and_update_config`, we use
`compilation_config.level` to update `compilation_config.cudagraph_mode`
to ensure the config is correct.

Add `compilation_config.cudagraph_num_of_warmups = 1` when V1 is
enabled, cause this is also used in torchair graph mode. and this fixes
https://github.com/vllm-project/vllm-ascend/issues/2523

fix the bug that the `aclgraphmode` always be `NONE` while running
forward in aclgraph mode

### How was this patch tested?
CI passed with new added/existing test.


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/f58675bfb36b67cdbca4d2356a2f580e7a706ec3

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-27 09:30:25 +08:00
wangxiyuan f22077daa6 [Embedding] Recover embedding function (#2483)
Fix broken embedding function. It's broken by
http://github.com/vllm-project/vllm/pull/23162

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/efc88cf64a399f5459cd6256223e99672c13614d

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-08-27 09:22:01 +08:00
s30076806 6a4ec186e7 [Qwen-moe] Remove the minor operation arange (#2373)
### What this PR does / why we need it?
Integrate the arange operator to reduce the time spent and improve
performance

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/56dcf4e7e965e34043acf20ca4e4aceda21d41ec

---------

Signed-off-by: s30076806 <songjiayang2@h-partners.com>
2025-08-27 09:13:31 +08:00
rjg-lyh 358ba68994 [main][bugfix] Fix MatmulNZ format bug on some machines (#2549)
### What this PR does / why we need it?
This PR fixes the bug on some machines where quantmatmul failed to run
with the NZ format. The change ensures proper execution under the
expected data layout.

### How was this patch tested?
CI passed with existing test.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/b5d34af3286ee0334d9f7bd729774ac55c5805e9

Signed-off-by: rjg-lyh <1318825571@qq.com>
2025-08-27 09:08:17 +08:00
Li Wang 042605f4b2 [Doc] Add stable modelslim branch (#2545)
### What this PR does / why we need it?
The branch `br_release_MindStudio_8.1.RC2_TR5_20260624` is commercial
delivery version of modelslim in Q3, and has been verified available
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/7d67a9d9f93f86b74066c64c373405aa088e4a16

Signed-off-by: wangli <wangli858794774@gmail.com>
2025-08-27 09:05:46 +08:00
zhanghw0354andzhanghaiwen 8151a9d5a4 [Test]Add unit test for worker_v1.py (#2547)
### What this PR does / why we need it?
According to issue
https://github.com/vllm-project/vllm-ascend/issues/1298 ,this pull
request adds unit test code for platform.py.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed with new added/existing test.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/b5d34af3286ee0334d9f7bd729774ac55c5805e9

---------

Signed-off-by: zhanghaiwen <zhanghaiwen@cmss.chinamobile.com>
Co-authored-by: zhanghaiwen <zhanghaiwen@cmss.chinamobile.com>
2025-08-26 22:00:49 +08:00
yiz-liu a6bb502e70 [2/N][Feat] Add MC2 communication method for MoE layers (#2469)
### What this PR does / why we need it?
This method replaces the previous all-gather approach for small numbers
of tokens.

The key changes include:
- A new `AscendFusedMoE` layer that handles token splitting, local
computation, and final aggregation via all-gather.
- Logic in the model runner to dynamically select between the new MC2
method and the existing all-gather method based on the number of input
tokens.
- Sharding the MoE communication mask across tensor-parallel ranks.

### Does this PR introduce _any_ user-facing change?
None.

### How was this patch tested?
Test case fixed.


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/b00e69f8ca55f4a82847d39466f57ceb748324c1

---------

Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
2025-08-26 19:05:23 +08:00
Wang Yixuan 5d8ec28009 [2/N][refactor] split torchair from fused_moe (#2503)
### What this PR does / why we need it?
After moved torchair related fused_moe section into torchair_fused_moe,
split the torchair from the origin fused_moe

### Does this PR introduce _any_ user-facing change?
NO

### How was this patch tested?
vLLM version: main
vLLM main:
https://github.com/vllm-project/vllm/commit/ab9f2cfd1942f7ddfee658ce86ea96b4789862af


- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/2a97ffc33de097f267f217132ced42f4714b7de5

Signed-off-by: hust17yixuan <303660421@qq.com>
2025-08-26 14:12:43 +08:00
lilinsiman cfe77e83ae [Bugfix]Support Qwen3-MOE on aclgraph mode in sizes capture and add new ut (#2511)
[Bugfix]Support Qwen3-MOE on aclgraph mode in sizes capture and add new
ut

What this PR does / why we need it?
This PR solves the problem of sizes capture and stream error caused by
using ACLgraph on the Qwen3-30B MOE model.
Add new ut.

Does this PR introduce any user-facing change?
no

How was this patch tested?
ut

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/6fad29b11b3680c44782cd6e5fe555779d620d6c

Signed-off-by: lilinsiman <lilinsiman@gmail.com>
2025-08-26 12:39:21 +08:00
zhanghw0354andzhanghaiwen b3fdd78a6b [Main][Refactor]Change ASCEND_QUATIZATION_METHOD to ASCEND_QUANTIZATION_METHOD (#2517)
### What this PR does / why we need it?
The constant ASCEND_QUATIZATION_METHOD in vllm_ascend/utils.py is
misspelled and should be corrected to ASCEND_QUANTIZATION_METHOD.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed with new added/existing test.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/c9abb104893b16958d8c73f927ea37e65985f4ca

Signed-off-by: zhanghaiwen <zhanghaiwen@cmss.chinamobile.com>
Co-authored-by: zhanghaiwen <zhanghaiwen@cmss.chinamobile.com>
2025-08-26 09:06:16 +08:00
Mengqing Cao 21b5727f9a [CI] Upgrade vllm in accuracy and performance CI (#2527)
### What this PR does / why we need it?
Upgrade vllm in accuracy and performance CI

### How was this patch tested?
CI passed with existing test.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/5c4b6e66fec0a59f702e32ec5c476cdd8a800566

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-26 08:49:49 +08:00
wangxiyuan 7e494e94a9 [CI] Fix broken ci (#2530)
vLLM commit https://github.com/vllm-project/vllm/pull/22711 changed the
encode cache entries logic, this PR adapt the same change for vllm
ascend to make CI happy.

Co-Authored-By: zhoux77899 <zhouxiang100@huawei.com>

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/0ff902f3b48376aadfe4e753ce0872b49c8f9f7e

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-08-26 07:42:24 +08:00
yiz-liu 99bf25af76 [Fix] Add operations in _dummy_run to maintain synchronization with _process_reqs, resolving a service hang (#2454)
### What this PR does / why we need it?
Fixes hang when batch size < DP size.

### Does this PR introduce _any_ user-facing change?
None.

### How was this patch tested?
After this change, the function in DP case will work now.

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/d9a55204bad7bde505624f4ffb0464d98c86914a

Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
2025-08-25 19:56:02 +08:00
wangxiyuan de7649492d [Refactor] cleanup converting_weight_acl_format_format (#2482)
move maybe_converting_weight_acl_format_format to torchair module, it's
only used with 310p+torchair

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/49ab23b3ccc2da9274c739d55f9b19206078c7a9

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-08-25 19:48:55 +08:00
Wang Yixuan 0f81e032f0 [1/N][refactor] torchair fused_moe refactor (#2438)
### What this PR does / why we need it?
Move torchair related fused_moe section into torchair_fused_moe to make
the code clear. Next step we'll remove all torchair related code outside
of torchair_fused_moe .

### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
vLLM version: v0.10.0
vLLM main:
https://github.com/vllm-project/vllm/commit/08d5f7113a024818b2867782c2539794b7aa162b

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/170e8ea9ea95294d1bdc4af4bea73741ea759e22

Signed-off-by: hust17yixuan <303660421@qq.com>
2025-08-25 15:46:10 +08:00
Shanshan Shen 334c44613a [Doc] Update release version info (#2518)
### What this PR does / why we need it?
Update release version info.

### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/712d0f88d89940291b6a6a4db635c60187398acd

Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2025-08-25 15:39:10 +08:00
Shanshan Shen 98c68220c1 [Doc] Update v0.9.1rc3 doc (#2512)
### What this PR does / why we need it?
Update `v0.9.1rc3` doc, which are supplements to
https://github.com/vllm-project/vllm-ascend/pull/2488.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/170e8ea9ea95294d1bdc4af4bea73741ea759e22

Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
2025-08-25 11:39:29 +08:00
Mengqing Cao 4c4ffeebe5 [Doc] update vllm version in ci (#2513)
### What this PR does / why we need it?
update vllm version in ci

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/170e8ea9ea95294d1bdc4af4bea73741ea759e22

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-25 11:35:37 +08:00
Shanshan Shen 0767d51dd5 [Structured Output][CI] Add test for outlines backend for structured output in CI (#2283)
### What this PR does / why we need it?
Add test for `outlines` backend for structured output in CI.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?

Tests have all passed with:

```bash
pytest -sv tests/e2e/singlecard/test_guided_decoding.py
```

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71

---------

Signed-off-by: shen-shanshan <467638484@qq.com>
2025-08-25 09:59:13 +08:00
Icey 891b2bfe71 Accuracy report formatting (#2279)
### What this PR does / why we need it?
Accuracy report formatting

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
CI passed with existing test.


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71

---------

Signed-off-by: Icey <1790571317@qq.com>
2025-08-25 09:39:30 +08:00
Icey f796e6280b [CustomOp] Register RotaryEmbedding instead of overwrite forward (#2385)
### What this PR does / why we need it?
Register RotaryEmbedding instead of overwrite forward

### Does this PR introduce _any_ user-facing change?
N/A

### How was this patch tested?
CI passed with new added/existing test.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/808d2e9aa0f302bf9667b09b9dcf297f86927dac

---------

Signed-off-by: Icey <1790571317@qq.com>
Signed-off-by: wxsIcey <1790571317@qq.com>
2025-08-25 09:32:35 +08:00
weichen 950c4b219a [main] refactor alltoallv in fused_moe (#2487)
### What this PR does / why we need it?
Refactor all2all-related fused_experts (both quantized/unquantized) into
TokenDispatcherWithAll2AllV, including dispatch & combine calculation.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
E2E & UT
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/65197a5fb37ef4d8b93e0b99ecc8b902fe948e97

Signed-off-by: Pr0Wh1teGivee <calvin_zhu0210@outlook.com>
2025-08-23 20:38:17 +08:00
linfeng-yuan 4af5b80606 [Scheduler] validate max_num_batched_tokens and max_model_len in AscendSchedulerConfig (#2434)
### What this PR does / why we need it?
Add configuration check logic for ascend scheduler: if chunked_prefill
is disabled, max_num_batched_tokens couldn't be less than max_model_len,
following vLLM;

### Does this PR introduce _any_ user-facing change?
users cannot set max_num_batched_tokens smaller than max_model_len with
ascend scheduler
### How was this patch tested?
CI and vllm serving passed

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/f77a0802b758a32c5b9f7bc04e9498d77e8d99e0

Signed-off-by: linfeng-yuan <1102311262@qq.com>
2025-08-23 19:39:44 +08:00
ZhaoJiangJiangand赵江江 3629bc4431 feat: add mtp ut and fix some bugs (#2453)
### What this PR does / why we need it?
Fix mtp mode ut

### Does this PR introduce _any_ user-facing change?
Nothing

### How was this patch tested?
This can be tested in the same way as a unit test.


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71

Signed-off-by: 赵江江 <zhaojiangjiang1@h-partners.com>
Co-authored-by: 赵江江 <zhaojiangjiang1@h-partners.com>
2025-08-22 17:09:08 +08:00
weiguihua2 dd04a96ee3 [Bugfix] Fix the bug of incorrect precision (#2479)
### What this PR does / why we need it?
Fix the bug of incorrect precision

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71

---------

Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
2025-08-22 17:08:56 +08:00
Shanshan Shen f0be3eed84 [Doc] Add release note for v0.9.1rc3 (#2488)
### What this PR does / why we need it?

Add release note for `v0.9.1rc3`.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71

---------

Signed-off-by: shen-shanshan <467638484@qq.com>
2025-08-22 16:06:29 +08:00
Mengqing Cao 60ac4fb576 [QuickFix] Skip failed ut to recover CI quickly (#2484)
### What this PR does / why we need it?
Skip failed ut to recover CI quickly
related ut:
- `test_embed_models_correctness`: revert me when pooler is adapted with
the latest vllm main
- `test_check_and_update_config_enforce_eager_mode`: revert me when the
occasional failed is fixed

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/8896eb72ebe90be6f1b83f32b3d3d0c379db4f82

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-22 14:14:51 +08:00
LookAround0301 e9fb895b10 [Doc] Add feature branch long_seq_optimization (#2477)
### What this PR does / why we need it?
Add cp/sp feature branch

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/0c6e40bbaa4707528286a1e7bf17c90c88a1d920

Signed-off-by: LookAround <lixushi@huawei.com>
2025-08-22 08:53:12 +08:00
Mengqing Cao b0403f8d8a [CI] fix ci (#2464)
### What this PR does / why we need it?
1. use action/checkout@v5 instead of v4
2. remove dbo test case because there is issue with it and will be
refactored later
3. make vllm-ascend compatible with vllm v0.10.1.1 and add CI for it
4. fix sampler api changes introduced by
https://github.com/vllm-project/vllm/pull/22387
6. fix qwen3 moe config changes intruoduced by
https://github.com/vllm-project/vllm/pull/20562
7. fix kvcache block changes introduced by
https://github.com/vllm-project/vllm/pull/23262

### Does this PR introduce _any_ user-facing change?
N/A

### How was this patch tested?
CI passed with existing test.


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/0c6e40bbaa4707528286a1e7bf17c90c88a1d920

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
2025-08-22 07:30:48 +08:00
linfeng-yuan 0ca3f48c90 [2/N][refactor] torchair deepseek mla backend refactor (#2459)
### What this PR does / why we need it?
This PR move current unified mla backend to torchair folder and remove
torchair-related code in attention/mla_v1.py (1.3k -> 0.9k).

 
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Running eager mode with mla backend, and torchair mode with code before
[2445](https://github.com/vllm-project/vllm-ascend/pull/2445)


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/f571ff8eb6d9117c6a418f7f925921968dff8ac8

Signed-off-by: linfeng-yuan <1102311262@qq.com>
2025-08-21 14:02:30 +08:00
Yikun Jiang 67a222c383 [Doc] Add feature branch policy (#2432)
### What this PR does / why we need it?

This patch add the feature branch policy.

After this patch: maintainers are allowed to create a feature branch.
Feature branches are used for collaboration and must include an RFC
link, merge plan and mentor info.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

CI passed

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/7be5d113d8784536b79f27f24cfa91958dc291b0

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-08-21 10:37:21 +08:00
sherieandwangxiaoxin-sherie 3fb80ee356 add mlp tp optimze (#2120)
### What this PR does / why we need it?
For dense models, by not applying tensor parallelism (TP) to the
attention module and applying TP to the MLP module, the allreduce
operations in the attention module can be eliminated, thereby reducing
computational overhead. However, this approach increases memory usage,
so the environment variable VLLM_ASCEND_ENABLE_MLP_OPTIMZE is used to
control this optimization.

- vLLM main:
https://github.com/vllm-project/vllm/commit/b17109beeafbf9577c319ab61530810943a7fc4b

Signed-off-by: wangxiaoxin-sherie <wangxiaoxin7@huawei.com>
Co-authored-by: wangxiaoxin-sherie <wangxiaoxin7@huawei.com>
2025-08-21 09:22:07 +08:00
yupeng 973a7cfdf0 [DOC] update doc: LoRA with ACLGraph (#2430)
### What this PR does / why we need it?
Update DOC. Guide users to run LoRA with ACLGraph.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
No.

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/de7b67a0232e35ae8e8ecd944aeddfc8cbc02631

---------

Signed-off-by: paulyu12 <507435917@qq.com>
2025-08-21 08:55:55 +08:00
weiguihua2 0dca4c6dbd refact runner model v1 (#2461)
refact model runner v1

### What this PR does / why we need it?
1. Separate the execute model logic from the prepare input logic
2. Disassemble the torchchair in model runner v1

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/68fcd3fa7313d00240f766f42affe931f1f379a7

---------

Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
2025-08-21 08:54:57 +08:00
Wang Kunpeng 1de16ead8e [main][bugfix] Modify the default value of the enable_shared_pert_dp to false (#2457)
### What this PR does / why we need it?
enable_shared_pert_dp is currently on by default. This optimization is
currently only valid for deepseek series models. The default opening
affects the accuracy of the qwen series models.
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?
use parameter --additional_config='{"enable_shared_expert_dp": true}'

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/d983769c41db224e0897fac2e9aefc5f57ad1122

Signed-off-by: Wang Kunpeng <1289706727@qq.com>
2025-08-20 20:25:53 +08:00
Wang Kunpeng c40d4171bc [main][quantization] Adapt to the new format of ds w4a8 weight (#2392)
### What this PR does / why we need it?

The deepseek w4a8 weights we supported before were in mindie-format
format. It uses int8 to represent int4, so the weight size is similar to
w8a8, and we need to do a few extra steps to make vllm-ascend load it
normally.

Now we can directly use the new weight format, which uses two int4 packs
to save the weight, the weight size is reduced, and there is no need to
do many extra operations to directly use it on vllm-ascend, but we are
also compatible with the weights of the previous mindie format.

The weight changes in the new version: 
1. The weight is packed (2 int4 pack to int8)
2. The bias required in the apply method is directly generated by
modelslim

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?

Adding ut case in `tests/ut/quantization/test_w4a8_dynamic.py`

#### 1.How to get weights using Modelslim

##### Installation steps

we can use the branch br_release_MindStudio_8.1.RC2_TR5_20260624
git clone -b br_release_MindStudio_8.1.RC2_TR5_20260624
https://gitee.com/ascend/msit.git
cd msit/msmodelslim
bash install.sh

##### Generate w4a8 weights

cd /example/DeepSeek
Command reference: msmodelslim/example/DeepSeek/README.md Execute the
[pre-check](https://gitee.com/ascend/msit/blob/br_release_MindStudio_8.1.RC2_TR5_20260624/msmodelslim/example/DeepSeek/README.md#%E8%BF%90%E8%A1%8C%E5%89%8D%E5%BF%85%E6%A3%80)
and [DeepSeek-R1 w4a8 mix
quantization](https://gitee.com/ascend/msit/blob/br_release_MindStudio_8.1.RC2_TR5_20260624/msmodelslim/example/DeepSeek/README.md#deepseek-r1-w4a8-%E6%B7%B7%E5%90%88%E9%87%8F%E5%8C%96%E5%89%8D%E4%B8%89%E5%B1%82-mlpw8a8-dynamic-%E9%87%8F%E5%8C%96mla%E5%85%B1%E4%BA%AB%E4%B8%93%E5%AE%B6w8a8%E9%87%8F%E5%8C%96%E8%B7%AF%E7%94%B1%E4%B8%93%E5%AE%B6w4a8-dynamic%E9%87%8F%E5%8C%96)
chapter
Reference command:python3 quant_deepseek_w4a8.py --model_path {Original
weight path} --save_path {Generate weight path}

##### Adapt to vllm-ascend

Modification in `config.json`:`"model_type":deepseekv2` is changed to
`"model_type":deepseek_v3`;

#### 2.How to run w4a8

##### a.How to run eager mode

export VLLM_ASCEND_MLA_PA=1

python -m vllm.entrypoints.openai.api_server --model=$1
--trust-remote-code -tp $2 -dp $3 --enable_expert_parallel
--quantization ascend --port $4 --max-model-len $5 --max-num-seqs $6
--enforce-eager
eg: python -m vllm.entrypoints.openai.api_server
--model=/weightpath/w4a8_4_layer --trust-remote-code -tp 4 -dp 4
--enable_expert_parallel --quantization ascend --port 8002
--max-model-len 5120 --max-num-seqs 128 --enforce-eager

##### b.How to run graph mode

export HCCL_BUFFSIZE=1024

python -m vllm.entrypoints.openai.api_server --model=$1
--trust-remote-code -tp $2 -dp $3 --enable_expert_parallel
--quantization ascend --port $4 --max-model-len $5
--additional_config='{"ascend_scheduler_config":{"enabled":true},"torchair_graph_config":{"enabled":true}}'
eg: python -m vllm.entrypoints.openai.api_server
--model=/weight/dsr1_w4a8_vllm --trust-remote-code -tp 4 -dp 4
--enable_expert_parallel --quantization ascend --port 8002
--max-model-len 5120
--additional_config='{"ascend_scheduler_config":{"enabled":true},"torchair_graph_config":{"enabled":true}}'


- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/103f1ec8d348a5f336f11d972d6285c4fb4736d4

---------

Signed-off-by: Wang Kunpeng <1289706727@qq.com>
2025-08-20 20:25:18 +08:00
wangxiyuanandMengqingCao eccfb715f6 [CI] Fix UT (#2452)
Make UT CI happy 

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/d983769c41db224e0897fac2e9aefc5f57ad1122

---------

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: MengqingCao <cmq0113@163.com>
Co-authored-by: MengqingCao <cmq0113@163.com>
2025-08-20 16:26:07 +08:00
sherieandwangxiaoxin-sherie 3f867ee708 refactor allgather/mc2-related fused_experts (#2369)
### What this PR does / why we need it?
refactor allgather/mc2-related fused_experts

- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/de7b67a0232e35ae8e8ecd944aeddfc8cbc02631

Signed-off-by: wangxiaoxin-sherie <wangxiaoxin7@huawei.com>
Co-authored-by: wangxiaoxin-sherie <wangxiaoxin7@huawei.com>
2025-08-20 14:20:46 +08:00
wangxiyuan 73acdcfc3b [PD] Correct the ip and port env (#2450)
1. rename `VLLM_LLMDD_RPC_PORT` to `VLLM_ASCEND_LLMDD_RPC_PORT` to make
the prefix the same in vllm-ascend
2. enable `VLLM_ASCEND_LLMDD_RPC_IP` env for PD feature.

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2025-08-20 11:39:05 +08:00