Mengqing Cao
5d59bf8ca0
[CI] unblock CI on suffix spec decoding ( #4813 )
...
### What this PR does / why we need it?
unblock CI on suffix spec decoding
### How was this patch tested?
CI passed with existing test.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: MengqingCao <cmq0113@163.com >
2025-12-20 14:54:49 +08:00
wangxiyuan
758d81dcb1
Drop 0.12.0 support ( #5146 )
...
We decided to release v0.13.0 soon. So no need to support 0.12.0 now.
Let's drop it.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2025-12-20 09:38:53 +08:00
Li Wang
243ab7d720
[CI] Use offline mode for nightly test ( #5187 )
...
### What this PR does / why we need it?
For single node test, the lack of a retry mechanism for accessing
ModelScope resulted in an HTTP 400 error sometimes. I recommend using a
local offline cache instead.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: wangli <wangli858794774@gmail.com >
2025-12-19 21:21:42 +08:00
XiaoxinWang and wangxiaoxin-sherie
0cc3fc357f
[pref] qwen3_next add triton ops : fused_sigmoid_gating_delta_rule_update ( #4818 )
...
### What this PR does / why we need it?
qwen3_next add fused_sigmoid_gating_delta_rule_update op which fused
fused_gdn_gating+fused_recurrent_gated_delta_rule
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: wangxiaoxin-sherie <wangxiaoxin7@huawei.com >
Co-authored-by: wangxiaoxin-sherie <wangxiaoxin7@huawei.com >
2025-12-19 16:34:11 +08:00
wangqiankun13
118b0ed346
[Feature] Add token mask for DispatchGmmCombineDecode operator ( #5171 )
...
### What this PR does / why we need it?
In this PR, DispatchGmmCombineDecode add an optional input
x_active_mask, with which
only token masked True will be dispatched and handle.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: wangqiankun <wangqiankun13@huawei.com >
2025-12-19 16:31:48 +08:00
zzzzwwjj
cc23067f1e
[refactor] refactor weight trans nz and transpose ( #4878 )
...
### What this PR does / why we need it?
Now `VLLM_ASCEND_ENABLE_NZ` will have three options:
0: disable nz;
1: only quant case enable nz;
2: enable nz as long as possible;
And `VLLM_ASCEND_ENABLE_NZ`=1 by default.
All cases are shown in the table below:
| | W4A4 | W4A8 | W8A8 | fp16/bf16 | fp32 |
|---|---|---|---|---|---|
| trans nz | can't support nz | trans nz by default | trans nz by
default | trans nz when VLLM_ASCEND_ENABLE_NZ is 2 | can't support nz |
| transpose | only support not transpose case | only support transpose
case | only support transpose case | linear: only support not transpose
case<br>gmm: only support transpose case | same to fp16/bf16 |
Some exceptional cases:
1. MLAPO op need to do some additional processing on the weights,
including trans nz. If use MLAPO op, some weight will be transformed to
nz forcely;
2. MLA/SFA's weight `W_UV` will be used by op
`torch.ops._C_ascend.batch_matmul_transpose`, and this op can't support
nz currently;
### Does this PR introduce _any_ user-facing change?
Now fp16/bf16 weight will not trans nz by default.
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: zzzzwwjj <1183291235@qq.com >
2025-12-19 14:27:24 +08:00
LookAround0301
76e58d66be
support basic long_seq feature st ( #5140 )
...
### What this PR does / why we need it?
support basic long_seq feature st
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: LookAround <lixushi@huawei.com >
2025-12-19 10:50:01 +08:00
weichen
ca6f631cba
[2/N][Pangu][MoE] Remove Pangu Related Code ( #5130 )
...
### What this PR does / why we need it?
Remove Pangu Related Code
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
e2e & ut
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: weichen <calvin_zhu0210@outlook.com >
2025-12-19 09:00:07 +08:00
zhaomingyu13
73e4b4f496
[BugFix] Fix top_p,top_k issue with EAGLE and add top_p,top_k in EAGLE e2e ( #5131 )
...
### What this PR does / why we need it?
Add top_p,top_k in EAGLE e2e
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: zhaomingyu <zhaomingyu13@h-partners.com >
2025-12-18 23:07:14 +08:00
zxr2333
073a3a6e6c
[Doc][P/D] Fix MooncakeConnector's name ( #5172 )
...
### What this PR does / why we need it?
vLLM community has integrated their MooncakeConnector. The original
scripts will now find this MooncakeConnector instead of the one from
vLLM-Ascend. All scripts that involve using the MooncakeConnector need
to be modified to another name.
### Does this PR introduce _any_ user-facing change?
Yes, users need to use a new name to load vLLM-Ascend MooncakeConnector.
### How was this patch tested?
By CI.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: nwpu-zxr <zhouxuerong2@huawei.com >
2025-12-18 22:29:19 +08:00
ZT-AIA
6cb76ecd02
[Nightly] Avoid max_model_len being smaller than the decoder prompt to prevent single-node-accuray-tests from failing ( #5174 )
...
### What this PR does / why we need it?
[Nightly] Avoid max_model_len being smaller than the decoder prompt to
prevent single-node-accuray-tests from failing
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: ZT-AIA <1028681969@qq.com >
Signed-off-by: ZT-AIA <63220130+ZT-AIA@users.noreply.github.com >
2025-12-18 22:25:45 +08:00
Angazenn
acc3578f58
[Graph][Fusion]Add new pattern for AddRmsnormQuant with SP. ( #5077 )
...
### What this PR does / why we need it?
1. In addition to
[#4168 ](https://github.com/vllm-project/vllm-ascend/pull/4168 ),
[#5011 ](https://github.com/vllm-project/vllm-ascend/pull/5011 ), this PR
adds two more pattern for AddRmsnormQuant with SP enabled. The key
difference is to insert an additional `maybe_all_gather_and_maybe_unpad`
between `addrmsnorm` and `quantize`.
2. This PR also introduce another api `torch.ops.vllm.quantize`, so that
we pass `input_scale` and `input_scale_reciprocal` at the same time.
This is because `npu_add_rms_norm_quant` and `npu_quantize` requires
different `div_mode`. To avoid introducing additional reciprocal
calculation in runtime, we have to pass both of them to quantize api.
3. Removes redundant `AscendQuantRmsnorm`.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: Angazenn <supperccell@163.com >
2025-12-18 20:25:44 +08:00
a74a1196c5
[Feat] Support MLP_TP feature, exclude MOE layer ( #4999 )
...
#4257 This PR implements the dense_ffn TP of the first three layers of
the deepseek model, I have refactored this PR and used very little code
to support the implementation of this feature.
This PR adds a function `is_moe_layer` to mlp_tp, which supports MLP TP
in models with both mlp and moe, such as deepseek or chat GLM.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: zzhx1 <zzh_201018@outlook.com >
Co-authored-by: 子潜 <ziqian@U-DMKXH32D-2015.local >
Co-authored-by: chenxiao <Jaychou1620@Gmail.com >
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com >
2025-12-18 20:06:53 +08:00
Ronald
b69b04d3a9
implement model runner v2 basic framework ( #5051 )
...
### What this PR does / why we need it?
This PR aim to implement model runner v2 basic framework in vllm-ascend,
the e2e function is not guaranteed by this pr.
### Does this PR introduce _any_ user-facing change?
use envs.VLLM_USE_V2_MODEL_RUNNER to decide if choose model_runenr_v2.
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
2025-12-18 15:51:54 +08:00
78602eab4f
[UT] Add mooncake ut test ( #5080 )
...
### What this PR does / why we need it?
Add UT for mooncake
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: tongyuzhou <tongyuzhou1@huawei.com >
Signed-off-by: wangxiaochao <w00642655@china.huawei.com >
Co-authored-by: tongyuzhou <tongyuzhou1@huawei.com >
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com >
2025-12-18 15:07:14 +08:00
9045843c90
[UT]Ut for function cumsum_group_list in moe_mlp (ref #5025 ) ( #5036 )
...
### What this PR does / why we need it?
Add ut for the cumsum_group_list function, which is related to the
precision issues stemming from the moe_mlp.py .
The ralated PR is https://github.com/vllm-project/vllm-ascend/pull/5025
### Does this PR introduce _any_ user-facing change?
No
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: tanqingshan (A) <50050625@china.huawei.com >
Co-authored-by: tanqingshan (A) <50050625@china.huawei.com >
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com >
Co-authored-by: weijinqian0 <1184188277@qq.com >
2025-12-18 15:00:16 +08:00
ck-hw-1018 and cuikai
71e544e259
[test] add w4a8 accuracy case ( #5110 )
...
### What this PR does / why we need it?
This PR add w4a8 accuracy testcase for e2e test
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
By running the test
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: cuikai (C) <c00827167@china.huawei.com >
Co-authored-by: cuikai (C) <c00827167@china.huawei.com >
2025-12-18 14:10:14 +08:00
zhangsicheng5 and wangxiyuan
07014e2101
[UT] Add model_runner pcp related UTs ( #4951 )
...
1. Add some uts for pcp related functions in NPUModelRunner
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: zhangsicheng5 <zhangsicheng5@huawei.com >
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com >
2025-12-18 10:54:57 +08:00
8069442b41
enable npugraph_ex ( #5120 )
...
### What this PR does / why we need it?
We will expose the enabling switch for npugraph_ex to better facilitate
subsequent optimization.
### Does this PR introduce _any_ user-facing change?
Previously, the enable_npugraph_ex switch would trigger an error; now we
have removed the error reporting mechanism to better facilitate
subsequent optimization efforts.
Basic functionalities are available in CANN and torch_npu for Q3, while
advanced optimizations will depend on the Q4 release.
### How was this patch tested?
llm =LLM(
model=model,
enforce_eager=False ,
additional_config={
"enable_npugraph_ex": True
},
compilation_config={
"cudagraph_mode": "FULL_DECODE_ONLY",
"cudagraph_capture_sizes": [16],
},
}
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: p00465316 <panchao13@huawei.com >
Co-authored-by: p00465316 <panchao13@huawei.com >
Co-authored-by: weijinqian0 <1184188277@qq.com >
2025-12-18 09:08:40 +08:00
shaopeng-666
39bdd4cfaa
fix profile run for vl model ( #5136 )
...
### What this PR does / why we need it?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: 李少鹏 <lishaopeng21@huawei.com >
2025-12-17 23:51:31 +08:00
zhenwenqi2024 and Mengqing Cao
950570f8d1
[Bugfix]delele profile_run in model_runner ( #5122 )
...
### What this PR does / why we need it?
delete sekf.in_profile_run in model_runner to make EPLB works as expect
### Does this PR introduce _any_ user-facing change?
NO
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: zhenwenqi2024 <zhenwenqi_2022@qq.com >
Signed-off-by: Mengqing Cao <cmq0113@163.com >
Co-authored-by: Mengqing Cao <cmq0113@163.com >
2025-12-17 23:48:34 +08:00
weijinqian0 and weijinqian_v1
98e6e57622
[Refactor] 4/N Distinguish the branches based on the applicable scenarios of PA and FIA Ops. ( #5081 )
...
RFC: https://github.com/vllm-project/vllm-ascend/issues/4629
Reason:
We distinguish the branches based on the applicable scenarios of
pagedAttention and fusedInferAttention, making the code more clear.
At the same time, it is convenient for the subsequent iterations of
sliding_window and sinks and removePA ops after FIA is ready.
Todo:
remove PA ops after FIA is ready
add slidingwindow and ops for gpt_oss
replace FIA with FIA_v2
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com >
Co-authored-by: weijinqian_v1 <weijinqian@huawei.com >
2025-12-17 23:14:02 +08:00
dsxsteven
97537709ae
[BugFix] Fix mooncake bug in PCP scenario ( #5055 )
...
### What this PR does / why we need it?
The mooncake_connector.py file was importing the wrong arguments to the
file, which could cause errors when use PCP; this issue has been
corrected.
### Does this PR introduce _any_ user-facing change?
NO
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: daishixun <dsxsteven@sina.com >
2025-12-17 16:32:16 +08:00
Feng Liu and F.Liu
eda3cabf5b
[UT] add pcp&dcp UT for mla_cp ( #4953 )
...
### What this PR does / why we need it?
Add UT of mla_cp, which include:
- test_compute_prefill_context_with_dcp_pcp
- test_reorg_kvcache_with_dcp_pcp
- test_out_lse_reshape
- test_npu_attention_update_with_dcp_pcp
- test_attention_with_mask_and_nomask_with_dcp_pcp
- test_process_attn_out_lse_with_dcp_pcp
- test_forward_prefill_cp_with_dcp_pcp
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: F.Liu <liufeng248@huawei.com >
Co-authored-by: F.Liu <liufeng248@huawei.com >
2025-12-17 16:19:27 +08:00
weichen
f0060fc822
[Pangu][MoE] Remove PanguProMoEV1 related code ( #5088 )
...
### What this PR does / why we need it?
PanguProMoEV1 is no longer supported in vllm-ascend, remove related
code.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
e2e & ut
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: weichen <calvin_zhu0210@outlook.com >
2025-12-17 16:14:42 +08:00
zzzzwwjj
06b82e7503
[main] rename device type ( #5099 )
...
### What this PR does / why we need it?
Rename `_910B` to `A2`;
Rename `_910_93` to `A3`;
Rename `_910_95` to `A5`;
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: zzzzwwjj <1183291235@qq.com >
2025-12-17 14:08:19 +08:00
wangxiyuan
4144376e88
[CI] Fix UT ( #5106 )
...
Fix broken ut introduced by #5053
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2025-12-17 09:52:20 +08:00
pichangping
06f33540c4
[UT]add the UT of pcp and dcp in the attention_cp file ( #5054 )
...
### What this PR does / why we need it?
add the UT of pcp and dcp in the attention_cp file
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: pichangping <1337510399@qq.com >
2025-12-17 09:11:33 +08:00
Icey
cadfa5ddc1
[Fusion] [Graph] Add qknorm rope fusion operator ( #4711 )
...
### What this PR does / why we need it?
This PR add `qkv_rmsnorm_rope` operator and introduces a graph fusion
pass for `qknorm_rope` operations. The implementation includes a new
configuration flag, a pattern matching pass using
`torch._inductor.pattern_matcher`, and a custom Triton kernel for the
fused operation.
Co-authored-by: Angazenn
[supperccell@163.com ](mailto:supperccell@163.com )
### Does this PR introduce _any_ user-facing change?
Yes, add new additional_config
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: wxsIcey <1790571317@qq.com >
2025-12-17 08:53:44 +08:00
ZixuanWang and leo-pony
b1a853b0f6
Upgrade vllm commit hash to 1216 ( #5053 )
...
### What this PR does / why we need it?
Upstream vLLM PR #30212 https://github.com/vllm-project/vllm/pull/30212
refactored the attention backend selection interface, This PR adapts
vllm-ascend's get_attn_backend_cls to align with the new upstream
standard, ensuring compatibility and reducing maintenance overhead.
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
co-author:[leo-pony][nengjunma@outlook.com ](mailto:nengjunma@outlook.com )
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: zxwang <1476209578@qq.com >
Signed-off-by: leo-pony <nengjunma@outlook.com >
Co-authored-by: leo-pony <nengjunma@outlook.com >
2025-12-17 08:48:36 +08:00
anon189Ty and drslark
5b1da4e914
[Feat] Support async_scheduler and disable_padded_drafter_batch in eagle ( #4893 )
...
### What this PR does / why we need it?
We refactored the eagle_proposer.py to adapt the framework of eagle.py
in vllm-v0.12.0, to support the logit of padded drafter batch and
async-scheduler.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: anon189Ty <Stari_Falcon@outlook.com >
Co-authored-by: drslark <slarksblood@qq.com >
2025-12-16 22:06:40 +08:00
zhenwenqi2024
4ed2951400
【Feature】refactor npu_modelrunner for profile_run ( #4993 )
...
### What this PR does / why we need it?
(1)refactor npu_model_runner for profile_run
(2) move _select_moe_comm_method to ascend_forward_context
(3) delete _init_model_kwargs in npu_model_runner
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Na
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: zhenwenqi2024 <zhenwenqi_2022@qq.com >
Signed-off-by: zhenwenqi2024 <155598497+zhenwenqi2024@users.noreply.github.com >
2025-12-16 17:44:04 +08:00
realliujiaxu
9e24bdd44c
[Feat] Refactor rejection sampler ( #4975 )
...
### What this PR does / why we need it?
Currently, we are using `AscendRejctionSampler` that extends from
`RejctionSampler` in spec decoding. `AscendRejctionSampler` override
`forward` of `RejctionSampler`, only aming to replace `rejection_sample`
func. This
causes a lot of code of `RejctionSampler` cannot be reused, for example:
- https://github.com/vllm-project/vllm/pull/19482
- https://github.com/vllm-project/vllm/pull/26060
- https://github.com/vllm-project/vllm/pull/29223
#### Proposed Change:
- Delete `AscendRejctionSampler` and use `RejctionSampler` directly in
model runner.
- Patch `RejctionSampler.expand_batch_to_tokens` and
`RejctionSampler.rejection_sample`, maybe a better way is to make them
as custom ops.
- Modify `NPUModelRunner` following
https://github.com/vllm-project/vllm/pull/26060
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- [x] test logits processor for spec decoding
- [x] test logprobs for spec decoding
- [x] test logprobs for spec decoding + async shcheduling (test with
https://github.com/vllm-project/vllm-ascend/pull/4893/ )
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: realliujiaxu <realliujiaxu@163.com >
2025-12-16 11:32:26 +08:00
SILONG ZENG and Mengqing Cao
237fad635c
[Fix]Revert temporary skip on mtp1/mtp2 correctness tests (aclgraph fix) ( #5039 )
...
### What this PR does / why we need it?
This Pull Request removes the @pytest.mark.skip decorators from
test_mtp1_correctness_piecewise_graph and
test_mtp2_correctness_piecewise_graph.
These tests were temporarily skipped because of an issue with the MTP
ACL Graph (as per the original TODO comment). Since the relevant
bug/issue has been resolved, these tests are now re-enabled to ensure
full correctness coverage for MTP functionality.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: MrZ20 <2609716663@qq.com >
Co-authored-by: Mengqing Cao <cmq0113@163.com >
2025-12-16 10:40:00 +08:00
Jade Zheng
c064d11fd7
[Cleanup] Remove unused attn_metadata parameter from Proposer classes ( #4862 )
...
The `attn_metadata` is not used by any draft proposer, so we can remove
it.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com >
2025-12-15 21:21:38 +08:00
Li Wang
8d2998d0e4
[Misc] Upgrade vllm hash to 12_14 ( #5000 )
...
### What this PR does / why we need it?
### Does this PR introduce _any_ user-facing change?
1. fix https://github.com/vllm-project/vllm/pull/27938
2. fix https://github.com/vllm-project/vllm/pull/27145
pooling models now supports chunked prefill and prefix caching,
3. fix https://github.com/vllm-project/vllm/pull/30181
define the CPU fields in the field config where they really belong.
4. fix https://github.com/vllm-project/vllm/pull/28168
define the CPU fields in the field config where they really belong.
5. fix https://github.com/vllm-project/vllm/pull/30201
some moudle rename
6. fix https://github.com/vllm-project/vllm/pull/29067
fusedmoe moudle refactor
7. fix https://github.com/vllm-project/vllm/pull/29066
fusedmoe moudle refactor
8. fix https://github.com/vllm-project/vllm/pull/29624
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: wangli <wangli858794774@gmail.com >
2025-12-15 19:54:23 +08:00
zengzengran
6029bea480
[UT]add pcp dcp ut ( #4949 )
...
### What this PR does / why we need it?
Adding UT for DCP/PCP
-vLLM version: v0.12.0
-vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: zengran <zengran2@huawei.com >
2025-12-15 18:41:38 +08:00
Icey
5fae65f3a8
[Graph][Fusion] Add AddRMSNorm(with bias) and Quant Fusion Pattern ( #5011 )
...
### What this PR does / why we need it?
AddRMSNorm(with bias) and Quant Fusion Pattern
### Does this PR introduce _any_ user-facing change?
N/A
### How was this patch tested?
CI passed with new added/existing test.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: wxsIcey <1790571317@qq.com >
2025-12-15 18:37:56 +08:00
SILONG ZENG
70606e0bb9
[Test]update accuracy test of models ( #4911 )
...
### What this PR does / why we need it?
Delete accuracy tests for models that are no longer retained:
- Meta-Llama-3.1-8B-Instruct
- llava-1.5-7b-hf
- InternVL2-8B.yaml
- InternVL2_5-8B.yaml
- InternVL3-8B.yaml
Add accuracy tests for the new models:
- Llama-3.2-3B-Instruct
- llava-onevision-qwen2-0.5b-ov-hf
- Qwen3-VL-30B-A3B-Instruct
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: MrZ20 <2609716663@qq.com >
2025-12-15 15:04:20 +08:00
drslark
8fb0ef5ffa
[main][BugFix] Fixed an accuracy bug of Qwen3-next-MTP when batched inferring ( #4932 )
...
### What this PR does / why we need it?
Fixes an accuracy bug of Qwen3-next-MTP when batched inferring.
It is descibed in
https://github.com/vllm-project/vllm-ascend/issues/4930 .
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: drslark <slarksblood@qq.com >
2025-12-15 13:22:30 +08:00
545e856971
[Refactor]3/N Refactor mla_v1.py & extract mla_cp ( #4933 )
...
RFC: https://github.com/vllm-project/vllm-ascend/issues/4629
Reason:
The functions related to Cp differ significantly from those of normal
MLA-Attention, but the coupling is quite severe.
Steps:
Isolate PCP and DCP
(1) create a new python file: mla_cp.py
(2) add classes AscendMlaCPImpl and
AscendMlaCPMetadataBuilder,Inheritance AscendMLAImpl and
AscendMLAMetadataBuilder
(3) Remove PCP and DCP-related methods from mla_v1.py to mla_cp.py
vLLM version: v0.12.0
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: wujinyuan1 <wjy9595@qq.com >
Co-authored-by: wujinyuan1 <wjy9595@qq.com >
Co-authored-by: weijinqian0 <1184188277@qq.com >
2025-12-15 12:59:18 +08:00
LookAround0301
bb7b74c14f
add ut for model runner ( #4991 )
...
### What this PR does / why we need it?
add ut for model runner
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: LookAround <lixushi@huawei.com >
2025-12-14 11:16:20 +08:00
ba28d54f35
[Perf]enable prefill flashcommon3 ( #4065 )
...
### What this PR does / why we need it?
moe multistream overlap to improve the performance.
### How was this patch tested?
--additional-config '{"multistream_overlap_gate": true}'
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: AlvisGong <gwly0401@163.com >
Signed-off-by: chenxiao <Jaychou1620@Gmail.com >
Co-authored-by: clrs97 <524936896@qq.com >
Co-authored-by: zzhx1 <zzh_201018@outlook.com >
Co-authored-by: chenxiao <Jaychou1620@Gmail.com >
2025-12-14 09:34:13 +08:00
Li Wang
0f92d34a70
[CI] Pull latest vllm-ascend src before tests ( #4988 )
...
### What this PR does / why we need it?
Currently, our image build suffers from errors during cross-compilation,
which causing the image to fail to build sometimes(see
https://github.com/vllm-project/vllm-ascend/actions/runs/20152861650/job/57849208186 ).
This results in the nightly test code not being the latest version.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: wangli <wangli858794774@gmail.com >
2025-12-13 19:04:14 +08:00
wangxiyuan and linfeng-yuan
fd7c929145
[perf] replace all_reduce for kv_consumer and support different num_tokens among all ranks ( #4983 )
...
pick from https://github.com/vllm-project/vllm-ascend/pull/4736 to fix
the merge conflict
### What this PR does / why we need it?
Currently, the all_reduce operation in _sync_metadata_across_dp is
performed with gloo backend which is extremely time-consuming when
DPEngineCores are in different nodes. This operation cannot be ignored
by async scheduling in multi-node-scenarios with speculative decoding
(e.g., EAGLE, mtp).
This pr eliminates the all_reduce operation for D Nodes and change the
input parameter of MoEDispatch & MoeCombine operators to make MC2EP
support different num_tokens across all ranks.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tested with PD disaggregation (2P: DP2TP8EP16 1D: DP8TP4EP32) scenarios
while enabling async scheduling. This pr can remove cross-node
all_reduce with gloo backend and further reduce latency with correct
accuracy.
---------
Signed-off-by: linfeng-yuan <1102311262@qq.com >
Co-authored-by: linfeng-yuan <1102311262@qq.com >
2025-12-13 18:59:54 +08:00
wangxiyuan
5211e991ad
Revert "[Bugfix] support mtp kv transfer and pp partition by hand in kv transfer ( #4892 )" ( #4981 )
...
This reverts commit 332b547728 .
This break deepseek3.2 in PD case.
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
2025-12-13 18:58:55 +08:00
zhenwenqi2024
4721e4f53f
[bugfix] asyncscheduler bug fix ( #4968 )
...
### What this PR does / why we need it?
now vllm-ascend uses AsyncGPUModelRunnerOutput
,AsyncNPUModelRunnerOutput before is outdated, so we should fix it
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: zhenwenqi2024 <zhenwenqi_2022@qq.com >
2025-12-13 17:04:54 +08:00
MengLong Chen
fa367e3b1a
[CI] Add mtp_proposer ut ( #4397 )
...
### What this PR does / why we need it?
Add mtp_proposer ut
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: chenmenglong <chenmenglong1@huawei.com >
2025-12-12 20:41:31 +08:00
zhenwenqi2024
f708d919f8
[Feature] model_runner refactor ( #4764 )
...
### What this PR does / why we need it?
refactor npu_modelrunner, we should be close to gpu_modelrunner
### Does this PR introduce _any_ user-facing change?
NO
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: zhenwenqi2024 <zhenwenqi_2022@qq.com >
Signed-off-by: zhenwenqi2024 <155598497+zhenwenqi2024@users.noreply.github.com >
2025-12-12 17:27:09 +08:00
Li Wang
5b12c068f9
[Nightly] Remove gen_ranktable logic ( #4941 )
...
### What this PR does / why we need it?
Since the `llmdatadist` has sunset, the logic gen_ranktable should also
be removed
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: wangli <wangli858794774@gmail.com >
2025-12-12 17:20:18 +08:00