Li Wang
02f89d166f
[CI] Update vllm version to 20250922(5aeb925) ( #3091 )
...
### What this PR does / why we need it?
This pr bump vllm commit hash to
https://github.com/vllm-project/vllm/commit/5aeb9254521023f97aca292b3478aa7ff485ffb2
fix issues:
1. https://github.com/vllm-project/vllm/pull/25345 has remove v0
metadata
2. https://github.com/vllm-project/vllm/pull/25332
3. https://github.com/vllm-project/vllm/pull/25334
4. https://github.com/vllm-project/vllm/pull/23558 , note that this vllm
commit update the model register logic, which will check all the model
registered have the `vllm.model_executor.models` path , which breaks our
custom registration of the deepseek_v3 model (it doesn't exist in the
vllm model path). so I move deepseek_v3 model registy to deepseek_v2 to
solve temporary
### How was this patch tested?
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/9607d5eb449711b349d4c2bee0a9c94afcc7ed14
---------
Signed-off-by: wangli <wangli858794774@gmail.com >
2025-09-22 22:18:13 +08:00
fan2956 and zhoufan2956
c5a502fd2e
main add ascend scheduler support multimodal ( #2844 )
...
### What this PR does / why we need it?
On main, AscendScheduler does not support Multimodels, becuse of lacking
of scheduled_encoder_inputs which is need on multimodels inference
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
vLLM version: main@93e28e6862669e3b5cf47cea9f782a65ec47e155
- vLLM version: v0.10.2rc2
- vLLM main:
https://github.com/vllm-project/vllm/commit/15b8fef453b373b84406207a947005a4d9d68acc
---------
Signed-off-by: fan2956 <zhoufan53@huawei.com >
Co-authored-by: zhoufan2956 <zhoufan2956@163.com >
2025-09-14 09:38:51 +08:00
rjg-lyh
585a494baa
[Core] Disable the chunked prefill feature in Non-MLA LLMs ( #2894 )
...
### What this PR does / why we need it?
This PR enforces the forcible disabling of the chunked prefill feature
in Non-MLA models, as the performance of operators supporting this
functionality is currently suboptimal. Unless the user has enabled
chunked prefill in the ascend_scheduler_config, we would allow this
feature.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
CI passed with new added/existing test.
Related: https://github.com/vllm-project/vllm-ascend/pull/2659
- vLLM version: main
- vLLM main:
https://github.com/vllm-project/vllm/commit/d21a36f5f9569949dd6c313deed609d77e393850
Signed-off-by: rjg-lyh <1318825571@qq.com >
2025-09-12 23:17:09 +08:00
Li Wang
22b425765a
[Bugfix] Fix broken CI ( #2825 )
...
### What this PR does / why we need it?
1. Initial support disable tp for integrating with
[vllm-commit](https://github.com/vllm-project/vllm/pull/23024 )
2. [vllm@commit](https://github.com/vllm-project/vllm/pull/23673 ) now
use `bytes` to save the `BlockHash` to reduce GC overhead, this pr add
the integration
- vLLM version: main
- vLLM main:
https://github.com/vllm-project/vllm/commit/e40827280b225bf0a0797cc9842fc3cdfea8ebdf
---------
Signed-off-by: wangli <wangli858794774@gmail.com >
2025-09-10 13:29:29 +08:00
CaranLic
168ad600b5
[main] add pd transfer for ascend scheduler ( #2753 )
...
### What this PR does / why we need it?
For offline scenarios, adjust the scheduling process to prioritize the
prefill phase of all requests, then process the decode phase of all
requests.
### How was this patch tested?
```
max_num_seqs=24,
additional_config={
"ascend_scheduler_config":{
"enabled": True,
"enable_pd_transfer": True,
"decode_max_num_seqs": 24,
"enable_chunked_prefill": False
}
},
```
| input | output | num prompts | max_num_seqs | dp | tp | scheduler |
tps |
| ------ | ------ | ---------- | ---------------- | ---- | ---- |
---------------- | --------------- |
| dapo-math-17K | 2K | 384 | 24 | 2 | 1 | v1 | 234.06 |
| dapo-math-17K | 2K | 384 | 24 | 2 | 1 | pd transfer | 239.59(+2.4%) |
| dapo-math-17K| 2K | 384 | 24 | 4 | 1 | v1 | 222.85 |
| dapo-math-17K| 2K | 384 | 24 | 4 | 1 | pd transfer | 225.81(+1.3%) |
- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/6fb27881634d89c2e70e9e5fbad1b918c0d916cf
---------
Signed-off-by: CaranLic <740821011@qq.com >
2025-09-10 08:46:39 +08:00
Mengqing Cao
edf1f600ad
[CI] Remove compatibility maintenance for vllm v0.10.1 and v0.10.1.1 ( #2840 )
...
### What this PR does / why we need it?
Remove compatibility maintenance for vllm v0.10.1 and v0.10.1.1
### Does this PR introduce _any_ user-facing change?
branch main of vllm-ascend will not be compatible with vllm v0.10.1 and
v0.10.1.1
### How was this patch tested?
CI passed with existing test.
- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/6fb27881634d89c2e70e9e5fbad1b918c0d916cf
---------
Signed-off-by: MengqingCao <cmq0113@163.com >
2025-09-10 08:43:10 +08:00
weijinqian0 and weijinqian_v1
6f1047d5fd
[CI] fix UT error. ( #2644 )
...
https://github.com/vllm-project/vllm/commit/69f46359dd5b36c1a059a0a8b729be1bd86394e8 changed the vl input usage, this PR fix the related UT failure.
- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/d660c98c1b59580af97d6c7dd162c7c8894d40ed
---------
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com >
Co-authored-by: weijinqian_v1 <weijinqian@huawei.com >
2025-08-30 12:04:01 +08:00
Yikun Jiang
175f6bc445
Support v0.10.1 ( #2584 )
...
### What this PR does / why we need it?
This patch also supports v0.10.1
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- CI passed
- test 0.10.1: https://github.com/vllm-project/vllm-ascend/pull/2583
- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/321938e9ac4000e0cb37e328359a7fd3026bc672
Signed-off-by: Yikun Jiang <yikunkero@gmail.com >
2025-08-28 18:47:53 +08:00
wangxiyuan
c0e12143a3
[CI] Fix UT failure ( #2563 )
...
UT is broken by vLLM commit
https://github.com/vllm-project/vllm/pull/23664
This PR mock the related config to recover the CI
- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/6dab89b8ece7e022bd3df5774c9ddf309e2eb2d9
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2025-08-27 11:24:35 +08:00
wangxiyuan
7e494e94a9
[CI] Fix broken ci ( #2530 )
...
vLLM commit https://github.com/vllm-project/vllm/pull/22711 changed the
encode cache entries logic, this PR adapt the same change for vllm
ascend to make CI happy.
Co-Authored-By: zhoux77899 <zhouxiang100@huawei.com >
- vLLM version: v0.10.1.1
- vLLM main:
https://github.com/vllm-project/vllm/commit/0ff902f3b48376aadfe4e753ce0872b49c8f9f7e
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2025-08-26 07:42:24 +08:00
linfeng-yuan
4af5b80606
[Scheduler] validate max_num_batched_tokens and max_model_len in AscendSchedulerConfig ( #2434 )
...
### What this PR does / why we need it?
Add configuration check logic for ascend scheduler: if chunked_prefill
is disabled, max_num_batched_tokens couldn't be less than max_model_len,
following vLLM;
### Does this PR introduce _any_ user-facing change?
users cannot set max_num_batched_tokens smaller than max_model_len with
ascend scheduler
### How was this patch tested?
CI and vllm serving passed
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/f77a0802b758a32c5b9f7bc04e9498d77e8d99e0
Signed-off-by: linfeng-yuan <1102311262@qq.com >
2025-08-23 19:39:44 +08:00
Mengqing Cao
b0403f8d8a
[CI] fix ci ( #2464 )
...
### What this PR does / why we need it?
1. use action/checkout@v5 instead of v4
2. remove dbo test case because there is issue with it and will be
refactored later
3. make vllm-ascend compatible with vllm v0.10.1.1 and add CI for it
4. fix sampler api changes introduced by
https://github.com/vllm-project/vllm/pull/22387
6. fix qwen3 moe config changes intruoduced by
https://github.com/vllm-project/vllm/pull/20562
7. fix kvcache block changes introduced by
https://github.com/vllm-project/vllm/pull/23262
### Does this PR introduce _any_ user-facing change?
N/A
### How was this patch tested?
CI passed with existing test.
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/0c6e40bbaa4707528286a1e7bf17c90c88a1d920
---------
Signed-off-by: MengqingCao <cmq0113@163.com >
2025-08-22 07:30:48 +08:00
wangxiyuan and MengqingCao
eccfb715f6
[CI] Fix UT ( #2452 )
...
Make UT CI happy
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/d983769c41db224e0897fac2e9aefc5f57ad1122
---------
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Signed-off-by: MengqingCao <cmq0113@163.com >
Co-authored-by: MengqingCao <cmq0113@163.com >
2025-08-20 16:26:07 +08:00
1327f9be1c
Fix some ci issue and refactor modelrunner ( #2445 )
...
### What this PR does / why we need it?
Fix some ci issue and refactor modelrunner
### Does this PR introduce _any_ user-facing change?
N/A
### How was this patch tested?
CI passed with existing test.
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/4d9c61993ac4209c97b3afef237b2387f2cd9b97
---------
Signed-off-by: wangli <wangli858794774@gmail.com >
Signed-off-by: MengqingCao <cmq0113@163.com >
Signed-off-by: weiguihua2 <weiguihua2@huawei.com >
Co-authored-by: wangli <wangli858794774@gmail.com >
Co-authored-by: weiguihua2 <weiguihua2@huawei.com >
2025-08-20 09:01:04 +08:00
Mengqing Cao
61866b8ac6
[Quickfix] update CachedRequestState as NewRequestData changed ( #2367 )
...
### What this PR does / why we need it?
1. update `CachedRequestState` as `NewRequestData` changed in
https://github.com/vllm-project/vllm/pull/22570
2. drop maintenance of vllm v0.10.0 in the branch main
### Does this PR introduce _any_ user-facing change?
N/A
### How was this patch tested?
CI passed with existing test.
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/92ff41abea130a3217faa54abb89ccc27aef3f06
---------
Signed-off-by: MengqingCao <cmq0113@163.com >
2025-08-15 07:35:27 +08:00
SunnyLee151064 and lwq
ae560f7131
[Test] Add uts for files in /core ( #1957 )
...
### What this PR does / why we need it?
Add uts for files in folder /core
### Does this PR introduce _any_ user-facing change?
No
- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/5a19a6c6705fe83db2e3517a2d2f473586901743
---------
Signed-off-by: lwq <liwenquan5@huawei.com >
Co-authored-by: lwq <liwenquan5@huawei.com >
2025-07-25 09:48:19 +08:00
JohnJan
ce4970eee0
[Test] Add unit test for schedule_config.py ( #1590 )
...
What this PR does / why we need it?
According to issue
https://github.com/vllm-project/vllm-ascend/issues/1298 , this pull
request adds unit test code for schedule_config.py.
Does this PR introduce any user-facing change?
No
How was this patch tested?
CI passed with new added/existing test.
- vLLM version: v0.9.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/8d0a01a5f2b53794e4bc6b734d7b63cb8a9b7d7d
2025-07-22 11:43:25 +08:00