SILONG ZENG and wangxiyuan
43df2cb2fc
[Lint]Style: Convert test/ to ruff format(Batch #1 ) ( #6738 )
...
### What this PR does / why we need it?
**Scope of Changes**:
| File Path |
| :--- |
| `tests/e2e/310p/multicard/test_vl_model_multicard.py` |
| `tests/e2e/310p/singlecard/test_vl_model_singlecard.py` |
| `tests/e2e/310p/test_utils.py` |
| `tests/e2e/conftest.py` |
| `tests/e2e/model_utils.py` |
| `tests/e2e/models/conftest.py` |
| `tests/e2e/models/test_lm_eval_correctness.py` |
| `tests/e2e/multicard/2-cards/spec_decode/test_spec_decode.py` |
| `tests/e2e/multicard/2-cards/test_aclgraph_capture_replay.py` |
| `tests/e2e/multicard/2-cards/test_data_parallel.py` |
| `tests/e2e/multicard/2-cards/test_disaggregated_encoder.py` |
| `tests/e2e/multicard/2-cards/test_expert_parallel.py` |
| `tests/e2e/multicard/2-cards/test_external_launcher.py` |
| `tests/e2e/multicard/2-cards/test_full_graph_mode.py` |
| `tests/e2e/multicard/2-cards/test_ilama_lora_tp2.py` |
| `tests/e2e/multicard/2-cards/test_offline_inference_distributed.py` |
| `tests/e2e/multicard/2-cards/test_offline_weight_load.py` |
| `tests/e2e/multicard/2-cards/test_pipeline_parallel.py` |
| `tests/e2e/multicard/2-cards/test_prefix_caching.py` |
| `tests/e2e/multicard/2-cards/test_quantization.py` |
| `tests/e2e/multicard/2-cards/test_qwen3_moe.py` |
| `tests/e2e/multicard/2-cards/test_qwen3_moe_routing_replay.py` |
| `tests/e2e/multicard/2-cards/test_qwen3_performance.py` |
| `tests/e2e/multicard/2-cards/test_shared_expert_dp.py` |
| `tests/e2e/multicard/2-cards/test_single_request_aclgraph.py` |
| `tests/e2e/multicard/2-cards/test_sp_pass.py` |
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.15.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/9562912cead1f11e8540fb91306c5cbda66f0007
Signed-off-by: MrZ20 <2609716663@qq.com >
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-03-10 09:52:50 +08:00
zhangxinyuehfad
81f3c09d6d
[CI] Change A2 runner ( #6557 )
...
### What this PR does / why we need it?
This PR updates the CI runner from `linux-aarch64-a2-*` to
`linux-aarch64-a2b3-*` in various test configuration files. This change
is necessary to adapt to updates in the CI infrastructure.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The changes are configuration updates for CI tests. The correctness will
be verified by the CI pipeline.
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2026-02-05 23:43:57 +08:00
Shanshan Shen
efa0f64f22
[Doc] Add tutorials for Qwen3-VL-30B-A3B-Instruct ( #5331 )
...
### What this PR does / why we need it?
Add tutorials for `Qwen3-VL-30B-A3B-Instruct`.
- vLLM version: release/v0.13.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/bc0a5a0c089844b17cb93f3294348f411e523586
---------
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-01-15 10:56:19 +08:00
SILONG ZENG and root
fa0c212bfa
[test]Corrected the Qwen3-Omni-30B-A3B-Instruct accuracy test configuration in nightly tests. ( #5195 )
...
### What this PR does / why we need it?
Corrected the Qwen3-Omni-30B-A3B-Instruct accuracy test configuration in
nightly tests.
link: https://github.com/vllm-project/vllm-ascend/pull/4911
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: MrZ20 <2609716663@qq.com >
Signed-off-by: root <root@LAPTOP-VQKDDVMG.localdomain >
Co-authored-by: root <root@LAPTOP-VQKDDVMG.localdomain >
2025-12-23 14:17:27 +08:00
ZT-AIA
6cb76ecd02
[Nightly] Avoid max_model_len being smaller than the decoder prompt to prevent single-node-accuray-tests from failing ( #5174 )
...
### What this PR does / why we need it?
[Nightly] Avoid max_model_len being smaller than the decoder prompt to
prevent single-node-accuray-tests from failing
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: ZT-AIA <1028681969@qq.com >
Signed-off-by: ZT-AIA <63220130+ZT-AIA@users.noreply.github.com >
2025-12-18 22:25:45 +08:00
shaopeng-666
39bdd4cfaa
fix profile run for vl model ( #5136 )
...
### What this PR does / why we need it?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: 李少鹏 <lishaopeng21@huawei.com >
2025-12-17 23:51:31 +08:00
SILONG ZENG
70606e0bb9
[Test]update accuracy test of models ( #4911 )
...
### What this PR does / why we need it?
Delete accuracy tests for models that are no longer retained:
- Meta-Llama-3.1-8B-Instruct
- llava-1.5-7b-hf
- InternVL2-8B.yaml
- InternVL2_5-8B.yaml
- InternVL3-8B.yaml
Add accuracy tests for the new models:
- Llama-3.2-3B-Instruct
- llava-onevision-qwen2-0.5b-ov-hf
- Qwen3-VL-30B-A3B-Instruct
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: MrZ20 <2609716663@qq.com >
2025-12-15 15:04:20 +08:00
zhangyiming
c95c271538
[E2E] Optimize nightly testcase. ( #4886 )
...
### What this PR does / why we need it?
Optimize nightly testcase.
Changes:
- tests/e2e/nightly/multi_node/config/models/Qwen3-235B-A3B.yaml: Add
accuracy and performance benchmark
- tests/e2e/models/configs/Qwen3-8B-Base.yaml: Delete
- tests/e2e/models/configs/internlm-7b.yaml: Change to
internlm3-8b-instruct
- tests/e2e/nightly/models/test_deepseek_r1_w8a8_eplb.py: Change to
DeepSeek-R1-0528-W8A8 model
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: menogrey <1299267905@qq.com >
2025-12-11 10:15:39 +08:00
zhangyiming
66b0781840
[E2E] Refactor the e2e testcases. ( #4789 )
...
### What this PR does / why we need it?
Refactor the e2e testcases.
- tests/e2e/multicard/test_weight_loader.py: Remove the unused code.
- tests/e2e/singlecard/multi-modal/test_internvl.py: Move to accuracy
test.
- tests/e2e/singlecard/test_aclgraph.py: Rename the file.
- tests/e2e/singlecard/test_embedding_aclgraph.py : Combine with
tests/e2e/singlecard/test_bge_model.py
- tests/e2e/singlecard/test_completion_with_prompt_embeds.py: Delete
eager mode and modify model to Qwen3-0.6B
- tests/e2e/singlecard/test_quantization.py: Modify model to
Qwen3-0.6B-W8A8
- tests/e2e/singlecard/test_vlm.py: Modify model to Qwen3-VL-8B
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
---------
Signed-off-by: menogrey <1299267905@qq.com >
2025-12-11 10:15:00 +08:00
SILONG ZENG
7132ae8532
[CI]Cleanup accurary test ( #4861 )
...
### What this PR does / why we need it?
Delete accuracy testing of some models:
- Qwen2-VL-7B-Instruct
- Qwen2.5-VL-7B-Instruct
- gemma-2-9b-it
- DeepSeek-V2-Lite
- vLLM version: v0.12.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/ad32e3e19ccf0526cb6744a5fed09a138a5fb2f9
Signed-off-by: MrZ20 <2609716663@qq.com >
2025-12-10 14:13:56 +08:00
zhangxinyuehfad
b6afec73e1
[Test] Add accuracy nightly test for new models ( #4262 )
...
### What this PR does / why we need it?
Add accuracy nightly test for new models:
PaddlePaddle/ERNIE-4.5-21B-A3B-PT
LLM-Research/Molmo-7B-D-0924
LLM-Research/gemma-2-9b-it
LLM-Research/gemma-3-4b-it
Shanghai_AI_Laboratory/internlm-7b
llava-hf/llava-1.5-7b-hf
- vLLM version: v0.11.2
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-12-01 22:28:46 +08:00
Canlin Guo
1c677c3b87
[Test][Accuracy] Add accuracy evaluation config for InternVL3_5-8B ( #3964 )
...
### What this PR does / why we need it?
To continuously monitor the accuracy of the InternVL3_5-8B model, this
PR adds the corresponding configuration file to the CI. We need to add
the `-hf` suffix to avoid incompatibility with the `lm-eval`
preprocessor.
### How was this patch tested?
`pytest -sv ./tests/e2e/models/test_lm_eval_correctness.py --config
./tests/e2e/models/configs/InternVL3_5-8B.yaml`
- vLLM version: v0.11.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-11-12 09:05:55 +08:00
zhangxinyuehfad
d40ba52454
[Fix] fix Qwen2-Audio-7B-Instruct accuracy test ( #4017 )
...
### What this PR does / why we need it?
fix Qwen2-Audio-7B-Instruct accuracy test
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.11.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-11-10 11:54:18 +08:00
zhangxinyuehfad
49e6983b3b
[Test] Add accuracy test for qwen3-30b-a3b-w8a8 ( #3807 )
...
### What this PR does / why we need it?
Add accuracy test for qwen3-30b-a3b-w8a8
This PR depends on https://github.com/vllm-project/vllm-ascend/pull/3799
### How was this patch tested?
qwen3-30b-a3b-w8a8 accuarcy test ok:
https://github.com/vllm-project/vllm-ascend/actions/runs/19062045267/job/54443732877?pr=3807
- vLLM version: v0.11.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-11-04 18:56:31 +08:00
ZengSilong
dc1a6cb503
[Test]Add accuracy test for multiple models ( #3823 )
...
### What this PR does / why we need it?
Add accuracy test for multiple models:
- Meta_Llama_3.1_8B_Instruct
- Qwen2.5-Omni-7B
- Qwen3-VL-8B-Instruct
- vLLM version: v0.11.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac
---------
Signed-off-by: MrZ20 <2609716663@qq.com >
2025-11-04 14:46:39 +08:00
zhangxinyuehfad
646fbac7a9
[Test] Add accuracy test for qwen3-8b-w8a8 ( #3799 )
...
### What this PR does / why we need it?
Add accuracy test for qwen3-8b-w8a8
- vLLM version: v0.11.0rc3
- vLLM main:
https://github.com/vllm-project/vllm/commit/c9461e05a4ed3557cfbf4b15ded1e26761cc39ca
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-11-04 09:23:11 +08:00
zhangxinyuehfad
8f6f967028
[Test] Add e2e test and accuracy test for Qwen3-Next-80B-A3B-Instruct ( #3450 )
...
### What this PR does / why we need it?
Add e2e test and accuracy test for Qwen3-Next-80B-A3B-Instruct
### How was this patch tested?
accuracy test:
https://github.com/vllm-project/vllm-ascend/actions/runs/18771221544/job/53556027634?pr=3450
ci test:
https://github.com/vllm-project/vllm-ascend/actions/runs/18771221530/job/53556027614?pr=3450
<img width="1703" height="562" alt="image"
src="https://github.com/user-attachments/assets/973b6cfa-8240-41e3-893a-5024ff8d0693 "
/>
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-10-25 10:57:56 +08:00
lilinsiman
1b424fb7f1
ACLgraph enable: Test cases revisions for all features ( #3388 )
...
### What this PR does / why we need it?
This PR revise the test cases of various features on the warehouse which
add the enablement of aclgraph to the test cases.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
ut
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0
Signed-off-by: lilinsiman <lilinsiman@gmail.com >
2025-10-17 17:15:19 +08:00
Yikun Jiang and hfadzxy
cd69385dab
Add models test and add serval new models yaml ( #3394 )
...
### What this PR does / why we need it?
This PR added Add accuracy CI for servals new models
- `ascend test / accuracy` is for PR triggered check popluar models
accuracy
- `ascedn test / models` is for accuracy report, full models test,
nightly model test
- Add Qwen2-Audio-7B-Instruct, Qwen2-VL-7B-Instruct, Qwen3-8B,
Qwen3-VL-30B-A3B-Instruct
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
CI passed
Closes: https://github.com/vllm-project/vllm-ascend/pull/2330
Closes: https://github.com/vllm-project/vllm-ascend/pull/3362
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0
---------
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
Signed-off-by: Yikun Jiang <yikunkero@gmail.com >
Co-authored-by: hfadzxy <starmoon_zhang@163.com >
2025-10-12 17:27:50 +08:00
zhangxinyuehfad
601a37aeff
[Fixbug] Fix accuarcy template ( #3088 )
...
### What this PR does / why we need it?
Fix empty lines between lm_eval command lines for accuarcy template
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/9607d5eb449711b349d4c2bee0a9c94afcc7ed14
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-10-10 09:03:21 +08:00
zhangxinyuehfad
c90a6d3658
[Test] Update the format of the accuracy report ( #3081 )
...
### What this PR does / why we need it?
Update the format of the accuracy report
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/c60e6137f0bf2034853919b3a9d705d7e06b93cf
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-09-22 14:10:03 +08:00
zhangxinyuehfad
e26fe1caf1
[TEST] Speed up DS V2 accuracy test and turn up accuracy baseline ( #3047 )
...
### What this PR does / why we need it?
1. update expected accuracy for DeepSeek-V2-Lite
2. add batch size
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Accuracy CI passed
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/838d7116ba59db528647b29f0d000742f4af9d4b
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-09-20 00:40:33 +08:00
zhangxinyuehfad
0c04bf1e36
[Fixbug] Fix accuracy for DeepSeek-V2-Lite ( #3016 )
...
### What this PR does / why we need it?
Fix accuracy for DeepSeek-V2-Lite
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
CI passed
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/66072b36dbf1707440ff43d57273d9e9974349d7
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-09-18 23:58:23 +08:00
Icey
891b2bfe71
Accuracy report formatting ( #2279 )
...
### What this PR does / why we need it?
Accuracy report formatting
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
CI passed with existing test.
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71
---------
Signed-off-by: Icey <1790571317@qq.com >
2025-08-25 09:39:30 +08:00
Icey
0bd5ff5299
Fix accuracy test config and add DeepSeek-V2-Lite test ( #2261 )
...
### What this PR does / why we need it?
This PR fix accuracy test related to
https://github.com/vllm-project/vllm-ascend/pull/2073 , users can now
perform accuracy tests on multiple models simultaneously and generate
different report files by running:
```bash
cd ~/vllm-ascend
pytest -sv ./tests/e2e/models/test_lm_eval_correctness.py \
--config-list-file ./tests/e2e/models/configs/accuracy.txt
```
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
<img width="1648" height="511" alt="image"
src="https://github.com/user-attachments/assets/1757e3b8-a6b7-44e5-b701-80940dc756cd "
/>
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/766bc8162cb37ad32605eee051d4f049ec325926
---------
Signed-off-by: Icey <1790571317@qq.com >
2025-08-08 11:09:16 +08:00