zhangxinyuehfad
d40ba52454
[Fix] fix Qwen2-Audio-7B-Instruct accuracy test ( #4017 )
...
### What this PR does / why we need it?
fix Qwen2-Audio-7B-Instruct accuracy test
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.11.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-11-10 11:54:18 +08:00
zhangxinyuehfad
49e6983b3b
[Test] Add accuracy test for qwen3-30b-a3b-w8a8 ( #3807 )
...
### What this PR does / why we need it?
Add accuracy test for qwen3-30b-a3b-w8a8
This PR depends on https://github.com/vllm-project/vllm-ascend/pull/3799
### How was this patch tested?
qwen3-30b-a3b-w8a8 accuarcy test ok:
https://github.com/vllm-project/vllm-ascend/actions/runs/19062045267/job/54443732877?pr=3807
- vLLM version: v0.11.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-11-04 18:56:31 +08:00
ZengSilong
dc1a6cb503
[Test]Add accuracy test for multiple models ( #3823 )
...
### What this PR does / why we need it?
Add accuracy test for multiple models:
- Meta_Llama_3.1_8B_Instruct
- Qwen2.5-Omni-7B
- Qwen3-VL-8B-Instruct
- vLLM version: v0.11.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac
---------
Signed-off-by: MrZ20 <2609716663@qq.com >
2025-11-04 14:46:39 +08:00
zhangxinyuehfad
646fbac7a9
[Test] Add accuracy test for qwen3-8b-w8a8 ( #3799 )
...
### What this PR does / why we need it?
Add accuracy test for qwen3-8b-w8a8
- vLLM version: v0.11.0rc3
- vLLM main:
https://github.com/vllm-project/vllm/commit/c9461e05a4ed3557cfbf4b15ded1e26761cc39ca
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-11-04 09:23:11 +08:00
zhangxinyuehfad
8f6f967028
[Test] Add e2e test and accuracy test for Qwen3-Next-80B-A3B-Instruct ( #3450 )
...
### What this PR does / why we need it?
Add e2e test and accuracy test for Qwen3-Next-80B-A3B-Instruct
### How was this patch tested?
accuracy test:
https://github.com/vllm-project/vllm-ascend/actions/runs/18771221544/job/53556027634?pr=3450
ci test:
https://github.com/vllm-project/vllm-ascend/actions/runs/18771221530/job/53556027614?pr=3450
<img width="1703" height="562" alt="image"
src="https://github.com/user-attachments/assets/973b6cfa-8240-41e3-893a-5024ff8d0693 "
/>
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-10-25 10:57:56 +08:00
lilinsiman
1b424fb7f1
ACLgraph enable: Test cases revisions for all features ( #3388 )
...
### What this PR does / why we need it?
This PR revise the test cases of various features on the warehouse which
add the enablement of aclgraph to the test cases.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
ut
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0
Signed-off-by: lilinsiman <lilinsiman@gmail.com >
2025-10-17 17:15:19 +08:00
Yikun Jiang and hfadzxy
cd69385dab
Add models test and add serval new models yaml ( #3394 )
...
### What this PR does / why we need it?
This PR added Add accuracy CI for servals new models
- `ascend test / accuracy` is for PR triggered check popluar models
accuracy
- `ascedn test / models` is for accuracy report, full models test,
nightly model test
- Add Qwen2-Audio-7B-Instruct, Qwen2-VL-7B-Instruct, Qwen3-8B,
Qwen3-VL-30B-A3B-Instruct
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
CI passed
Closes: https://github.com/vllm-project/vllm-ascend/pull/2330
Closes: https://github.com/vllm-project/vllm-ascend/pull/3362
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0
---------
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
Signed-off-by: Yikun Jiang <yikunkero@gmail.com >
Co-authored-by: hfadzxy <starmoon_zhang@163.com >
2025-10-12 17:27:50 +08:00
zhangxinyuehfad
601a37aeff
[Fixbug] Fix accuarcy template ( #3088 )
...
### What this PR does / why we need it?
Fix empty lines between lm_eval command lines for accuarcy template
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/9607d5eb449711b349d4c2bee0a9c94afcc7ed14
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-10-10 09:03:21 +08:00
zhangxinyuehfad
c90a6d3658
[Test] Update the format of the accuracy report ( #3081 )
...
### What this PR does / why we need it?
Update the format of the accuracy report
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/c60e6137f0bf2034853919b3a9d705d7e06b93cf
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-09-22 14:10:03 +08:00
zhangxinyuehfad
e26fe1caf1
[TEST] Speed up DS V2 accuracy test and turn up accuracy baseline ( #3047 )
...
### What this PR does / why we need it?
1. update expected accuracy for DeepSeek-V2-Lite
2. add batch size
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Accuracy CI passed
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/838d7116ba59db528647b29f0d000742f4af9d4b
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-09-20 00:40:33 +08:00
zhangxinyuehfad
0c04bf1e36
[Fixbug] Fix accuracy for DeepSeek-V2-Lite ( #3016 )
...
### What this PR does / why we need it?
Fix accuracy for DeepSeek-V2-Lite
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
CI passed
- vLLM version: v0.10.2
- vLLM main:
https://github.com/vllm-project/vllm/commit/66072b36dbf1707440ff43d57273d9e9974349d7
Signed-off-by: hfadzxy <starmoon_zhang@163.com >
2025-09-18 23:58:23 +08:00
Icey
891b2bfe71
Accuracy report formatting ( #2279 )
...
### What this PR does / why we need it?
Accuracy report formatting
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
CI passed with existing test.
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/53415653ff24be03e7c90f5b42ef9cb3f72aad71
---------
Signed-off-by: Icey <1790571317@qq.com >
2025-08-25 09:39:30 +08:00
Icey
0bd5ff5299
Fix accuracy test config and add DeepSeek-V2-Lite test ( #2261 )
...
### What this PR does / why we need it?
This PR fix accuracy test related to
https://github.com/vllm-project/vllm-ascend/pull/2073 , users can now
perform accuracy tests on multiple models simultaneously and generate
different report files by running:
```bash
cd ~/vllm-ascend
pytest -sv ./tests/e2e/models/test_lm_eval_correctness.py \
--config-list-file ./tests/e2e/models/configs/accuracy.txt
```
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
<img width="1648" height="511" alt="image"
src="https://github.com/user-attachments/assets/1757e3b8-a6b7-44e5-b701-80940dc756cd "
/>
- vLLM version: v0.10.0
- vLLM main:
https://github.com/vllm-project/vllm/commit/766bc8162cb37ad32605eee051d4f049ec325926
---------
Signed-off-by: Icey <1790571317@qq.com >
2025-08-08 11:09:16 +08:00