EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Enable pytest and yaml style accuracy test (#2073)

Browse Source 
### What this PR does / why we need it?

This PR enabled pytest and yaml style accuracy test, users now can
enable accuracy test by running:

```bash
cd ~/vllm-ascend
pytest -sv ./tests/e2e/singlecard/models/test_lm_eval_correctness.py \
          --config ./tests/e2e/singlecard/models/configs/Qwen3-8B-Base.yaml \
          --report_output ./benchmarks/accuracy/Qwen3-8B-Base.md

pytest -sv ./tests/e2e/singlecard/models/test_lm_eval_correctness.py \
          --config-list-file ./tests/e2e/singlecard/models/configs/accuracy.txt
```

Closes: https://github.com/vllm-project/vllm-ascend/issues/1970

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?


- vLLM version: v0.10.0
- vLLM main:
2836dd73f1

---------

Signed-off-by: Icey <1790571317@qq.com>
This commit is contained in:
Icey
2025-07-31 21:39:13 +08:00
committed by GitHub
parent 9c9a7cd90b
commit 86bdde1ca8
10 changed files with 336 additions and 446 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
requirements-dev.txt
View File

@@ -5,7 +5,7 @@ openai
pytest >= 6.0
pytest-asyncio
pytest-mock
lm-eval
lm-eval==0.4.8
types-jsonschema
xgrammar
zmq