init v0.23.0

Signed-off-by: Sun Ruoxi <sunruoxi@4paradigm.com>
This commit is contained in:
2026-08-27 15:11:51 +08:00
parent b582a8e7d1
commit 7f8a1b1f7a
2849 changed files with 712887 additions and 22001 deletions

View File

@@ -1,9 +1,10 @@
# Using OpenCompass
This document will guide you have a accuracy testing using [OpenCompass](https://github.com/open-compass/opencompass).
## 1. Online Serving
This document guides you to conduct accuracy testing using [OpenCompass](https://github.com/open-compass/opencompass).
You can run docker container to start the vLLM server on a single NPU:
## 1. Online Server
You can run a docker container to start the vLLM server on a single NPU:
```{code-block} bash
:substitutions:
@@ -13,6 +14,7 @@ export DEVICE=/dev/davinci7
export IMAGE=quay.io/ascend/vllm-ascend:|vllm_ascend_version|
docker run --rm \
--name vllm-ascend \
--shm-size=1g \
--device $DEVICE \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
@@ -30,29 +32,30 @@ docker run --rm \
vllm serve Qwen/Qwen2.5-7B-Instruct --max_model_len 26240
```
If your service start successfully, you can see the info shown below:
The vLLM server is started successfully, if you see information as below:
```
```shell
INFO: Started server process [6873]
INFO: Waiting for application startup.
INFO: Application startup complete.
```
Once your server is started, you can query the model with input prompts in new terminal:
Once your server is started, you can query the model with input prompts in a new terminal.
```
```shell
curl http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen2.5-7B-Instruct",
"prompt": "The future of AI is",
"max_tokens": 7,
"max_completion_tokens": 7,
"temperature": 0
}'
```
## 2. Run ceval accuracy test using OpenCompass
Install OpenCompass and configure the environment variables in the container.
## 2. Run C-Eval (a Chinese language model evaluation benchmark) using OpenCompass for accuracy testing
Install OpenCompass and configure the environment variables in the container:
```bash
# Pin Python 3.10 due to:
@@ -64,7 +67,7 @@ export DATASET_SOURCE=ModelScope
git clone https://github.com/open-compass/opencompass.git
```
Add `opencompass/configs/eval_vllm_ascend_demo.py` with the following content:
Add the following content to `opencompass/configs/eval_vllm_ascend_demo.py`:
```python
from mmengine.config import read_base
@@ -106,14 +109,14 @@ models = [
Run the following command:
```
```shell
python3 run.py opencompass/configs/eval_vllm_ascend_demo.py --debug
```
After 1-2 mins, the output is as shown below:
After 1 to 2 minutes, the output is shown below:
```
The markdown format results is as below:
```shell
The markdown format results are as below:
| dataset | version | metric | mode | Qwen2.5-7B-Instruct-vLLM-API |
|----- | ----- | ----- | ----- | -----|