[Lint]Style: reformat markdown files via markdownlint (#5884)
### What this PR does / why we need it?
reformat markdown files via markdownlint
- vLLM version: v0.13.0
- vLLM main:
bde38c11df
---------
Signed-off-by: root <root@LAPTOP-VQKDDVMG.localdomain>
Signed-off-by: MrZ20 <2609716663@qq.com>
Co-authored-by: root <root@LAPTOP-VQKDDVMG.localdomain>
This commit is contained in:
@@ -1,8 +1,11 @@
|
||||
# Using lm-eval
|
||||
|
||||
This document guides you to conduct accuracy testing using [lm-eval][1].
|
||||
|
||||
## Online Server
|
||||
|
||||
### 1. Start the vLLM server
|
||||
|
||||
You can run docker container to start the vLLM server on a single NPU:
|
||||
|
||||
```{code-block} bash
|
||||
@@ -34,7 +37,7 @@ vllm serve Qwen/Qwen2.5-0.5B-Instruct --max_model_len 4096 &
|
||||
|
||||
The vLLM server is started successfully, if you see logs as below:
|
||||
|
||||
```
|
||||
```shell
|
||||
INFO: Started server process [9446]
|
||||
INFO: Waiting for application startup.
|
||||
INFO: Application startup complete.
|
||||
@@ -44,7 +47,7 @@ INFO: Application startup complete.
|
||||
|
||||
You can query the result with input prompts:
|
||||
|
||||
```
|
||||
```shell
|
||||
curl http://localhost:8000/v1/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
@@ -71,7 +74,7 @@ curl http://localhost:8000/v1/completions \
|
||||
|
||||
The output format matches the following:
|
||||
|
||||
```
|
||||
```json
|
||||
{
|
||||
"id": "cmpl-2f678e8bdf5a4b209a3f2c1fa5832e25",
|
||||
"object": "text_completion",
|
||||
@@ -108,7 +111,7 @@ pip install lm-eval[api]
|
||||
|
||||
Run the following command:
|
||||
|
||||
```
|
||||
```shell
|
||||
# Only test gsm8k dataset in this demo
|
||||
lm_eval \
|
||||
--model local-completions \
|
||||
@@ -119,7 +122,7 @@ lm_eval \
|
||||
|
||||
After 30 minutes, the output is as shown below:
|
||||
|
||||
```
|
||||
```shell
|
||||
The markdown format results is as below:
|
||||
|
||||
|Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr|
|
||||
@@ -130,6 +133,7 @@ The markdown format results is as below:
|
||||
```
|
||||
|
||||
## Offline Server
|
||||
|
||||
### 1. Run docker container
|
||||
|
||||
You can run docker container on a single NPU:
|
||||
@@ -161,6 +165,7 @@ docker run --rm \
|
||||
```
|
||||
|
||||
### 2. Run GSM8K using lm-eval for accuracy testing
|
||||
|
||||
Install lm-eval in the container:
|
||||
|
||||
```bash
|
||||
@@ -170,7 +175,7 @@ pip install lm-eval
|
||||
|
||||
Run the following command:
|
||||
|
||||
```
|
||||
```shell
|
||||
# Only test gsm8k dataset in this demo
|
||||
lm_eval \
|
||||
--model vllm \
|
||||
@@ -181,7 +186,7 @@ lm_eval \
|
||||
|
||||
After 1 to 2 minutes, the output is shown below:
|
||||
|
||||
```
|
||||
```shell
|
||||
The markdown format results is as below:
|
||||
|
||||
Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr|
|
||||
|
||||
Reference in New Issue
Block a user