EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,183 Commits 2 Branches 0 Tags
39b994a987888f7ba78df28b1ccb41a5e8d6eaf5
Commit Graph

9 Commits

Author SHA1 Message Date
Li Wang
802c574532 [Benchmark] Upgrade benchmark args for new vllm version (#3218) 
### What this PR does / why we need it?
Since the newest vllm commit has deprecated the arg `--endpoint-type`,
we should use `--backend` instead
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?
test it locally:
```shell
export VLLM_USE_MODELSCOPE=true
export DATASET_PATH=/root/.cache/datasets/ShareGPT_V3_unfiltered_cleaned_split.json

vllm serve Qwen/Qwen2.5-7B-Instruct --load-format dummy

wget -O ${DATASET_PATH}  /root/.cache/datasets/ShareGPT_V3_unfiltered_cleaned_split.json https://hf-mirror.com/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json

vllm bench serve --model Qwen/Qwen2.5-7B-Instruct --backend vllm --dataset-name sharegpt --dataset-path ${DATASET_PATH}  --num-prompt 200
```
and the result looks good:
```shell
============ Serving Benchmark Result ============
Successful requests:                     200
Benchmark duration (s):                  20.36
Total input tokens:                      43560
Total generated tokens:                  44697
Request throughput (req/s):              9.82
Output token throughput (tok/s):         2194.88
Peak output token throughput (tok/s):    4676.00
Peak concurrent requests:                200.00
Total Token throughput (tok/s):          4333.93
---------------Time to First Token----------------
Mean TTFT (ms):                          2143.85
Median TTFT (ms):                        2486.17
P99 TTFT (ms):                           2530.36
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          43.50
Median TPOT (ms):                        30.75
P99 TPOT (ms):                           309.22
---------------Inter-token Latency----------------
Mean ITL (ms):                           28.15
Median ITL (ms):                         25.42
P99 ITL (ms):                            38.30
==================================================
```
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-10-24 11:18:19 +08:00
Li Wang
2967e5e22a [Benchmark] Correctly kill vllm process in performance benchamrk (#2782) 
### What this PR does / why we need it?
vLLM now names the process with VLLM prefix after
https://github.com/vllm-project/vllm/pull/21445, we should kill the
correct process name after one iteration benchmark to avoid OOM issue
### Does this PR introduce _any_ user-facing change?

### How was this patch tested?

- vLLM version: v0.10.1.1
- vLLM main:
e599e2c65e

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-09-07 10:36:34 +08:00
Li Wang
6db7dc2c85 [Benchmark] Refactor perf script to use benchmark cli (#1524) 
### What this PR does / why we need it?

Since, `vllm bench` cli has optimized enough for production use(support
more datasets), we are now do not need to copy vllm codes, now , with
vllm installed, we can easily use the benchmark cli
### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-06-30 23:42:04 +08:00
Li Wang
dd207cb261 [CI][Benchmark] Add new model and v1 test to perf benchmarks (#1099) 
### What this PR does / why we need it?
- Add qwen2.5-7b-instruct test
- Add v1 test
---------

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-06-12 10:46:41 +08:00
Li Wang
76dacf3fa0 [CI][Benchmark] Optimize performance benchmark workflow (#1039) 
### What this PR does / why we need it?

This is a post patch of #1014, for some convenience optimization
- Set cached dataset path for speed
- Use pypi to install escli-tool
- Add benchmark results convert script to have a developer-friendly
result
- Patch the `benchmark_dataset.py` to disable streaming load for
internet
- Add more trigger ways for different purpose, `pr` for debug,
`schedule` for daily test, `dispatch` and `pr-labled` for manual testing
of a single(current) commit
- Disable latency test for `qwen-2.5-vl`, (This script does not support
multi-modal yet)

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-06-03 23:38:34 +08:00
Li Wang
d9fb027068 [CI] Add benchmark workflows (#1014) 
### What this PR does / why we need it?

Add benchmark workflows

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Run locally

---------

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-05-30 22:42:44 +08:00
Li Wang
218f21de21 [Benchmarks] Add qwen2.5-7b test (#763) 
### What this PR does / why we need it?
- Add qwen2.5-7b test
- Optimize the documentation to be more developer-friendly 

Signed-off-by: xuedinge233 <damow890@gmail.com>
Co-authored-by: xuedinge233 <damow890@gmail.com>
 2025-05-10 09:47:42 +08:00
Li Wang
866ce7168c [Benchmark] Download model from modelscope (#634) 
### What this PR does / why we need it?
-  Run benchmark scripts will Download model from modelscope

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-04-24 14:48:24 +08:00
Li Wang
9a175ca0fc [Doc]Add benchmark scripts (#74) 
### What this PR does / why we need it?
The purpose of this PR is to add benchmark scripts for npu, developers
can easily run performance tests on their own machines with one line of
code .


---------

Signed-off-by: wangli <wangli858794774@gmail.com>
 2025-03-21 15:54:34 +08:00