EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[Doc]Add benchmark scripts (#74)

Browse Source 
### What this PR does / why we need it?
The purpose of this PR is to add benchmark scripts for npu, developers
can easily run performance tests on their own machines with one line of
code .


---------

Signed-off-by: wangli <wangli858794774@gmail.com>
This commit is contained in:
Li Wang
2025-03-21 15:54:34 +08:00
committed by GitHub
parent befbee5883
commit 9a175ca0fc
6 changed files with 397 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
benchmarks/tests/throughput-tests.json Normal file
View File

@@ -0,0 +1,14 @@
[
{
"test_name": "throughput_llama8B_tp1",
"parameters": {
"model": "meta-llama/Llama-3.1-8B-Instruct",
"tensor_parallel_size": 1,
"load_format": "dummy",
"dataset_path": "./ShareGPT_V3_unfiltered_cleaned_split.json",
"num_prompts": 200,
"backend": "vllm"
}
}
]