Update version to v0.1.13 (#280)

This commit is contained in:
Lianmin Zheng
2024-03-11 05:49:27 -07:00
committed by GitHub
parent 13662fd533
commit 4aa5dd2c5f
11 changed files with 35 additions and 21 deletions

View File

@@ -37,6 +37,23 @@ python3 bench_sglang.py --nsub 3
# Average accuracy: 0.413
```
#### GSM-8K
```
cd benchmark/gsm8k
```
Follow README.md to download the data.
```
python3 bench_sglang.py --num-q 200
# Expected performance on A10G
# Latency: 32.103
# Accuracy: 0.250
```
#### More
Please also test `benchmark/hellaswag`, `benchmark/latency_throughput`.
### More Models
#### LLaVA
@@ -48,6 +65,9 @@ python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenize
```
cd benchmark/llava_bench
python3 bench_sglang.py
# Expected performance on A10G
# Latency: 50.031
```
## SGLang Unit Tests