benchmark json schema (#2030)

This commit is contained in:
DarkSharpness
2024-11-15 22:06:19 +09:00
committed by GitHub
parent 2558d6a675
commit 954f4e6bd6
2 changed files with 153 additions and 0 deletions

View File

@@ -0,0 +1,15 @@
## Run benchmark
### Benchmark sglang
Run Llama-8b
```bash
python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --port 30000
```
Benchmark
```bash
python3 bench_sglang.py
```