EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Add sglang.bench_latency to CI (#1243)

Browse Source
This commit is contained in:
Lianmin Zheng
2024-08-28 04:20:54 -07:00
committed by GitHub
parent f25f4dfde5
commit 6c49831394
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.github/workflows/e2e-test.yml vendored
View File

@@ -38,6 +38,11 @@ jobs:
cd test/srt
python3 -m unittest test_serving_throughput.TestServingThroughput.test_default
- name: Benchmark Serving Latency
timeout-minutes: 10
run: |
python3 -m sglang.bench_latency --model meta-llama/Meta-Llama-3.1-8B-Instruct --batch-size 1 --input 128 --output 8
- name: Benchmark Serving Throughput (w/o RadixAttention)
timeout-minutes: 10
run: |