EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[Bench] Flush cache before benchmarking (#2566)

Browse Source
This commit is contained in:
Ying Sheng
2024-12-23 19:21:21 -08:00
committed by GitHub
parent e835a50021
commit 8a56b43175
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/bench_serving.py
View File

@@ -924,6 +924,7 @@ async def benchmark(
f"are correctly specified. Error: {test_output.error}" f"are correctly specified. Error: {test_output.error}"
) )
else: else:
requests.post(base_url + "/flush_cache")
print("Initial test run completed. Starting main benchmark run...") print("Initial test run completed. Starting main benchmark run...")
time.sleep(1.5) time.sleep(1.5)