EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Minor bugfix on benchmark serving (#1005)

Browse Source
This commit is contained in:
Roger Wang
2024-08-09 09:53:50 -07:00
committed by GitHub
parent b568df5d03
commit 05c50a82b8
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
python/sglang/bench_serving.py
View File

@@ -195,7 +195,8 @@ async def async_request_openai_completions(
output.ttft = ttft output.ttft = ttft
# Decoding phase # Decoding phase
output.itl.append(timestamp - most_recent_timestamp) else:
output.itl.append(timestamp - most_recent_timestamp)
most_recent_timestamp = timestamp most_recent_timestamp = timestamp
generated_text += data["choices"][0]["text"] generated_text += data["choices"][0]["text"]