EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[FIX] Fix concatenation error in capture_bs when open --disable-cuda-graph-padding and without MTP (#5412)

Browse Source
This commit is contained in:
Zhaoyang Hao
2025-04-15 16:42:27 +08:00
committed by GitHub
parent f88f7e1943
commit 5d13440162
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/sglang/srt/model_executor/cuda_graph_runner.py
View File

@@ -116,7 +116,7 @@ def get_batch_sizes_to_capture(model_runner: ModelRunner):
if capture_bs is None:
if server_args.speculative_algorithm is None:
if server_args.disable_cuda_graph_padding:
capture_bs = list(range(1, 33)) + range(40, 161, 16)
capture_bs = list(range(1, 33)) + list(range(40, 161, 16))
else:
capture_bs = [1, 2, 4, 8] + list(range(16, 161, 8))
else: