EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Raise error when topk>1 and page>1 for paged attention backends. (#9784)

Browse Source 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
Liangsheng Yin
2025-08-29 12:47:34 +08:00
committed by GitHub
parent a7d825fccc
commit a23c30205d
1 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
python/sglang/srt/server_args.py
View File

@@ -747,6 +747,15 @@ class ServerArgs:
) )
self.speculative_num_draft_tokens = self.speculative_num_steps + 1 self.speculative_num_draft_tokens = self.speculative_num_steps + 1
if (
self.speculative_eagle_topk > 1
and self.page_size > 1
and self.attention_backend != "flashinfer"
):
raise ValueError(
"speculative_eagle_topk > 1 with page_size > 1 is unstable and produces incorrect results for paged attention backends. This combination is only supported for the 'flashinfer' backend."
)
# The token generated from the verify step is counted. # The token generated from the verify step is counted.
# If sepculative_num_steps >= speculative_num_draft_tokens, the additional tokens will definitely be discarded. # If sepculative_num_steps >= speculative_num_draft_tokens, the additional tokens will definitely be discarded.
# assert self.speculative_num_steps < self.speculative_num_draft_tokens # assert self.speculative_num_steps < self.speculative_num_draft_tokens