EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Fixed the issue where eagle3 TPOT was not as good as without eagle3. (#9404)

Browse Source
This commit is contained in:
jiapingW
2025-08-21 07:42:01 +08:00
committed by GitHub
parent c10b8e6a0f
commit e99729c9f3
1 changed files with 7 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/speculative/eagle_utils.py
View File

@@ -453,6 +453,7 @@ class EagleVerifyInput:
sampling_info.top_ks, self.draft_token_num, dim=0 sampling_info.top_ks, self.draft_token_num, dim=0
), ),
) # (bs * draft_token_num, vocab_size) ) # (bs * draft_token_num, vocab_size)
if not torch.all(sampling_info.top_ps == 1.0):
target_probs = top_p_renorm_prob( target_probs = top_p_renorm_prob(
target_probs, target_probs,
torch.repeat_interleave( torch.repeat_interleave(