EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Fix logprob_start_len for multi modal models (#2597)

Browse Source 
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
This commit is contained in:
Lianmin Zheng
2024-12-26 06:27:45 -08:00
committed by GitHub
parent 637de9e8ce
commit 773951548d
4 changed files with 10 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python/sglang/srt/managers/scheduler.py
View File

@@ -565,7 +565,7 @@ class Scheduler:
if req.logprob_start_len == -1:
# By default, only return the logprobs for output tokens
req.logprob_start_len = len(recv_req.input_ids) - 1
req.logprob_start_len = len(req.origin_input_ids) - 1
# Truncate prompts that are too long
if len(req.origin_input_ids) > self.max_req_input_len: