EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

bugfix: penalizers to be merged before reqs (#1001)

Browse Source
This commit is contained in:
Juwan Yoo
2024-08-09 04:46:24 -07:00
committed by GitHub
parent b91a4cb1b1
commit 10bca45bc6
3 changed files with 44 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
python/sglang/srt/sampling/penaltylib/orchestrator.py
View File

@@ -133,6 +133,10 @@ class BatchedPenalizerOrchestrator:
"""
Merge the penalizers of another orchestrator into this one.
Note that this function **must** be called _before_ self.batch.reqs is updated (filtered).
Each unprepared penalizers would have to be prepared (creating tensors, etc.) first before merging.
This step requires the original batch.reqs, before it gets merged with other batch.reqs.
Args:
their (BatchedPenalizerOrchestrator): The orchestrator to merge into this one.
"""