EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

fix: draft model IMA by overide max_positional_embeddings (#10787)

Browse Source 
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
This commit is contained in:
Xinyuan Tong
2025-09-23 12:56:16 -07:00
committed by GitHub
parent 42aedb02af
commit aab35bccb4
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
python/sglang/srt/configs/model_config.py
View File

@@ -210,6 +210,11 @@ class ModelConfig:
): ):
logger.warning(msg) logger.warning(msg)
self.context_len = context_length self.context_len = context_length
if is_draft_model:
self.hf_text_config.max_position_embeddings = context_length
logger.warning(
f"Overriding the draft model's max_position_embeddings to {context_length}."
)
else: else:
raise ValueError( raise ValueError(
f"{msg} To allow overriding this maximum, set the env var SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1" f"{msg} To allow overriding this maximum, set the env var SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1"