EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[Bugfix] Correct method call for _set_cos_sin_cache (#774)

Browse Source 
This change ensures proper functionality for longer sequences by
correctly invoking the _set_cos_sin_cache method with self as the first
argument.

For example, with DeepSeek R1, if this change isn't made, the program
will crash when the input sequence exceeds 4096.

Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
This commit is contained in:
Jade Zheng
2025-05-09 12:55:57 +08:00
committed by GitHub
parent 5301649108
commit 2c685e3b61
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm_ascend/ops/rotary_embedding.py
View File

@@ -82,7 +82,7 @@ def native_rope_deepseek_forward(self,
offsets: Optional[torch.Tensor] = None, offsets: Optional[torch.Tensor] = None,
max_seq_len: Optional[int] = None): max_seq_len: Optional[int] = None):
if max_seq_len is not None and max_seq_len > self.max_seq_len: if max_seq_len is not None and max_seq_len > self.max_seq_len:
self._set_cos_sin_cache(max_seq_len, query.device, query.dtype) _set_cos_sin_cache(self, max_seq_len, query.device, query.dtype)
if len(key.shape) == 2: if len(key.shape) == 2:
key = key[:, None, :] key = key[:, None, :]
# Note: we implement the non neox_style method with shuffle the last dim and neox style # Note: we implement the non neox_style method with shuffle the last dim and neox style