[Bugfix]Modify NPU rotary encoding parameter fields,fix RopeOperation setup failed in condition of self.rotary_dim < self.head_size (#6310)
### What this PR does / why we need it?
change self.head_size to self.rotary_dim.
only the rotary part is processed here, the dimension should be rotary_dim.
Fix bug #6060
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Only a small section of code was modified to adjust the parameters, and
all standard tests were passed.
- vLLM version: v0.14.1
- vLLM main:
dc917cceb8
Signed-off-by: fengshi666 <fengshi666@adsl-99-12-210-25.dsl.hstntx.sbcglobal.net>
Co-authored-by: fengshi666 <fengshi666@adsl-99-12-210-25.dsl.hstntx.sbcglobal.net>
This commit is contained in:
@@ -240,11 +240,13 @@ def _rope_forward_oot(
|
||||
k_pass = key[..., self.rotary_dim:]
|
||||
q_rot = q_rot.contiguous().view(num_tokens, -1)
|
||||
k_rot = k_rot.contiguous().view(num_tokens, -1)
|
||||
# only the rotary part is processed here,
|
||||
# the dimension should be rotary_dim
|
||||
torch_npu._npu_rotary_embedding(
|
||||
positions,
|
||||
q_rot,
|
||||
k_rot,
|
||||
self.head_size,
|
||||
self.rotary_dim,
|
||||
self.cos_sin_cache,
|
||||
is_neox_style,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user