[Bugfix]Modify NPU rotary encoding parameter fields,fix RopeOperation setup failed in condition of self.rotary_dim < self.head_size (#6310)
### What this PR does / why we need it? change self.head_size to self.rotary_dim. only the rotary part is processed here, the dimension should be rotary_dim. Fix bug #6060 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Only a small section of code was modified to adjust the parameters, and all standard tests were passed. - vLLM version: v0.14.1 - vLLM main: https://github.com/vllm-project/vllm/commit/dc917cceb877dfd13f98c538c4c96158047d98bd Signed-off-by: fengshi666 <fengshi666@adsl-99-12-210-25.dsl.hstntx.sbcglobal.net> Co-authored-by: fengshi666 <fengshi666@adsl-99-12-210-25.dsl.hstntx.sbcglobal.net>
This commit is contained in:
1 parent
77ea873224
commit
4230bc8646
1 file changed
+3
-1
@@ -240,11 +240,13 @@ def _rope_forward_oot(
|
||||
k_pass = key[..., self.rotary_dim:]
|
||||
q_rot = q_rot.contiguous().view(num_tokens, -1)
|
||||
k_rot = k_rot.contiguous().view(num_tokens, -1)
|
||||
# only the rotary part is processed here,
|
||||
# the dimension should be rotary_dim
|
||||
torch_npu._npu_rotary_embedding(
|
||||
positions,
|
||||
q_rot,
|
||||
k_rot,
|
||||
self.head_size,
|
||||
self.rotary_dim,
|
||||
self.cos_sin_cache,
|
||||
is_neox_style,
|
||||
)
|
||||
|
||||
Reference in new issue
Block a user