[BugFix] Fix kv_no_split not contiguous (#3711)
allgather need contiguous data, split operation return uncontiguous data. - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 Signed-off-by: zhaozx-cn <zhaozx2116@163.com>
This commit is contained in:
@@ -1166,6 +1166,8 @@ class AscendMLAImpl(MLAAttentionImpl):
|
||||
dim=-1,
|
||||
)
|
||||
q_c = self.q_a_layernorm(q_c)
|
||||
# allgather need contiguous data
|
||||
kv_no_split = kv_no_split.contiguous()
|
||||
else:
|
||||
q_c = hidden_states
|
||||
kv_no_split = self.kv_a_proj_with_mqa(hidden_states)[0]
|
||||
|
||||
Reference in New Issue
Block a user