EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[bugfix][accuracy] Fix ds indexer accuracy problem caused by k rope (#7341)

Browse Source 
### What this PR does / why we need it?
The rotary algorithm in deepseek indexer should be neox-style instead of
gptj style. PR #4641 fix this accuracy bug in original pytorch version.
But PR #5701 accidentally removed the fixed code line and reverted the
implementation back to the problematic version. This PR fixes it.

Signed-off-by: rjg-lyh <1318825571@qq.com>
This commit is contained in:
rjg-lyh
2026-03-18 14:20:21 +08:00
committed by GitHub
parent c7157af8f7
commit c1392a6ce6
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm_ascend/attention/sfa_v1.py
View File

@@ -884,7 +884,7 @@ class AscendSFAImpl(MLAAttentionImpl):
sin = sin.view(-1, 1, 1, self.qk_rope_head_dim) sin = sin.view(-1, 1, 1, self.qk_rope_head_dim)
k_li_pe = k_li_pe.unsqueeze(2) k_li_pe = k_li_pe.unsqueeze(2)
k_li_pe = torch_npu.npu_interleave_rope(k_li_pe, cos, sin) k_li_pe = torch_npu.npu_rotary_mul(k_li_pe, cos, sin)
k_li_pe = k_li_pe.squeeze(2) k_li_pe = k_li_pe.squeeze(2)
k_li = torch.cat([k_li_pe, k_li_nope], dim=-1) # [b*s,128] k_li = torch.cat([k_li_pe, k_li_nope], dim=-1) # [b*s,128]