EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[main] v_proj combining transpose and matmul (#3545)

Browse Source 
### What this PR does / why we need it?

v_proj combining transpose and matmul

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

CI passed with new added/existing test.

- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0

Signed-off-by: GDzhu1 <809721801@qq.com>
This commit is contained in:
Zhu Yi Lin
2025-10-20 19:53:32 +08:00
committed by GitHub
parent e04a5e3dd3
commit 34c2996ab8
1 changed files with 7 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
vllm_ascend/attention/mla_v1.py
View File

@@ -559,12 +559,13 @@ class AscendMLAImpl(MLAAttentionImpl):
self.speculative_config = vllm_config.speculative_config self.speculative_config = vllm_config.speculative_config
def _v_up_proj(self, x): def _v_up_proj(self, x):
# Convert from (B, N, L) to (N, B, L) x = x.view(-1, self.num_heads, self.kv_lora_rank)
x = x.view(-1, self.num_heads, self.kv_lora_rank).transpose(0, 1) x = torch_npu.npu_transpose_batchmatmul(x,
# Multiply (N, B, L) x (N, L, V) -> (N, B, V) self.W_UV,
x = torch.bmm(x, self.W_UV) perm_x1=[1, 0, 2],
# Convert from (N, B, V) to (B, N * V) perm_x2=[0, 1, 2],
x = x.transpose(0, 1).reshape(-1, self.num_heads * self.v_head_dim) perm_y=[1, 0, 2])
x = x.reshape(-1, self.num_heads * self.v_head_dim)
return x return x
# Return `ql_nope`, `q_pe` # Return `ql_nope`, `q_pe`