EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[0.18.0][BugFix] Add PrefillNoCache state in mla _forward_decode for short prompt (#8264)

Browse Source 
### What this PR does / why we need it?
This PR is cherry-pick from #8263.

This PR aims to fix short prompt problem. The root cause can be found in
#8029. Since the previous pr may miss mixed long and short prompt batch,
after discussion, we decide to add PrefillNoCache state in mla
_forward_decode now instead.

Signed-off-by: Zetong Li <slippersss@126.com>
This commit is contained in:
Zetong Li
2026-04-15 09:23:52 +08:00
committed by GitHub
parent 70713c3fd4
commit b6aa5bbdbf
2 changed files with 1 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
vllm_ascend/attention/mla_v1.py
View File

@@ -1270,6 +1270,7 @@ class AscendMLAImpl(MLAAttentionImpl):
AscendAttentionState.SpecDecoding,
AscendAttentionState.ChunkedPrefill,
AscendAttentionState.DecodeOnly,
AscendAttentionState.PrefillNoCache, # for extremely short prefills
]
and self.speculative_config is not None
):