EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add note for deepseek related docs and remove unnecessary comments (#590)

Browse Source 
### What this PR does / why we need it?
Add notes for deepseek's patch and remove some of the unnecessary
comments

---------

Signed-off-by: ganyi <pleaplusone.gy@gmail.com>
This commit is contained in:
Pleaplusone
2025-04-22 09:59:09 +08:00
committed by GitHub
parent c5850d302d
commit d12a057df8
6 changed files with 78 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
vllm_ascend/worker/model_runner_v1.py
View File

@@ -835,7 +835,6 @@ class NPUModelRunner:
assert num_blocks >= kv_cache_config.num_blocks
# TODO: remove this after the OOM issue is located and fixed, otherwise, some model may
# encounter OOM issue
num_blocks = num_blocks // 4
if isinstance(kv_cache_spec, FullAttentionSpec):
kv_cache_shape = self.attn_backend.get_kv_cache_shape(
num_blocks, kv_cache_spec.block_size,