dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix: patched_reshape_and_cache signature 7→8 args (k_scale, v_scale)

Browse Source 
Original vllm ops.reshape_and_cache passes 8 args:
  key, value, key_cache, value_cache, slot_mapping, kv_cache_dtype, k_scale, v_scale
Our patch had 7 (single kv_scale) → TypeError on first request → engine dead.
This commit is contained in:
Claude
2026-08-17 18:10:52 +00:00
parent 8d969822b4
commit f7b1b2d119
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
qwen3_6_scripts/ex_engine/python/patch_vllm_hot_path.py
View File

@@ -164,7 +164,8 @@ def apply(strict=True):
import vllm._custom_ops as ops import vllm._custom_ops as ops
def patched_reshape_and_cache(key, value, key_cache, value_cache, def patched_reshape_and_cache(key, value, key_cache, value_cache,
slot_mapping, kv_cache_dtype, kv_scale): slot_mapping, kv_cache_dtype,
k_scale, v_scale):
xllm_ops.reshape_and_cache(key, value, key_cache, value_cache, xllm_ops.reshape_and_cache(key, value, key_cache, value_cache,
slot_mapping) slot_mapping)