Compare commits

...

2 Commits

Author SHA1 Message Date
root
ae27d0405b Merge branch 'main' of https://github.com/dylanyunlon/project_6 2026-08-17 18:11:57 +00:00
Claude
f7b1b2d119 fix: patched_reshape_and_cache signature 7→8 args (k_scale, v_scale)
Original vllm ops.reshape_and_cache passes 8 args:
  key, value, key_cache, value_cache, slot_mapping, kv_cache_dtype, k_scale, v_scale
Our patch had 7 (single kv_scale) → TypeError on first request → engine dead.
2026-08-17 18:10:52 +00:00

View File

@@ -164,7 +164,8 @@ def apply(strict=True):
import vllm._custom_ops as ops
def patched_reshape_and_cache(key, value, key_cache, value_cache,
slot_mapping, kv_cache_dtype, kv_scale):
slot_mapping, kv_cache_dtype,
k_scale, v_scale):
xllm_ops.reshape_and_cache(key, value, key_cache, value_cache,
slot_mapping)