EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[feat] add LMCacheAscendConnector (#6882)

Browse Source 
### What this PR does / why we need it?

LMCache-Ascend is LMCache's solution on the Ascend platform and one of
the KVCache pooling solutions for Ascend. We hope to integrate
LMCache-Ascend into the vLLM-Ascend community as one of the official
KVCache pooling solutions for vLLM-Ascend.

We added a new LMCacheAscendConnector in vLLM-Ascend and registered it.

### Does this PR introduce _any_ user-facing change?

Users can specify the kvconnector using `--kv-transfer-config`, allowing
them to freely choose which kvconnector to use, without any user-facing
change.

### How was this patch tested?

Test by specifying `--kv-transfer-config
'{"kv_connector":"LMCacheAscendConnector","kv_role":"kv_both"}'`

- vLLM version: v0.16.0
- vLLM main:
15d76f74e2

---------

Signed-off-by: chloroethylene <jjysama@gmail.com>
This commit is contained in:
Junyuan
2026-03-13 17:41:35 +08:00
committed by GitHub
parent 986cd45397
commit 6852a2e267
5 changed files with 109 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
vllm_ascend/distributed/kv_transfer/__init__.py
View File

@@ -51,3 +51,9 @@ def register_connector():
KVConnectorFactory.register_connector(
"UCMConnector", "vllm_ascend.distributed.kv_transfer.kv_pool.ucm_connector", "UCMConnectorV1"
)
KVConnectorFactory.register_connector(
"LMCacheAscendConnector",
"vllm_ascend.distributed.kv_transfer.kv_pool.lmcache_ascend_connector",
"LMCacheConnectorV1",
)