EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[BugFix][P/D] Modify the recalculation logic to prevent waiting requests from filling up the D node KVCache (#3641)

Browse Source 
### What this PR does / why we need it?
Modify the recalculation logic to prevent waiting requests from filling
up the D node KVCache

- vLLM version: v0.11.0rc3
- vLLM main:
17c540a993

Signed-off-by: underfituu <hzhucong@163.com>
This commit is contained in:
hucong
2025-10-25 09:14:20 +08:00
committed by GitHub
parent 39b994a987
commit 292cf339c3
3 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py
View File

@@ -565,7 +565,7 @@ async def _handle_completions(api: str, request: Request):
chunk_json = json.loads(chunk_str)
except json.JSONDecodeError:
# if chunk is [done], skip it.
logger.warning(
logger.debug(
f"Skipping chunk: {chunk_str}")
yield chunk
continue

2
examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py
View File

@@ -548,7 +548,7 @@ async def _handle_completions(api: str, request: Request):
chunk_json = json.loads(chunk_str)
except json.JSONDecodeError:
# if chunk is [done], skip it.
logger.warning(
logger.debug(
f"Skipping chunk: {chunk_str}")
yield chunk
continue