EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[v0.11.0][BugFix][P/D] Modify the recalculation logic to prevent waiting requests from filling up the D node KVCache (#3686)

Browse Source 
### What this PR does / why we need it?
Modify the recalculation logic to prevent waiting requests from filling
up the D node KVCache

Signed-off-by: underfituu <hzhucong@163.com>
This commit is contained in:
hucong
2025-10-25 09:15:42 +08:00
committed by GitHub
parent 5c0a23f98b
commit 12bc78d252
3 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/disaggregated_prefill_v1/load_balance_proxy_layerwise_server_example.py
View File

@@ -565,7 +565,7 @@ async def _handle_completions(api: str, request: Request):
chunk_json = json.loads(chunk_str)
except json.JSONDecodeError:
# if chunk is [done], skip it.
logger.warning(
logger.debug(
f"Skipping chunk: {chunk_str}")
yield chunk
continue

2
examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py
View File

@@ -548,7 +548,7 @@ async def _handle_completions(api: str, request: Request):
chunk_json = json.loads(chunk_str)
except json.JSONDecodeError:
# if chunk is [done], skip it.
logger.warning(
logger.debug(
f"Skipping chunk: {chunk_str}")
yield chunk
continue