EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[Feat][Doc] Add a load_balance_dp_proxy in examples and external dp doc. (#4265)

Browse Source 
### What this PR does / why we need it?
This PR adds a load-balance dp proxy server which can be used in
external DP scenario without Disaggregated-Prefill enabled. What's more,
add a doc of external dp and load-balance dp proxy server.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
See the new doc.

- vLLM version: v0.11.0
- vLLM main:
2918c1b49c

---------

Signed-off-by: whx-sjtu <2952154980@qq.com>
This commit is contained in:
whx
2025-11-21 16:33:23 +08:00
committed by GitHub
parent 6c157cb75a
commit a5554b6661
6 changed files with 514 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/disaggregated_prefill_v1/load_balance_proxy_server_example.py
View File

@@ -15,7 +15,7 @@
# - Streams responses from backend servers to clients.
#
# Prerequisites:
# - Python 3.8+
# - Python 3.10+
# - Install dependencies:
# pip install fastapi httpx uvicorn vllm
#