[Disaggregated Prefill] P2P Disaggregated Prefill based on llm_datadist (#694)
### What this PR does / why we need it? - This PR proposes a P2P version of Disaggregated Prefill based on llm_datadist which manages data transfer. - This solution reconstructs previous offline single-node Disaggregated Prefill solution, and supports multi-node and online serveing now. - Currently this solution supports 1P1D situation of Deepseek hybrid parallelism (P: TP+EP, D: DP+EP). Note that xPyD situation is considered in the solution design, and will be supported soon within v1 engine. --------- Signed-off-by: hw_whx <wanghexiang7@huawei.com> Signed-off-by: ganyi <pleaplusone.gy@gmail.com> Co-authored-by: hw_whx <wanghexiang7@huawei.com> Co-authored-by: ganyi <pleaplusone.gy@gmail.com>
This commit is contained in:
13
.github/workflows/vllm_ascend_test.yaml
vendored
13
.github/workflows/vllm_ascend_test.yaml
vendored
@@ -136,18 +136,9 @@ jobs:
|
||||
id: filter_spec_decode
|
||||
uses: dorny/paths-filter@v3
|
||||
with:
|
||||
# speculative decode seems will cause oom issue, disable it now on ci test
|
||||
filters: |
|
||||
speculative_tests_changed:
|
||||
- "tests/singlecard/spec_decode/**"
|
||||
- "tests/multicard/spec_decode_e2e/**"
|
||||
- "vllm_ascend/worker/worker.py"
|
||||
- "vllm_ascend/worker/model_runner.py"
|
||||
- "vllm_ascend/worker/multi_step_runner.py"
|
||||
- "vllm_ascend/worker/multi_step_worker.py"
|
||||
- "vllm_ascend/worker/draft_model_runner.py"
|
||||
- "vllm_ascend/patch/worker/patch_common/patch_metrics.py"
|
||||
- "vllm_ascend/patch/worker/patch_common/patch_spec_decode_worker.py"
|
||||
- "vllm_ascend/patch/worker/patch_common/patch_multi_step_worker.py"
|
||||
speculative_tests_changed: 'false'
|
||||
|
||||
- name: Run vllm-project/vllm-ascend Speculative Decode test
|
||||
if: steps.filter_spec_decode.outputs.speculative_tests_changed == 'true' || github.event_name == 'schedule'
|
||||
|
||||
Reference in New Issue
Block a user