EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[BugFix] Add async communication check for capturing mode (#8149)

Browse Source 
### What this PR does / why we need it?
Introduce a check to not using asynchronous communication under
`enable_dsa_cp_with_layer_shard` branch on capturing mode. This change
prevents potential stream and event issues when operating in
graph/capturing mode, ensuring safer communication practices.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
E2E test with dsv32 + FC1 + FULL_DECODE_ONLY +
kv_transfer_config(kv_both)

---------

Signed-off-by: chenchuw886 <chenchuw@huawei.com>
Co-authored-by: chenchuw886 <chenchuw@huawei.com>
This commit is contained in:
Frank Chen
2026-04-12 21:52:54 +08:00
committed by GitHub
parent c1f323ee46
commit 31186a3a9d
8 changed files with 85 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/user_guide/feature_guide/layer_sharding.md
View File

@@ -37,11 +37,15 @@ To enable **Layer Shard Linear**, specify the target linear layers using the `--
}'
```
> **Restriction**
> In PD-disaggregated deployments, Layer Sharding can only be enabled on the **P node** with `kv_role="kv_producer"`.
> `kv_role="kv_consumer"` and `kv_role="kv_both"` are not supported.
---
## Supported Scenarios
This feature can be enabled in any scenario, but delivers the greatest benefit in the following cases:
This feature delivers the greatest benefit in the following cases:
### FlashComm2-enabled
@@ -62,6 +66,8 @@ vllm serve \
With [DSA-CP](https://github.com/vllm-project/vllm-ascend/pull/4702), both `q_b_proj` and `o_proj` layers require large weight matrices to be stored per layer. Sharding these layers across NPUs helps fit extremely deep models (e.g., 61-layer architectures) into limited device memory.
In PD-disaggregated deployments, this mode is supported only on the **P node** with `kv_role="kv_producer"`.
**Example configuration:**
```bash