EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1,275 Commits 1 Branch 0 Tags
main
Commit Graph

2 Commits

Author SHA1 Message Date
whx
a58ff9e92f [Cherry-pick] Port MoE multi-stream fix to v0.11.0-dev (#3753) 
This PR moves the communication operation of shared experts out of extra
stream because I found that this might cause rtMemcpy related errors
when running shared experts multistream with aclgraph.

Furthermore, I utilize a global variable as extra stream object to avoid
allocating streams for each layer in full-graph mode.

Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-10-25 15:51:43 +08:00
whx
0a526768f5 [Feature] Support moe multi-stream for aclgraph. (#2946) 
This PR puts the calculation of shared experts into a separate stream,
overlaping with routing experts.

- vLLM version: v0.10.2
- vLLM main:
fbd6523ac0

---------

Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-09-19 11:06:45 +08:00