[feat] support dispatch_v2/combine_v2 hierarchy communication (#7698)
### What this PR does / why we need it? This PR adds support for hierarchical communication for `dispatch_v2` and `combine_v2` MoE operations. This is achieved by introducing a new configuration `enable_mc2_hierarchy_comm`. When enabled, the communication algorithm is set to "hierarchy", which support mc2 op comm between two super pod. The changes include: - Adding `enable_mc2_hierarchy_comm` to `AscendConfig`. - Modifying `TokenDispatcherWithMC2` to pass `comm_alg: "hierarchy"` to the underlying `torch_npu` ops when the new config is enabled. - Adding validation to ensure that this feature is only used with compatible PTA/CANN versions and is not used with the conflicting `fused_mc2` op. - Updating `is_hierarchical_communication_enabled` to respect the new configuration flag. ### Does this PR introduce _any_ user-facing change? Yes, this PR introduces a new user-facing configuration option `enable_mc2_hierarchy_comm` in `additional_config` to enable hierarchical communication for MoE. ### How was this patch tested? - vLLM version: v0.18.0 Signed-off-by: zzzzwwjj <1183291235@qq.com>
This commit is contained in:
@@ -165,6 +165,9 @@ class AscendConfig:
|
||||
and vllm_config.compilation_config.pass_config.enable_sp
|
||||
)
|
||||
|
||||
# Enable dispatch/combine op inter-node communication by ROCE
|
||||
self.enable_mc2_hierarchy_comm = additional_config.get("enable_mc2_hierarchy_comm", False)
|
||||
|
||||
@staticmethod
|
||||
def _get_compile_ranges(compilation_config):
|
||||
return compilation_config.compile_ranges_endpoints or []
|
||||
|
||||
Reference in New Issue
Block a user