[feat] support customized and separated hccl_buffer_size for process group initialization (#3073)

### What this PR does / why we need it?
Currently, users have to set `HCCL_BUFFSIZE` to 512~1024 to perform mc2
operators (dispatch and combine) while running moe models with large
`ep_size` and `batch_size`. This environmental variable not only affects
allocated VRAM for mc2 group, but also increases VRAM allocation for dp,
tp & ep groups, leading to significant kvcache and free_memory drops.
This PR supports to automatically calculate and set `hccl_buffer_size`
for each process group **(except mc2 group)** separately when users set
`HCCL_BUFFSIZE` for mc2 group. This can significantly reduce wasted
buffer_size set for dp, tp & ep groups.

Note that current mc2 operators can only perform communication space
partitioning based on `HCCL_BUFFSIZE` configuration. Once they support
`hccl_buffer_size` configuration with `pg_options` while initializing
process group, we'll caculate the required buffer size and users would
avoid set `HCCL_BUFFSIZE` themselves.

### Does this PR introduce _any_ user-facing change?
No. 

### How was this patch tested?
We performed E2E serving with deepseek_r1 initializing DP/TP/EP/MC2
process group and observed significant kv_cache and free_memory
increase!


- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0

---------

Signed-off-by: linfeng-yuan <1102311262@qq.com>
This commit is contained in:
linfeng-yuan
2025-10-11 15:55:22 +08:00
committed by GitHub
parent 9eb103607f
commit e4acb2dfc7
4 changed files with 143 additions and 6 deletions

View File

@@ -87,6 +87,19 @@
# ** File: worker/patch_common/patch_distributed.py **
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# 1. `vllm.distributed.parallel_state.GroupCoordinator`
# (1) __init__()
# Why:
# The original GroupCoordinator initialization lacks pg_options to generate new
# process group with customized options.
# How:
# Inject HCCL options during process group initialization.
# Related PR (if no, explain why):
# Need a PR to vllm to support a dictionary as input while initializing distributed
# environment (e.g., Dict[str, torch.distributed.ProcessGroupHCCL.Options])
# https://github.com/vllm-project/vllm/pull/25417
# Future Plan:
# Remove this patch when vllm merges this PR.
# (2) all_to_all()
# Why:
# vllm doesn't support all_to_all for GroupCoordinator.
# How