zzhxxx a74a1196c5 [Feat] Support MLP_TP feature, exclude MOE layer (#4999) ...

#4257 This PR implements the dense_ffn TP of the first three layers of
the deepseek model, I have refactored this PR and used very little code
to support the implementation of this feature.
This PR adds a function `is_moe_layer` to mlp_tp, which supports MLP TP
in models with both mlp and moe, such as deepseek or chat GLM.


- vLLM version: v0.12.0
- vLLM main:
ad32e3e19c

Signed-off-by: zzhx1 <zzh_201018@outlook.com>
Co-authored-by: 子潜 <ziqian@U-DMKXH32D-2015.local>
Co-authored-by: chenxiao <Jaychou1620@Gmail.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>

2025-12-18 20:06:53 +08:00

..

device_communicators

[Test] Add ut for files in /distributed (#1951)

2025-07-24 10:36:11 +08:00

mooncake

[Feature][main]reconstruction kvpool connector to ascend connector (#4438)

2025-11-28 18:08:37 +08:00

test_communicator.py

[2/N][Feat] Add MC2 communication method for MoE layers (#2469)

2025-08-26 19:05:23 +08:00

test_determin_expert_map_all.py

Dynamic Expert Load Balance with Zero-like-overhead (#2956)

2025-09-17 10:36:43 +08:00

test_parallel_state.py

[Feat] Support MLP_TP feature, exclude MOE layer (#4999)

2025-12-18 20:06:53 +08:00