[BugFix]Fix group list type of mc2. (#4047)
### What this PR does / why we need it?
Fix accrucy problem of eplb because of PTA upgrade.
### How was this patch tested?
Main:
baseline:
| dataset | version | metric | mode | vllm-api-general-chat |
|----- | ----- | ----- | ----- | -----|
| aime2024 | 604a78 | accuracy | gen | 87.50 |
EPLB:
| dataset | version | metric | mode | vllm-api-general-chat |
|----- | ----- | ----- | ----- | -----|
| aime2024 | 604a78 | accuracy | gen | 87.50 |
- vLLM version: v0.11.0
- vLLM main:
83f478bb19
Signed-off-by: offline0806 <3337230449@qq.com>
Co-authored-by: offline0806 <3337230449@qq.com>
This commit is contained in:
@@ -225,7 +225,7 @@ class TokenDispatcherWithMC2(MoETokenDispatcher):
|
||||
"expand_scales": expand_scales
|
||||
}
|
||||
|
||||
group_list_type = 1 if dynamic_eplb else 0
|
||||
group_list_type = 0
|
||||
|
||||
return {
|
||||
"group_list_type": group_list_type,
|
||||
|
||||
Reference in New Issue
Block a user