Bug: apply final_hidden_states*=self.routed_scaling_factor at MoE lay… (#8511)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
This commit is contained in:
@@ -304,7 +304,7 @@ class EPMoE(FusedMoE):
|
||||
m_max * self.start_expert_id,
|
||||
BLOCK_SIZE=512,
|
||||
)
|
||||
return output
|
||||
return output * self.routed_scaling_factor
|
||||
|
||||
|
||||
class DeepEPMoE(EPMoE):
|
||||
|
||||
Reference in New Issue
Block a user