EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836)

Browse Source
This commit is contained in:
fzyzcjy
2025-04-18 12:38:26 +08:00
committed by GitHub
parent 1effba4c70
commit 53dcf38876
4 changed files with 31 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/managers/schedule_batch.py
View File

@@ -78,6 +78,7 @@ global_server_args_dict = {
"speculative_accept_threshold_acc": ServerArgs.speculative_accept_threshold_acc,
"disable_radix_cache": ServerArgs.disable_radix_cache,
"flashinfer_mla_disable_ragged": ServerArgs.flashinfer_mla_disable_ragged,
"moe_dense_tp_size": ServerArgs.moe_dense_tp_size,
"chunked_prefill_size": ServerArgs.chunked_prefill_size,
"n_share_experts_fusion": ServerArgs.n_share_experts_fusion,
"disable_shared_experts_fusion": ServerArgs.disable_shared_experts_fusion,