EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Small improvement of native api docs (#5139)

Browse Source 
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
This commit is contained in:
simveit
2025-04-08 21:09:26 +02:00
committed by GitHub
parent 6d3b35fae9
commit f8194b267c
1 changed files with 10 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/backend/native_api.ipynb
View File

@@ -371,7 +371,9 @@
"source": [ "source": [
"## Capture expert selection distribution in MoE models\n", "## Capture expert selection distribution in MoE models\n",
"\n", "\n",
"SGLang Runtime supports recording the number of times an expert is selected in a MoE model run for each expert in the model. This is useful when analyzing the throughput of the model and plan for optimization." "SGLang Runtime supports recording the number of times an expert is selected in a MoE model run for each expert in the model. This is useful when analyzing the throughput of the model and plan for optimization.\n",
"\n",
"*Note: We only print out the first 10 lines of the csv below for better readability. Please adjust accordingly if you want to analyze the results more deeply.*"
] ]
}, },
{ {
@@ -412,9 +414,13 @@
"\n", "\n",
"output_file = glob.glob(\"expert_distribution_*.csv\")[0]\n", "output_file = glob.glob(\"expert_distribution_*.csv\")[0]\n",
"with open(output_file, \"r\") as f:\n", "with open(output_file, \"r\") as f:\n",
" print_highlight(\"Content of dumped record:\")\n", " print_highlight(\"\\n| Layer ID | Expert ID | Count |\")\n",
" for line in f:\n", " print_highlight(\"|----------|-----------|--------|\")\n",
" print_highlight(line.strip())" " next(f)\n",
" for i, line in enumerate(f):\n",
" if i < 9:\n",
" layer_id, expert_id, count = line.strip().split(\",\")\n",
" print_highlight(f\"| {layer_id:8} | {expert_id:9} | {count:6} |\")"
] ]
}, },
{ {