EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

fix draft cuda graph capture failure (#3431)

Browse Source
This commit is contained in:
Yineng Zhang
2025-02-09 23:16:20 +08:00
committed by GitHub
parent bc72e5bd32
commit 1646149a83
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/speculative/eagle_draft_cuda_graph_runner.py
View File

@@ -85,6 +85,7 @@ class EAGLEDraftCudaGraphRunner:
"1. disable cuda graph by --disable-cuda-graph\n" "1. disable cuda graph by --disable-cuda-graph\n"
"2. set --mem-fraction-static to a smaller value (e.g., 0.8 or 0.7)\n" "2. set --mem-fraction-static to a smaller value (e.g., 0.8 or 0.7)\n"
"3. disable torch compile by not using --enable-torch-compile\n" "3. disable torch compile by not using --enable-torch-compile\n"
"4. specify --dtype to the same dtype (e.g. bfloat16)\n"
"Open an issue on GitHub https://github.com/sgl-project/sglang/issues/new/choose \n" "Open an issue on GitHub https://github.com/sgl-project/sglang/issues/new/choose \n"
) )