EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Docs: Quick fix for Speculative_decoding doc (#3228)

Browse Source 
Co-authored-by: Chayenne <zhaochenyang@ucla.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
This commit is contained in:
Jhin
2025-01-31 10:30:40 -06:00
committed by GitHub
parent cf0f7eafe6
commit 656f7fc1bc
1 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/backend/speculative_decoding.ipynb
View File

@@ -8,10 +8,11 @@
"\n", "\n",
"SGLang now provides an EAGLE-based speculative decoding option. The implementation aims to maximize speed and efficiency and is considered to be among the fastest in open-source LLM engines.\n", "SGLang now provides an EAGLE-based speculative decoding option. The implementation aims to maximize speed and efficiency and is considered to be among the fastest in open-source LLM engines.\n",
"\n", "\n",
"**Note:** Currently, Speculative Decoding in SGLang does not support radix cache.\n",
"\n",
"To run the following tests or benchmarks, you also need to install [**cutex**](https://pypi.org/project/cutex/): \n", "To run the following tests or benchmarks, you also need to install [**cutex**](https://pypi.org/project/cutex/): \n",
"> ```bash\n", "\n",
"> pip install cutex\n", "`pip install cutex`\n",
"> ```\n",
"\n", "\n",
"### Performance Highlights\n", "### Performance Highlights\n",
"\n", "\n",