EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Cleaning codes for speculative attention mode (#10149)

Browse Source
This commit is contained in:
Baizhou Zhang
2025-09-08 17:38:06 -07:00
committed by GitHub
parent 148022fc36
commit 8ad700f735
7 changed files with 14 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/advanced_features/server_arguments.md
View File

@@ -209,6 +209,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
| `--speculative-accept-threshold-single` | Accept a draft token if its probability in the target model is greater than this threshold. | 1.0 |
| `--speculative-accept-threshold-acc` | The accept probability of a draft token is raised from its target probability p to min(1, p / threshold_acc). | 1.0 |
| `--speculative-token-map` | The path of the draft model's small vocab table. | None |
| `--speculative-attention-mode` | Attention backend for speculative decoding operations (both target verify and draft extend). Can be one of 'prefill' (default) or 'decode'. | Prefill |
## Expert parallelism