EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[Fix] Add speculative_draft_model_revision to server_args (#5255)

Browse Source 
Signed-off-by: Devashish Lal <devashish@rivosinc.com>
This commit is contained in:
DevashishLal-CB
2025-09-05 04:45:46 -07:00
committed by GitHub
parent df97b31f37
commit 13705dae06
13 changed files with 68 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
benchmark/mtbench/README.md
View File

@@ -18,7 +18,7 @@ python3 bench_sglang.py --num-questions 80
### Benchmark sglang EAGLE
```
python3 -m sglang.launch_server --model meta-llama/Meta-Llama-3-8B-Instruct --speculative-algo EAGLE \
--speculative-draft lmsys/sglang-EAGLE-LLaMA3-Instruct-8B --speculative-num-steps 5 \
--speculative-draft-model-path lmsys/sglang-EAGLE-LLaMA3-Instruct-8B --speculative-num-steps 5 \
--speculative-eagle-topk 8 --speculative-num-draft-tokens 64 --dtype float16 --port 30000
```