EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Rename sglang.bench_latency to sglang.bench_one_batch (#2118)

Browse Source
This commit is contained in:
Lianmin Zheng
2024-11-21 20:07:48 -08:00
committed by GitHub
parent 8048c28c11
commit dfec7fca06
16 changed files with 521 additions and 599 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/references/supported_models.md
View File

@@ -59,7 +59,7 @@ For interactive debugging, you can compare the outputs of huggingface/transforme
The following two commands should give the same text output and very similar prefill logits.
- Get the reference output by `python3 scripts/playground/reference_hf.py --model [new model]`
- Get the SGLang output by `python3 -m sglang.bench_latency --correct --model [new model]`
- Get the SGLang output by `python3 -m sglang.bench_one_batch --correct --model [new model]`
#### Add the model to the test suite
To make sure the new model is well maintained in the future, it is better to add it to the test suite.