EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Fix logprob_start_len for multi modal models (#2597)

Browse Source 
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
This commit is contained in:
Lianmin Zheng
2024-12-26 06:27:45 -08:00
committed by GitHub
parent 637de9e8ce
commit 773951548d
4 changed files with 10 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/references/sampling_params.md
View File

@@ -1,8 +1,7 @@
# Sampling Parameters in SGLang Runtime
This doc describes the sampling parameters of the SGLang Runtime.
It is the low-level endpoint of the runtime.
If you want a high-level endpoint that can automatically handle chat templates, consider using the [OpenAI Compatible API
](https://github.com/sgl-project/sglang?tab=readme-ov-file#openai-compatible-api).
If you want a high-level endpoint that can automatically handle chat templates, consider using the [OpenAI Compatible API](../backend/openai_api_completions.ipynb).
The `/generate` endpoint accepts the following arguments in the JSON format.