Update Readme (#660)

Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
This commit is contained in:
Ying Sheng
2024-07-19 09:54:01 -07:00
committed by GitHub
parent dc4e4a6acc
commit 51fda1439f
25 changed files with 200 additions and 185 deletions

View File

@@ -1,4 +1,4 @@
## Sampling Parameters of SGLang Runtime
# Sampling Parameters in SGLang Runtime
This doc describes the sampling parameters of the SGLang Runtime.
The `/generate` endpoint accepts the following arguments in the JSON format.
@@ -6,11 +6,11 @@ The `/generate` endpoint accepts the following arguments in the JSON format.
```python
@dataclass
class GenerateReqInput:
# The input prompt
# The input prompt. It can be a single prompt or a batch of prompts.
text: Union[List[str], str]
# The token ids for text; one can either specify text or input_ids
input_ids: Optional[Union[List[List[int]], List[int]]] = None
# The image input
# The image input. It can be a file name.
image_data: Optional[Union[List[str], str]] = None
# The sampling_params
sampling_params: Union[List[Dict], Dict] = None