EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Update docs

Browse Source
This commit is contained in:
Ying Sheng
2024-07-19 11:40:06 -07:00
parent 11c8efff73
commit 50a53887be
2 changed files with 3 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -79,7 +79,7 @@ pip install -U --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/
## Backend: SGLang Runtime (SRT) ## Backend: SGLang Runtime (SRT)
The SGLang Runtime (SRT) is an efficient serving engine. The SGLang Runtime (SRT) is an efficient serving engine.
### Launching a Server ### Quick Start
Launch a server Launch a server
``` ```
python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3-8B-Instruct --port 30000 python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3-8B-Instruct --port 30000

3
docs/sampling_params.md
View File

@@ -121,6 +121,7 @@ Download an image
curl -o example_image.png -L https://github.com/sgl-project/sglang/blob/main/test/lang/example_image.png?raw=true curl -o example_image.png -L https://github.com/sgl-project/sglang/blob/main/test/lang/example_image.png?raw=true
``` ```
Send a request
```python ```python
import requests import requests
@@ -139,4 +140,4 @@ print(response.json())
``` ```
The `image_data` can be a file name, a URL, or a base64 encoded string. See also `python/sglang/srt/utils.py:load_image`. The `image_data` can be a file name, a URL, or a base64 encoded string. See also `python/sglang/srt/utils.py:load_image`.
Streaming is supported in a similar manner as [above](#streaming). Streaming is supported in a similar manner as [above](#streaming).