From 50a53887bee1d6c565532c40f9bfc3c12b86040e Mon Sep 17 00:00:00 2001 From: Ying Sheng Date: Fri, 19 Jul 2024 11:40:06 -0700 Subject: [PATCH] Update docs --- README.md | 2 +- docs/sampling_params.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3e0cdffc3..e416bafaf 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,7 @@ pip install -U --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/ ## Backend: SGLang Runtime (SRT) The SGLang Runtime (SRT) is an efficient serving engine. -### Launching a Server +### Quick Start Launch a server ``` python -m sglang.launch_server --model-path meta-llama/Meta-Llama-3-8B-Instruct --port 30000 diff --git a/docs/sampling_params.md b/docs/sampling_params.md index e9b3140e3..979815ffb 100644 --- a/docs/sampling_params.md +++ b/docs/sampling_params.md @@ -121,6 +121,7 @@ Download an image curl -o example_image.png -L https://github.com/sgl-project/sglang/blob/main/test/lang/example_image.png?raw=true ``` +Send a request ```python import requests @@ -139,4 +140,4 @@ print(response.json()) ``` The `image_data` can be a file name, a URL, or a base64 encoded string. See also `python/sglang/srt/utils.py:load_image`. -Streaming is supported in a similar manner as [above](#streaming). \ No newline at end of file +Streaming is supported in a similar manner as [above](#streaming).