EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Update Readme (#11)

Browse Source
This commit is contained in:
Lianmin Zheng
2024-01-16 02:46:27 -08:00
parent 2ccd9fd8c5
commit fbf42263f1
1 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@@ -36,6 +36,12 @@ pip install -e "python[all]"
The example below shows how to use sglang to answer a mulit-turn question. The example below shows how to use sglang to answer a mulit-turn question.
### Using OpenAI Models ### Using OpenAI Models
Set the OpenAI API Key
```
export OPENAI_API_KEY=sk-xxxxxx
```
Then, answer a multi-turn question.
```python ```python
from sglang import function, system, user, assistant, gen, set_default_backend, OpenAI from sglang import function, system, user, assistant, gen, set_default_backend, OpenAI
@@ -99,7 +105,7 @@ To begin with, import sglang.
import sglang as sgl import sglang as sgl
``` ```
`sglang` provides some simple primitives such as `gen`, `select`, `fork`. `sglang` provides some simple primitives such as `gen`, `select`, `fork`, `image`.
You can implement your prompt flow in a function decorated by `sgl.function`. You can implement your prompt flow in a function decorated by `sgl.function`.
You can then invoke the function with `run` or `run_batch`. You can then invoke the function with `run` or `run_batch`.
The system will manage the state, chat template, and parallelism for you. The system will manage the state, chat template, and parallelism for you.