EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Update readme.md

Browse Source
This commit is contained in:
Lianmin Zheng
2024-01-08 21:20:23 +00:00
parent ead5b39f82
commit 93eeb543ba
1 changed files with 4 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@@ -98,11 +98,13 @@ You can find more examples at [examples/quick_start](examples/quick_start).
### Multi Modality ### Multi Modality
```python ```python
@sgl.function @sgl.function
def multi_turn_question(s, image_file, question): def image_qa(s, image_file, question):
s += sgl.user(sgl.image(image_file) + question) s += sgl.user(sgl.image(image_file) + question)
s += sgl.assistant(sgl.gen("answer_1", max_tokens=256)) s += sgl.assistant(sgl.gen("answer_1", max_tokens=256))
``` ```
### Constrained decoding
### Batching ### Batching
### Streaming ### Streaming
@@ -142,12 +144,12 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
- Mistral - Mistral
- Mixtral - Mixtral
- LLaVA - LLaVA
- `python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000`
## Benchmark And Performance ## Benchmark And Performance
## Roadmap ## Roadmap
- [ ] Function call - [ ] Function call
- [ ] Constrained decoding
- [ ] Quantization - [ ] Quantization
- [ ] S-LoRA - [ ] S-LoRA
- [ ] More models - [ ] More models