From 93eeb543bacf2d7f5f56dd7e7dd4767884026b38 Mon Sep 17 00:00:00 2001 From: Lianmin Zheng Date: Mon, 8 Jan 2024 21:20:23 +0000 Subject: [PATCH] Update readme.md --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 40fe5d5c3..fbfeacc40 100644 --- a/README.md +++ b/README.md @@ -98,11 +98,13 @@ You can find more examples at [examples/quick_start](examples/quick_start). ### Multi Modality ```python @sgl.function -def multi_turn_question(s, image_file, question): +def image_qa(s, image_file, question): s += sgl.user(sgl.image(image_file) + question) s += sgl.assistant(sgl.gen("answer_1", max_tokens=256)) ``` +### Constrained decoding + ### Batching ### Streaming @@ -142,12 +144,12 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port - Mistral - Mixtral - LLaVA + - `python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000` ## Benchmark And Performance ## Roadmap - [ ] Function call -- [ ] Constrained decoding - [ ] Quantization - [ ] S-LoRA - [ ] More models