[Feature] Adds basic support for image content in OpenAI chat routes (#113)

2024-01-30 23:12:33 +09:00
parent 97aa9b3284
commit 1d0fbe8e43
8 changed files with 220 additions and 11 deletions
--- a/README.md
+++ b/README.md
@@ -357,7 +357,7 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
 - Mistral
 - Mixtral
 - LLaVA
-  - `python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000`
+  - `python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --chat-template vicuna_v1.1 --port 30000`
 - Qwen / Qwen 2
 - AWQ quantization