EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

doc: emphasize and notify the usage of chat_template (#3589)

Browse Source 
Co-authored-by: Chayenne <zhaochen20@outlook.com>
This commit is contained in:
Mick
2025-02-15 16:10:32 +08:00
committed by GitHub
parent 7443197a63
commit 7711ac6ed0
6 changed files with 41 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/backend/openai_api_vision.ipynb
View File

@@ -24,7 +24,8 @@
"\n",
"Launch the server in your terminal and wait for it to initialize.\n",
"\n",
"Remember to add `--chat-template llama_3_vision` to specify the vision chat template, otherwise the server only supports text.\n",
"**Remember to add `--chat-template llama_3_vision` to specify the vision chat template, otherwise the server only supports text, and performance degradation may occur.**\n",
"\n",
"We need to specify `--chat-template` for vision language models because the chat template provided in Hugging Face tokenizer only supports text."
]
},

3
docs/backend/server_arguments.md
View File

@@ -56,6 +56,9 @@ Please consult the documentation below to learn more about the parameters you ma
* `json_model_override_args`: Override model config with the provided JSON.
* `delete_ckpt_after_loading`: Delete the model checkpoint after loading the model.
> [!IMPORTANT]
> **Make sure the correct `chat_template` is passed, or performance degradation may occur.**
## Serving: HTTP & API
### HTTP Server configuration