EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Fix formatting in long code blocks (#10528)

Browse Source
This commit is contained in:
Philip Kiely - Baseten
2025-09-16 12:02:05 -07:00
committed by GitHub
parent 0abb41c70d
commit 7f028b07c4
11 changed files with 345 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/references/custom_chat_template.md
View File

@@ -8,7 +8,10 @@ It should just work for most official models such as Llama-2/Llama-3.
If needed, you can also override the chat template when launching the server:
```bash
python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --chat-template llama-2
python -m sglang.launch_server \
--model-path meta-llama/Llama-2-7b-chat-hf \
--port 30000 \
--chat-template llama-2
```
If the chat template you are looking for is missing, you are welcome to contribute it or load it from a file.
@@ -30,7 +33,10 @@ You can load the JSON format, which is defined by `conversation.py`.
```
```bash
python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --chat-template ./my_model_template.json
python -m sglang.launch_server \
--model-path meta-llama/Llama-2-7b-chat-hf \
--port 30000 \
--chat-template ./my_model_template.json
```
## Jinja Format
@@ -38,5 +44,8 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
You can also use the [Jinja template format](https://huggingface.co/docs/transformers/main/en/chat_templating) as defined by Hugging Face Transformers.
```bash
python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --chat-template ./my_model_template.jinja
python -m sglang.launch_server \
--model-path meta-llama/Llama-2-7b-chat-hf \
--port 30000 \
--chat-template ./my_model_template.jinja
```