EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Enables force reasoning based on chat template for Qwen3-Thinking (#8369)

Browse Source 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Chang Su <csu272@usc.edu>
This commit is contained in:
Xinyuan Tong
2025-08-06 20:02:47 -07:00
committed by GitHub
parent 6210e2c4f0
commit 3fa3c6cd6a
6 changed files with 88 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/backend/openai_api_completions.ipynb
View File

@@ -101,19 +101,16 @@
"\n",
"**Reasoning Parser Options:**\n",
"- `--reasoning-parser deepseek-r1`: For DeepSeek-R1 family models (R1, R1-0528, R1-Distill)\n",
"- `--reasoning-parser qwen3`: For standard Qwen3 models that support `enable_thinking` parameter\n",
"- `--reasoning-parser qwen3-thinking`: For Qwen3-Thinking models (e.g., Qwen/Qwen3-235B-A22B-Thinking-2507) that always generate thinking content\n",
"- `--reasoning-parser qwen3`: For both standard Qwen3 models that support `enable_thinking` parameter and Qwen3-Thinking models\n",
"- `--reasoning-parser qwen3-thinking`: For Qwen3-Thinking models, force reasoning version of qwen3 parser\n",
"- `--reasoning-parser kimi`: For Kimi thinking models\n",
"\n",
"Here's an example demonstrating how to enable thinking and retrieve the reasoning content separately (using `separate_reasoning: True`):\n",
"\n",
"```python\n",
"# For standard Qwen3 models with enable_thinking support:\n",
"# For Qwen3 models with enable_thinking support:\n",
"# python3 -m sglang.launch_server --model-path QwQ/Qwen3-32B-250415 --reasoning-parser qwen3 ...\n",
"\n",
"# For Qwen3-Thinking models that always think:\n",
"# python3 -m sglang.launch_server --model-path Qwen/Qwen3-235B-A22B-Thinking-2507 --reasoning-parser qwen3-thinking ...\n",
"\n",
"from openai import OpenAI\n",
"\n",
"# Modify OpenAI's API key and API base to use SGLang's API server.\n",
@@ -132,7 +129,7 @@
" model=model,\n",
" messages=messages,\n",
" extra_body={\n",
" \"chat_template_kwargs\": {\"enable_thinking\": True}, # Only for standard Qwen3 models\n",
" \"chat_template_kwargs\": {\"enable_thinking\": True},\n",
" \"separate_reasoning\": True\n",
" }\n",
")\n",
@@ -158,7 +155,7 @@
"\n",
"Setting `\"enable_thinking\": False` (or omitting it) will result in `reasoning_content` being `None`.\n",
"\n",
"**Note for Qwen3-Thinking models:** These models always generate thinking content and do not support the `enable_thinking` parameter. When using `--reasoning-parser qwen3-thinking`, the model will always produce reasoning content regardless of the `enable_thinking` setting.\n",
"**Note for Qwen3-Thinking models:** These models always generate thinking content and do not support the `enable_thinking` parameter. Use `--reasoning-parser qwen3-thinking` or `--reasoning-parser qwen3` to parse the thinking content.\n",
"\n",
"Here is an example of a detailed chat completion request using standard OpenAI parameters:"
]