EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Add note to add supported model to documentation (#6640)

Browse Source
This commit is contained in:
Brayden Zhong
2025-05-27 01:18:46 -04:00
committed by GitHub
parent 7a0bbe6a64
commit 1aa0fbf416
3 changed files with 14 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/supported_models/generative_models.md
View File

@@ -15,11 +15,13 @@ python3 -m sglang.launch_server \
Below the supported models are summarized in a table. Below the supported models are summarized in a table.
If unsure if a specific architecture is implemented you may look it up via github search for example to search for `Qwen3ForCausalLm` use the expression If you are unsure if a specific architecture is implemented, you can search for it via GitHub. For example, to search for `Qwen3ForCausalLM`, use the expression:
``` ```
repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen3ForCausalLm repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen3ForCausalLM
``` ```
in Github search.
in the GitHub search bar.
| Model Family (Variants) | Example HuggingFace Identifier | Description | | Model Family (Variants) | Example HuggingFace Identifier | Description |
|-------------------------------------|--------------------------------------------------|----------------------------------------------------------------------------------------| |-------------------------------------|--------------------------------------------------|----------------------------------------------------------------------------------------|

6
docs/supported_models/multimodal_language_models.md
View File

@@ -15,11 +15,13 @@ python3 -m sglang.launch_server \
Below the supported models are summarized in a table. Below the supported models are summarized in a table.
If unsure if a specific architecture is implemented you may look it up via Github search. For example to search for `Qwen2_5_VLForConditionalGeneration` use the expression If you are unsure if a specific architecture is implemented, you can search for it via GitHub. For example, to search for `Qwen2_5_VLForConditionalGeneration`, use the expression:
``` ```
repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen2_5_VLForConditionalGeneration repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen2_5_VLForConditionalGeneration
``` ```
in Github search.
in the GitHub search bar.
| Model Family (Variants) | Example HuggingFace Identifier | Chat Template | Description | | Model Family (Variants) | Example HuggingFace Identifier | Chat Template | Description |

8
docs/supported_models/support_new_models.md
View File

@@ -1,9 +1,9 @@
# How to Support New Models # How to Support New Models
This document explains how to add support for new language models and multimodal large language models (mllms) in This document explains how to add support for new language models and multimodal large language models (MLLMs) in
SGLang. It also covers how to test new models and register external implementations. SGLang. It also covers how to test new models and register external implementations.
## How to Support a new Language Model ## How to Support a New Language Model
To support a new model in SGLang, you only need to add a single file under To support a new model in SGLang, you only need to add a single file under
the [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models). You can learn the [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models). You can learn
@@ -11,7 +11,7 @@ from existing model implementations and create a new file for your model. For mo
similar model to start with (e.g., starting from Llama). Also refer how similar model to start with (e.g., starting from Llama). Also refer how
to [port a Model from vLLM to SGLang](#port-a-model-from-vllm-to-sglang) to [port a Model from vLLM to SGLang](#port-a-model-from-vllm-to-sglang)
## How to Support a new Multimodal Large Language Model ## How to Support a New Multimodal Large Language Model
To support a new multimodal large language model (MLLM) in SGLang, there are several key components in addition to the To support a new multimodal large language model (MLLM) in SGLang, there are several key components in addition to the
standard LLM support: standard LLM support:
@@ -94,6 +94,8 @@ To port a model from vLLM to SGLang:
- **Add `EntryClass`** at the end. - **Add `EntryClass`** at the end.
- **Ensure that the new implementation uses only SGLang components** and does not rely on any vLLM components. - **Ensure that the new implementation uses only SGLang components** and does not rely on any vLLM components.
Note: make sure you add your new model to the supported models list in the supported models documentation.
## Registering an External Model Implementation ## Registering an External Model Implementation
In addition to the methods above, you can register your new model with the `ModelRegistry` before launching the server. In addition to the methods above, you can register your new model with the `ModelRegistry` before launching the server.