From 1aa0fbf416a55f09a0b2e48bedcab572e8d7f877 Mon Sep 17 00:00:00 2001 From: Brayden Zhong Date: Tue, 27 May 2025 01:18:46 -0400 Subject: [PATCH] Add note to add supported model to documentation (#6640) --- docs/supported_models/generative_models.md | 8 +++++--- docs/supported_models/multimodal_language_models.md | 6 ++++-- docs/supported_models/support_new_models.md | 8 +++++--- 3 files changed, 14 insertions(+), 8 deletions(-) diff --git a/docs/supported_models/generative_models.md b/docs/supported_models/generative_models.md index e7147ad8c..3ffd6a883 100644 --- a/docs/supported_models/generative_models.md +++ b/docs/supported_models/generative_models.md @@ -15,11 +15,13 @@ python3 -m sglang.launch_server \ Below the supported models are summarized in a table. -If unsure if a specific architecture is implemented you may look it up via github search for example to search for `Qwen3ForCausalLm` use the expression +If you are unsure if a specific architecture is implemented, you can search for it via GitHub. For example, to search for `Qwen3ForCausalLM`, use the expression: + ``` -repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen3ForCausalLm +repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen3ForCausalLM ``` -in Github search. + +in the GitHub search bar. | Model Family (Variants) | Example HuggingFace Identifier | Description | |-------------------------------------|--------------------------------------------------|----------------------------------------------------------------------------------------| diff --git a/docs/supported_models/multimodal_language_models.md b/docs/supported_models/multimodal_language_models.md index 9e6ddaece..50e0e311a 100644 --- a/docs/supported_models/multimodal_language_models.md +++ b/docs/supported_models/multimodal_language_models.md @@ -15,11 +15,13 @@ python3 -m sglang.launch_server \ Below the supported models are summarized in a table. -If unsure if a specific architecture is implemented you may look it up via Github search. For example to search for `Qwen2_5_VLForConditionalGeneration` use the expression +If you are unsure if a specific architecture is implemented, you can search for it via GitHub. For example, to search for `Qwen2_5_VLForConditionalGeneration`, use the expression: + ``` repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen2_5_VLForConditionalGeneration ``` -in Github search. + +in the GitHub search bar. | Model Family (Variants) | Example HuggingFace Identifier | Chat Template | Description | diff --git a/docs/supported_models/support_new_models.md b/docs/supported_models/support_new_models.md index d7610d600..2223254d9 100644 --- a/docs/supported_models/support_new_models.md +++ b/docs/supported_models/support_new_models.md @@ -1,9 +1,9 @@ # How to Support New Models -This document explains how to add support for new language models and multimodal large language models (mllms) in +This document explains how to add support for new language models and multimodal large language models (MLLMs) in SGLang. It also covers how to test new models and register external implementations. -## How to Support a new Language Model +## How to Support a New Language Model To support a new model in SGLang, you only need to add a single file under the [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models). You can learn @@ -11,7 +11,7 @@ from existing model implementations and create a new file for your model. For mo similar model to start with (e.g., starting from Llama). Also refer how to [port a Model from vLLM to SGLang](#port-a-model-from-vllm-to-sglang) -## How to Support a new Multimodal Large Language Model +## How to Support a New Multimodal Large Language Model To support a new multimodal large language model (MLLM) in SGLang, there are several key components in addition to the standard LLM support: @@ -94,6 +94,8 @@ To port a model from vLLM to SGLang: - **Add `EntryClass`** at the end. - **Ensure that the new implementation uses only SGLang components** and does not rely on any vLLM components. +Note: make sure you add your new model to the supported models list in the supported models documentation. + ## Registering an External Model Implementation In addition to the methods above, you can register your new model with the `ModelRegistry` before launching the server.