diff --git a/docs/references/supported_models.md b/docs/references/supported_models.md index dbf4f71a0..13572e437 100644 --- a/docs/references/supported_models.md +++ b/docs/references/supported_models.md @@ -80,3 +80,30 @@ To port a model from vLLM to SGLang, you can compare these two files [SGLang Lla - Remove `Sample`. - Change `forward()` functions, and add `forward_batch`. - Add `EntryClass` at the end. + +### Registering an external model implementation + +In addition to the methods described above, you can also register your new model with the `ModelRegistry` before launching the server. This approach is useful if you want to integrate your model without needing to modify the source code. + +Here is how you can do it: + +```python +from sglang.srt.models.registry import ModelRegistry +from sglang.srt.server import launch_server + +# for a single model, you can add it to the registry +ModelRegistry.models[model_name] = model_class + +# for multiple models, you can imitate the import_model_classes() function in sglang/srt/models/registry.py +from functools import lru_cache + +@lru_cache() +def import_new_model_classes(): + model_arch_name_to_cls = {} + ... + return model_arch_name_to_cls + +ModelRegistry.models.update(import_new_model_classes()) + +launch_server(server_args) +``` \ No newline at end of file