EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Model: Support Qwen 72B RM model. (#3772)

Browse Source
This commit is contained in:
simveit
2025-02-21 23:38:21 +01:00
committed by GitHub
parent e310722266
commit 20b765a26e
4 changed files with 75 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/references/supported_models.md
View File

@@ -47,7 +47,8 @@
- `python -m sglang.launch_server --model-path Skywork/Skywork-Reward-Gemma-2-27B-v0.2 --is-embedding`
- InternLM2ForRewardModel
- `python -m sglang.launch_server --model-path internlm/internlm2-7b-reward --is-embedding --trust-remote-code`
- Qwen2ForRewardModel
- `python -m sglang.launch_server --model-path Qwen/Qwen2.5-Math-RM-72B --is-embedding --trust-remote-code --tp-size=4`
## How to Support a New Language Model
To support a new model in SGLang, you only need to add a single file under [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models).