EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Add OpenAI backend to the CI test (#869)

Browse Source
This commit is contained in:
Ying Sheng
2024-08-01 09:25:24 -07:00
committed by GitHub
parent 9e8d2c7f74
commit 4075677621
16 changed files with 30 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/en/model_support.md
View File

@@ -11,6 +11,6 @@ To port a model from vLLM to SGLang, you can compare these two files [SGLang LLa
- Change `forward()` functions, and add `input_metadata`.
- Add `EntryClass` at the end.
- Test correctness by comparing the final logits and outputs of the two following commands:
- `python3 playground/reference_hf.py --model [new model]`
- `python3 scripts/playground/reference_hf.py --model [new model]`
- `python3 -m sglang.bench_latency --model [new model] --correct --output-len 16 --trust-remote-code`
- Update [Supported Models](https://github.com/sgl-project/sglang/tree/main?tab=readme-ov-file#supported-models) at [README](../README.md).