EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

feat: support internlm2 (#636)

Browse Source
This commit is contained in:
zhyncs
2024-07-17 15:40:03 +10:00
committed by GitHub
parent a470e60c97
commit a8552cb18b
3 changed files with 323 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/model_support.md
View File

@@ -12,4 +12,4 @@ To port a model from vLLM to SGLang, you can compare these two files [SGLang LLa
- Add `EntryClass` at the end.
- Test correctness by comparing the final logits and outputs of the two following commands:
- `python3 playground/reference_hf.py --model [new model]`
- `python3 -m sglang.bench_latency --model [new model] --correct --output-len 16`
- `python3 -m sglang.bench_latency --model [new model] --correct --output-len 16 --trust-remote-code`