Suppport qwen model and solve some problems (#75)

2024-01-23 12:14:51 +08:00
parent e08bca2840
commit 63e97e5e4c
7 changed files with 274 additions and 4 deletions
--- a/README.md
+++ b/README.md
@@ -316,6 +316,7 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
 - Mixtral
 - LLaVA
  - `python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000`
+- Qwen
 - AWQ quantization

 ## Benchmark And Performance