Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a0d9bdf96e |
20
README.md
20
README.md
@@ -57,7 +57,25 @@ docker run -dit --name <container_name> \
|
|||||||
--trust-remote-code --max-model-len 2048 --enforce-eager -tp 1
|
--trust-remote-code --max-model-len 2048 --enforce-eager -tp 1
|
||||||
```
|
```
|
||||||
|
|
||||||
## 构建
|
## 测试
|
||||||
|
|
||||||
|
如果发送请求时出现以下报错:
|
||||||
|
|
||||||
|
```
|
||||||
|
ValueError: default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one.
|
||||||
|
```
|
||||||
|
|
||||||
|
请使用 completion 接口测试,而非 chat completion:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8000/v1/completions \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"model": "llm",
|
||||||
|
"prompt": "Hello, how are you?",
|
||||||
|
"max_tokens": 100
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker build -t vllm-ascend-fix-tokenizer:latest .
|
docker build -t vllm-ascend-fix-tokenizer:latest .
|
||||||
|
|||||||
Reference in New Issue
Block a user