Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a0d9bdf96e | ||
|
|
750992c82d | ||
|
|
500f2bba0d |
@@ -7,7 +7,7 @@ on:
|
||||
|
||||
jobs:
|
||||
docker:
|
||||
runs-on: amd64-ubuntu-24.04
|
||||
runs-on: arm64-ubuntu-24.04
|
||||
|
||||
steps:
|
||||
- name: Clone repository
|
||||
|
||||
28
README.md
28
README.md
@@ -6,10 +6,10 @@
|
||||
|
||||
部分模型的 `tokenizer_config.json` 存在以下问题,导致 vLLM 服务启动失败(错误码 `TOKENIZER_FAILED`):
|
||||
|
||||
| 错误 | 原因 | 受影响模型数 |
|
||||
|---|---|---|
|
||||
| `ValueError: Tokenizer class TokenizersBackend does not exist` | `tokenizer_class` 不是 transformers 合法类名 | 10 |
|
||||
| `AttributeError: 'list' object has no attribute 'keys'` | `extra_special_tokens` 为 list 格式,transformers 要求 dict | 15 |
|
||||
| 错误 | 原因 |
|
||||
|---|---|
|
||||
| `ValueError: Tokenizer class TokenizersBackend does not exist` | `tokenizer_class` 不是 transformers 合法类名 |
|
||||
| `AttributeError: 'list' object has no attribute 'keys'` | `extra_special_tokens` 为 list 格式,transformers 要求 dict |
|
||||
|
||||
详见 `tokenizer_error_classification_summary.md`。
|
||||
|
||||
@@ -57,7 +57,25 @@ docker run -dit --name <container_name> \
|
||||
--trust-remote-code --max-model-len 2048 --enforce-eager -tp 1
|
||||
```
|
||||
|
||||
## 构建
|
||||
## 测试
|
||||
|
||||
如果发送请求时出现以下报错:
|
||||
|
||||
```
|
||||
ValueError: default chat template is no longer allowed, so you must provide a chat template if the tokenizer does not define one.
|
||||
```
|
||||
|
||||
请使用 completion 接口测试,而非 chat completion:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "llm",
|
||||
"prompt": "Hello, how are you?",
|
||||
"max_tokens": 100
|
||||
}'
|
||||
```
|
||||
|
||||
```bash
|
||||
docker build -t vllm-ascend-fix-tokenizer:latest .
|
||||
|
||||
Reference in New Issue
Block a user