This commit is contained in:
23
README.md
23
README.md
@@ -6,10 +6,10 @@
|
||||
|
||||
部分模型的 `tokenizer_config.json` 存在以下问题,导致 vLLM 0.11.0 服务启动失败:
|
||||
|
||||
| 错误 | 原因 | 数量 |
|
||||
|---|---|---|
|
||||
| `ValueError: Tokenizer class TokenizersBackend does not exist` | `tokenizer_class` 不是 transformers 合法类名,环境中缺少 `tokenizers` 包 | 23 |
|
||||
| `AttributeError: 'list' object has no attribute 'keys'` | `extra_special_tokens` 为 list 格式,transformers 要求 dict | 4 |
|
||||
| 错误 | 原因 |
|
||||
|---|---|
|
||||
| `ValueError: Tokenizer class TokenizersBackend does not exist` | `tokenizer_class` 不是 transformers 合法类名,环境中缺少 `tokenizers` 包 |
|
||||
| `AttributeError: 'list' object has no attribute 'keys'` | `extra_special_tokens` 为 list 格式,transformers 要求 dict |
|
||||
|
||||
## 修复方式
|
||||
|
||||
@@ -35,13 +35,18 @@ harbor.4pd.io/modelhubxc/enginex/xc-llm-biren166m:26.01
|
||||
|
||||
示例:
|
||||
```bash
|
||||
docker run -dit --name <container_name> \
|
||||
-p 38079:8000 \
|
||||
--privileged=true \
|
||||
docker run -dit --name biren_sut_submit_task_<task_id> \
|
||||
--ipc=host \
|
||||
-v /dev:/dev \
|
||||
-v /lib/modules:/lib/modules \
|
||||
--privileged \
|
||||
--pid=host \
|
||||
-v /path/to/model:/model \
|
||||
-e SUPA_VISIBLE_DEVICES=0 \
|
||||
--entrypoint /bin/bash <this-image> \
|
||||
-p 35055:8000 \
|
||||
-e MAX_MODEL_LEN=2048 \
|
||||
-e SUPA_VISIBLE_DEVICES=2 \
|
||||
--entrypoint /bin/bash \
|
||||
<this-image> \
|
||||
-ic 'vllm serve /model --port 8000 --served-model-name llm --max-model-len 2048 --gpu-memory-utilization 0.9 --enforce-eager --trust-remote-code -tp 1 --host 0.0.0.0'
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user