Initial commit: bi100 tokenizer patch
All checks were successful
Docker Build and Push / docker (push) Successful in 1m14s
All checks were successful
Docker Build and Push / docker (push) Successful in 1m14s
Add fix_tokenizer.py, vllm_wrapper.sh, Dockerfile, and README for automatic tokenizer_config.json repair on Iluvatar BI-100 vLLM images.
This commit is contained in:
17
vllm_wrapper.sh
Normal file
17
vllm_wrapper.sh
Normal file
@@ -0,0 +1,17 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# 只拦截 "serve <model_dir>" 子命令,其他子命令直接透传
|
||||
if [ "$1" = "serve" ] && [ -n "$2" ]; then
|
||||
MODEL_DIR="$2"
|
||||
shift 2
|
||||
|
||||
FIXED_DIR=$(python3 /opt/fix_tokenizer.py "$MODEL_DIR")
|
||||
if [ -n "$FIXED_DIR" ]; then
|
||||
exec /usr/local/corex/lib64/python3/dist-packages/bin/vllm_real serve "$MODEL_DIR" --tokenizer "$FIXED_DIR" "$@"
|
||||
else
|
||||
exec /usr/local/corex/lib64/python3/dist-packages/bin/vllm_real serve "$MODEL_DIR" "$@"
|
||||
fi
|
||||
fi
|
||||
|
||||
exec /usr/local/corex/lib64/python3/dist-packages/bin/vllm_real "$@"
|
||||
Reference in New Issue
Block a user