feature:add head size detect and patch some ops
All checks were successful
Docker Build and Push / docker (push) Successful in 1m5s
All checks were successful
Docker Build and Push / docker (push) Successful in 1m5s
Signed-off-by: Sun Ruoxi <sunruoxi@4paradigm.com>
This commit is contained in:
@@ -34,6 +34,14 @@ else
|
||||
TOKENIZER_ARG=""
|
||||
fi
|
||||
|
||||
# 检查并patch head_size支持(即使失败也不影响启动)
|
||||
echo "[entrypoint] checking model head_size..."
|
||||
python3 /opt/detect_head_size.py || echo "[entrypoint] head_size check failed, but continuing with vllm startup"
|
||||
|
||||
# Patch ixformer ops(即使失败也不影响启动)
|
||||
echo "[entrypoint] patching ixformer ops..."
|
||||
python3 /opt/patch_ops.py || echo "[entrypoint] ixformer ops patch failed, but continuing with vllm startup"
|
||||
|
||||
echo "[entrypoint] starting vllm..."
|
||||
|
||||
exec vllm serve "$MODEL_DIR" $TOKENIZER_ARG "$@"
|
||||
|
||||
Reference in New Issue
Block a user