fix(critical): remove pip install transformers — was breaking corex kernel loading
ROOT CAUSE FOUND from competitor sub168 docker log comparison: Sub168 (competitor, works): - corex_gdn.py:56] Loaded fused CoreX GDN decode operator ✓ - corex_moe.py:339] Using CoreX fused MoE prefill operator ✓ - corex_fa2.py:333] Using CoreX FA2 packed prefill ✓ - NO NaN warnings, NO MoE fallback - max_model_len=256000, gpu_mem=0.95, max_num_seqs=2 (yaml params work) Sub509 (ours, broken): - NaN in prefill GatedDeltaNet layer 0 (frac=0.9998) ✗ - FusedMoE native kernel failed, falling back to PyTorch ✗ - NO corex_gdn/corex_moe/corex_fa2 loading logs at all - max_model_len=100000, gpu_mem=0.9, max_num_seqs=1 (yaml params ignored) The pip install transformers==4.55.3 in patch_ops.sh was the likely cause: it changed dependencies that broke corex kernel loading paths. Without corex_gdn, GatedDeltaNet falls back to Python → NaN. Without corex_moe, MoE falls back to PyTorch → 10x slower. Fix: Remove pip install, use base image's transformers version. Only register qwen3_5 config files without upgrading the package.
This commit is contained in:
@@ -51,14 +51,32 @@ deploy() {
|
||||
|
||||
# ============================================================
|
||||
# 1. Transformers: register Qwen3_5 / Qwen3_5_MoE model types
|
||||
# CRITICAL: Do NOT pip install transformers — it breaks corex
|
||||
# kernel dependencies. Competitor sub168's docker log shows
|
||||
# corex_gdn/corex_moe/corex_fa2 all loaded successfully.
|
||||
# Our sub509 failed to load any corex kernel.
|
||||
# The pip install transformers==4.55.3 likely caused this.
|
||||
# ============================================================
|
||||
pip install transformers==4.55.3 -i https://pypi.tuna.tsinghua.edu.cn/simple 2>/dev/null || \
|
||||
pip install transformers==4.55.3 2>/dev/null || \
|
||||
echo "[patch_ops] WARNING: pip install transformers failed, using pre-installed version"
|
||||
cp -r ./qwen3_5 /usr/local/lib/python3.10/site-packages/transformers/models/
|
||||
cp -r ./qwen3_5_moe /usr/local/lib/python3.10/site-packages/transformers/models/
|
||||
python3 ./patch_transformers_qwen3_5.py
|
||||
echo "[patch_ops] transformers Qwen3_5 models installed"
|
||||
# Use base image transformers — just add config files
|
||||
TRANSFORMERS_MODELS=""
|
||||
for P in /usr/local/lib/python3.10/site-packages/transformers/models \
|
||||
/usr/local/corex/lib/python3/dist-packages/transformers/models \
|
||||
/usr/local/corex/lib64/python3/dist-packages/transformers/models; do
|
||||
if [ -d "$P" ]; then
|
||||
TRANSFORMERS_MODELS="$P"
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -n "$TRANSFORMERS_MODELS" ]; then
|
||||
cp -r ./qwen3_5 "$TRANSFORMERS_MODELS/"
|
||||
cp -r ./qwen3_5_moe "$TRANSFORMERS_MODELS/"
|
||||
python3 ./patch_transformers_qwen3_5.py 2>&1 || \
|
||||
echo "[patch_ops] WARNING: patch_transformers failed (may work at runtime)"
|
||||
echo "[patch_ops] transformers Qwen3_5 configs registered (no pip install)"
|
||||
else
|
||||
echo "[patch_ops] WARNING: transformers/models not found — skipping config registration"
|
||||
fi
|
||||
|
||||
# ============================================================
|
||||
# 2. Model registry: ensure qwen3_5 is registered in vllm
|
||||
|
||||
@@ -12,7 +12,16 @@ Target: pip-installed transformers at /usr/local/lib/python3.10/site-packages/tr
|
||||
|
||||
import sys
|
||||
|
||||
TRANSFORMERS_ROOT = "/usr/local/lib/python3.10/site-packages/transformers"
|
||||
TRANSFORMERS_ROOT = None
|
||||
for _p in ["/usr/local/lib/python3.10/site-packages/transformers",
|
||||
"/usr/local/corex/lib/python3/dist-packages/transformers",
|
||||
"/usr/local/corex/lib64/python3/dist-packages/transformers"]:
|
||||
import os
|
||||
if os.path.isdir(_p):
|
||||
TRANSFORMERS_ROOT = _p
|
||||
break
|
||||
if TRANSFORMERS_ROOT is None:
|
||||
TRANSFORMERS_ROOT = "/usr/local/lib/python3.10/site-packages/transformers"
|
||||
AUTO_CONFIG = f"{TRANSFORMERS_ROOT}/models/auto/configuration_auto.py"
|
||||
MODELS_INIT = f"{TRANSFORMERS_ROOT}/models/__init__.py"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user