dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[CRITICAL/deploy] fix 3 deployment gaps found from docker crash log

Browse Source 
1. paged_attention_v2_pytorch.py was missing from container
   - _custom_ops.py imports it but Dockerfile only COPYs qwen3_6_scripts/
   - Now: copied into qwen3_6_scripts/ + patch_ops deploys to both $V/ and /workspace/

2. prefix_prefill.py was not deployed by patch_ops.sh
   - xformers.py may try to import context_attention_fwd from it
   - Now: patch_ops copies it to $V/attention/ops/

3. _custom_ops.py paged_attention_v2 import path hardened
   - Try 3 locations: vllm package, /workspace/, repo root
   - Prevents ImportError in container where file locations differ

CCCL source read: cub/block/block_exchange.cuh (blocked↔striped data rearrangement)
→ identified missing file deployment as analogous to incorrect data layout mapping
This commit is contained in:
dylanyunlon
2026-08-06 07:01:14 +00:00
parent b075b015b1
commit 5ba9c1e731
3 changed files with 368 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
qwen3_6_scripts/patch_ops.sh
View File

@@ -42,6 +42,14 @@ echo "[patch_ops] _custom_ops.py → / (SMEM 32KB→48KB fix)"
cp ./paged_attn.py $V/attention/ops/paged_attn.py
echo "[patch_ops] paged_attn.py → attention/ops/"
# --- prefix_prefill.py: Triton-free prefix attention -------------------------
# On BI-V100, Triton is not installed. This file provides the context_attention_fwd
# function that paged_attn.py imports. Even though our paged_attn.py comments out
# the Triton import, the base xformers.py may still try to import it.
# Deploy it so the import doesn't crash — the function itself won't be called.
cp ./prefix_prefill.py $V/attention/ops/prefix_prefill.py
echo "[patch_ops] prefix_prefill.py → attention/ops/"
# --- model_runner.py: prefix_cache_hit fix -----------------------------------
# Bug: Case 1 (prefix_cache_len <= context_len) leaves prefix_cache_hit=True,
# causing undersized block_tables in chunked prefill chunk 2+.
@@ -99,6 +107,14 @@ cp ./qwen3_5.py $V/model_executor/models/qwen3_5.py
cp ./registry.py $V/model_executor/models/registry.py
echo "[patch_ops] qwen3_5.py + registry.py deployed"
# --- paged_attention_v2_pytorch.py: PyTorch V2 attention fallback ------------
# _custom_ops.py imports this from the parent of its own directory.
# In docker, vllm lives at $V/, so we place it one level up AND next to _custom_ops.
# Belt-and-suspenders: also copy to /workspace/ where _custom_ops.py's _repo_root points.
cp ./paged_attention_v2_pytorch.py $V/paged_attention_v2_pytorch.py
cp ./paged_attention_v2_pytorch.py /workspace/paged_attention_v2_pytorch.py
echo "[patch_ops] paged_attention_v2_pytorch.py → $V/ + /workspace/"
# --- sequence.py: fix completion_tokens inflation ----------------------------
cp ./sequence.py $V/sequence.py
echo "[patch_ops] sequence.py → /"