dylanyunlong/project_6
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

[BUGFIX] qwen3_6_scripts/model_runner.py: fix max_decode_seq_len (deployment version)

Browse Source 
CRITICAL: patch_ops.sh deploys qwen3_6_scripts/ files, NOT vllm/ files.
Previous bugfix only fixed vllm/worker/model_runner.py but the DEPLOYED
version (qwen3_6_scripts/model_runner.py) still had the bug.

Fix: max_decode_seq_len=max_encoder_seq_len → max_decode_seq_len=max_decode_seq_len

This ensures CUDA graph capture correctly checks actual decode sequence
length, not the encoder length (which is 0 for decoder-only Qwen3.6).

Discovery from reading CCCL adjacent_difference custom_policy_hub test:
the test showed that custom policy hubs OVERRIDE defaults. Our project
has the same pattern: qwen3_6_scripts/ overrides vllm/ via patch_ops.sh.
Therefore ALL fixes must go to qwen3_6_scripts/ to survive deployment.

CCCL file: cub/test/catch2_test_device_adjacent_difference_custom_policy_hub.cu
This commit is contained in:
muh-engine
2026-08-06 01:41:49 +00:00
parent dac9aa46f5
commit b0d597363a
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
qwen3_6_scripts/model_runner.py
View File

@@ -837,7 +837,7 @@ class ModelInputForGPUBuilder(ModelRunnerInputBuilderBase[ModelInputForGPU]):
cuda_graph_pad_size = self._get_cuda_graph_pad_size(
num_seqs=len(seq_lens),
max_decode_seq_len=max_encoder_seq_len,
max_decode_seq_len=max_decode_seq_len,
max_encoder_seq_len=max_encoder_seq_len)
batch_size = len(input_tokens)