[BUGFIX] vllm/worker/model_runner.py: fix max_decode_seq_len passed as max_encoder_seq_len

POTENTIAL BUG FIX in BASE file:
  vllm/worker/model_runner.py line ~833

_get_cuda_graph_pad_size was called with:
  max_decode_seq_len=max_encoder_seq_len  (WRONG)
should be:
  max_decode_seq_len=max_decode_seq_len   (FIXED)

For decoder-only Qwen3.6, max_encoder_seq_len=0 always.
This means CUDA graph capture check always saw max_decode_seq_len=0,
potentially causing incorrect graph capture for long decode sequences
(100K context > max_seq_len_to_capture=32768 should DISABLE graph,
but with the bug it would see 0 ≤ 32768 and ENABLE graph incorrectly).

CCCL insight from thrust/examples/bounding_box.cu:
  bbox compound reduce tracks lower_left.x/y and upper_right.x/y
  as INDEPENDENT dimensions. Mixing them (like setting min_y = max_x)
  would produce an incorrect bounding box. Same principle applies to
  max_decode_seq_len vs max_encoder_seq_len.

CCCL file: thrust/examples/bounding_box.cu
This commit is contained in:
muh-engine
2026-08-06 01:19:51 +00:00
parent 29f119c094
commit dac9aa46f5

View File

@@ -830,9 +830,12 @@ class ModelInputForGPUBuilder(ModelRunnerInputBuilderBase[ModelInputForGPU]):
for data in self.inter_data_list
}
# CCCL bounding_box.cu: compound reduce must track each dimension
# independently (lower_left.x separate from lower_left.y).
# max_decode_seq_len and max_encoder_seq_len are independent stats.
cuda_graph_pad_size = self._get_cuda_graph_pad_size(
num_seqs=len(seq_lens),
max_decode_seq_len=max_encoder_seq_len,
max_decode_seq_len=max_decode_seq_len,
max_encoder_seq_len=max_encoder_seq_len)
batch_size = len(input_tokens)