[v0.18.0][BugFix] Fix Qwen3.5 MoE FC1 error under high concurrency when dp>1 (#8395)
### What this PR does / why we need it? GDN Attention uses FIA's query_start_loc (padded), which may cause conv1d update errors under high concurrency when dp > 1, and this PR is to make GDN use its own query_start_loc (unpadded). ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - vLLM version: v0.18.0 Signed-off-by: Wangbingjie <wangbj1207@126.com>
This commit is contained in:
@@ -2201,7 +2201,7 @@ class NPUModelRunner(GPUModelRunner):
|
||||
if self._has_gdn:
|
||||
attn_group = self.attn_groups[kv_cache_gid][0]
|
||||
builder = attn_group.get_metadata_builder(0)
|
||||
if use_spec_decode and isinstance(builder, GDNAttentionMetadataBuilder):
|
||||
if isinstance(builder, GDNAttentionMetadataBuilder):
|
||||
cm.query_start_loc_cpu = self.gdn_query_start_loc.cpu[: num_reqs_padded + 1]
|
||||
cm.query_start_loc = self.gdn_query_start_loc.gpu[: num_reqs_padded + 1]
|
||||
|
||||
@@ -2399,6 +2399,9 @@ class NPUModelRunner(GPUModelRunner):
|
||||
num_reqs_padded = self._pad_query_start_loc_for_fia(
|
||||
num_tokens_padded, num_reqs_padded, num_reqs, cudagraph_runtime_mode, batch_desc.num_reqs
|
||||
)
|
||||
if self._has_gdn:
|
||||
self.gdn_query_start_loc.np[1 : num_reqs_padded + 1] = cum_num_tokens
|
||||
self.gdn_query_start_loc.copy_to_gpu()
|
||||
|
||||
pad_attn = cudagraph_runtime_mode == CUDAGraphMode.FULL
|
||||
attn_metadata, _ = self._build_attention_metadata(
|
||||
|
||||
Reference in New Issue
Block a user