[v0.11.0-dev][bugfix] fix valueError in static_forward_context when prefix is empty (#3929)
### What this PR does / why we need it?
This PR temporarily bypasses the scenario where some models in vLLM
trigger a `ValueError` during the process of storing values in
`static_forward_context` when no `prefix` is specified for the linear
layers, which is a bug in some models in vLLM. The official fix will be
addressed by submitting a PR to the vLLM community that specifies a
prefix for the linear layers in each model.
- vLLM version: v0.11.0
- vLLM main:
83f478bb19
### How was this patch tested?
CI passed with new added/existing test.
Signed-off-by: rjg-lyh <1318825571@qq.com>
This commit is contained in:
@@ -237,7 +237,9 @@ class AscendRowParallelLinear(RowParallelLinear):
|
||||
):
|
||||
compilation_config = get_current_vllm_config().compilation_config
|
||||
# TODO(shaopeng-666): Remove the visual check after the mm model reconstruction is complete.
|
||||
# TODO(MengqingCao): Remove the empty string check, after specifying the prefix in linear layers of some models in the vLLM.
|
||||
if prefix in compilation_config.static_forward_context and \
|
||||
prefix != "" and \
|
||||
"visual" not in prefix:
|
||||
raise ValueError(f"Duplicate layer name: {prefix}")
|
||||
compilation_config.static_forward_context[prefix] = self
|
||||
|
||||
Reference in New Issue
Block a user