[v0.11.0-dev][bugfix] fix valueError in static_forward_context when prefix is empty (#3929)
### What this PR does / why we need it? This PR temporarily bypasses the scenario where some models in vLLM trigger a `ValueError` during the process of storing values in `static_forward_context` when no `prefix` is specified for the linear layers, which is a bug in some models in vLLM. The official fix will be addressed by submitting a PR to the vLLM community that specifies a prefix for the linear layers in each model. - vLLM version: v0.11.0 - vLLM main: https://github.com/vllm-project/vllm/commit/83f478bb19489b41e9d208b47b4bb5a95ac171ac ### How was this patch tested? CI passed with new added/existing test. Signed-off-by: rjg-lyh <1318825571@qq.com>
This commit is contained in:
1 parent
9f7de45b75
commit
3d81ea03ed
1 file changed
+2
@@ -237,7 +237,9 @@ class AscendRowParallelLinear(RowParallelLinear):
|
||||
):
|
||||
compilation_config = get_current_vllm_config().compilation_config
|
||||
# TODO(shaopeng-666): Remove the visual check after the mm model reconstruction is complete.
|
||||
# TODO(MengqingCao): Remove the empty string check, after specifying the prefix in linear layers of some models in the vLLM.
|
||||
if prefix in compilation_config.static_forward_context and \
|
||||
prefix != "" and \
|
||||
"visual" not in prefix:
|
||||
raise ValueError(f"Duplicate layer name: {prefix}")
|
||||
compilation_config.static_forward_context[prefix] = self
|
||||
|
||||
Reference in new issue
Block a user