This PR port optimization in PR #2002 to main and makes it cleaner. - vLLM version: v0.10.0 - vLLM main: afa5b7ca0b --------- Signed-off-by: whx-sjtu <2952154980@qq.com>
afa5b7ca0b