From 243ed4da695479c252a21f14398bb2a79639f1c7 Mon Sep 17 00:00:00 2001 From: Yikun Jiang Date: Fri, 21 Mar 2025 09:32:42 +0800 Subject: [PATCH] Add vLLM forum info and update readme (#366) ### What this PR does / why we need it? Add vLLM forum info and update readme ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI passed --------- Signed-off-by: Yikun Jiang Co-authored-by: wangxiyuan --- README.md | 39 +++++++-------------------------------- README.zh.md | 36 ++++++------------------------------ 2 files changed, 13 insertions(+), 62 deletions(-) diff --git a/README.md b/README.md index 66737e7..ded9f91 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ vLLM Ascend Plugin

-| About Ascend | Documentation | Developer Slack (#sig-ascend) | +| About Ascend | Documentation | Devs Slack (#sig-ascend) | Users forum |

@@ -19,14 +19,15 @@ vLLM Ascend Plugin --- *Latest News* 🔥 - +- [2025/03] We hosted the [vLLM Beijing Meetup](https://mp.weixin.qq.com/s/VtxO9WXa5fC-mKqlxNUJUQ) with vLLM team! Please find the meetup slides [here](https://drive.google.com/drive/folders/1Pid6NSFLU43DZRi0EaTcPgXsAzDvbBqF). +- [2025/02] vLLM community officially created [vllm-project/vllm-ascend](https://github.com/vllm-project/vllm-ascend) repo for running vLLM seamlessly on the Ascend NPU. - [2024/12] We are working with the vLLM community to support [[RFC]: Hardware pluggable](https://github.com/vllm-project/vllm/issues/11162). --- ## Overview -vLLM Ascend plugin (`vllm-ascend`) is a backend plugin for running vLLM on the Ascend NPU. +vLLM Ascend (`vllm-ascend`) is a community maintained hardware plugin for running vLLM seamlessly on the Ascend NPU. -This plugin is the recommended approach for supporting the Ascend backend within the vLLM community. It adheres to the principles outlined in the [[RFC]: Hardware pluggable](https://github.com/vllm-project/vllm/issues/11162), providing a hardware-pluggable interface that decouples the integration of the Ascend NPU with vLLM. +It is the recommended approach for supporting the Ascend backend within the vLLM community. It adheres to the principles outlined in the [[RFC]: Hardware pluggable](https://github.com/vllm-project/vllm/issues/11162), providing a hardware-pluggable interface that decouples the integration of the Ascend NPU with vLLM. By using vLLM Ascend plugin, popular open-source models, including Transformer-like, Mixture-of-Expert, Embedding, Multi-modal LLMs can run seamlessly on the Ascend NPU. @@ -43,40 +44,14 @@ Find more about how to setup your environment step by step in [here](docs/source ## Getting Started -> [!NOTE] -> Currently, we are actively collaborating with the vLLM community to support the Ascend backend plugin, once supported you can use one line command `pip install vllm vllm-ascend` to complete installation. - -Installation from source code: -```bash -# Install vllm main branch according: -# https://docs.vllm.ai/en/latest/getting_started/installation/cpu/index.html#build-wheel-from-source -git clone --depth 1 https://github.com/vllm-project/vllm.git -cd vllm -pip install -r requirements/build.txt -VLLM_TARGET_DEVICE=empty pip install . - -# Install vllm-ascend main branch -git clone https://github.com/vllm-project/vllm-ascend.git -cd vllm-ascend -pip install -e . -``` - -Run the following command to start the vLLM server with the [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) model: - -```bash -# export VLLM_USE_MODELSCOPE=true to speed up download -vllm serve Qwen/Qwen2.5-0.5B-Instruct -curl http://localhost:8000/v1/models -``` - Please refer to [QuickStart](https://vllm-ascend.readthedocs.io/en/latest/quick_start.html) and [Installation](https://vllm-ascend.readthedocs.io/en/latest/installation.html) for more details. ## Contributing See [CONTRIBUTING](docs/source/developer_guide/contributing.md) for more details, which is a step-by-step guide to help you set up development environment, build and test. We welcome and value any contributions and collaborations: -- Please feel free comments [here](https://github.com/vllm-project/vllm-ascend/issues/19) about your usage of vLLM Ascend Plugin. -- Please let us know if you encounter a bug by [filing an issue](https://github.com/vllm-project/vllm-ascend/issues). +- Please let us know if you encounter a bug by [filing an issue](https://github.com/vllm-project/vllm-ascend/issues) +- Please use [User forum] for usage questions and help. ## Branch diff --git a/README.zh.md b/README.zh.md index 2f3a5d8..6ece548 100644 --- a/README.zh.md +++ b/README.zh.md @@ -10,7 +10,7 @@ vLLM Ascend Plugin

-| 关于昇腾 | 官方文档 | 开发者 Slack (#sig-ascend) | +| 关于昇腾 | 官方文档 | 开发者 Slack (#sig-ascend) | 用户论坛 |

@@ -20,11 +20,13 @@ vLLM Ascend Plugin --- *最新消息* 🔥 +- [2025/03] 我们和vLLM团队举办了[vLLM Beijing Meetup](https://mp.weixin.qq.com/s/CGDuMoB301Uytnrkc2oyjg)! 你可以在[这里](https://drive.google.com/drive/folders/1Pid6NSFLU43DZRi0EaTcPgXsAzDvbBqF)找到演讲材料. +- [2025/02] vLLM社区正式创建了[vllm-project/vllm-ascend](https://github.com/vllm-project/vllm-ascend)仓库，让vLLM可以无缝运行在Ascend NPU。 - [2024/12] 我们正在与 vLLM 社区合作，以支持 [[RFC]: Hardware pluggable](https://github.com/vllm-project/vllm/issues/11162). --- ## 总览 -vLLM 昇腾插件 (`vllm-ascend`) 是一个让vLLM在Ascend NPU无缝运行的后端插件。 +vLLM 昇腾插件 (`vllm-ascend`) 是一个由社区维护的让vLLM在Ascend NPU无缝运行的后端插件。 此插件是 vLLM 社区中支持昇腾后端的推荐方式。它遵循[[RFC]: Hardware pluggable](https://github.com/vllm-project/vllm/issues/11162)所述原则：通过解耦的方式提供了vLLM对Ascend NPU的支持。 @@ -43,32 +45,6 @@ vLLM 昇腾插件 (`vllm-ascend`) 是一个让vLLM在Ascend NPU无缝运行的 ## 开始使用 -> [!NOTE] -> 目前，我们正在积极与 vLLM 社区合作以支持 Ascend 后端插件，一旦支持，您可以使用一行命令: `pip install vllm vllm-ascend` 来完成安装。 - -通过源码安装: -```bash -# 安装vllm main 分支参考文档: -# https://docs.vllm.ai/en/latest/getting_started/installation/cpu/index.html#build-wheel-from-source -git clone --depth 1 https://github.com/vllm-project/vllm.git -cd vllm -pip install -r requirements/build.txt -VLLM_TARGET_DEVICE=empty pip install . - -# 安装vllm-ascend main 分支 -git clone https://github.com/vllm-project/vllm-ascend.git -cd vllm-ascend -pip install -e . -``` - -运行如下命令使用 [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) 模型启动服务: - -```bash -# 设置环境变量 VLLM_USE_MODELSCOPE=true 加速下载 -vllm serve Qwen/Qwen2.5-0.5B-Instruct -curl http://localhost:8000/v1/models -``` - 请查看[快速开始](https://vllm-ascend.readthedocs.io/en/latest/quick_start.html)和[安装指南](https://vllm-ascend.readthedocs.io/en/latest/installation.html)了解更多. ## 分支 @@ -92,8 +68,8 @@ vllm-ascend有主干分支和开发分支。 有关更多详细信息，请参阅 [CONTRIBUTING](docs/source/developer_guide/contributing.zh.md)，可以更详细的帮助您部署开发环境、构建和测试。 我们欢迎并重视任何形式的贡献与合作： -- 您可以在[这里](https://github.com/vllm-project/vllm-ascend/issues/19)反馈您的使用体验。 -- 请通过[提交问题](https://github.com/vllm-project/vllm-ascend/issues)来告知我们您遇到的任何错误。 +- 请通过[Issue](https://github.com/vllm-project/vllm-ascend/issues)来告知我们您遇到的任何Bug。 +- 请通过[用户论坛](https://github.com/vllm-project/vllm-ascend/issues)来交流使用问题和寻求帮助。 ## 许可证