EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
398 Commits 1 Branch 0 Tags
2009fdb8da54fd1ea2f6ade0ff3268c0917e8d0c
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Yikun Jiang 2009fdb8da [Test] Enable code cov for V1 and enable push trigger (#1164) 
### What this PR does / why we need it?
- Enable code cov for V1
- Enable push triggered job

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
2025-06-21 00:01:05 +08:00
.github
[Test] Enable code cov for V1 and enable push trigger (#1164)
2025-06-21 00:01:05 +08:00
benchmarks
[CI][Benchmark] Add new model and v1 test to perf benchmarks (#1099)
2025-06-12 10:46:41 +08:00
cmake
[core] Support custom ascendc kernels in vllm-ascend (#233)
2025-04-03 14:52:34 +08:00
csrc
add custom ascendc kernel vocabparallelembedding (#796)
2025-06-12 10:44:33 +08:00
docs
[CI] Add codespell check for doc (#1314)
2025-06-20 16:48:14 +08:00
examples
Fix the device error when using ray as vllm-acend backend (#884)
2025-06-16 21:03:16 +08:00
tests
[V1][eagle3] Support eagle3 proposer for v1 (#1032)
2025-06-20 17:19:54 +08:00
tools
add workflow to build and release wheel (#775)
2025-05-26 14:18:26 +08:00
vllm_ascend
Support Pangu Pro MoE model (#1204)
2025-06-20 23:59:59 +08:00
.gitignore
[Misc] version control by setuptools_scm (#21)
2025-02-10 09:36:09 +08:00
.readthedocs.yaml
[Doc] Add sphinx build for vllm-ascend (#55)
2025-02-13 18:44:17 +08:00
CMakeLists.txt
add custom ascendc kernel vocabparallelembedding (#796)
2025-06-12 10:44:33 +08:00
CODE_OF_CONDUCT.md
[Core] Init vllm-ascend (#3)
2025-02-05 10:53:12 +08:00
codecov.yml
[Test] Enable code cov for V1 and enable push trigger (#1164)
2025-06-21 00:01:05 +08:00
collect_env.py
[CI]Add model basic accuracy test(Qwen2.5-0.5B-Instruct) (#460)
2025-04-17 14:59:56 +08:00
DCO
[Core] Init vllm-ascend (#3)
2025-02-05 10:53:12 +08:00
Dockerfile
[DP][V1] Fix rank set in DP scenario & Bump torch-npu version to 2.5.1.post1.dev20250528 (#1235)
2025-06-16 23:09:53 +08:00
Dockerfile.openEuler
[DP][V1] Fix rank set in DP scenario & Bump torch-npu version to 2.5.1.post1.dev20250528 (#1235)
2025-06-16 23:09:53 +08:00
format.sh
[CI] Add unit test framework (#1201)
2025-06-16 18:32:28 +08:00
LICENSE
Initial commit
2025-01-29 02:44:13 -08:00
mypy.ini
Support multistream of shared experts in FusedMoE (#997)
2025-06-11 09:18:38 +08:00
packages.txt
[CI/UT][PD Disaggreate] Initialize PD Disaggreate UT (#889)
2025-05-29 10:17:12 +08:00
pyproject.toml
[DP][V1] Fix rank set in DP scenario & Bump torch-npu version to 2.5.1.post1.dev20250528 (#1235)
2025-06-16 23:09:53 +08:00
pytest.ini
[CI/UT] Ignore vllm/tests/test_vllm_port.py (#887)
2025-05-16 18:52:59 +08:00
README.md
[DP][V1] Fix rank set in DP scenario & Bump torch-npu version to 2.5.1.post1.dev20250528 (#1235)
2025-06-16 23:09:53 +08:00
README.zh.md
[DP][V1] Fix rank set in DP scenario & Bump torch-npu version to 2.5.1.post1.dev20250528 (#1235)
2025-06-16 23:09:53 +08:00
requirements-dev.txt
[Test] Enable code cov for V1 and enable push trigger (#1164)
2025-06-21 00:01:05 +08:00
requirements-lint.txt
[CI] Fix mypy CI (#443)
2025-04-01 09:25:33 +08:00
requirements.txt
[DP][V1] Fix rank set in DP scenario & Bump torch-npu version to 2.5.1.post1.dev20250528 (#1235)
2025-06-16 23:09:53 +08:00
setup.py
[DP][V1] Fix rank set in DP scenario & Bump torch-npu version to 2.5.1.post1.dev20250528 (#1235)
2025-06-16 23:09:53 +08:00

README.md

vllm-ascend

vLLM Ascend Plugin

| About Ascend | Documentation | #sig-ascend | Users Forum | Weekly Meeting |

English | 中文

Latest News 🔥

Overview

vLLM Ascend (vllm-ascend) is a community maintained hardware plugin for running vLLM seamlessly on the Ascend NPU.

It is the recommended approach for supporting the Ascend backend within the vLLM community. It adheres to the principles outlined in the [RFC]: Hardware pluggable, providing a hardware-pluggable interface that decouples the integration of the Ascend NPU with vLLM.

By using vLLM Ascend plugin, popular open-source models, including Transformer-like, Mixture-of-Expert, Embedding, Multi-modal LLMs can run seamlessly on the Ascend NPU.

Prerequisites

  • Hardware: Atlas 800I A2 Inference series, Atlas A2 Training series
  • OS: Linux
  • Software:
    • Python >= 3.9, < 3.12
    • CANN >= 8.1.RC1
    • PyTorch >= 2.5.1, torch-npu >= 2.5.1.post1.dev20250528
    • vLLM (the same version as vllm-ascend)

Getting Started

Please refer to QuickStart and Installation for more details.

Contributing

See CONTRIBUTING for more details, which is a step-by-step guide to help you set up development environment, build and test.

We welcome and value any contributions and collaborations:

Branch

vllm-ascend has main branch and dev branch.

  • main: main branch，corresponds to the vLLM main branch, and is continuously monitored for quality through Ascend CI.
  • vX.Y.Z-dev: development branch, created with part of new releases of vLLM. For example, v0.7.3-dev is the dev branch for vLLM v0.7.3 version.

Below is maintained branches:

Branch Status Note
main Maintained CI commitment for vLLM main branch and vLLM 0.9.x branch
v0.7.1-dev Unmaintained Only doc fixed is allowed
v0.7.3-dev Maintained CI commitment for vLLM 0.7.3 version

Please refer to Versioning policy for more details.

Weekly Meeting

License

Apache License 2.0, as found in the LICENSE file.

Description
XC-LLM: A Specially Optimized LLM Inference Engine for ModelHub XC
Readme Apache-2.0 8.6 MiB
Languages
Python 66.8%
C++ 31.8%
Shell 1%
CMake 0.2%
Dockerfile 0.1%
Other 0.1%