e1ab6d318e
More and more config options are added to additional_config. This PR provide a new AscendConfig to manage these config options by an easier way to make code cleaner and readable. This PR also added the `additional_config` doc for users. Added the test_ascend_config.py to make sure the new AscendConfig works as expect. TODO: Add e2e test with torchair and deepseek once the CI resource is available. Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2.2 KiB
Welcome to vLLM Ascend Plugin
:::{figure} ./logos/vllm-ascend-logo-text-light.png :align: center :alt: vLLM :class: no-scaled-link :width: 70% :::
:::{raw} html
vLLM Ascend Plugin
<script async defer src="https://buttons.github.io/buttons.js"></script> Star Watch Fork
:::vLLM Ascend plugin (vllm-ascend) is a community maintained hardware plugin for running vLLM on the Ascend NPU.
This plugin is the recommended approach for supporting the Ascend backend within the vLLM community. It adheres to the principles outlined in the [RFC]: Hardware pluggable, providing a hardware-pluggable interface that decouples the integration of the Ascend NPU with vLLM.
By using vLLM Ascend plugin, popular open-source models, including Transformer-like, Mixture-of-Expert, Embedding, Multi-modal LLMs can run seamlessly on the Ascend NPU.
Documentation
% How to start using vLLM on Ascend NPU? :::{toctree} :caption: Getting Started :maxdepth: 1 quick_start installation tutorials/index.md faqs :::
% What does vLLM Ascend Plugin support? :::{toctree} :caption: User Guide :maxdepth: 1 user_guide/suppoted_features user_guide/supported_models user_guide/env_vars user_guide/additional_config user_guide/release_notes :::
% How to contribute to the vLLM Ascend project :::{toctree} :caption: Developer Guide :maxdepth: 1 developer_guide/contributing developer_guide/versioning_policy developer_guide/evaluation/index :::
% How to involve vLLM Ascend :::{toctree} :caption: Community :maxdepth: 1 community/governance community/contributors :::
% User stories about vLLM Ascend project :::{toctree} :caption: User Story :maxdepth: 1 user_stories/index :::