Add a Mooncake installation tutorial for kv pool and update Mooncake installation tutorial (#5069)
### What this PR does / why we need it?
Add a Mooncake installation tutorial for kv pool and update Mooncake
installation tutorial
- vLLM version: v0.12.0
- vLLM main:
ad32e3e19c
Signed-off-by: liziyu <liziyu16@huawei.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
This commit is contained in:
@@ -79,7 +79,8 @@ docker run --rm \
|
||||
|
||||
## Install Mooncake
|
||||
|
||||
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. First, we need to obtain the Mooncake project. Refer to the following command:
|
||||
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.Installation and Compilation Guide: https://github.com/kvcache-ai/Mooncake?tab=readme-ov-file#build-and-use-binaries.
|
||||
First, we need to obtain the Mooncake project. Refer to the following command:
|
||||
|
||||
```shell
|
||||
git clone -b v0.3.7.post2 --depth 1 https://github.com/kvcache-ai/Mooncake.git
|
||||
@@ -114,6 +115,17 @@ make -j
|
||||
make install
|
||||
```
|
||||
|
||||
Set environment variables
|
||||
|
||||
**Note:**
|
||||
|
||||
- Adjust the Python path according to your specific Python installation
|
||||
- Ensure `/usr/local/lib` and `/usr/local/lib64` are in your `LD_LIBRARY_PATH`
|
||||
|
||||
```shell
|
||||
export LD_LIBRARY_PATH=/usr/local/lib64/python3.11/site-packages/mooncake:$LD_LIBRARY_PATH
|
||||
```
|
||||
|
||||
## Prefiller/Decoder Deployment
|
||||
|
||||
We can run the following scripts to launch a server on the prefiller/decoder NPU, respectively.
|
||||
|
||||
Reference in New Issue
Block a user