ref(upstream): FULL TREE — Deep-Spark xllm (1470) + ds_vllm csrc/models (703)

Replaces cherry-picked upstream_ref with complete source trees.

xllm/ — Iluvatar official C++ inference engine (15MB, 1470 files)
  Complete: kernels → layers → models → runtime → scheduler → api
  Excluded: .git, binary images, third_party submodule checkouts

ds_vllm/ — Iluvatar official vllm fork (8MB, 703 files)
  Included: csrc/ (ALL CUDA kernels), fused_moe/, qwen3_5 model, _custom_ops
  Excluded: tests, benchmarks, docs, examples (not needed for reference)

Critical call chains now fully traceable:
  MoE: moe_topk_softmax_kernels.cuh → ixformer.h → fused_moe.cpp → layer
  GDN: qwen3_gated_delta_net_base.cpp → qwen3_5_gated_delta_net.cpp
  Attention: ixformer.h → xllm_paged_attention → attention.cpp
This commit is contained in:
EX Engine
2026-08-10 02:53:54 +00:00
parent 9e4fb3712f
commit 002f9879b2
2179 changed files with 494021 additions and 79 deletions

View File

@@ -0,0 +1,34 @@
# 全局多级KV Cache
## 背景
大型语言模型LLM解码阶段因自回归生成需频繁访问历史KV缓存导致显存带宽成为瓶颈。随着模型规模与上下文窗口扩大如128K Token消耗超40GB显存单卡显存压力剧增。现有方案如vLLM在长上下文场景下存在明显局限预填充耗时激增、解码阶段显存带宽争抢严重为满足SLOTTFT<2s, TBT<100ms常需过量预留资源致使GPU利用率不足40%且难以利用跨服务器资源为此我们提出分布式全局多级KV缓存管理系统采用存算一体架构以突破单机资源限制
## 功能介绍
xLLM 全局KV Cache功能主要通过以下三个模块实现
- etcd: 集群服务注册负载信息同步及全局缓存状态管理
- xLLM Service: 调度请求和管理所有计算实例
- xLLM: 请求计算实例
整体架构图如下
![xLLM 全局多级KV Cache](../../assets/globalkvcache_architecture.png)
## 功能使用示例
### 使用准备
#### 安装相关依赖
- **xLLM**: 参见[快速开始](../getting_started/quick_start.md)
- **xLLM Service**: 参见[PD分离部署](../getting_started/disagg_pd.md)
### 使用方式
1. etcd启动配置
```bash
./etcd --listen-peer-urls=http://0.0.0.0:10999 --listen-client-urls=http://0.0.0.0:10998
```
2. xLLM Service启动配置
```bash
./xllm_master_serving --etcd_addr="127.0.0.1:10998" --http_server_port 28888 --rpc_server_port 28889 --tokenizer_path=/path/to/tokenizer_config_dir/
```
3. xLLM启动添加上下面的 gflag 参数即可
```bash
--enable_service_routing=true
--enable_cache_upload=true
# PD分离暂时不支持全局KVCache管理
--enable_disagg_pd=false
```