diff --git a/docs/references/deepseek.md b/docs/references/deepseek.md index 3aac5e077..2ed666088 100644 --- a/docs/references/deepseek.md +++ b/docs/references/deepseek.md @@ -8,74 +8,46 @@ Special thanks to Meituan's Search & Recommend Platform Team and Baseten's Model SGLang is recognized as one of the top engines for [DeepSeek model inference](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3). To run DeepSeek V3/R1 models, the requirements are as follows: - - - - - - Weight Configurations - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Weight TypeConfiguration
Full precision FP8 (recommended)8 x H200
8 x MI300X
2 x 8 x H100/800/20
Full precision BF162 x 8 x H200
2 x 8 x MI300X
4 x 8 x H100/800/20
4 x 8 x A100/A800
Quantized weights (AWQ)8 x H100/800/20
8 x A100/A800
- - +| Weight Type | Configuration | +|------------|-------------------| +| **Full precision FP8**
*(recommended)* | 8 x H200 | +| | 8 x MI300X | +| | 2 x 8 x H100/800/20 | +| **Full precision BF16** | 2 x 8 x H200 | +| | 2 x 8 x MI300X | +| | 4 x 8 x H100/800/20 | +| | 4 x 8 x A100/A800 | +| **Quantized weights (AWQ)** | 8 x H100/800/20 | +| | 8 x A100/A800 | + + Detailed commands for reference: