From d8a98a2cad6dcddaa1e7b7ec21fa8ffca88b08ba Mon Sep 17 00:00:00 2001 From: Stefan He Date: Thu, 27 Feb 2025 00:13:04 -0800 Subject: [PATCH] [Docs] Improve DPSK docs in dark mode (#3914) --- docs/references/deepseek.md | 108 +++++++++++++----------------------- 1 file changed, 40 insertions(+), 68 deletions(-) diff --git a/docs/references/deepseek.md b/docs/references/deepseek.md index 3aac5e077..2ed666088 100644 --- a/docs/references/deepseek.md +++ b/docs/references/deepseek.md @@ -8,74 +8,46 @@ Special thanks to Meituan's Search & Recommend Platform Team and Baseten's Model SGLang is recognized as one of the top engines for [DeepSeek model inference](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3). To run DeepSeek V3/R1 models, the requirements are as follows: - - - - - - Weight Configurations - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Weight TypeConfiguration
Full precision FP8 (recommended)8 x H200
8 x MI300X
2 x 8 x H100/800/20
Full precision BF162 x 8 x H200
2 x 8 x MI300X
4 x 8 x H100/800/20
4 x 8 x A100/A800
Quantized weights (AWQ)8 x H100/800/20
8 x A100/A800
- - +| Weight Type | Configuration | +|------------|-------------------| +| **Full precision FP8**
*(recommended)* | 8 x H200 | +| | 8 x MI300X | +| | 2 x 8 x H100/800/20 | +| **Full precision BF16** | 2 x 8 x H200 | +| | 2 x 8 x MI300X | +| | 4 x 8 x H100/800/20 | +| | 4 x 8 x A100/A800 | +| **Quantized weights (AWQ)** | 8 x H100/800/20 | +| | 8 x A100/A800 | + + Detailed commands for reference: