add deepseek-v3 amd docker command (#3495)

This commit is contained in:
Zachary Streeter
2025-02-11 13:03:08 -06:00
committed by GitHub
parent 67c5de9286
commit 2491cc928d

View File

@@ -98,3 +98,21 @@ drun sglang_image \
``` ```
With your AMD system properly configured and SGLang installed, you can now fully leverage AMD hardware to power SGLangs machine learning capabilities. With your AMD system properly configured and SGLang installed, you can now fully leverage AMD hardware to power SGLangs machine learning capabilities.
## Running DeepSeek-V3
The only difference in running DeepSeek-V3 is when starting the server.
```bash
drun -p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--ipc=host \
--env "HF_TOKEN=<secret>" \
sglang_image \
python3 -m sglang.launch_server \
--model deepseek-ai/DeepSeek-V3 # <- here \
--tp 8 \
--trust-remote-code \
--host 0.0.0.0 \
--port 30000
```