add deepseek-v3 amd docker command (#3495)

This commit is contained in:
Zachary Streeter
2025-02-11 13:03:08 -06:00
committed by GitHub
parent 67c5de9286
commit 2491cc928d

View File

@@ -98,3 +98,21 @@ drun sglang_image \
```
With your AMD system properly configured and SGLang installed, you can now fully leverage AMD hardware to power SGLangs machine learning capabilities.
## Running DeepSeek-V3
The only difference in running DeepSeek-V3 is when starting the server.
```bash
drun -p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--ipc=host \
--env "HF_TOKEN=<secret>" \
sglang_image \
python3 -m sglang.launch_server \
--model deepseek-ai/DeepSeek-V3 # <- here \
--tp 8 \
--trust-remote-code \
--host 0.0.0.0 \
--port 30000
```