EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Docs: Fix layout to docs (#3733)

Browse Source
This commit is contained in:
simveit
2025-02-21 20:24:13 +01:00
committed by GitHub
parent 9af0e21ef5
commit 4592afc27d
7 changed files with 11 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/backend/quantization.md
View File

@@ -10,7 +10,7 @@ Online quantization dynamically computes scaling parameters—such as the maximu
## Offline Quantization
To load already quantized models, simply load the model weights and config. **Again, if the model has been quantized offline, there's no need to add "--quantization" argument when starting the engine. The quantization method will be parsed from the downloaded Hugging Face config. For example, DeepSeek V3/R1 models are already in FP8, so do not add redundant parameters.**
To load already quantized models, simply load the model weights and config. **Again, if the model has been quantized offline, there's no need to add `--quantization` argument when starting the engine. The quantization method will be parsed from the downloaded Hugging Face config. For example, DeepSeek V3/R1 models are already in FP8, so do not add redundant parameters.**
```bash
python3 -m sglang.launch_server \