[Doc][releases/v0.18.0] fix documentation error or non-standard description (#8626)
### What this PR does / why we need it? fix documentation error or non-standard description in releases/v0.18.0 branch ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Documentation check. --------- Signed-off-by: linfeng-yuan <1102311262@qq.com>
This commit is contained in:
@@ -105,8 +105,8 @@ vllm serve vllm-ascend/Qwen3-235B-A22B-w8a8 \
|
||||
|
||||
**Notice:**
|
||||
|
||||
- for vllm version below `v0.12.0` use parameter: `--rope_scaling '{"rope_type":"yarn","factor":4,"original_max_position_embeddings":32768}' \`
|
||||
- for vllm version `v0.12.0` use parameter: `--hf-overrides '{"rope_parameters": {"rope_type":"yarn","rope_theta":1000000,"factor":4,"original_max_position_embeddings":32768}}' \`
|
||||
- for vllm version below `v0.12.0` use parameter: `--rope-scaling '{"rope_type":"yarn","factor":4,"original_max_position_embeddings":32768}' \`
|
||||
- for vllm version same as or newer than `v0.12.0` use parameter: `--hf-overrides '{"rope_parameters": {"rope_type":"yarn","rope_theta":1000000,"factor":4,"original_max_position_embeddings":32768}}' \`
|
||||
|
||||
The parameters are explained as follows:
|
||||
|
||||
|
||||
@@ -244,7 +244,7 @@ python load_balance_proxy_server_example.py \
|
||||
--host 192.0.0.1 \
|
||||
--port 8080 \
|
||||
--prefiller-hosts 192.0.0.1 \
|
||||
--prefiller-port 13700 \
|
||||
--prefiller-ports 13700 \
|
||||
--decoder-hosts 192.0.0.1 \
|
||||
--decoder-ports 13701
|
||||
```
|
||||
@@ -252,7 +252,7 @@ python load_balance_proxy_server_example.py \
|
||||
|Parameter | Meaning |
|
||||
| --- | --- |
|
||||
| --port | Port of proxy |
|
||||
| --prefiller-port | All ports of prefill |
|
||||
| --prefiller-ports | All ports of prefill |
|
||||
| --decoder-ports | All ports of decoder |
|
||||
|
||||
## Verification
|
||||
|
||||
Reference in New Issue
Block a user