Add --enable-p2p-check option (#599)

2024-07-06 23:34:10 -07:00
parent 26908d9568
commit 5304b4ef58
4 changed files with 13 additions and 9 deletions
--- a/README.md
+++ b/README.md
@@ -362,7 +362,7 @@ python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port
 ```

 ### Additional Arguments
- Add `--tp 2` to enable tensor parallelism.
+- Add `--tp 2` to enable tensor parallelism. If it indicates `peer access is not supported between these two devices`, add `--enable-p2p-check` option.
 ```
 python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --tp 2
 ```