[Docs]Add instruction for manually stopping nsys profiler (#3795)

Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
This commit is contained in:
Baizhou Zhang
2025-02-23 13:21:48 -08:00
committed by GitHub
parent 45360b2fa9
commit 4d2a88bdff

View File

@@ -71,14 +71,29 @@ apt install nsight-systems-cli
2. To profile a server, e.g.
```bash
# server
# set the delay and duration times according to needs
# launch the server, set the delay and duration times according to needs
# after the duration time has been used up, server will be killed by nsys
nsys profile --trace-fork-before-exec=true --cuda-graph-trace=node -o sglang.out --delay 60 --duration 70 python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --disable-radix-cache
# client
python3 -m sglang.bench_serving --backend sglang --num-prompts 1000 --dataset-name random --random-input 1024 --random-output 512
```
In practice, we recommend users to set `--duration` argument to a large value. Whenever user wants the server to stop profiling. Firstly run:
```bash
nsys sessions list
```
to get the session id in the form of `profile-XXXXX`, then run:
```bash
nsys stop --session=profile-XXXXX
```
to manually kill the profiler and generate `nsys-rep` files instantly.
3. Use NVTX to annotate code regions, e.g. to see their execution time.
```bash