EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

change file tree (#1859)

Browse Source 
Co-authored-by: Chayenne <zhaochenyang@g.ucla.edu>
This commit is contained in:
Chayenne
2024-10-31 20:10:16 -07:00
committed by GitHub
parent b9fd178f1b
commit 61cf00e112
24 changed files with 1177 additions and 456 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/references/troubleshooting.md Normal file
View File

@@ -0,0 +1,13 @@
# Troubleshooting
This page lists some common errors and tips for fixing them.
## CUDA error: an illegal memory access was encountered
This error may be due to kernel errors or out-of-memory issues.
- If it is a kernel error, it is not easy to fix.
- If it is out-of-memory, sometimes it will report this error instead of "Out-of-memory." In this case, try setting a smaller value for `--mem-fraction-static`. The default value of `--mem-fraction-static` is around 0.8 - 0.9.
## The server hangs
If the server hangs, try disabling some optimizations when launching the server.
- Add `--disable-cuda-graph`.
- Add `--sampling-backend pytorch`.