From 90286d857638f539ea47df1e0604fcc4f341234d Mon Sep 17 00:00:00 2001 From: Ying Sheng Date: Thu, 1 Aug 2024 00:05:26 -0700 Subject: [PATCH] Add troubleshooting doc (#856) --- docs/en/troubleshooting.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 docs/en/troubleshooting.md diff --git a/docs/en/troubleshooting.md b/docs/en/troubleshooting.md new file mode 100644 index 000000000..c6c016fd1 --- /dev/null +++ b/docs/en/troubleshooting.md @@ -0,0 +1,13 @@ +# Troubleshooting + +This page lists some common errors and tips for fixing them. + +## CUDA error: an illegal memory access was encountered +This error may be due to kernel errors or out-of-memory issues. +- If it is a kernel error, it is not easy to fix. +- If it is out-of-memory, sometimes it will report this error instead of "Out-of-memory." In this case, try setting a smaller value for `--mem-fraction-static`. The default value of `--mem-fraction-static` is around 0.8 - 0.9. https://github.com/sgl-project/sglang/blob/1edd4e07d6ad52f4f63e7f6beaa5987c1e1cf621/python/sglang/srt/server_args.py#L92-L102 + +## The server hangs +If the server hangs, try disabling some optimizations when launching the server. +- Add `--disable-cuda-graph`. +- Add `--disable-flashinfer-sampling`.