Turn on flashinfer by default (#578)

2024-07-02 02:25:07 -07:00
parent 95dc093b19
commit 9380f50ff9
5 changed files with 14 additions and 27 deletions
--- a/docs/flashinfer.md
+++ b/docs/flashinfer.md
@@ -1,18 +0,0 @@
-## Flashinfer Mode
-
-[flashinfer](https://github.com/flashinfer-ai/flashinfer) is a kernel library for LLM serving.
-It can be used in SGLang runtime to accelerate attention computation.
-
-### Install flashinfer
-
-See https://docs.flashinfer.ai/installation.html.
-
-### Run a Server With Flashinfer Mode
-
-Add `--enable-flashinfer` argument to enable flashinfer when launching a server.
-
-Example:
-
-```bash
-python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 --enable-flashinfer
-```