EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[Docs] Update runtime/engine/readme.md (#5737)

Browse Source 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
This commit is contained in:
Michael Yao
2025-04-26 07:39:29 +08:00
committed by GitHub
parent 18ce468d56
commit 269c457e05
1 changed files with 18 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
examples/runtime/engine/readme.md
View File

@@ -1,12 +1,12 @@
# SGLang Engine # SGLang Engine
## Introduction SGLang provides a direct inference engine without the need for an HTTP server. There are generally these use cases:
SGLang provides a direct inference engine without the need for an HTTP server. There are generally two use cases:
1. **Offline Batch Inference** - [Offline Batch Inference](#offline-batch-inference)
2. **Embedding Generation** - [Embedding Generation](#embedding-generation)
3. **Custom Server on Top of the Engine** - [Custom Server](#custom-server)
4. **Inference Using FastAPI** - [Token-In-Token-Out for RLHF](#token-in-token-out-for-rlhf)
- [Inference Using FastAPI](#inference-using-fastapi)
## Examples ## Examples
@@ -22,7 +22,7 @@ In this example, we launch an SGLang engine and feed a batch of inputs for embed
This example demonstrates how to create a custom server on top of the SGLang Engine. We use [Sanic](https://sanic.dev/en/) as an example. The server supports both non-streaming and streaming endpoints. This example demonstrates how to create a custom server on top of the SGLang Engine. We use [Sanic](https://sanic.dev/en/) as an example. The server supports both non-streaming and streaming endpoints.
#### Steps: #### Steps
1. Install Sanic: 1. Install Sanic: