EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Update readme and workflow (#1716)

Browse Source
This commit is contained in:
Lianmin Zheng
2024-10-19 12:58:55 -07:00
parent 087257ea03
commit b6cd903604
3 changed files with 6 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.github/workflows/pr-test-amd.yml vendored
View File

@@ -14,7 +14,7 @@ on:
workflow_dispatch: workflow_dispatch:
concurrency: concurrency:
group: pr-test-${{ github.ref }} group: pr-test-amd-${{ github.ref }}
cancel-in-progress: true cancel-in-progress: true
jobs: jobs:
@@ -28,7 +28,8 @@ jobs:
- name: Install dependencies - name: Install dependencies
run: | run: |
pip install --upgrade pip pip install --upgrade pip
pip install -e "python[all]" --no-deps pip install -e "python[runtime_common, test]"
pip install -e "python" --no-deps
git clone https://github.com/merrymercy/human-eval.git git clone https://github.com/merrymercy/human-eval.git
cd human-eval cd human-eval

2
README.md
View File

@@ -18,11 +18,11 @@
- [2024/10] 🔥 The First SGLang Online Meetup ([slides](https://github.com/sgl-project/sgl-learning-materials?tab=readme-ov-file#the-first-sglang-online-meetup)). - [2024/10] 🔥 The First SGLang Online Meetup ([slides](https://github.com/sgl-project/sgl-learning-materials?tab=readme-ov-file#the-first-sglang-online-meetup)).
- [2024/09] SGLang v0.3 Release: 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision ([blog](https://lmsys.org/blog/2024-09-04-sglang-v0-3/)). - [2024/09] SGLang v0.3 Release: 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision ([blog](https://lmsys.org/blog/2024-09-04-sglang-v0-3/)).
- [2024/07] Faster Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM) ([blog](https://lmsys.org/blog/2024-07-25-sglang-llama3/)). - [2024/07] Faster Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM) ([blog](https://lmsys.org/blog/2024-07-25-sglang-llama3/)).
- [2024/02] SGLang enables **3x faster JSON decoding** with compressed finite state machine ([blog](https://lmsys.org/blog/2024-02-05-compressed-fsm/)).
<details> <details>
<summary>More</summary> <summary>More</summary>
- [2024/02] SGLang enables **3x faster JSON decoding** with compressed finite state machine ([blog](https://lmsys.org/blog/2024-02-05-compressed-fsm/)).
- [2024/04] SGLang is used by the official **LLaVA-NeXT (video)** release ([blog](https://llava-vl.github.io/blog/2024-04-30-llava-next-video/)). - [2024/04] SGLang is used by the official **LLaVA-NeXT (video)** release ([blog](https://llava-vl.github.io/blog/2024-04-30-llava-next-video/)).
- [2024/01] SGLang provides up to **5x faster inference** with RadixAttention ([blog](https://lmsys.org/blog/2024-01-17-sglang/)). - [2024/01] SGLang provides up to **5x faster inference** with RadixAttention ([blog](https://lmsys.org/blog/2024-01-17-sglang/)).
- [2024/01] SGLang powers the serving of the official **LLaVA v1.6** release demo ([usage](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#demo)). - [2024/01] SGLang powers the serving of the official **LLaVA v1.6** release demo ([usage](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#demo)).

3
python/sglang/README.md
View File

@@ -4,7 +4,8 @@
- `srt`: The backend engine for running local models. (SRT = SGLang Runtime). - `srt`: The backend engine for running local models. (SRT = SGLang Runtime).
- `test`: The test utilities. - `test`: The test utilities.
- `api.py`: The public APIs. - `api.py`: The public APIs.
- `bench_latency.py`: Benchmark a single static batch. - `bench_latency.py`: Benchmark the latency of running a single static batch.
- `bench_server_latency.py`: Benchmark the latency of serving a single batch with a real server.
- `bench_serving.py`: Benchmark online serving with dynamic requests. - `bench_serving.py`: Benchmark online serving with dynamic requests.
- `global_config.py`: The global configs and constants. - `global_config.py`: The global configs and constants.
- `launch_server.py`: The entry point for launching the local server. - `launch_server.py`: The entry point for launching the local server.