Files

Lianmin Zheng 2369e88209 [minor] Clean up unused imports (#2122 )

Co-authored-by: rinrin32 <rinrin.int@gmail.com>

2024-11-22 01:50:42 -08:00

lang

Unify the model type checking (#1905 )

2024-11-03 12:25:39 -08:00

srt

[minor] Clean up unused imports (#2122 )

2024-11-22 01:50:42 -08:00

test

Benchmark with Pytorch Profiler easily (#2110 )

2024-11-21 23:29:50 -08:00

__init__.py

Provide an offline engine API (#1567 )

2024-10-06 20:27:03 -07:00

api.py

Update links (#1805 )

2024-10-26 04:46:01 -07:00

bench_latency.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

bench_offline_throughput.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

bench_one_batch_server.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

bench_one_batch.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

bench_serving.py

Benchmark with Pytorch Profiler easily (#2110 )

2024-11-21 23:29:50 -08:00

check_env.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

global_config.py

Improve the user control of new_token_ratio (#1811 )

2024-10-26 16:39:41 -07:00

launch_server_llavavid.py

Update vllm to 0.6.3 (#1711 ) (#1720 )

2024-10-19 20:45:41 -07:00

launch_server.py

Add a watch dog thread (#1816 )

2024-10-27 02:00:50 -07:00

README.md

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

utils.py

fix black in pre-commit (#1940 )

2024-11-08 07:42:47 +08:00

version.py

Release v0.3.5.post2 (#2046 )

2024-11-15 06:54:00 -08:00

README.md

Code Structures

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the throughput in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching the local server.
utils.py: Common utilities.