EngineX-Hygon/sglang

Files

History

Lianmin Zheng dc3bee4815 Fix test and benchmark scripts (#2598 )

2024-12-26 07:56:26 -08:00

..

[Feature] Support new parameter - EBNF in xgrammar (#2526 )

2024-12-26 05:12:41 -08:00

Fix test and benchmark scripts (#2598 )

2024-12-26 07:56:26 -08:00

Refactor MoE (#2575 )

2024-12-26 00:02:14 +08:00

__init__.py

minor: limit the range of vllm versions (#2350 )

2024-12-05 02:00:34 +08:00

api.py

Merged three native APIs into one: get_server_info (#2152 )

2024-11-24 01:37:58 -08:00

bench_latency.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

bench_offline_throughput.py

Fix gemlite import (#2553 )

2024-12-22 20:21:17 -08:00

bench_one_batch_server.py

Crash the server correctly during error (#2231 )

2024-11-28 00:22:39 -08:00

bench_one_batch.py

Fix gemlite import (#2553 )

2024-12-22 20:21:17 -08:00

bench_serving.py

Fix test and benchmark scripts (#2598 )

2024-12-26 07:56:26 -08:00

check_env.py

ROCm support for sglang.check_env (#2426 )

2024-12-17 03:45:14 -08:00

global_config.py

Improve the user control of new_token_ratio (#1811 )

2024-10-26 16:39:41 -07:00

launch_server_llavavid.py

Update vllm to 0.6.3 (#1711 ) (#1720 )

2024-10-19 20:45:41 -07:00

launch_server.py

fix missing launch server import (#2242 )

2024-11-28 21:24:47 +08:00

llama3_eval.py

docs: update README (#2516 )

2024-12-19 13:44:02 +08:00

README.md

docs: update README (#2516 )

2024-12-19 13:44:02 +08:00

utils.py

[Core] in batch prefix caching by delay scheduling (#2442 )

2024-12-11 12:51:50 -08:00

version.py

chore: bump v0.4.1 (#2582 )

2024-12-26 07:14:51 +08:00

README.md

Code Structures

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the throughput in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching the local server.
llama3_eval.py: Llama 3.1 evaluation with meta-llama dataset.
utils.py: Common utilities.