EngineX-Hygon/sglang

Files

History

Jerry Zhang feb2b768ba Add integration with gemlite weight only quant (#2528 )

2024-12-21 00:25:25 +08:00

..

[Core] in batch prefix caching by delay scheduling (#2442 )

2024-12-11 12:51:50 -08:00

Add integration with gemlite weight only quant (#2528 )

2024-12-21 00:25:25 +08:00

Simplify stream_output (#2398 )

2024-12-08 12:27:13 -08:00

__init__.py

minor: limit the range of vllm versions (#2350 )

2024-12-05 02:00:34 +08:00

api.py

Merged three native APIs into one: get_server_info (#2152 )

2024-11-24 01:37:58 -08:00

bench_latency.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

bench_offline_throughput.py

Add integration with gemlite weight only quant (#2528 )

2024-12-21 00:25:25 +08:00

bench_one_batch_server.py

Crash the server correctly during error (#2231 )

2024-11-28 00:22:39 -08:00

bench_one_batch.py

Add integration with gemlite weight only quant (#2528 )

2024-12-21 00:25:25 +08:00

bench_serving.py

Fix a bug with logprob streaming + chunked prefill (#2403 )

2024-12-08 03:55:27 -08:00

check_env.py

ROCm support for sglang.check_env (#2426 )

2024-12-17 03:45:14 -08:00

global_config.py

Improve the user control of new_token_ratio (#1811 )

2024-10-26 16:39:41 -07:00

launch_server_llavavid.py

Update vllm to 0.6.3 (#1711 ) (#1720 )

2024-10-19 20:45:41 -07:00

launch_server.py

fix missing launch server import (#2242 )

2024-11-28 21:24:47 +08:00

llama3_eval.py

docs: update README (#2516 )

2024-12-19 13:44:02 +08:00

README.md

docs: update README (#2516 )

2024-12-19 13:44:02 +08:00

utils.py

[Core] in batch prefix caching by delay scheduling (#2442 )

2024-12-11 12:51:50 -08:00

version.py

Release v0.4.0.post1 (#2375 )

2024-12-06 06:08:19 -08:00

README.md

Code Structures

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the throughput in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching the local server.
llama3_eval.py: Llama 3.1 evaluation with meta-llama dataset.
utils.py: Common utilities.