EngineX-Hygon/sglang

Files

History

Lianmin Zheng 93b77c8e8a Fix the request loggings to make it fully able to be easily replayed (#2973 )

2025-01-18 21:45:00 -08:00

..

[Feature] Support minicpmv v2.6 (#2785 )

2025-01-18 14:14:19 -08:00

Fix the request loggings to make it fully able to be easily replayed (#2973 )

2025-01-18 21:45:00 -08:00

[Feature] Support minicpmv v2.6 (#2785 )

2025-01-18 14:14:19 -08:00

__init__.py

minor: limit the range of vllm versions (#2350 )

2024-12-05 02:00:34 +08:00

api.py

Merged three native APIs into one: get_server_info (#2152 )

2024-11-24 01:37:58 -08:00

bench_latency.py

Rename sglang.bench_latency to sglang.bench_one_batch (#2118 )

2024-11-21 20:07:48 -08:00

bench_offline_throughput.py

Improve benchmark scripts and error message printing (#2922 )

2025-01-16 06:24:31 -08:00

bench_one_batch_server.py

Crash the server correctly during error (#2231 )

2024-11-28 00:22:39 -08:00

bench_one_batch.py

add profiling to bench_one_batch script (#2821 )

2025-01-16 07:24:24 -08:00

bench_serving.py

Improve benchmark scripts and error message printing (#2922 )

2025-01-16 06:24:31 -08:00

check_env.py

ROCm support for sglang.check_env (#2426 )

2024-12-17 03:45:14 -08:00

global_config.py

Improve the user control of new_token_ratio (#1811 )

2024-10-26 16:39:41 -07:00

launch_server_llavavid.py

Update vllm to 0.6.3 (#1711 ) (#1720 )

2024-10-19 20:45:41 -07:00

launch_server.py

fix missing launch server import (#2242 )

2024-11-28 21:24:47 +08:00

llama3_eval.py

docs: update README (#2516 )

2024-12-19 13:44:02 +08:00

README.md

feat: use CUDA 12.4 by default (for FA3) (#2682 )

2024-12-31 15:52:09 +08:00

utils.py

Refactor to add TypeBasedDispatcher to simplify dispatching (#2958 )

2025-01-18 20:13:27 -08:00

version.py

chore: bump v0.4.1.post6 (#2899 )

2025-01-15 16:23:42 +08:00

README.md

Code Structures

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the throughput in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching the local server.
llama3_eval.py: Evaluation of Llama 3.1 using the Meta Llama dataset.
utils.py: Common utilities.