EngineX-Hygon/sglang

Files

History

lambert0312 91dda4cd06 Add A800 fused moe config for qwen3 30b (#5880 )

2025-04-29 02:02:24 -07:00

..

Support o1 model on Azure (#4980 )

2025-04-21 00:46:09 -07:00

Add A800 fused moe config for qwen3 30b (#5880 )

2025-04-29 02:02:24 -07:00

[CI] test chunked prefill more (#5798 )

2025-04-28 10:57:17 -07:00

__init__.py

Clean up imports (#5467 )

2025-04-16 15:26:49 -07:00

api.py

Support n in OpenAI API completions (#3446 )

2025-03-20 13:46:46 +08:00

bench_offline_throughput.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988 )

2025-03-03 00:12:04 -08:00

bench_one_batch_server.py

Revert "[fix] fix bench_one_batch_server" (#5785 )

2025-04-27 03:48:33 -07:00

bench_one_batch.py

[CI] Fix ci tests (#5769 )

2025-04-27 07:18:10 -07:00

bench_serving.py

[CI] fix port conflicts (#5789 )

2025-04-27 05:17:44 -07:00

check_env.py

Use metadata to detect version of package (#4782 )

2025-03-26 00:41:43 -07:00

compile_deep_gemm.py

Resolves the 404 Not Found error when running compile_deep_gemm.py in multi-node setups (#5720 )

2025-04-26 18:13:13 -07:00

global_config.py

Misc clean up; Remove the support of jump forward (#4032 )

2025-03-03 07:02:14 -08:00

launch_server.py

Separate two entry points: Engine and HTTP server (#2996 )

2025-01-19 22:09:24 -08:00

llama3_eval.py

docs: update README (#2516 )

2024-12-19 13:44:02 +08:00

README.md

Misc clean up; Remove the support of jump forward (#4032 )

2025-03-03 07:02:14 -08:00

utils.py

Fix sglang frontend's incorrect dependency on torch (#4931 )

2025-03-30 13:00:24 -07:00

version.py

chore: bump v0.4.6.post1 (#5845 )

2025-04-28 12:57:08 -07:00

README.md

Code Structures

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the throughput in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables and dependencies.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching the local server.
llama3_eval.py: Evaluation of Llama 3 using the Meta Llama dataset.
utils.py: Common utilities.
version.py: Version info.