Files

Chang Su 51c38163c1 model: support Step3V (#8583 )

Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>

2025-07-31 02:41:00 -07:00

eval

[script] update loogle test (#7975 )

2025-07-12 00:06:17 -07:00

lang

model: support intern-s1 (#8350 )

2025-07-26 13:48:51 -07:00

srt

model: support Step3V (#8583 )

2025-07-31 02:41:00 -07:00

test

feat: throttle requests at scheduler based on --max_queued_requests (#7565 )

2025-07-28 22:32:33 +08:00

__init__.py

Frontend language separate reasoning support (#6031 )

2025-06-10 17:11:29 -07:00

api.py

Frontend language separate reasoning support (#6031 )

2025-06-10 17:11:29 -07:00

bench_offline_throughput.py

Fix torch profiler bugs for bench_offline_throughput.py (#6557 )

2025-06-09 20:33:41 +08:00

bench_one_batch_server.py

[CI][Router] Fix bench_one_batch_server for pd router test (#7731 )

2025-07-02 23:18:24 -07:00

bench_one_batch.py

Use device_group for all_gather when disabling overlap scheduling (#8001 )

2025-07-15 19:38:58 -07:00

bench_serving.py

[Multimodal][Perf] Use pybase64 instead of base64 (#7724 )

2025-07-08 14:00:58 -07:00

check_env.py

Revert "feat: replace Decord with video_reader-rs" (#8077 )

2025-07-15 20:04:56 -07:00

compile_deep_gemm.py

Use monotonic clock for interval measurement (#6211 )

2025-05-17 16:49:18 -07:00

global_config.py

Misc clean up; Remove the support of jump forward (#4032 )

2025-03-03 07:02:14 -08:00

launch_server.py

Separate two entry points: Engine and HTTP server (#2996 )

2025-01-19 22:09:24 -08:00

profiler.py

Improve profiler and integrate profiler in bench_one_batch_server (#6787 )

2025-05-31 15:53:55 -07:00

README.md

feat: support loogle eval (#6190 )

2025-05-10 23:52:44 -07:00

utils.py

Split the scheduler into multiple mixin classes to reduce the file size (#8483 )

2025-07-29 12:46:50 -07:00

version.py

chore: bump v0.4.9.post6 (#8517 )

2025-07-29 02:30:07 -07:00

README.md

Code Structures

eval: The evaluation utilities.
lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the throughput in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables and dependencies.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching the local server.
utils.py: Common utilities.
version.py: Version info.