Files

Nan Jiang ba69c153f6 [RL]: Fix error tagging in multi-stage wake up (#7812 )

Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>

2025-07-06 16:51:29 -07:00

eval

chore: add hf_xet dep (#6243 )

2025-05-12 13:08:40 -07:00

lang

Frontend language separate reasoning support (#6031 )

2025-06-10 17:11:29 -07:00

srt

[RL]: Fix error tagging in multi-stage wake up (#7812 )

2025-07-06 16:51:29 -07:00

test

Add V2-lite model test (#7390 )

2025-07-03 22:25:50 -07:00

__init__.py

Frontend language separate reasoning support (#6031 )

2025-06-10 17:11:29 -07:00

api.py

Frontend language separate reasoning support (#6031 )

2025-06-10 17:11:29 -07:00

bench_offline_throughput.py

Fix torch profiler bugs for bench_offline_throughput.py (#6557 )

2025-06-09 20:33:41 +08:00

bench_one_batch_server.py

[CI][Router] Fix bench_one_batch_server for pd router test (#7731 )

2025-07-02 23:18:24 -07:00

bench_one_batch.py

[Feature] Comprehensive Hybrid Parallelism Support (#6389 )

2025-06-20 14:43:11 -07:00

bench_serving.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632 )

2025-06-29 23:16:19 -07:00

check_env.py

Fix check_env script (#5901 )

2025-04-29 18:54:54 -07:00

compile_deep_gemm.py

Use monotonic clock for interval measurement (#6211 )

2025-05-17 16:49:18 -07:00

global_config.py

Misc clean up; Remove the support of jump forward (#4032 )

2025-03-03 07:02:14 -08:00

launch_server.py

Separate two entry points: Engine and HTTP server (#2996 )

2025-01-19 22:09:24 -08:00

profiler.py

Improve profiler and integrate profiler in bench_one_batch_server (#6787 )

2025-05-31 15:53:55 -07:00

README.md

feat: support loogle eval (#6190 )

2025-05-10 23:52:44 -07:00

utils.py

Feat/support rerank (#6058 )

2025-06-16 10:50:01 -07:00

version.py

chore: bump v0.4.9 (#7802 )

2025-07-05 17:40:29 -07:00

README.md

Code Structures

eval: The evaluation utilities.
lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the throughput in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables and dependencies.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching the local server.
utils.py: Common utilities.
version.py: Version info.