EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
Files
6b847a9a05b3c0870968bd5a274207349034cf4e
sglang/python/sglang
History
JiLi 6b847a9a05 Optimize: Cache CUDA device to reduce redundant calls during tensor l… (#8996)
2025-08-10 00:32:57 -07:00
..
eval
[script] update loogle test (#7975)
2025-07-12 00:06:17 -07:00
lang
Support glm4.1v and glm4.5v (#8798)
2025-08-09 00:59:13 -07:00
srt
Optimize: Cache CUDA device to reduce redundant calls during tensor l… (#8996)
2025-08-10 00:32:57 -07:00
test
Add CI for gpt-oss model on hopper (#8851)
2025-08-09 00:34:23 -07:00
__init__.py
Frontend language separate reasoning support (#6031)
2025-06-10 17:11:29 -07:00
api.py
Frontend language separate reasoning support (#6031)
2025-06-10 17:11:29 -07:00
bench_offline_throughput.py
Feature/modelscope model download (#8083)
2025-07-31 17:29:31 -07:00
bench_one_batch_server.py
[CI][Router] Fix bench_one_batch_server for pd router test (#7731)
2025-07-02 23:18:24 -07:00
bench_one_batch.py
[bugfix] Fix missing args in bench one batch (#8877)
2025-08-09 01:34:03 -07:00
bench_serving.py
[Multimodal][Perf] Use pybase64 instead of base64 (#7724)
2025-07-08 14:00:58 -07:00
check_env.py
Revert "feat: replace Decord with video_reader-rs" (#8077)
2025-07-15 20:04:56 -07:00
compile_deep_gemm.py
[bugfix] Add 'disaggregation_mode' parameter to warmup function when compile deep_gemm manually (#8618)
2025-08-01 16:02:53 -07:00
global_config.py
Reduce memory accumulation in long-running server (#8306)
2025-08-03 15:03:16 +08:00
launch_server.py
Separate two entry points: Engine and HTTP server (#2996)
2025-01-19 22:09:24 -08:00
profiler.py
Improve profiler and integrate profiler in bench_one_batch_server (#6787)
2025-05-31 15:53:55 -07:00
README.md
feat: support loogle eval (#6190)
2025-05-10 23:52:44 -07:00
utils.py
Split the scheduler into multiple mixin classes to reduce the file size (#8483)
2025-07-29 12:46:50 -07:00
version.py
chore: bump v0.5.0rc0 (#8959)
2025-08-08 09:16:18 -07:00

README.md

Code Structures

  • eval: The evaluation utilities.
  • lang: The frontend language.
  • srt: The backend engine for running local models. (SRT = SGLang Runtime).
  • test: The test utilities.
  • api.py: The public APIs.
  • bench_offline_throughput.py: Benchmark the throughput in the offline mode.
  • bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
  • bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
  • bench_serving.py: Benchmark online serving with dynamic requests.
  • check_env.py: Check the environment variables and dependencies.
  • global_config.py: The global configs and constants.
  • launch_server.py: The entry point for launching the local server.
  • utils.py: Common utilities.
  • version.py: Version info.