EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity
Files
a1c1ebe9357418ae8cf8ffdf66e3eaec066170e8
sglang/python/sglang
History
Yuhong Guo a1c1ebe935 Fix FP8 KV Cache Support in FA3 Backend (#7148)
2025-06-25 02:14:40 -07:00
..
eval
chore: add hf_xet dep (#6243)
2025-05-12 13:08:40 -07:00
lang
Frontend language separate reasoning support (#6031)
2025-06-10 17:11:29 -07:00
srt
Fix FP8 KV Cache Support in FA3 Backend (#7148)
2025-06-25 02:14:40 -07:00
test
feat: integrate deepgemm into EPMoE (#6821)
2025-06-23 01:38:58 -07:00
__init__.py
Frontend language separate reasoning support (#6031)
2025-06-10 17:11:29 -07:00
api.py
Frontend language separate reasoning support (#6031)
2025-06-10 17:11:29 -07:00
bench_offline_throughput.py
Fix torch profiler bugs for bench_offline_throughput.py (#6557)
2025-06-09 20:33:41 +08:00
bench_one_batch_server.py
Improve profiler and integrate profiler in bench_one_batch_server (#6787)
2025-05-31 15:53:55 -07:00
bench_one_batch.py
[Feature] Comprehensive Hybrid Parallelism Support (#6389)
2025-06-20 14:43:11 -07:00
bench_serving.py
Support VILA models (#6106)
2025-06-11 11:47:25 -07:00
check_env.py
Fix check_env script (#5901)
2025-04-29 18:54:54 -07:00
compile_deep_gemm.py
Use monotonic clock for interval measurement (#6211)
2025-05-17 16:49:18 -07:00
global_config.py
Misc clean up; Remove the support of jump forward (#4032)
2025-03-03 07:02:14 -08:00
launch_server.py
Separate two entry points: Engine and HTTP server (#2996)
2025-01-19 22:09:24 -08:00
math_utils.py
Re-quantize DeepSeek model weights to support DeepGEMM new input format (#7156)
2025-06-13 15:57:45 -07:00
profiler.py
Improve profiler and integrate profiler in bench_one_batch_server (#6787)
2025-05-31 15:53:55 -07:00
README.md
feat: support loogle eval (#6190)
2025-05-10 23:52:44 -07:00
utils.py
Feat/support rerank (#6058)
2025-06-16 10:50:01 -07:00
version.py
chore: bump v0.4.8 (#7493)
2025-06-23 23:14:22 -07:00

README.md

Code Structures

  • eval: The evaluation utilities.
  • lang: The frontend language.
  • srt: The backend engine for running local models. (SRT = SGLang Runtime).
  • test: The test utilities.
  • api.py: The public APIs.
  • bench_offline_throughput.py: Benchmark the throughput in the offline mode.
  • bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
  • bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
  • bench_serving.py: Benchmark online serving with dynamic requests.
  • check_env.py: Check the environment variables and dependencies.
  • global_config.py: The global configs and constants.
  • launch_server.py: The entry point for launching the local server.
  • utils.py: Common utilities.
  • version.py: Version info.