2024-08-02 03:12:20 +08:00
|
|
|
# Code Structures
|
|
|
|
|
|
2024-07-13 15:00:26 -07:00
|
|
|
- `lang`: The frontend language.
|
2024-07-19 09:54:01 -07:00
|
|
|
- `srt`: The backend engine for running local models. (SRT = SGLang Runtime).
|
2024-09-01 02:36:56 -07:00
|
|
|
- `test`: The test utilities.
|
|
|
|
|
- `api.py`: The public APIs.
|
2024-11-21 20:07:48 -08:00
|
|
|
- `bench_offline_throughput.py`: Benchmark the throughput in the offline mode.
|
|
|
|
|
- `bench_one_batch.py`: Benchmark the latency of running a single static batch without a server.
|
|
|
|
|
- `bench_one_batch_server.py`: Benchmark the latency of running a single batch with a server.
|
2024-07-19 10:58:03 -07:00
|
|
|
- `bench_serving.py`: Benchmark online serving with dynamic requests.
|
2025-03-03 00:12:04 -08:00
|
|
|
- `check_env.py`: Check the environment variables and dependencies.
|
2024-07-13 15:00:26 -07:00
|
|
|
- `global_config.py`: The global configs and constants.
|
2024-09-12 23:29:44 -07:00
|
|
|
- `launch_server.py`: The entry point for launching the local server.
|
2025-02-18 17:23:43 +08:00
|
|
|
- `llama3_eval.py`: Evaluation of Llama 3 using the Meta Llama dataset.
|
2024-07-13 15:00:26 -07:00
|
|
|
- `utils.py`: Common utilities.
|
2025-03-03 00:12:04 -08:00
|
|
|
- `version.py`: Version info.
|
