EngineX-Hygon/sglang

Files

History

Kaichen Zhang - NTU 8234e663e9 [Minor Fix] Fix llava modalities issue for single-image (#1402 )

2024-09-12 01:10:26 -07:00

..

[Fix] Reduce memory usage for loading llava model & Remove EntryClassRemapping (#1308 )

2024-09-02 21:44:45 -07:00

[Minor Fix] Fix llava modalities issue for single-image (#1402 )

2024-09-12 01:10:26 -07:00

[CI] Return output logprobs in unit test (#1361 )

2024-09-09 13:05:13 -07:00

__init__.py

Feat: add alternative choices selection methods (#835 )

2024-08-05 03:27:49 -07:00

api.py

make json_schema usable from gen (#1254 )

2024-08-28 18:57:10 -07:00

bench_latency.py

[Minor] move triton attention kernels into a separate folder (#1379 )

2024-09-10 15:15:08 -07:00

bench_serving.py

[Minor] Many cleanup (#1357 )

2024-09-09 04:14:11 -07:00

check_env.py

misc: add hypervisor vendor (#1165 )

2024-08-21 02:14:51 +10:00

global_config.py

Remove useless fields in global_config.py (#1328 )

2024-09-04 05:37:32 -07:00

launch_server_llavavid.py

[Minor] Many cleanup (#1357 )

2024-09-09 04:14:11 -07:00

launch_server.py

[Minor] Many cleanup (#1357 )

2024-09-09 04:14:11 -07:00

README.md

[CI] merge all ci tests into one file (#1289 )

2024-09-01 02:36:56 -07:00

utils.py

[Minor] Many cleanup (#1357 )

2024-09-09 04:14:11 -07:00

version.py

chore: bump v0.3.0 (#1320 )

2024-09-04 04:32:18 +08:00

README.md

Code Structures

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_latency.py: Benchmark a single static batch.
bench_serving.py: Benchmark online serving with dynamic requests.
global_config.py: The global configs and constants.
launch_server.py: The entry point of launching local server.
utils.py: Common utilities.