EngineX-Hygon/sglang

Files

History

zhyncs 6a4dc99697 misc: rm rpyc from PACKAGE_LIST (#649 )

2024-07-18 02:35:38 -07:00

..

Fix vertexai (#633 )

2024-07-16 16:07:19 -07:00

Optimize mem indices mangement (#619 )

2024-07-13 23:39:37 -07:00

Remove the dependency of rpyc (#646 )

2024-07-18 02:13:54 -07:00

Higher priority for user input of max_prefill_tokens & format (#540 )

2024-06-12 21:48:40 -07:00

__init__.py

Bump version to 0.1.21 (#626 )

2024-07-15 13:10:53 -07:00

api.py

[Feat] Expose logprob options to sgl.gen API (#503 )

2024-07-09 00:35:39 -07:00

bench_latency.py

Flashinfer sample kernel (#617 )

2024-07-17 13:24:43 -07:00

check_env.py

misc: rm rpyc from PACKAGE_LIST (#649 )

2024-07-18 02:35:38 -07:00

global_config.py

Simplify mem state (#623 )

2024-07-15 02:01:09 -07:00

launch_server_llavavid.py

Remove the dependency of rpyc (#646 )

2024-07-18 02:13:54 -07:00

launch_server.py

Remove the dependency of rpyc (#646 )

2024-07-18 02:13:54 -07:00

README.md

Improve tensor parallel performance (#625 )

2024-07-15 07:10:51 -07:00

utils.py

Higher priority for user input of max_prefill_tokens & format (#540 )

2024-06-12 21:48:40 -07:00

README.md

Code Structure

backend: Various backends for the language interpreter.
lang: The frontend language.
srt: The serving engine for running local models. (SRT = SGLang Runtime).
test: Test utilities.
api.py: Public API.
bench_latency.py: Benchmark utilities.
global_config.py: The global configs and constants.
launch_server.py: The entry point of launching local server.
utils.py: Common utilities.