EngineX-Hygon/sglang

Files

History

Ke Bao 0ac94c36cb Fallback when sampling failed (#678 )

2024-07-20 10:44:54 -07:00

..

Update Readme (#660 )

2024-07-19 09:54:01 -07:00

Fallback when sampling failed (#678 )

2024-07-20 10:44:54 -07:00

Update OpenAI API (#667 )

2024-07-19 23:20:54 -07:00

__init__.py

Update version to 0.1.22 (#677 )

2024-07-20 03:39:50 -07:00

api.py

Update Readme (#660 )

2024-07-19 09:54:01 -07:00

bench_latency.py

Flashinfer sample kernel (#617 )

2024-07-17 13:24:43 -07:00

bench_serving.py

Fix random dataset (#671 )

2024-07-20 01:57:43 -07:00

check_env.py

Improve docs (#662 )

2024-07-19 10:58:03 -07:00

global_config.py

Simplify mem state (#623 )

2024-07-15 02:01:09 -07:00

launch_server_llavavid.py

Remove the dependency of rpyc (#646 )

2024-07-18 02:13:54 -07:00

launch_server.py

Remove the dependency of rpyc (#646 )

2024-07-18 02:13:54 -07:00

README.md

Improve docs (#662 )

2024-07-19 10:58:03 -07:00

utils.py

Higher priority for user input of max_prefill_tokens & format (#540 )

2024-06-12 21:48:40 -07:00

README.md

Code Structure

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: Test utilities.
api.py: Public API.
bench_latency.py: Benchmark a single static batch.
bench_serving.py: Benchmark online serving with dynamic requests.
global_config.py: The global configs and constants.
launch_server.py: The entry point of launching local server.
utils.py: Common utilities.