EngineX-Hygon/sglang

Files

History

Ying Sheng 83d2b30d75 format

2024-07-24 10:53:07 +00:00

..

Fix StreamExecutor.fork() losing the current role start index. (#684 )

2024-07-20 23:32:11 -07:00

format

2024-07-24 10:53:07 +00:00

Update OpenAI API (#667 )

2024-07-19 23:20:54 -07:00

__init__.py

bump version to 0.1.23 (#706 )

2024-07-23 13:53:19 -07:00

api.py

Update Readme (#660 )

2024-07-19 09:54:01 -07:00

bench_latency.py

Flashinfer sample kernel (#617 )

2024-07-17 13:24:43 -07:00

bench_serving.py

feat: add e2e latency (#704 )

2024-07-24 05:51:10 +10:00

check_env.py

Improve docs (#662 )

2024-07-19 10:58:03 -07:00

global_config.py

Auto adjust new ratio (#708 )

2024-07-23 22:06:02 -07:00

launch_server_llavavid.py

Remove the dependency of rpyc (#646 )

2024-07-18 02:13:54 -07:00

launch_server.py

Remove the dependency of rpyc (#646 )

2024-07-18 02:13:54 -07:00

README.md

Improve docs (#662 )

2024-07-19 10:58:03 -07:00

utils.py

Higher priority for user input of max_prefill_tokens & format (#540 )

2024-06-12 21:48:40 -07:00

README.md

Code Structure

lang: The frontend language.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: Test utilities.
api.py: Public API.
bench_latency.py: Benchmark a single static batch.
bench_serving.py: Benchmark online serving with dynamic requests.
global_config.py: The global configs and constants.
launch_server.py: The entry point of launching local server.
utils.py: Common utilities.