Qiaolin Yu 69f453e5a4 Use device_group for all_gather when disabling overlap scheduling (#8001)

2025-07-15 19:38:58 -07:00

..

multimodal_processors

add model: qwen2-audio (#7596)

2025-07-04 21:13:10 -07:00

cache_controller.py

Kernels for efficient KV cache IO (#7313)

2025-07-06 22:53:36 -07:00

configure_logging.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

data_parallel_controller.py

feat: add direct routing strategy to DP worker (#6884)

2025-06-09 11:44:05 -07:00

detokenizer_manager.py

Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225)

2025-05-12 14:33:38 -07:00

io_struct.py

feat: update multimodal data handling in engine entrypoint (#8002)

2025-07-15 00:12:22 -07:00

mm_utils.py

Fix flaky CI: test_vlm_models (#8006)

2025-07-14 14:56:41 -07:00

multimodal_processor.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

schedule_batch.py

SWA Prefix Cache (#7367)

2025-07-13 12:31:07 -07:00

schedule_policy.py

SWA Prefix Cache (#7367)

2025-07-13 12:31:07 -07:00

scheduler_output_processor_mixin.py

prevent server crash from potential invalid grammar (#7897)

2025-07-15 11:21:45 +08:00

scheduler.py

Use device_group for all_gather when disabling overlap scheduling (#8001)

2025-07-15 19:38:58 -07:00

session_controller.py

Move mem_fraction_static adjustment for multimodal models to server_args.py & Fix session control & Other cleanups (#7748)

2025-07-04 16:33:33 -07:00

template_manager.py

feat(oai refactor): Replace openai_api with entrypoints/openai (#7351)

2025-06-21 13:21:06 -07:00

tokenizer_manager.py

[BugFix] add verify logit_bias to avoid crash because of IndexError (#7749)

2025-07-14 02:44:12 +08:00

tp_worker_overlap_thread.py

SWA Prefix Cache (#7367)

2025-07-13 12:31:07 -07:00

tp_worker.py

Fix different device type adjustment in PP (#7760)

2025-07-15 19:37:14 -07:00

utils.py

Correctly abort the failed grammar requests & Improve the handling of abort (#6803)

2025-06-01 19:00:07 -07:00