Zhihao Zhang e7bc600304 [Feature] Speculative decoding support lookahead (#9873) ...

Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>

2025-09-18 16:42:41 -07:00

..

async_dynamic_batch_tokenizer.py

[Performance] Dynamic Batch Tokenizer (#9382)

2025-09-14 01:56:04 +08:00

cache_controller.py

support memory_pool_host page first direct layout (#10031)

2025-09-11 23:19:44 -07:00

configure_logging.py

Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)

2025-06-29 23:16:19 -07:00

data_parallel_controller.py

[4/N]DP refactor: support watching mode get_load and shortest queue strategy (#10201)

2025-09-15 10:06:08 +08:00

detokenizer_manager.py

[4/N]DP refactor: support watching mode get_load and shortest queue strategy (#10201)

2025-09-15 10:06:08 +08:00

disagg_service.py

[1/2] Refactor multi-tokenizer manager (#10074)

2025-09-07 19:13:34 +08:00

io_struct.py

[RL] Add destroy process group api (#9979)

2025-09-19 00:31:56 +08:00

mm_utils.py

Qwen2.5-VL eagle3 infer (#8801)

2025-09-07 20:44:34 -07:00

multi_tokenizer_mixin.py

[4/N]DP refactor: support watching mode get_load and shortest queue strategy (#10201)

2025-09-15 10:06:08 +08:00

multimodal_processor.py

Fix CI (#9012)

2025-08-09 13:33:42 -07:00

schedule_batch.py

[Feature] Speculative decoding support lookahead (#9873)

2025-09-18 16:42:41 -07:00

schedule_policy.py

feat: add priority based scheduling with priority based request acceptance and preemption (#8746)

2025-09-16 17:10:10 -07:00

scheduler_input_blocker.py

Support colocating requests (#7973)

2025-07-28 22:51:49 -07:00

scheduler_metrics_mixin.py

[4/N]DP refactor: support watching mode get_load and shortest queue strategy (#10201)

2025-09-15 10:06:08 +08:00

scheduler_output_processor_mixin.py

[Generative Score API] Scoring(Prefill-only) optimizations. (#9748)

2025-09-14 01:57:06 +08:00

scheduler_profiler_mixin.py

[Auto Sync] Update scheduler_profiler_mixin.py, rpd_utils.p... (20250916) (#10494)

2025-09-16 17:02:20 -07:00

scheduler_recv_skipper.py

Reduce scheduler recv requests overhead (#8947)

2025-08-08 00:10:05 -07:00

scheduler_update_weights_mixin.py

[RL] Add destroy process group api (#9979)

2025-09-19 00:31:56 +08:00

scheduler.py

[Feature] Speculative decoding support lookahead (#9873)

2025-09-18 16:42:41 -07:00

session_controller.py

Fix DP load for embedding (#9165)

2025-08-15 23:58:44 -07:00

template_manager.py

Move parsers under a single folder (#9912)

2025-09-02 18:25:04 -07:00

tokenizer_communicator_mixin.py

[RL] Add destroy process group api (#9979)

2025-09-19 00:31:56 +08:00

tokenizer_manager.py

[Feature] Speculative decoding support lookahead (#9873)

2025-09-18 16:42:41 -07:00

tp_worker_overlap_thread.py

[RL] Add destroy process group api (#9979)

2025-09-19 00:31:56 +08:00

tp_worker.py

[RL] Add destroy process group api (#9979)

2025-09-19 00:31:56 +08:00

utils.py

[PD] Support PD disaggregation with Prefill PP (#8846)

2025-08-16 18:31:31 -07:00