Lianmin Zheng ad20b7957e Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) ...

Co-authored-by: kavioyu <kavioyu@tencent.com>

2025-01-02 02:09:08 -08:00

..

data_parallel_controller.py

Crash the server correctly during error (#2231)

2024-11-28 00:22:39 -08:00

detokenizer_manager.py

[Feature] Get Token IDs with Engine.generate() (#2636)

2024-12-29 12:28:27 -08:00

image_processor.py

Simplify tokenizer manager (#2254)

2024-11-29 02:18:51 -08:00

io_struct.py

Speed up update_weights_from_tensor (#2695)

2025-01-02 02:05:19 -08:00

schedule_batch.py

Eagle speculative decoding part 3: small modifications to the general scheduler (#2709)

2025-01-02 02:09:08 -08:00

schedule_policy.py

Fix cache hit rate when chunked prefill (#2555)

2024-12-26 03:14:28 -08:00

scheduler.py

Eagle speculative decoding part 3: small modifications to the general scheduler (#2709)

2025-01-02 02:09:08 -08:00

session_controller.py

[Session] Update session control interface (#2635)

2024-12-29 02:10:27 -08:00

tokenizer_manager.py

Improve the computation for time_per_output_token Prometheus metrics (#2674)

2024-12-30 21:40:14 -08:00

tp_worker_overlap_thread.py

Refactor logprob computation to return the real logprob used in sampling (#2664)

2024-12-30 04:51:38 -08:00

tp_worker.py

Eagle speculative decoding part 3: small modifications to the general scheduler (#2709)

2025-01-02 02:09:08 -08:00