Yineng Zhang 714f3e6362 feat: support flashinfer mla with prefix cache (#3643)

2025-02-18 02:06:43 +08:00

..

cache_controller.py

Cache controller for hierarchical caching (#2804)

2025-01-10 20:22:01 -08:00

configure_logging.py

Fix the request loggings to make it fully able to be easily replayed (#2973)

2025-01-18 21:45:00 -08:00

data_parallel_controller.py

[Feature] Support minicpmv v2.6 (#2785)

2025-01-18 14:14:19 -08:00

detokenizer_manager.py

[Feat] Return hidden states (experimental) (#3364)

2025-02-10 15:54:37 -08:00

image_processor.py

Model: Support Qwen 2.5 vl (#3258)

2025-02-16 00:58:53 -08:00

io_struct.py

[Feat] Return hidden states (experimental) (#3364)

2025-02-10 15:54:37 -08:00

schedule_batch.py

feat: support flashinfer mla with prefix cache (#3643)

2025-02-18 02:06:43 +08:00

schedule_policy.py

Query remaining memory dynamically for PrefillAdder (#2941)

2025-01-17 20:20:26 -08:00

scheduler.py

[Feat] Return hidden states (experimental) (#3364)

2025-02-10 15:54:37 -08:00

session_controller.py

[Feature] Add sampler custom logits processor (#2396)

2025-01-19 14:46:53 -08:00

tokenizer_manager.py

[Feat] Return hidden states (experimental) (#3364)

2025-02-10 15:54:37 -08:00

tp_worker_overlap_thread.py

[Feat] Return hidden states (experimental) (#3364)

2025-02-10 15:54:37 -08:00

tp_worker.py

fix missing revision arg when loading tokenizer (#2982)

2025-01-19 11:36:07 -08:00

utils.py

Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956)

2025-01-18 19:37:30 -08:00