Cheng Wan b2e95f62b4 Fix two issues related to --moe-dense-tp-size=1 (#5657) ...

Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: 颉沆 <xiehang.lsy@alibaba-inc.com>

2025-05-12 23:51:39 -07:00

..

multimodal_processors

[Fix] Incorrect Memory Allocation on CUDA:0 by Non-Zero CUDA Processes in TP/DP (#5745)

2025-05-08 17:52:26 -07:00

cache_controller.py

fix bug that gpu0 occupies more memory when hicache is turned on (#5778)

2025-05-09 15:36:08 -07:00

configure_logging.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00

data_parallel_controller.py

Add typo checker in pre-commit (#6179)

2025-05-11 12:55:00 +08:00

detokenizer_manager.py

Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225)

2025-05-12 14:33:38 -07:00

expert_distribution.py

Support compressed tensors fp8w8a8 (#4743)

2025-03-26 13:21:25 -07:00

io_struct.py

Fix start_profile does not support with_stack and record_shapes (#6043)

2025-05-11 23:11:32 -07:00

mm_utils.py

Add typo checker in pre-commit (#6179)

2025-05-11 12:55:00 +08:00

multimodal_processor.py

Clean up imports (#5467)

2025-04-16 15:26:49 -07:00

schedule_batch.py

Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225)

2025-05-12 14:33:38 -07:00

schedule_policy.py

[PP] Fix init_memory_pool desync & add PP for mixtral (#6223)

2025-05-12 12:38:09 -07:00

scheduler_output_processor_mixin.py

Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225)

2025-05-12 14:33:38 -07:00

scheduler.py

Fix two issues related to --moe-dense-tp-size=1 (#5657)

2025-05-12 23:51:39 -07:00

session_controller.py

model: Minicpmo (#3023)

2025-03-24 20:08:40 -07:00

tokenizer_manager.py

Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225)

2025-05-12 14:33:38 -07:00

tp_worker_overlap_thread.py

Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225)

2025-05-12 14:33:38 -07:00

tp_worker.py

Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201)

2025-05-12 00:17:33 -07:00

utils.py

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00