fzyzcjy 61970b08d8 Let bench_one_batch support enable_dp_attention (#4058)

2025-04-08 23:44:25 -07:00

..

configs

FP4 weight loading and inference (2/2) (#3972)

2025-04-08 17:26:21 -07:00

connector

feat(remote_model): support variable remote backend for model loader (#3964)

2025-03-14 00:40:44 -07:00

constrained

feat: disable grammar restrictions within reasoning sections (#4984)

2025-04-07 21:46:47 -07:00

disaggregation

[PD] Simplify mini LB (#4911)

2025-04-08 09:42:34 -07:00

distributed

python transfer custom allreduce from trt kernel to vllm kernel (#5080)

2025-04-05 15:35:55 -07:00

entrypoints

[feat&refactor] Enhance multimodal input support with refactor io_struct (#4938)

2025-04-08 14:48:07 -07:00

layers

[DeepEP] fix: import buffer error (#5179)

2025-04-08 22:12:14 -07:00

lora

[Feature] add multi-rank support for Lora (#4492)

2025-03-28 09:38:44 -07:00

managers

Let bench_one_batch support enable_dp_attention (#4058)

2025-04-08 23:44:25 -07:00

mem_cache

bug fix for hicache host eviction (#4989)

2025-04-02 00:33:50 -07:00

metrics

Fix retract for page size > 1 (#4914)

2025-03-30 02:57:15 -07:00

model_executor

[Fix] DeepEP Compatibility with Low Latency (#5068)

2025-04-08 20:31:31 -07:00

model_loader

fix empty_cache error in pt_weights_iterator (#5151)

2025-04-08 01:22:10 -07:00

models

fix: use DeepEPDispatcher on CUDA (#5180)

2025-04-08 21:56:53 -07:00

openai_api

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

platforms

[feat] interface for platforms abstraction (#4928)

2025-03-31 00:04:21 -07:00

sampling

[Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466)

2025-03-16 00:02:47 -07:00

speculative

[Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052)

2025-04-05 01:23:02 -07:00

_custom_ops.py

python transfer custom allreduce from trt kernel to vllm kernel (#5080)

2025-04-05 15:35:55 -07:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

code_completion_parser.py

Feature: support code completion (#3612)

2025-03-16 18:26:19 -07:00

conversation.py

Add Llama4 support (#5092)

2025-04-07 00:29:36 -07:00

custom_op.py

sgl scaled_fp8_quant support output padding (#4861)

2025-04-02 23:53:57 +08:00

function_call_parser.py

Remove empty tool function name (#4704)

2025-03-27 22:23:30 -07:00

hf_transformers_utils.py

[Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984)

2025-03-22 14:27:39 -07:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

patch_torch.py

Patch PyTorch's bug that cross-process tensor transfer will lead to wrong device (#4565)

2025-03-27 00:22:33 -07:00

reasoning_parser.py

Reasoning parser (#4000)

2025-03-03 21:16:36 -08:00

server_args.py

[Fix] DeepEP Compatibility with Low Latency (#5068)

2025-04-08 20:31:31 -07:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Warn users when release_memory_occupation is called without memory saver enabled (#4566)

2025-03-26 00:18:14 -07:00

utils.py

[fix] remove cuda_device_count_stateless (#5060)

2025-04-04 00:18:26 -07:00

warmup.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00