Yineng Zhang 7d3b7c87f5 fix: determine if flashinfer is installed (#5336)

2025-04-12 19:59:13 -07:00

..

configs

Support --enable-llama4-multimodal (#5254)

2025-04-11 01:24:14 -07:00

connector

feat(remote_model): support variable remote backend for model loader (#3964)

2025-03-14 00:40:44 -07:00

constrained

feat: disable grammar restrictions within reasoning sections (#4984)

2025-04-07 21:46:47 -07:00

disaggregation

[PD] Support KV transfer with mooncake (#4880)

2025-04-10 14:23:23 +08:00

distributed

python transfer custom allreduce from trt kernel to vllm kernel (#5080)

2025-04-05 15:35:55 -07:00

entrypoints

Support server based rollout in Verlengine (#4848)

2025-04-12 10:07:52 -07:00

layers

refine fused_moe tuning docs (#5294)

2025-04-12 10:01:13 -07:00

lora

[Feature] add multi-rank support for Lora (#4492)

2025-03-28 09:38:44 -07:00

managers

Support server based rollout in Verlengine (#4848)

2025-04-12 10:07:52 -07:00

mem_cache

[PD] Add get_contiguous_buf_infos interface for MLATokenToKVPool (#5204)

2025-04-10 20:05:34 +08:00

metrics

[metrics] Add in queue metrics (#4444)

2025-04-09 17:19:27 -07:00

model_executor

feat: use fa3 mla by default on hopper (#5210)

2025-04-12 01:09:25 -07:00

model_loader

Support Llama4 fp8 inference (#5194)

2025-04-09 20:14:34 +08:00

models

Fix w8a8_int8 model shared experts fusion load weights error (#5120)

2025-04-11 23:33:51 -07:00

openai_api

[fix] fix mrope positions not picked up (#5265)

2025-04-11 01:29:45 -07:00

platforms

[feat] interface for platforms abstraction (#4928)

2025-03-31 00:04:21 -07:00

sampling

[Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466)

2025-03-16 00:02:47 -07:00

speculative

fix: log warning when disable cuda graph (#5209)

2025-04-09 14:16:13 -07:00

_custom_ops.py

python transfer custom allreduce from trt kernel to vllm kernel (#5080)

2025-04-05 15:35:55 -07:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

code_completion_parser.py

Feature: support code completion (#3612)

2025-03-16 18:26:19 -07:00

conversation.py

Add Llama4 support (#5092)

2025-04-07 00:29:36 -07:00

custom_op.py

sgl scaled_fp8_quant support output padding (#4861)

2025-04-02 23:53:57 +08:00

function_call_parser.py

Remove empty tool function name (#4704)

2025-03-27 22:23:30 -07:00

hf_transformers_utils.py

[VLM] Adopt fast image processor by default (#5065)

2025-04-11 21:46:58 -07:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

patch_torch.py

Fix torch.compile cacheing (#5259)

2025-04-10 18:08:45 -07:00

reasoning_parser.py

Reasoning parser (#4000)

2025-03-03 21:16:36 -08:00

server_args.py

[VLM] Adopt fast image processor by default (#5065)

2025-04-11 21:46:58 -07:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Warn users when release_memory_occupation is called without memory saver enabled (#4566)

2025-03-26 00:18:14 -07:00

utils.py

fix: determine if flashinfer is installed (#5336)

2025-04-12 19:59:13 -07:00

warmup.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00