Yineng Zhang 1c63e79756 use fa3 in sgl-kernel (#4954)

2025-03-31 16:14:49 -07:00

..

configs

Clean up import vllm in quantization/__init__.py (#4834)

2025-03-28 10:34:10 -07:00

connector

feat(remote_model): support variable remote backend for model loader (#3964)

2025-03-14 00:40:44 -07:00

constrained

Fix retract for page size > 1 (#4914)

2025-03-30 02:57:15 -07:00

disaggregation

[PD] Remove invalid parameter (#4721)

2025-03-24 13:14:16 -07:00

distributed

Fix error due to CustomAllreduce setup failure (#4815)

2025-03-27 18:52:10 -07:00

entrypoints

Include context length in /v1/models response. (#4809)

2025-03-27 20:23:18 -07:00

layers

use fa3 in sgl-kernel (#4954)

2025-03-31 16:14:49 -07:00

lora

[Feature] add multi-rank support for Lora (#4492)

2025-03-28 09:38:44 -07:00

managers

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

mem_cache

Support page size > 1 + eagle (#4908)

2025-03-30 00:46:23 -07:00

metrics

Fix retract for page size > 1 (#4914)

2025-03-30 02:57:15 -07:00

model_executor

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

model_loader

[Misc] Fix issues reported by torchfix (#4837)

2025-03-27 20:10:32 -07:00

models

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

openai_api

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

platforms

[feat] interface for platforms abstraction (#4928)

2025-03-31 00:04:21 -07:00

sampling

[Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466)

2025-03-16 00:02:47 -07:00

speculative

[Fix] avoid stream sync and torch compile in prefill for fa3 backend (#4932)

2025-03-30 13:53:44 -07:00

_custom_ops.py

use sgl custom all reduce (#4441)

2025-03-18 00:46:41 -07:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

code_completion_parser.py

Feature: support code completion (#3612)

2025-03-16 18:26:19 -07:00

conversation.py

model: Minicpmo (#3023)

2025-03-24 20:08:40 -07:00

custom_op.py

Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215)

2025-03-12 00:08:03 -07:00

function_call_parser.py

Remove empty tool function name (#4704)

2025-03-27 22:23:30 -07:00

hf_transformers_utils.py

[Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984)

2025-03-22 14:27:39 -07:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

patch_torch.py

Patch PyTorch's bug that cross-process tensor transfer will lead to wrong device (#4565)

2025-03-27 00:22:33 -07:00

reasoning_parser.py

Reasoning parser (#4000)

2025-03-03 21:16:36 -08:00

server_args.py

Fix retract for page size > 1 (#4914)

2025-03-30 02:57:15 -07:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Warn users when release_memory_occupation is called without memory saver enabled (#4566)

2025-03-26 00:18:14 -07:00

utils.py

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

warmup.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00