Wen-Heng (Jack) Chung 871a4aa1bf [ROCm] Add ROCm tuning configs for AMD Instinct MI325X. (#3536)

2025-02-12 20:09:36 -08:00

..

configs

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

constrained

compatible with new outlines (#3435)

2025-02-10 01:51:30 +08:00

distributed

[Fix] Not skip NVML Check on AMD Platform (#3135)

2025-01-25 21:33:51 -08:00

entrypoints

Make NCCL NVLS configurable (#3502)

2025-02-12 03:25:06 +08:00

layers

[ROCm] Add ROCm tuning configs for AMD Instinct MI325X. (#3536)

2025-02-12 20:09:36 -08:00

lora

[Fix] Fix accuracy bug and refactor codes for lora (#3413)

2025-02-10 13:29:00 +08:00

managers

[Feat] Return hidden states (experimental) (#3364)

2025-02-10 15:54:37 -08:00

mem_cache

Sanity check to prevent performance regression (#3171)

2025-01-27 12:28:17 -08:00

metrics

Improve metrics, logging, and importing orders (#2992)

2025-01-19 18:36:59 -08:00

model_executor

MI30x: More graph captures for larger batch sizes and concurrencies (#3420)

2025-02-12 03:04:38 +08:00

model_loader

Improve weight loading and code style (#3174)

2025-01-27 03:00:41 -08:00

models

Fix deepseek awq v3 (#3450)

2025-02-12 22:09:52 +08:00

openai_api

Feature/function calling update (#2700)

2025-01-26 09:57:51 -08:00

sampling

Fix repetition penalty (#3139)

2025-01-25 21:48:58 -08:00

speculative

[Eagle] reduce one draft forward (#3468)

2025-02-10 20:21:49 +08:00

_custom_ops.py

Roll back to use vllm custom allreduce (#3006)

2025-01-20 04:03:15 -08:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

conversation.py

[Feature] Support minicpmv v2.6 (#2785)

2025-01-18 14:14:19 -08:00

custom_op.py

Use forward_cuda to execute custom op for hip platform (#3305)

2025-02-05 02:58:17 +00:00

function_call_parser.py

Fix: Runtime error for function calling (#3300)

2025-02-06 20:52:01 -08:00

hf_transformers_utils.py

Remove vllm dependency in model config (#2809)

2025-01-09 17:51:56 +08:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

server_args.py

Make NCCL NVLS configurable (#3502)

2025-02-12 03:25:06 +08:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

utils.py

[ROCm] Logic to decide whether to used manually unrolled kernel. (#3306)

2025-02-04 19:12:20 -08:00