Ke Bao 20c8119915 Fix eagle hang issue for max_new_tokens=1 (#4185)

2025-03-07 12:11:18 -08:00

..

configs

example: add vlm to token in & out example (#3941)

2025-03-04 22:18:26 -08:00

constrained

[Fix & Style] Refactor the grammar backend to reduce human errors and improve readability (#4030)

2025-03-04 03:56:45 -08:00

distributed

Clean up custom allreduce (#4029)

2025-03-03 04:59:53 -08:00

entrypoints

Add Support for Qwen2-VL Multi-modal Embedding Models (#3694)

2025-03-06 16:46:20 -08:00

layers

Revert "ROCm: Flex Attention Enablement with custom backends (#4178)" (#4186)

2025-03-07 10:27:52 -08:00

lora

[Fix] Fix bugs and refactor codes in lora for better scalability. (#3652)

2025-02-20 11:51:57 -08:00

managers

Fix eagle hang issue for max_new_tokens=1 (#4185)

2025-03-07 12:11:18 -08:00

mem_cache

Memory pool fix for upstream change about eagle (#4170)

2025-03-07 00:58:20 -08:00

metrics

Split the __init__ of scheduler as smaller functions. Improve the eagle tests (#4128)

2025-03-06 00:13:20 -08:00

model_executor

Revert "ROCm: Flex Attention Enablement with custom backends (#4178)" (#4186)

2025-03-07 10:27:52 -08:00

model_loader

example: add vlm to token in & out example (#3941)

2025-03-04 22:18:26 -08:00

models

[Feature] DeepSeek V3/R1 INT8 Quantization (channel-wise) (#3888)

2025-03-06 20:54:52 -08:00

openai_api

Add Support for Qwen2-VL Multi-modal Embedding Models (#3694)

2025-03-06 16:46:20 -08:00

sampling

Remove outdated test utils and fix links for the doc of sampling params (#3999)

2025-03-03 09:41:38 -08:00

speculative

Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134)

2025-03-06 06:13:59 -08:00

_custom_ops.py

Fix breakage problem when using custom_ar (#4052)

2025-03-04 02:59:03 -08:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

conversation.py

Add Support for Qwen2-VL Multi-modal Embedding Models (#3694)

2025-03-06 16:46:20 -08:00

custom_op.py

Use forward_cuda to execute custom op for hip platform (#3305)

2025-02-05 02:58:17 +00:00

function_call_parser.py

[Minor] more code cleanup (#4077)

2025-03-04 21:23:47 -08:00

hf_transformers_utils.py

Model: Support Qwen 2.5 vl (#3258)

2025-02-16 00:58:53 -08:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

reasoning_parser.py

Reasoning parser (#4000)

2025-03-03 21:16:36 -08:00

server_args.py

Revert "ROCm: Flex Attention Enablement with custom backends (#4178)" (#4186)

2025-03-07 10:27:52 -08:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

utils.py

ROCm: enable trillion-parameter MoE models with INT4-FP8 single node (#4152)

2025-03-06 15:33:02 -08:00

warmup.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00