Ke Wen 862bcff833 Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

..

configs

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

constrained

Move sgl.Runtime under sglang/lang (#2990)

2025-01-19 17:10:29 -08:00

distributed

disable custom allreduce on HIP (#3058)

2025-01-22 13:57:22 -08:00

entrypoints

Separate two entry points: Engine and HTTP server (#2996)

2025-01-19 22:09:24 -08:00

layers

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

lora

cleanup models dependencies 1/n (#2948)

2025-01-17 23:46:48 +08:00

managers

Use int64 as indices for set_kv_buffer (#3039)

2025-01-21 19:46:09 -08:00

mem_cache

Fix perf regression on small batch sizes (#3008)

2025-01-20 03:39:49 -08:00

metrics

Improve metrics, logging, and importing orders (#2992)

2025-01-19 18:36:59 -08:00

model_executor

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

model_loader

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

models

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

openai_api

Improve type annotation and styles (#2926)

2025-01-16 12:51:11 -08:00

sampling

[Enhancement] Custom Logit Processor Improvement (#2998)

2025-01-20 02:00:35 -08:00

speculative

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

_custom_ops.py

Roll back to use vllm custom allreduce (#3006)

2025-01-20 04:03:15 -08:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

conversation.py

[Feature] Support minicpmv v2.6 (#2785)

2025-01-18 14:14:19 -08:00

hf_transformers_utils.py

Remove vllm dependency in model config (#2809)

2025-01-09 17:51:56 +08:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

server_args.py

Support loading of larger models with on-the-fly quantization (#3061)

2025-01-22 21:33:17 -08:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

utils.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00