Yineng Zhang bf8d07a6f9 feat: patch linear base (#2915)

2025-01-16 18:00:03 +08:00

..

configs

Support w8a8 int8 quantization config (#2881)

2025-01-14 17:07:49 +08:00

constrained

[Feature] Support new parameter - EBNF in xgrammar (#2526)

2024-12-26 05:12:41 -08:00

distributed

adapt custom allreduce for tensorrt llm (#2511)

2025-01-16 04:57:35 +08:00

layers

feat: patch linear base (#2915)

2025-01-16 18:00:03 +08:00

lora

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

managers

Remove normalized_prompt_logprobs from the engine to make code easier to maintain (#2902)

2025-01-15 04:54:14 -08:00

mem_cache

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

metrics

Collect more metrics: num_requests_total (#2859)

2025-01-13 03:57:39 -08:00

model_executor

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

model_loader

Error occurs when loading the gemma model in bitsandbytes format. (#2557)

2024-12-26 05:10:37 -08:00

models

feat: support internlm 3 dense (#2888)

2025-01-14 19:23:26 +08:00

openai_api

[Fix] fix openai adapter (#2685)

2024-12-31 10:48:19 +00:00

sampling

Sampling penalties memory interface (#2870)

2025-01-13 23:09:00 +08:00

speculative

[Fix]eagle2 health_generate is first request,apiserver will core (#2853)

2025-01-13 01:01:21 -08:00

_custom_ops.py

adapt custom allreduce for tensorrt llm (#2511)

2025-01-16 04:57:35 +08:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

conversation.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

hf_transformers_utils.py

Remove vllm dependency in model config (#2809)

2025-01-09 17:51:56 +08:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

server_args.py

Support w8a8 int8 quantization config (#2881)

2025-01-14 17:07:49 +08:00

server.py

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

torch_memory_saver_adapter.py

Add a new api configure_logging to allow dumping the requests (#2875)

2025-01-13 14:24:00 -08:00

utils.py

feat: patch linear base (#2915)

2025-01-16 18:00:03 +08:00