Ke Bao 7dc66fcb40 Optimize Triton decoding kernel for long context (#2394)

2024-12-08 01:17:37 -08:00

..

configs

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

constrained

Fix the overlap for xgrammar (#2377)

2024-12-06 05:49:29 -08:00

distributed

adapt vllm distributed module to sglang (#2244)

2024-12-01 15:54:52 +08:00

layers

Optimize Triton decoding kernel for long context (#2394)

2024-12-08 01:17:37 -08:00

lora

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

managers

nit: Remove busy waiting on scheduler (#2382)

2024-12-08 01:06:15 -08:00

mem_cache

Add more support for intel Gaudi accelerators (#2357)

2024-12-06 01:16:33 -08:00

metrics

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_executor

Resubmit MoE-EP (#2371)

2024-12-06 15:05:21 +08:00

model_loader

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

models

Fix shape error that occurred when loading lora weight of gemma2 model. (#2330)

2024-12-08 01:04:08 -08:00

openai_api

Fix logprob for completions (#2301)

2024-12-01 05:17:05 -08:00

sampling

Fix the overlap for xgrammar (#2377)

2024-12-06 05:49:29 -08:00

_custom_ops.py

adapt vllm distributed module to sglang (#2244)

2024-12-01 15:54:52 +08:00

conversation.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

hf_transformers_utils.py

Update model_loader deps and qqq quantization deps (#2220) (#2318)

2024-12-02 23:22:13 +08:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torch.compile (#2352)

2024-12-04 17:26:00 -08:00

server_args.py

Optimize Triton decoding kernel for long context (#2394)

2024-12-08 01:17:37 -08:00

server.py

Use proc.join instead of busy waiting (#2374)

2024-12-06 02:01:23 -08:00

utils.py

Add more support for intel Gaudi accelerators (#2357)

2024-12-06 01:16:33 -08:00