Ying Sheng 8b48496aaf Revert "Revert "Add simple CPU offloading support"" (#2253) ...

Co-authored-by: Jani Monoses <jani.monoses@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

2024-11-28 23:58:54 -08:00

..

configs

Use an env var SGLANG_SET_CPU_AFFINITY to set cpu affinity; turn it off by default (#2222)

2024-11-27 02:52:46 -08:00

constrained

Update XGrammar to the latest API (#2176)

2024-11-25 15:58:30 -08:00

layers

Rename tuned MI300X config files for fused_moe_triton (#2228)

2024-11-27 21:18:51 -08:00

lora

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

managers

[fix] Fix prefix caching for multi-image/video (#2239)

2024-11-28 12:08:13 -08:00

mem_cache

Update docs (#1839)

2024-10-30 02:49:08 -07:00

metrics

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_executor

Revert "Revert "Add simple CPU offloading support"" (#2253)

2024-11-28 23:58:54 -08:00

models

Revert "Revert "Add simple CPU offloading support"" (#2253)

2024-11-28 23:58:54 -08:00

openai_api

fix typo prompts (#2224)

2024-11-27 12:07:00 -08:00

sampling

[Fix] Avoid calling fill_vocab_mask for terminated requests (#2175)

2024-11-25 17:31:25 +08:00

conversation.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

hf_transformers_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

[minor] Clean up unused imports (#2122)

2024-11-22 01:50:42 -08:00

server_args.py

Revert "Revert "Add simple CPU offloading support"" (#2253)

2024-11-28 23:58:54 -08:00

server.py

Crash the server correctly during error (#2231)

2024-11-28 00:22:39 -08:00

utils.py

Revert "Revert "Add simple CPU offloading support"" (#2253)

2024-11-28 23:58:54 -08:00