Liangsheng Yin b1a3a454ee add --disable-disk-cache (#160) ...

Co-authored-by: Ja1Zhou <50169346+Ja1Zhou@users.noreply.github.com>

2024-02-08 00:50:12 +08:00

..

infer_batch.py

Support decode token logprobs (#130)

2024-02-06 12:24:55 -08:00

manager.py

Improve the control of streaming and improve the first token latency in streaming (#117)

2024-01-29 17:05:42 -08:00

model_rpc.py

add --disable-disk-cache (#160)

2024-02-08 00:50:12 +08:00

model_runner.py

[Submodule] Change FlashInfer to import (#156)

2024-02-06 19:28:29 -08:00

radix_cache.py

Flush Cache API (#103)

2024-01-25 21:32:59 -08:00

scheduler.py

Fix the possible bug of decode out of memory (#36)

2024-01-19 11:01:15 -08:00