Fix typos (#4368)
This commit is contained in:
@@ -14,7 +14,7 @@
|
||||
"""
|
||||
The entry point of inference server. (SRT = SGLang Runtime)
|
||||
|
||||
This file implements HTTP APIs for the inferenc engine via fastapi.
|
||||
This file implements HTTP APIs for the inference engine via fastapi.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
@@ -19,7 +19,7 @@ from sglang.srt.torch_memory_saver_adapter import TorchMemorySaverAdapter
|
||||
Memory pool.
|
||||
|
||||
SGLang has two levels of memory pool.
|
||||
ReqToTokenPool maps a a request to its token locations.
|
||||
ReqToTokenPool maps a request to its token locations.
|
||||
TokenToKVPoolAllocator manages the indices to kv cache data.
|
||||
KVCache actually holds the physical kv cache.
|
||||
"""
|
||||
|
||||
Reference in New Issue
Block a user