EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Garbage collector regression in the online server (#10621)

Browse Source
This commit is contained in:
brayden-hai
2025-09-18 11:52:35 -07:00
committed by GitHub
parent 9a5c42f9ad
commit 67073dde85
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/sglang/srt/model_executor/cuda_graph_runner.py
View File

@@ -100,6 +100,7 @@ def freeze_gc(enable_cudagraph_gc: bool):
finally: finally:
if should_freeze: if should_freeze:
gc.unfreeze() gc.unfreeze()
gc.collect()
def _to_torch(model: torch.nn.Module, reverse: bool, num_tokens: int): def _to_torch(model: torch.nn.Module, reverse: bool, num_tokens: int):