EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

[Fix] Fix OOM in llava base class (#1249)

Browse Source
This commit is contained in:
Lianmin Zheng
2024-08-28 08:38:50 -07:00
parent c411f32e1c
commit 0a97d7962d
3 changed files with 30 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/runtime/llava_onevision/http_llava_onevision_test.py
View File

@@ -1,7 +1,7 @@
"""
Usage:
python3 -m sglang.launch_server --model-path lmms-lab/llava-onevision-qwen2-72b-ov --port=30000 --tp-size=8 --chat-template=chatml-llava --chunked-prefill-size=16384
python3 -m sglang.launch_server --model-path lmms-lab/llava-onevision-qwen2-72b-ov --port=30000 --tp-size=8 --chat-template=chatml-llava
python3 http_llava_onevision_test.py
"""