EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Update MMMU Benchmark instructions (#4694)

Browse Source
This commit is contained in:
Ravi Theja
2025-03-28 03:14:16 +05:30
committed by GitHub
parent 188105a21b
commit e6e4d02245
2 changed files with 11 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
benchmark/mmmu/README.md
View File

@@ -2,12 +2,16 @@
### Evaluate sglang
```
python -m sglang.launch_server --model-path Qwen/Qwen2-VL-7B-Instruct --port 30000
```
Host the VLM:
```
python benchmark/mmmu/bench_sglang.py --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl --port 30000
python -m sglang.launch_server --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl --port 30000
```
Benchmark:
```
python benchmark/mmmu/bench_sglang.py --port 30000
```
It's recommended to reduce the memory usage by appending something ike `--mem-fraction-static 0.6` to the command above.

4
benchmark/mmmu/bench_sglang.py
View File

@@ -2,7 +2,9 @@
Bench the sglang-hosted vLM with benchmark MMMU
Usage:
python benchmark/mmmu/bench_sglang.py --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl
Host the VLM: python -m sglang.launch_server --model-path Qwen/Qwen2-VL-7B-Instruct --chat-template qwen2-vl --port 30000
Benchmark: python benchmark/mmmu/bench_sglang.py --port 30000
The eval output will be logged
"""