EngineX-Hygon/sglang
5
0
Fork 0
Code Issues Pull Requests Actions 7 Projects Releases Wiki Activity

Fix engine unit test (#1701)

Browse Source
This commit is contained in:
Lianmin Zheng
2024-10-17 09:53:32 -07:00
committed by GitHub
parent 7feba41584
commit dd3809fad8
4 changed files with 12 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
benchmark/llava_bench/README.md
View File

@@ -17,7 +17,7 @@ pip3 install "torch>=2.1.2" "transformers>=4.36" pillow
### Benchmark sglang
Launch a server
```
python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000
python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.6-vicuna-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000
```
Run benchmark

2
benchmark/llava_bench/bench_sglang.py
View File

@@ -20,7 +20,7 @@ def image_qa(s, image_file, question):
def main(args):
lines = read_jsonl(args.question_file)[: args.num_questions]
lines = list(read_jsonl(args.question_file))[: args.num_questions]
arguments = [
{
"image_file": os.path.abspath(args.image_folder + "/" + l["image"]),