Cleanup readme, llava examples, usage examples and nccl init (#1194)
This commit is contained in:
@@ -1,5 +1,8 @@
|
||||
# Sampling Parameters in SGLang Runtime
|
||||
This doc describes the sampling parameters of the SGLang Runtime.
|
||||
It is the low-level endpoint of the runtime.
|
||||
If you want a high-level endpoint that can automatically handle chat templates, consider using the [OpenAI Compatible API
|
||||
](https://github.com/sgl-project/sglang?tab=readme-ov-file#openai-compatible-api).
|
||||
|
||||
The `/generate` endpoint accepts the following arguments in the JSON format.
|
||||
|
||||
@@ -140,7 +143,7 @@ print("")
|
||||
|
||||
Launch a server
|
||||
```
|
||||
python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.6-vicuna-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --chat-template vicuna_v1.1 --port 30000
|
||||
python3 -m sglang.launch_server --model-path lmms-lab/llava-onevision-qwen2-7b-ov --chat-template chatml-llava
|
||||
```
|
||||
|
||||
Download an image
|
||||
@@ -155,7 +158,9 @@ import requests
|
||||
response = requests.post(
|
||||
"http://localhost:30000/generate",
|
||||
json={
|
||||
"text": "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions. USER: <image>\nDescribe this picture ASSISTANT:",
|
||||
"text": "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n"
|
||||
"<|im_start|>user\n<image>\nDescribe this image in a very short sentence.<|im_end|>\n"
|
||||
"<|im_start|>assistant\n",
|
||||
"image_data": "example_image.png",
|
||||
"sampling_params": {
|
||||
"temperature": 0,
|
||||
|
||||
Reference in New Issue
Block a user