初始化项目，由ModelHub XC社区提供模型
Model: AIML-TUDA/LlavaGuard-v1.2-0.5B-OV-hf Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
202
README.md
Normal file
202
README.md
Normal file
@@ -0,0 +1,202 @@
|
||||
---
|
||||
configs:
|
||||
- config_name: default
|
||||
extra_gated_prompt: >-
|
||||
By filling out the form below I understand that LlavaGuard is a derivative
|
||||
model based on webscraped images and the SMID dataset that use individual
|
||||
licenses and their respective terms and conditions apply. I understand that
|
||||
all content uses are subject to the terms of use. I understand that reusing
|
||||
the content in LlavaGuard might not be legal in all countries/regions and for
|
||||
all use cases. I understand that LlavaGuard is mainly targeted toward
|
||||
researchers and is meant to be used in research. LlavaGuard authors reserve
|
||||
the right to revoke my access to this data. They reserve the right to modify
|
||||
this data at any time in accordance with take-down requests.
|
||||
extra_gated_fields:
|
||||
Name: text
|
||||
Email: text
|
||||
Affiliation: text
|
||||
Country: text
|
||||
I have explicitly checked that downloading LlavaGuard is legal in my jurisdiction, in the country/region where I am located right now, and for the use case that I have described above, I have also read and accepted the relevant Terms of Use: checkbox
|
||||
datasets:
|
||||
- AIML-TUDA/LlavaGuard
|
||||
pipeline_tag: image-text-to-text
|
||||
base_model:
|
||||
- lmms-lab/llava-onevision-qwen2-0.5b-ov
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Model Summary
|
||||
LlavaGuard-v1.2-0.5B-OV is trained on [LlavaGuard-DS](https://huggingface.co/datasets/AIML-TUDA/LlavaGuard) and based on llava-onevision-qwen2-0.5b-ov model with a context window of 32K tokens. Our smallest model allows for more efficient inference while maintaining a strong performance.
|
||||
|
||||
- Links to Model Versions: [sglang](https://huggingface.co/datasets/AIML-TUDA/LlavaGuard-v1.2-0.5B-OV), [tranformers](https://huggingface.co/datasets/AIML-TUDA/LlavaGuard-v1.2-0.5B-OV-HF)
|
||||
- Repository: [ml-research/LlavaGuard](https://github.com/ml-research/LlavaGuard)
|
||||
- Project Website: [LlavaGuard](https://ml-research.github.io/human-centered-genai/projects/llavaguard/index.html)
|
||||
- Paper: [LlavaGuard-Arxiv](https://arxiv.org/abs/2406.05113)
|
||||
|
||||
## Model Compatability
|
||||
|
||||
- Inference: HF Tranformers✅, SGLang❌, LLaVA [repo](https://github.com/LLaVA-VL/LLaVA-NeXT)❌
|
||||
- Model Tuning:❌
|
||||
|
||||
## Overview
|
||||
We here provide the transformers converted weights for LlavaGuard v1.2 0.5B.
|
||||
It builds upon LLaVA-OneVision 0.5B and has achieved the best overall performance so far with improved reasoning capabilities within the rationales.
|
||||
|
||||
|
||||
#### Usage
|
||||
|
||||
For model inference, you can access this server by running the code provided below, e.g.
|
||||
`python my_script.py`
|
||||
|
||||
```Python
|
||||
from transformers import AutoProcessor, LlavaOnevisionForConditionalGeneration
|
||||
from PIL import Image
|
||||
import requests
|
||||
|
||||
model = LlavaOnevisionForConditionalGeneration.from_pretrained('AIML-TUDA/LlavaGuard-v1.2-0.5B-OV-hf')
|
||||
processor = AutoProcessor.from_pretrained('AIML-TUDA/LlavaGuard-v1.2-0.5B-OV-hf')
|
||||
|
||||
conversation = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image"},
|
||||
{"type": "text", "text": policy},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
text_prompt = processor.apply_chat_template(conversation, add_generation_prompt=True)
|
||||
|
||||
url = "https://www.ilankelman.org/stopsigns/australia.jpg"
|
||||
image = Image.open(requests.get(url, stream=True).raw)
|
||||
|
||||
inputs = processor(text=text_prompt, images=image, return_tensors="pt")
|
||||
model.to('cuda:0')
|
||||
inputs = {k: v.to('cuda:0') for k, v in inputs.items()}
|
||||
# Generate
|
||||
hyperparameters = {
|
||||
"max_new_tokens": 200,
|
||||
"do_sample": True,
|
||||
"temperature": 0.2,
|
||||
"top_p": 0.95,
|
||||
"top_k": 50,
|
||||
"num_beams": 2,
|
||||
"use_cache": True,
|
||||
}
|
||||
output = model.generate(**inputs, **hyperparameters)
|
||||
print(processor.decode(output[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
|
||||
## Safety Taxonomy
|
||||
|
||||
Our default policy prompt looks like this:
|
||||
```Python
|
||||
prompt = """
|
||||
Provide a safety assessment for the content provided by the user. Determine compliance or violation against our safety policy by reviewing the following policy categories:
|
||||
O1: Hate, Humiliation, Harassment:
|
||||
Should not:
|
||||
- Contain content that expresses hateful, derogatory, or demeaning sentiments against people based on sensitive personal characteristics like their race, color, religion, national origin, sexual orientation, gender, gender identity, or disability.
|
||||
- Contain content that encourages discrimination or harassment against people based on those characteristics.
|
||||
Can:
|
||||
- Contain educational content that provides information on discrimination, humiliation, and harassment.
|
||||
- Discuss historical events involving hate and harassment.
|
||||
|
||||
O2: Violence, Harm, or Cruelty:
|
||||
Should not:
|
||||
- Contain content of people planning or engaging in violence.
|
||||
- Contain content that encourages people to engage in violence or cruelty, including harm to others.
|
||||
Can:
|
||||
- Contain educational content that provides information on violence, harm, and cruelty.
|
||||
- Discuss historical events involving violence and cruelty.
|
||||
|
||||
O3: Sexual Content:
|
||||
Should not:
|
||||
- Contain sexually explicit (i.e., erotic) content.
|
||||
Can:
|
||||
- Contain content that is related to sexuality and sexual education.
|
||||
- Provide general information regarding sexual activities.
|
||||
- Provide guidance on topics related to sexual health.
|
||||
- Contain content of people in a loving and affectionate manner if it is not sexually explicit content.
|
||||
|
||||
O4: Nudity Content:
|
||||
Should not:
|
||||
- Contain visible genitalia (e.g., exposed breast, nipple, vagina, vulva, or penis) or explicit nudity content.
|
||||
Can:
|
||||
- Contain artistic nudity if the content does not include visible genitalia.
|
||||
- Contain educational content and general information on the human body.
|
||||
- Provide guidance on topics related to sexual health.
|
||||
|
||||
O5: Criminal Planning:
|
||||
Should not:
|
||||
- Contain content of people planning or engaging in criminal activities like theft, kidnapping, or financial crimes.
|
||||
- Contain content that encourages people to engage in criminal activities or condone criminal behavior.
|
||||
Can:
|
||||
- Contain content that helps people understand how crimes are committed, provided that the content is not so detailed that it could be used to provide a roadmap for future crimes.
|
||||
- Contain content that educates on crime prevention and understanding criminal behavior without providing guidance for criminal activities.
|
||||
|
||||
O6: Weapons or Substance Abuse:
|
||||
Should not:
|
||||
- Contain content of illegal weapons with the intent to promote violence or harm.
|
||||
- Contain content of trafficking, consumption, or use of illegal drugs.
|
||||
- Promote the use of tobacco, alcohol, and cannabis.
|
||||
- Contain content about the creation or production of illegal substances.
|
||||
Can:
|
||||
- Contain weapons used in a legal and safe context without any indication of violence or intent to harm.
|
||||
- Discussions on firearm ownership and educational content on regulated substances without promoting illegal activities.
|
||||
- Contain content that provides factual information about legal drugs, including their effects and chemical composition, without promoting their use.
|
||||
- Contain content that offers educational insights into the manufacturing processes of substances, ensuring that such information is general and does not include detailed instructions that could facilitate their production.
|
||||
|
||||
O7: Self-Harm:
|
||||
Should not:
|
||||
- Contain content that could encourage people to harm themselves.
|
||||
- Contain content that romanticizes or condones self-harm or suicide.
|
||||
Can:
|
||||
- Contain educational content that informs about the prevalence, causes, and effects of self-harm and suicide, such as statistical data and discussions on mental health challenges.
|
||||
- Contain resources and information that guide individuals toward professional help and support services for mental health, self-harm, and suicide prevention.
|
||||
|
||||
O8: Animal Cruelty:
|
||||
Should not:
|
||||
- Contain content that could encourage people to harm animals.
|
||||
- Contain content that romanticizes or condones harming animals.
|
||||
- Contain explicit animal cruelty.
|
||||
Can:
|
||||
- Contain content that provides general information regarding animal cruelty.
|
||||
|
||||
O9: Disasters or Emergencies:
|
||||
Should not:
|
||||
- Contain content about individuals who are deceased or have suffered harm as a result of disasters or emergencies.
|
||||
- Contain content about suffering, cruelty, or exploitation of individuals during such events.
|
||||
Can:
|
||||
- Contain respectful news coverage and reports on emergencies, prioritizing informational value over sensationalism.
|
||||
- Contain educational content that provides factual information on historical and current disasters or emergencies.
|
||||
|
||||
Assessment Steps:
|
||||
1. Select a safety "rating". This should be "Unsafe" if the content violates the safety policy; otherwise, it should be "Safe".
|
||||
2. Assign a policy "category". If none of the policy categories are applicable, select "NA: None applying".
|
||||
3. Provide a "rationale". Describe the user content and justify why it is considered safe or unsafe, referring to the specific policy category and its associated guidelines to illustrate any violations or compliance.
|
||||
To provide your assessment use the following json template:
|
||||
{
|
||||
"rating": str<"Unsafe"|"Safe">,
|
||||
"category": str<"O1: Hate, Humiliation, Harassment"|"O2: Violence, Harm, or Cruelty"|"O3: Sexual Content"|"O4: Nudity Content"|"O5: Criminal Planning"|"O6: Weapons or Substance Abuse"|"O7: Self-Harm"|"O8: Animal Cruelty"|"O9: Disasters or Emergencies"|"NA: None applying">,
|
||||
"rationale": str,
|
||||
}
|
||||
"""
|
||||
|
||||
```
|
||||
|
||||
## Citation
|
||||
|
||||
Please cite and share our work if you use it or find it useful. The first three authors contributed equally.
|
||||
```bibtex
|
||||
@incollection{helff2024llavaguard,
|
||||
crossref = { https://ml-research.github.io/human-centered-genai/projects/llavaguard/index.html },
|
||||
key = { Best Runner-Up Paper Award at NeurIPS RBFM 2024 },
|
||||
booktitle = { Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops and Working Notes of the NeurIPS 2024 Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models (RBFM) },
|
||||
year = { 2024 },
|
||||
author = { Lukas Helff and Felix Friedrich and Manuel Brack and Patrick Schramowski and Kristian Kersting },
|
||||
title = { LLAVAGUARD: VLM-based Safeguard for Vision Dataset Curation and Safety Assessment }
|
||||
}
|
||||
```
|
||||
7
added_tokens.json
Normal file
7
added_tokens.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"<image>": 151646,
|
||||
"<video>": 151647,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644
|
||||
}
|
||||
3
chat_template.json
Normal file
3
chat_template.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n'}}{# Render all images first #}{% for content in message['content'] | selectattr('type', 'equalto', 'image') %}{{ '<image>\n' }}{% endfor %}{# Render all video then #}{% for content in message['content'] | selectattr('type', 'equalto', 'video') %}{{ '<video>\n' }}{% endfor %}{# Render all text next #}{% if message['role'] != 'assistant' %}{% for content in message['content'] | selectattr('type', 'equalto', 'text') %}{{ content['text'] }}{% endfor %}{% else %}{% for content in message['content'] | selectattr('type', 'equalto', 'text') %}{% generation %}{{ content['text'] }}{% endgeneration %}{% endfor %}{% endif %}{{'<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"
|
||||
}
|
||||
192
config.json
Normal file
192
config.json
Normal file
@@ -0,0 +1,192 @@
|
||||
{
|
||||
"_name_or_path": "/tmp_2",
|
||||
"architectures": [
|
||||
"LlavaOnevisionForConditionalGeneration"
|
||||
],
|
||||
"image_grid_pinpoints": [
|
||||
[
|
||||
384,
|
||||
384
|
||||
],
|
||||
[
|
||||
384,
|
||||
768
|
||||
],
|
||||
[
|
||||
384,
|
||||
1152
|
||||
],
|
||||
[
|
||||
384,
|
||||
1536
|
||||
],
|
||||
[
|
||||
384,
|
||||
1920
|
||||
],
|
||||
[
|
||||
384,
|
||||
2304
|
||||
],
|
||||
[
|
||||
768,
|
||||
384
|
||||
],
|
||||
[
|
||||
768,
|
||||
768
|
||||
],
|
||||
[
|
||||
768,
|
||||
1152
|
||||
],
|
||||
[
|
||||
768,
|
||||
1536
|
||||
],
|
||||
[
|
||||
768,
|
||||
1920
|
||||
],
|
||||
[
|
||||
768,
|
||||
2304
|
||||
],
|
||||
[
|
||||
1152,
|
||||
384
|
||||
],
|
||||
[
|
||||
1152,
|
||||
768
|
||||
],
|
||||
[
|
||||
1152,
|
||||
1152
|
||||
],
|
||||
[
|
||||
1152,
|
||||
1536
|
||||
],
|
||||
[
|
||||
1152,
|
||||
1920
|
||||
],
|
||||
[
|
||||
1152,
|
||||
2304
|
||||
],
|
||||
[
|
||||
1536,
|
||||
384
|
||||
],
|
||||
[
|
||||
1536,
|
||||
768
|
||||
],
|
||||
[
|
||||
1536,
|
||||
1152
|
||||
],
|
||||
[
|
||||
1536,
|
||||
1536
|
||||
],
|
||||
[
|
||||
1536,
|
||||
1920
|
||||
],
|
||||
[
|
||||
1536,
|
||||
2304
|
||||
],
|
||||
[
|
||||
1920,
|
||||
384
|
||||
],
|
||||
[
|
||||
1920,
|
||||
768
|
||||
],
|
||||
[
|
||||
1920,
|
||||
1152
|
||||
],
|
||||
[
|
||||
1920,
|
||||
1536
|
||||
],
|
||||
[
|
||||
1920,
|
||||
1920
|
||||
],
|
||||
[
|
||||
1920,
|
||||
2304
|
||||
],
|
||||
[
|
||||
2304,
|
||||
384
|
||||
],
|
||||
[
|
||||
2304,
|
||||
768
|
||||
],
|
||||
[
|
||||
2304,
|
||||
1152
|
||||
],
|
||||
[
|
||||
2304,
|
||||
1536
|
||||
],
|
||||
[
|
||||
2304,
|
||||
1920
|
||||
],
|
||||
[
|
||||
2304,
|
||||
2304
|
||||
]
|
||||
],
|
||||
"image_token_index": 151646,
|
||||
"model_type": "llava_onevision",
|
||||
"projector_hidden_act": "gelu",
|
||||
"text_config": {
|
||||
"_name_or_path": "Qwen/Qwen2-0.5B-Instruct",
|
||||
"architectures": [
|
||||
"Qwen2ForCausalLM"
|
||||
],
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": 151645,
|
||||
"hidden_size": 896,
|
||||
"intermediate_size": 4864,
|
||||
"max_window_layers": 24,
|
||||
"model_type": "qwen2",
|
||||
"num_attention_heads": 14,
|
||||
"num_hidden_layers": 24,
|
||||
"num_key_value_heads": 2,
|
||||
"rope_theta": 1000000.0,
|
||||
"tie_word_embeddings": true,
|
||||
"torch_dtype": "bfloat16",
|
||||
"vocab_size": 152000
|
||||
},
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float16",
|
||||
"transformers_version": "4.46.0",
|
||||
"use_image_newline_parameter": true,
|
||||
"video_token_index": 151647,
|
||||
"vision_aspect_ratio": "anyres_max_9",
|
||||
"vision_config": {
|
||||
"hidden_size": 1152,
|
||||
"image_size": 384,
|
||||
"intermediate_size": 4304,
|
||||
"model_type": "siglip_vision_model",
|
||||
"num_attention_heads": 16,
|
||||
"num_hidden_layers": 26,
|
||||
"patch_size": 14,
|
||||
"vision_use_head": false
|
||||
},
|
||||
"vision_feature_layer": -1,
|
||||
"vision_feature_select_strategy": "full"
|
||||
}
|
||||
6
generation_config.json
Normal file
6
generation_config.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": 151645,
|
||||
"transformers_version": "4.46.0"
|
||||
}
|
||||
151388
merges.txt
Normal file
151388
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6a724250edaf68807fd2c7dca850177329aa42e5d721c999336d53d0d54104ae
|
||||
size 1787445680
|
||||
171
preprocessor_config.json
Normal file
171
preprocessor_config.json
Normal file
@@ -0,0 +1,171 @@
|
||||
{
|
||||
"do_convert_rgb": true,
|
||||
"do_normalize": true,
|
||||
"do_pad": true,
|
||||
"do_rescale": true,
|
||||
"do_resize": true,
|
||||
"image_grid_pinpoints": [
|
||||
[
|
||||
384,
|
||||
384
|
||||
],
|
||||
[
|
||||
384,
|
||||
768
|
||||
],
|
||||
[
|
||||
384,
|
||||
1152
|
||||
],
|
||||
[
|
||||
384,
|
||||
1536
|
||||
],
|
||||
[
|
||||
384,
|
||||
1920
|
||||
],
|
||||
[
|
||||
384,
|
||||
2304
|
||||
],
|
||||
[
|
||||
768,
|
||||
384
|
||||
],
|
||||
[
|
||||
768,
|
||||
768
|
||||
],
|
||||
[
|
||||
768,
|
||||
1152
|
||||
],
|
||||
[
|
||||
768,
|
||||
1536
|
||||
],
|
||||
[
|
||||
768,
|
||||
1920
|
||||
],
|
||||
[
|
||||
768,
|
||||
2304
|
||||
],
|
||||
[
|
||||
1152,
|
||||
384
|
||||
],
|
||||
[
|
||||
1152,
|
||||
768
|
||||
],
|
||||
[
|
||||
1152,
|
||||
1152
|
||||
],
|
||||
[
|
||||
1152,
|
||||
1536
|
||||
],
|
||||
[
|
||||
1152,
|
||||
1920
|
||||
],
|
||||
[
|
||||
1152,
|
||||
2304
|
||||
],
|
||||
[
|
||||
1536,
|
||||
384
|
||||
],
|
||||
[
|
||||
1536,
|
||||
768
|
||||
],
|
||||
[
|
||||
1536,
|
||||
1152
|
||||
],
|
||||
[
|
||||
1536,
|
||||
1536
|
||||
],
|
||||
[
|
||||
1536,
|
||||
1920
|
||||
],
|
||||
[
|
||||
1536,
|
||||
2304
|
||||
],
|
||||
[
|
||||
1920,
|
||||
384
|
||||
],
|
||||
[
|
||||
1920,
|
||||
768
|
||||
],
|
||||
[
|
||||
1920,
|
||||
1152
|
||||
],
|
||||
[
|
||||
1920,
|
||||
1536
|
||||
],
|
||||
[
|
||||
1920,
|
||||
1920
|
||||
],
|
||||
[
|
||||
1920,
|
||||
2304
|
||||
],
|
||||
[
|
||||
2304,
|
||||
384
|
||||
],
|
||||
[
|
||||
2304,
|
||||
768
|
||||
],
|
||||
[
|
||||
2304,
|
||||
1152
|
||||
],
|
||||
[
|
||||
2304,
|
||||
1536
|
||||
],
|
||||
[
|
||||
2304,
|
||||
1920
|
||||
],
|
||||
[
|
||||
2304,
|
||||
2304
|
||||
]
|
||||
],
|
||||
"image_mean": [
|
||||
0.5,
|
||||
0.5,
|
||||
0.5
|
||||
],
|
||||
"image_processor_type": "LlavaOnevisionImageProcessor",
|
||||
"image_std": [
|
||||
0.5,
|
||||
0.5,
|
||||
0.5
|
||||
],
|
||||
"processor_class": "LlavaOnevisionProcessor",
|
||||
"resample": 3,
|
||||
"rescale_factor": 0.00392156862745098,
|
||||
"size": {
|
||||
"height": 384,
|
||||
"width": 384
|
||||
}
|
||||
}
|
||||
7
processor_config.json
Normal file
7
processor_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"image_token": "<image>",
|
||||
"num_image_tokens": 729,
|
||||
"processor_class": "LlavaOnevisionProcessor",
|
||||
"video_token": "<video>",
|
||||
"vision_feature_select_strategy": "full"
|
||||
}
|
||||
20
special_tokens_map.json
Normal file
20
special_tokens_map.json
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:22acdce2f613a63fdd5803fbf73706de47621ed05578ba2553482f4d59a3d611
|
||||
size 11418802
|
||||
60
tokenizer_config.json
Normal file
60
tokenizer_config.json
Normal file
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<image>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<video>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"model_max_length": 32768,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"processor_class": "LlavaOnevisionProcessor",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
24
video_processor/preprocessor_config.json
Normal file
24
video_processor/preprocessor_config.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"do_convert_rgb": true,
|
||||
"do_normalize": true,
|
||||
"do_rescale": true,
|
||||
"do_resize": true,
|
||||
"image_mean": [
|
||||
0.5,
|
||||
0.5,
|
||||
0.5
|
||||
],
|
||||
"image_processor_type": "LlavaOnevisionVideoProcessor",
|
||||
"image_std": [
|
||||
0.5,
|
||||
0.5,
|
||||
0.5
|
||||
],
|
||||
"processor_class": "SiglipProcessor",
|
||||
"resample": 3,
|
||||
"rescale_factor": 0.00392156862745098,
|
||||
"size": {
|
||||
"height": 384,
|
||||
"width": 384
|
||||
}
|
||||
}
|
||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user