初始化项目，由ModelHub XC社区提供模型
Model: internlm/Intern-S1-mini Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
464
README.md
Normal file
464
README.md
Normal file
@@ -0,0 +1,464 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
pipeline_tag: image-text-to-text
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
## Intern-S1-mini
|
||||
|
||||
<div align="center">
|
||||
<img src="https://cdn-uploads.huggingface.co/production/uploads/642695e5274e7ad464c8a5ba/E43cgEXBRWjVJlU_-hdh6.png" />
|
||||
|
||||
<div> </div>
|
||||
|
||||
[💻Github Repo](https://github.com/InternLM/Intern-S1) • [🤗Model Collections](https://huggingface.co/collections/internlm/intern-s1-6882e325e8ac1c58ba108aa5) • [📜Technical Report](https://arxiv.org/abs/2508.15763) • [🏠Project Page](https://chat.intern-ai.org.cn/)
|
||||
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
👋 join us on <a href="https://discord.gg/xa29JuW87d" target="_blank">Discord</a> and <a href="https://cdn.vansin.top/intern-s1.jpg" target="_blank">WeChat</a>
|
||||
</p>
|
||||
|
||||
|
||||
|
||||
## Introduction
|
||||
|
||||
We introduce **Intern-S1-mini**, a lightweight open-source multimodal reasoning model based on the same techniques as **[Intern-S1](https://huggingface.co/internlm/Intern-S1)**.
|
||||
Built upon an 8B dense language model (Qwen3) and a 0.3B Vision encoder (InternViT), Intern-S1-mini has been further pretrained on **5 trillion tokens** of multimodal data, including over **2.5 trillion scientific-domain tokens**. This enables the model to retain strong general capabilities while excelling in specialized scientific domains such as **interpreting chemical structures, understanding protein sequences, and planning compound synthesis routes**, making Intern-S1-mini to be a capable research assistant for real-world scientific applications.
|
||||
|
||||
## Features
|
||||
|
||||
- Strong performance across language and vision reasoning benchmarks, especially scientific tasks.
|
||||
|
||||
- Continuously pretrained on a massive 5T token dataset, with over 50% specialized scientific data, embedding deep domain expertise.
|
||||
|
||||
- Dynamic tokenizer enables native understanding of molecular formulas and protein sequences.
|
||||
|
||||
## Performance
|
||||
|
||||
We evaluate the Intern-S1-mini on various benchmarks including general datasets and scientific datasets. We report the performance comparison with the recent VLMs and LLMs below.
|
||||
|
||||
|
||||
| | | Intern-S1-mini | Qwen3-8B | GLM-4.1V | MiMo-VL-7B-RL-2508 |
|
||||
|------------|----------------|-------------------|----------|----------|--------------------|
|
||||
| General | MMLU-Pro | **74.78** | 73.7 | 57.1 | 73.93 |
|
||||
| | MMMU | **72.33** | N/A | 69.9 | 70.4 |
|
||||
| | MMStar | 65.2 | N/A | 71.5 | 72.9 |
|
||||
| | GPQA | **65.15** | 62 | 50.32 | 60.35 |
|
||||
| | AIME2024 | **84.58** | 76 | 36.2 | 72.6 |
|
||||
| | AIME2025 | **80** | 67.3 | 32 | 64.4 |
|
||||
| | MathVision | 51.41 | N/A | 53.9 | 54.5 |
|
||||
| | MathVista | 70.3 | N/A | 80.7 | 79.4 |
|
||||
| | IFEval | 81.15 | 85 | 71.53 | 71.4 |
|
||||
| | | | | | |
|
||||
| Scientific | SFE | 35.84 | N/A | 43.2 | 43.9 |
|
||||
| | Physics | **28.76** | N/A | 28.3 | 28.2 |
|
||||
| | SmolInstruct | **32.2** | 17.6 | 18.1 | 16.11 |
|
||||
| | ChemBench | **76.47** | 61.1 | 56.2 | 66.78 |
|
||||
| | MatBench | **61.55** | 45.24 | 54.3 | 46.9 |
|
||||
| | MicroVQA | **56.62** | N/A | 50.2 | 50.96 |
|
||||
| | ProteinLMBench | 58.47 | 59.1 | 58.3 | 59.8 |
|
||||
| | MSEarthMCQ | **58.12** | N/A | 50.3 | 47.3 |
|
||||
| | XLRS-Bench | **51.63** | N/A | 49.8 | 12.29 |
|
||||
|
||||
|
||||
We use the [OpenCompass](https://github.com/open-compass/OpenCompass/) and [VLMEvalkit](https://github.com/open-compass/vlmevalkit) to evaluate all models.
|
||||
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Sampling Parameters
|
||||
|
||||
We recommend using the following hyperparameters to ensure better results
|
||||
|
||||
```python
|
||||
top_p = 1.0
|
||||
top_k = 50
|
||||
min_p = 0.0
|
||||
temperature = 0.8
|
||||
```
|
||||
|
||||
### Transformers
|
||||
|
||||
The following provides demo code illustrating how to generate based on text and multimodal inputs.
|
||||
|
||||
> **Please use transformers>=4.55.2 to ensure the model works normally.**
|
||||
|
||||
#### Text input
|
||||
|
||||
```python
|
||||
from transformers import AutoProcessor, AutoModelForCausalLM
|
||||
import torch
|
||||
|
||||
model_name = "internlm/Intern-S1-mini"
|
||||
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True)
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "tell me about an interesting physical phenomenon."},
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device, dtype=torch.bfloat16)
|
||||
|
||||
generate_ids = model.generate(**inputs, max_new_tokens=32768)
|
||||
decoded_output = processor.decode(generate_ids[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True)
|
||||
print(decoded_output)
|
||||
```
|
||||
|
||||
#### Image input
|
||||
|
||||
```python
|
||||
from transformers import AutoProcessor, AutoModelForCausalLM
|
||||
import torch
|
||||
|
||||
model_name = "internlm/Intern-S1-mini"
|
||||
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True)
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image", "url": "http://images.cocodataset.org/val2017/000000039769.jpg"},
|
||||
{"type": "text", "text": "Please describe the image explicitly."},
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device, dtype=torch.bfloat16)
|
||||
|
||||
generate_ids = model.generate(**inputs, max_new_tokens=32768)
|
||||
decoded_output = processor.decode(generate_ids[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True)
|
||||
print(decoded_output)
|
||||
```
|
||||
|
||||
#### Video input
|
||||
|
||||
Please ensure that the decord video decoding library is installed via `pip install decord`. To avoid OOM, please install flash_attention and use at least 2 GPUS.
|
||||
|
||||
```python
|
||||
from transformers import AutoProcessor, AutoModelForCausalLM
|
||||
import torch
|
||||
|
||||
model_name = "internlm/Intern-S1-mini"
|
||||
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True)
|
||||
|
||||
messages = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "video",
|
||||
"url": "https://huggingface.co/datasets/hf-internal-testing/fixtures_videos/resolve/main/tennis.mp4",
|
||||
},
|
||||
{"type": "text", "text": "What type of shot is the man performing?"},
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
inputs = processor.apply_chat_template(
|
||||
messages,
|
||||
return_tensors="pt",
|
||||
add_generation_prompt=True,
|
||||
video_load_backend="decord",
|
||||
tokenize=True,
|
||||
return_dict=True,
|
||||
).to(model.device, dtype=torch.float16)
|
||||
|
||||
generate_ids = model.generate(**inputs, max_new_tokens=32768)
|
||||
decoded_output = processor.decode(generate_ids[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True)
|
||||
print(decoded_output)
|
||||
```
|
||||
|
||||
### Serving
|
||||
|
||||
The minimum hardware requirements for deploying Intern-S1 series models are:
|
||||
|
||||
| Model | A100(GPUs) | H800(GPUs) | H100(GPUs) | H200(GPUs) |
|
||||
| :---------------------------------------------------------------------: | :--------: | :--------: | :--------: | :--------: |
|
||||
| [internlm/Intern-S1-mini](https://huggingface.co/internlm/Intern-S1-mini) | 1 | 1 | 1 | 1 |
|
||||
| [internlm/Intern-S1-mini-FP8](https://huggingface.co/internlm/Intern-S1-mini-FP8) | - | 1 | 1 | 1 |
|
||||
|
||||
You can utilize one of the following LLM inference frameworks to create an OpenAI compatible server:
|
||||
|
||||
#### [lmdeploy (>=0.9.2.post1)](https://github.com/InternLM/lmdeploy)
|
||||
|
||||
```bash
|
||||
lmdeploy serve api_server internlm/Intern-S1-mini --reasoning-parser intern-s1 --tool-call-parser intern-s1
|
||||
```
|
||||
|
||||
#### [vllm (>=0.10.1)](https://github.com/vllm-project/vllm)
|
||||
|
||||
```bash
|
||||
vllm serve internlm/Intern-S1-mini --trust-remote-code
|
||||
```
|
||||
|
||||
#### [sglang](https://github.com/sgl-project/sglang)
|
||||
|
||||
```bash
|
||||
python3 -m sglang.launch_server \
|
||||
--model-path internlm/Intern-S1-mini \
|
||||
--trust-remote-code \
|
||||
--grammar-backend none
|
||||
```
|
||||
|
||||
#### ollama for local deployment:
|
||||
|
||||
```bash
|
||||
# install ollama
|
||||
curl -fsSL https://ollama.com/install.sh | sh
|
||||
# fetch model
|
||||
ollama pull internlm/interns1-mini
|
||||
# run model
|
||||
ollama run internlm/interns1-mini
|
||||
# then use openai client to call on http://localhost:11434/v1
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Tool Calling
|
||||
|
||||
Many Large Language Models (LLMs) now feature **Tool Calling**, a powerful capability that allows them to extend their functionality by interacting with external tools and APIs. This enables models to perform tasks like fetching up-to-the-minute information, running code, or calling functions within other applications.
|
||||
|
||||
A key advantage for developers is that a growing number of open-source LLMs are designed to be compatible with the OpenAI API. This means you can leverage the same familiar syntax and structure from the OpenAI library to implement tool calling with these open-source models. As a result, the code demonstrated in this tutorial is versatile—it works not just with OpenAI models, but with any model that follows the same interface standard.
|
||||
|
||||
To illustrate how this works, let's dive into a practical code example that uses tool calling to get the latest weather forecast (based on lmdeploy api server).
|
||||
|
||||
```python
|
||||
|
||||
from openai import OpenAI
|
||||
import json
|
||||
|
||||
|
||||
def get_current_temperature(location: str, unit: str = "celsius"):
|
||||
"""Get current temperature at a location.
|
||||
|
||||
Args:
|
||||
location: The location to get the temperature for, in the format "City, State, Country".
|
||||
unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])
|
||||
|
||||
Returns:
|
||||
the temperature, the location, and the unit in a dict
|
||||
"""
|
||||
return {
|
||||
"temperature": 26.1,
|
||||
"location": location,
|
||||
"unit": unit,
|
||||
}
|
||||
|
||||
|
||||
def get_temperature_date(location: str, date: str, unit: str = "celsius"):
|
||||
"""Get temperature at a location and date.
|
||||
|
||||
Args:
|
||||
location: The location to get the temperature for, in the format "City, State, Country".
|
||||
date: The date to get the temperature for, in the format "Year-Month-Day".
|
||||
unit: The unit to return the temperature in. Defaults to "celsius". (choices: ["celsius", "fahrenheit"])
|
||||
|
||||
Returns:
|
||||
the temperature, the location, the date and the unit in a dict
|
||||
"""
|
||||
return {
|
||||
"temperature": 25.9,
|
||||
"location": location,
|
||||
"date": date,
|
||||
"unit": unit,
|
||||
}
|
||||
|
||||
def get_function_by_name(name):
|
||||
if name == "get_current_temperature":
|
||||
return get_current_temperature
|
||||
if name == "get_temperature_date":
|
||||
return get_temperature_date
|
||||
|
||||
tools = [{
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': 'get_current_temperature',
|
||||
'description': 'Get current temperature at a location.',
|
||||
'parameters': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'location': {
|
||||
'type': 'string',
|
||||
'description': 'The location to get the temperature for, in the format \'City, State, Country\'.'
|
||||
},
|
||||
'unit': {
|
||||
'type': 'string',
|
||||
'enum': [
|
||||
'celsius',
|
||||
'fahrenheit'
|
||||
],
|
||||
'description': 'The unit to return the temperature in. Defaults to \'celsius\'.'
|
||||
}
|
||||
},
|
||||
'required': [
|
||||
'location'
|
||||
]
|
||||
}
|
||||
}
|
||||
}, {
|
||||
'type': 'function',
|
||||
'function': {
|
||||
'name': 'get_temperature_date',
|
||||
'description': 'Get temperature at a location and date.',
|
||||
'parameters': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'location': {
|
||||
'type': 'string',
|
||||
'description': 'The location to get the temperature for, in the format \'City, State, Country\'.'
|
||||
},
|
||||
'date': {
|
||||
'type': 'string',
|
||||
'description': 'The date to get the temperature for, in the format \'Year-Month-Day\'.'
|
||||
},
|
||||
'unit': {
|
||||
'type': 'string',
|
||||
'enum': [
|
||||
'celsius',
|
||||
'fahrenheit'
|
||||
],
|
||||
'description': 'The unit to return the temperature in. Defaults to \'celsius\'.'
|
||||
}
|
||||
},
|
||||
'required': [
|
||||
'location',
|
||||
'date'
|
||||
]
|
||||
}
|
||||
}
|
||||
}]
|
||||
|
||||
|
||||
|
||||
messages = [
|
||||
{'role': 'user', 'content': 'Today is 2024-11-14, What\'s the temperature in San Francisco now? How about tomorrow?'}
|
||||
]
|
||||
|
||||
openai_api_key = "EMPTY"
|
||||
openai_api_base = "http://0.0.0.0:23333/v1"
|
||||
client = OpenAI(
|
||||
api_key=openai_api_key,
|
||||
base_url=openai_api_base,
|
||||
)
|
||||
model_name = client.models.list().data[0].id
|
||||
response = client.chat.completions.create(
|
||||
model=model_name,
|
||||
messages=messages,
|
||||
max_tokens=32768,
|
||||
temperature=0.8,
|
||||
top_p=0.8,
|
||||
stream=False,
|
||||
extra_body=dict(spaces_between_special_tokens=False, enable_thinking=False),
|
||||
tools=tools)
|
||||
print(response.choices[0].message)
|
||||
messages.append(response.choices[0].message)
|
||||
|
||||
for tool_call in response.choices[0].message.tool_calls:
|
||||
tool_call_args = json.loads(tool_call.function.arguments)
|
||||
tool_call_result = get_function_by_name(tool_call.function.name)(**tool_call_args)
|
||||
tool_call_result = json.dumps(tool_call_result, ensure_ascii=False)
|
||||
messages.append({
|
||||
'role': 'tool',
|
||||
'name': tool_call.function.name,
|
||||
'content': tool_call_result,
|
||||
'tool_call_id': tool_call.id
|
||||
})
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model=model_name,
|
||||
messages=messages,
|
||||
temperature=0.8,
|
||||
top_p=0.8,
|
||||
stream=False,
|
||||
extra_body=dict(spaces_between_special_tokens=False, enable_thinking=False),
|
||||
tools=tools)
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
### Switching Between Thinking and Non-Thinking Modes
|
||||
|
||||
Intern-S1-mini enables thinking mode by default, enhancing the model's reasoning capabilities to generate higher-quality responses. This feature can be disabled by setting `enable_thinking=False` in `tokenizer.apply_chat_template`
|
||||
|
||||
```python
|
||||
text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
enable_thinking=False # think mode indicator
|
||||
)
|
||||
```
|
||||
|
||||
With LMDeploy serving Intern-S1-mini models, you can dynamically control the thinking mode by adjusting the `enable_thinking` parameter in your requests.
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
import json
|
||||
|
||||
messages = [
|
||||
{
|
||||
'role': 'user',
|
||||
'content': 'who are you'
|
||||
}, {
|
||||
'role': 'assistant',
|
||||
'content': 'I am an AI'
|
||||
}, {
|
||||
'role': 'user',
|
||||
'content': 'AGI is?'
|
||||
}]
|
||||
|
||||
openai_api_key = "EMPTY"
|
||||
openai_api_base = "http://0.0.0.0:23333/v1"
|
||||
client = OpenAI(
|
||||
api_key=openai_api_key,
|
||||
base_url=openai_api_base,
|
||||
)
|
||||
model_name = client.models.list().data[0].id
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model=model_name,
|
||||
messages=messages,
|
||||
temperature=0.8,
|
||||
top_p=0.8,
|
||||
max_tokens=2048,
|
||||
extra_body={
|
||||
"enable_thinking": False,
|
||||
}
|
||||
)
|
||||
print(json.dumps(response.model_dump(), indent=2, ensure_ascii=False))
|
||||
```
|
||||
|
||||
For vllm and sglang users, configure this through,
|
||||
|
||||
```python
|
||||
extra_body={
|
||||
"chat_template_kwargs": {"enable_thinking": False}
|
||||
}
|
||||
```
|
||||
|
||||
## Fine-tuning
|
||||
|
||||
See this [documentation](https://github.com/InternLM/Intern-S1/blob/main/docs/sft.md) for more details.
|
||||
|
||||
## Citation
|
||||
|
||||
If you find this work useful, feel free to give us a cite.
|
||||
|
||||
```
|
||||
@misc{bai2025interns1scientificmultimodalfoundation,
|
||||
title={Intern-S1: A Scientific Multimodal Foundation Model},
|
||||
author={Lei Bai and Zhongrui Cai and Maosong Cao and Weihan Cao and Chiyu Chen and Haojiong Chen and Kai Chen and Pengcheng Chen and Ying Chen and Yongkang Chen and Yu Cheng and Yu Cheng and Pei Chu and Tao Chu and Erfei Cui and Ganqu Cui and Long Cui and Ziyun Cui and Nianchen Deng and Ning Ding and Nanqin Dong and Peijie Dong and Shihan Dou and Sinan Du and Haodong Duan and Caihua Fan and Ben Gao and Changjiang Gao and Jianfei Gao and Songyang Gao and Yang Gao and Zhangwei Gao and Jiaye Ge and Qiming Ge and Lixin Gu and Yuzhe Gu and Aijia Guo and Qipeng Guo and Xu Guo and Conghui He and Junjun He and Yili Hong and Siyuan Hou and Caiyu Hu and Hanglei Hu and Jucheng Hu and Ming Hu and Zhouqi Hua and Haian Huang and Junhao Huang and Xu Huang and Zixian Huang and Zhe Jiang and Lingkai Kong and Linyang Li and Peiji Li and Pengze Li and Shuaibin Li and Tianbin Li and Wei Li and Yuqiang Li and Dahua Lin and Junyao Lin and Tianyi Lin and Zhishan Lin and Hongwei Liu and Jiangning Liu and Jiyao Liu and Junnan Liu and Kai Liu and Kaiwen Liu and Kuikun Liu and Shichun Liu and Shudong Liu and Wei Liu and Xinyao Liu and Yuhong Liu and Zhan Liu and Yinquan Lu and Haijun Lv and Hongxia Lv and Huijie Lv and Qidang Lv and Ying Lv and Chengqi Lyu and Chenglong Ma and Jianpeng Ma and Ren Ma and Runmin Ma and Runyuan Ma and Xinzhu Ma and Yichuan Ma and Zihan Ma and Sixuan Mi and Junzhi Ning and Wenchang Ning and Xinle Pang and Jiahui Peng and Runyu Peng and Yu Qiao and Jiantao Qiu and Xiaoye Qu and Yuan Qu and Yuchen Ren and Fukai Shang and Wenqi Shao and Junhao Shen and Shuaike Shen and Chunfeng Song and Demin Song and Diping Song and Chenlin Su and Weijie Su and Weigao Sun and Yu Sun and Qian Tan and Cheng Tang and Huanze Tang and Kexian Tang and Shixiang Tang and Jian Tong and Aoran Wang and Bin Wang and Dong Wang and Lintao Wang and Rui Wang and Weiyun Wang and Wenhai Wang and Yi Wang and Ziyi Wang and Ling-I Wu and Wen Wu and Yue Wu and Zijian Wu and Linchen Xiao and Shuhao Xing and Chao Xu and Huihui Xu and Jun Xu and Ruiliang Xu and Wanghan Xu and GanLin Yang and Yuming Yang and Haochen Ye and Jin Ye and Shenglong Ye and Jia Yu and Jiashuo Yu and Jing Yu and Fei Yuan and Bo Zhang and Chao Zhang and Chen Zhang and Hongjie Zhang and Jin Zhang and Qiaosheng Zhang and Qiuyinzhe Zhang and Songyang Zhang and Taolin Zhang and Wenlong Zhang and Wenwei Zhang and Yechen Zhang and Ziyang Zhang and Haiteng Zhao and Qian Zhao and Xiangyu Zhao and Xiangyu Zhao and Bowen Zhou and Dongzhan Zhou and Peiheng Zhou and Yuhao Zhou and Yunhua Zhou and Dongsheng Zhu and Lin Zhu and Yicheng Zou},
|
||||
year={2025},
|
||||
eprint={2508.15763},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.LG},
|
||||
url={https://arxiv.org/abs/2508.15763},
|
||||
}
|
||||
```
|
||||
120
chat_template.jinja
Normal file
120
chat_template.jinja
Normal file
@@ -0,0 +1,120 @@
|
||||
{% set default_thinking_sys %}You are an expert reasoner with extensive experience in all areas. You approach problems through systematic thinking and rigorous reasoning. Your response should reflect deep understanding and precise logical thinking, making your solution path and reasoning clear to others. Please put your thinking process within <think>...</think> tags.{% endset %}
|
||||
{%- set tool_instruction %}Your response should consist of a reasoning step (**thought**) followed immediately by a function call in valid JSON format. Wrap each function call using the `<|action_start|><|plugin|>` and `<|action_end|>` tags.
|
||||
|
||||
**Format example:**
|
||||
|
||||
```
|
||||
(Your thought goes here...)
|
||||
|
||||
<|action_start|><|plugin|>
|
||||
{
|
||||
"name": "tool_name",
|
||||
"parameters": {
|
||||
"parameter1": "value1",
|
||||
"parameter2": "value2"
|
||||
}
|
||||
}
|
||||
<|action_end|>
|
||||
```
|
||||
|
||||
# External Tools
|
||||
You have access to these tools:
|
||||
{% if tools %}{{ tools | tojson(indent=2) }}{% else %}[]{% endif %}{% endset %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- set role = message.role if message.role != 'tool' else 'environment' %}
|
||||
{%- set reasoning_content = '' %}
|
||||
{%- set content = message.content %}
|
||||
{%- set ns.tool_calls = '' %}
|
||||
{%- if role == 'assistant' %}
|
||||
{%- if message.reasoning_content is string %}
|
||||
{%- set reasoning_content = message.reasoning_content %}
|
||||
{%- elif '</think>' in content %}
|
||||
{%- set reasoning_content = content.split('</think>')[0].strip().split('<think>')[-1].strip() %}
|
||||
{%- set content = content.split('</think>')[-1].lstrip('
|
||||
') %}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if not loop.first %}
|
||||
{%- set ns.tool_calls = ns.tool_calls + '
|
||||
' %}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{%- set ns.tool_calls = ns.tool_calls + '<|action_start|><|plugin|>
|
||||
{"name": "' + tool_call.name + '", "parameters": ' %}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{%- set ns.tool_calls = ns.tool_calls + tool_call.arguments %}
|
||||
{%- else %}
|
||||
{%- set ns.tool_calls = ns.tool_calls + tool_call.arguments | tojson %}
|
||||
{%- endif %}
|
||||
{%- set ns.tool_calls = ns.tool_calls + '}
|
||||
<|action_end|>' %}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{%- set reasoning_content = '<think>
|
||||
' + reasoning_content.strip('
|
||||
') + '
|
||||
</think>
|
||||
' %}
|
||||
{%- endif %}
|
||||
{%- if not content is string %}
|
||||
{%- set ns.content = '' %}
|
||||
{%- for _content in message.content %}
|
||||
{%- if _content.type == 'image' %}
|
||||
{%- set ns.content = ns.content ~ '
|
||||
<IMG_CONTEXT>' %}
|
||||
{%- elif _content.type == 'video' %}
|
||||
{%- set ns.content = ns.content ~ '
|
||||
<video>' %}
|
||||
{%- elif _content.type == 'text' %}
|
||||
{%- set ns.content = ns.content ~ '
|
||||
' ~ _content.text %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- set content = ns.content %}
|
||||
{%- endif %}
|
||||
{%- set content = content.lstrip('
|
||||
') %}
|
||||
{%- if ns.tool_calls %}
|
||||
{%- set content = content + ns.tool_calls %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 == 0 %}
|
||||
{%- set system_prompt = '' %}
|
||||
{%- if role == 'system' %}
|
||||
{%- set system_prompt = system_prompt + content %}
|
||||
{%- elif enable_thinking is not defined or enable_thinking %}
|
||||
{%- set system_prompt = system_prompt + default_thinking_sys %}
|
||||
{%- endif %}
|
||||
{%- if tools %}
|
||||
{%- set system_prompt = system_prompt.rstrip('
|
||||
') + '
|
||||
|
||||
' + tool_instruction %}
|
||||
{%- endif %}
|
||||
{%- set system_prompt = system_prompt.strip('
|
||||
') %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 == 0 and system_prompt %}<|im_start|>system{% if tools %} name=<|plugin|>{% endif %}
|
||||
|
||||
{{ system_prompt }}<|im_end|>
|
||||
{% endif %}
|
||||
{%- if role != 'system' %}<|im_start|>{{ role }}{% if role == 'environment' or role == 'tool' %} name=<|plugin|>{% endif %}
|
||||
|
||||
{% if loop.index0 > ns.last_query_index and (loop.last or (not loop.last and reasoning_content)) %}{{ reasoning_content }}
|
||||
{%- endif %}{{ content }}<|im_end|>
|
||||
{% endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}<|im_start|>assistant
|
||||
{% if enable_thinking is not defined or enable_thinking %}
|
||||
<think>{% endif %}
|
||||
{% endif %}
|
||||
89
config.json
Normal file
89
config.json
Normal file
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"architectures": [
|
||||
"InternS1ForConditionalGeneration"
|
||||
],
|
||||
"downsample_ratio": 0.5,
|
||||
"image_seq_length": 256,
|
||||
"image_token_id": 152957,
|
||||
"model_type": "interns1",
|
||||
"projector_hidden_act": "gelu",
|
||||
"text_config": {
|
||||
"_attn_implementation_autoset": true,
|
||||
"architectures": [
|
||||
"Qwen3ForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": 151645,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 12288,
|
||||
"max_position_embeddings": 65536,
|
||||
"max_window_layers": 36,
|
||||
"model_type": "qwen3",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 36,
|
||||
"num_key_value_heads": 8,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000,
|
||||
"sliding_window": null,
|
||||
"torch_dtype": "bfloat16",
|
||||
"use_cache": true,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 153216
|
||||
},
|
||||
"torch_dtype": "bfloat16",
|
||||
"transformers_version": "4.53.0",
|
||||
"vision_config": {
|
||||
"_attn_implementation_autoset": true,
|
||||
"architectures": [
|
||||
"InternVisionModel"
|
||||
],
|
||||
"attention_bias": true,
|
||||
"attention_dropout": 0.0,
|
||||
"auto_map": {
|
||||
"AutoConfig": "configuration_interns1.InternS1VisionConfig",
|
||||
"AutoModel": "modeling_interns1.InternS1VisionModel"
|
||||
},
|
||||
"drop_path_rate": 0.0,
|
||||
"dropout": 0.0,
|
||||
"hidden_act": "gelu",
|
||||
"hidden_dropout_prob": 0.0,
|
||||
"hidden_size": 1024,
|
||||
"image_size": [
|
||||
448,
|
||||
448
|
||||
],
|
||||
"initializer_factor": 1.0,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 4096,
|
||||
"layer_norm_eps": 1e-06,
|
||||
"layer_scale_init_value": 0.1,
|
||||
"model_type": "interns1_vision",
|
||||
"norm_type": "layer_norm",
|
||||
"num_attention_heads": 16,
|
||||
"num_channels": 3,
|
||||
"num_hidden_layers": 24,
|
||||
"patch_size": [
|
||||
14,
|
||||
14
|
||||
],
|
||||
"projection_dropout": 0.0,
|
||||
"torch_dtype": "bfloat16",
|
||||
"use_absolute_position_embeddings": true,
|
||||
"use_mask_token": false,
|
||||
"use_mean_pooling": true,
|
||||
"use_qk_norm": false
|
||||
},
|
||||
"vision_feature_layer": -1,
|
||||
"vision_feature_select_strategy": "default",
|
||||
"auto_map": {
|
||||
"AutoConfig": "configuration_interns1.InternS1Config",
|
||||
"AutoModel": "modeling_interns1.InternS1Model",
|
||||
"AutoModelForCausalLM": "modeling_interns1.InternS1ForConditionalGeneration"
|
||||
}
|
||||
}
|
||||
225
configuration_interns1.py
Normal file
225
configuration_interns1.py
Normal file
@@ -0,0 +1,225 @@
|
||||
# coding=utf-8
|
||||
# Copyright 2025 HuggingFace Inc. team. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
|
||||
from transformers.configuration_utils import PretrainedConfig
|
||||
from transformers import AutoConfig
|
||||
|
||||
|
||||
class InternS1VisionConfig(PretrainedConfig):
|
||||
r"""
|
||||
This is the configuration class to store the configuration of a [`InternS1VisionModel`]. It is used to instantiate an InternS1VisionModel
|
||||
model according to the specified arguments, defining the model architecture.
|
||||
|
||||
Args:
|
||||
hidden_size (`int`, *optional*, defaults to 1024):
|
||||
Dimensionality of the encoder layers and the pooler layer.
|
||||
num_hidden_layers (`int`, *optional*, defaults to 24):
|
||||
Number of hidden layers in the Transformer encoder.
|
||||
num_attention_heads (`int`, *optional*, defaults to 16):
|
||||
Number of attention heads for each attention layer in the Transformer encoder.
|
||||
attention_bias (`bool`, *optional*, defaults to `False`):
|
||||
Whether to add a bias to the queries, keys and values.
|
||||
use_qk_norm (`bool`, *optional*, defaults to `False`):
|
||||
Whether to apply normalization to the queries and keys before the attention operation.
|
||||
intermediate_size (`int`, *optional*, defaults to 4096):
|
||||
Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder.
|
||||
hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`):
|
||||
The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,
|
||||
`"relu"`, `"selu"` and `"gelu_new"` are supported.
|
||||
hidden_dropout_prob (`float`, *optional*, defaults to 0.0):
|
||||
The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
|
||||
attention_dropout (`float`, *optional*, defaults to 0.0):
|
||||
Dropout probability for attention weights.
|
||||
projection_dropout (`float`, *optional*, defaults to 0.0):
|
||||
Dropout probability for the projection layer.
|
||||
initializer_range (`float`, *optional*, defaults to 0.02):
|
||||
The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
|
||||
norm_type (`str`, *optional*, defaults to `"layer_norm"`):
|
||||
The type of normalization to use in the encoder. Can be `"layer_norm"` or `"rms_norm"`.
|
||||
layer_norm_eps (`float`, *optional*, defaults to 1e-06):
|
||||
The epsilon used by the layer normalization layers.
|
||||
image_size (`int` or `list[int]`, *optional*, defaults to `[448, 448]`):
|
||||
The size (resolution) of each image.
|
||||
patch_size (`int` or `list[int]`, *optional*, defaults to `[14, 14]`):
|
||||
The size (resolution) of each patch.
|
||||
num_channels (`int`, *optional*, defaults to 3):
|
||||
The number of input channels.
|
||||
use_mask_token (`bool`, *optional*, defaults to `False`):
|
||||
Whether to use a mask token for masked image modeling.
|
||||
use_absolute_position_embeddings (`bool`, *optional*, defaults to `True`):
|
||||
Whether to use BERT-style absolute position embeddings.
|
||||
layer_scale_init_value (`float`, *optional*, defaults to 0.1):
|
||||
Scale to use in the self-attention layers. 0.1 for base, 1e-5 for large. Set 0 to disable layer scale.
|
||||
use_mean_pooling (`bool`, *optional*, defaults to `True`):
|
||||
Whether to mean pool the final hidden states of the patches instead of using the final hidden state of the
|
||||
CLS token, before applying the classification head.
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
>>> from transformers import InternS1VisionConfig, InternS1VisionModel
|
||||
|
||||
>>> # Initializing a InternS1VisionModel
|
||||
>>> configuration = InternS1VisionConfig()
|
||||
|
||||
>>> # Initializing a model (with random weights) from configuration
|
||||
>>> model = InternS1VisionModel(configuration)
|
||||
|
||||
>>> # Accessing the model configuration
|
||||
>>> configuration = model.config
|
||||
```"""
|
||||
|
||||
model_type = "interns1_vision"
|
||||
base_config_key = "vision_config"
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
hidden_size=1024,
|
||||
num_hidden_layers=24,
|
||||
num_attention_heads=16,
|
||||
attention_bias=False,
|
||||
use_qk_norm=False,
|
||||
intermediate_size=4096,
|
||||
hidden_act="gelu",
|
||||
hidden_dropout_prob=0.0,
|
||||
attention_dropout=0.0,
|
||||
projection_dropout=0.0,
|
||||
drop_path_rate=0.0,
|
||||
initializer_range=0.02,
|
||||
norm_type="layer_norm",
|
||||
layer_norm_eps=1e-06,
|
||||
image_size=[448, 448],
|
||||
patch_size=[14, 14],
|
||||
num_channels=3,
|
||||
use_mask_token=False,
|
||||
use_absolute_position_embeddings=True,
|
||||
layer_scale_init_value=0.1,
|
||||
use_mean_pooling=True,
|
||||
**kwargs,
|
||||
):
|
||||
super().__init__(**kwargs)
|
||||
|
||||
self.hidden_size = hidden_size
|
||||
self.num_hidden_layers = num_hidden_layers
|
||||
self.num_attention_heads = num_attention_heads
|
||||
self.attention_bias = attention_bias
|
||||
self.use_qk_norm = use_qk_norm
|
||||
self.intermediate_size = intermediate_size
|
||||
self.hidden_act = hidden_act
|
||||
self.hidden_dropout_prob = hidden_dropout_prob
|
||||
self.attention_dropout = attention_dropout
|
||||
self.projection_dropout = projection_dropout
|
||||
self.initializer_range = initializer_range
|
||||
self.norm_type = norm_type
|
||||
self.layer_norm_eps = layer_norm_eps
|
||||
self.drop_path_rate = drop_path_rate
|
||||
|
||||
image_size = image_size if isinstance(image_size, (list, tuple)) else (image_size, image_size)
|
||||
patch_size = patch_size if isinstance(patch_size, (list, tuple)) else (patch_size, patch_size)
|
||||
self.image_size = image_size
|
||||
self.patch_size = patch_size
|
||||
|
||||
self.num_channels = num_channels
|
||||
self.use_mask_token = use_mask_token
|
||||
self.use_absolute_position_embeddings = use_absolute_position_embeddings
|
||||
self.layer_scale_init_value = layer_scale_init_value
|
||||
self.use_mean_pooling = use_mean_pooling
|
||||
|
||||
|
||||
class InternS1Config(PretrainedConfig):
|
||||
r"""
|
||||
This is the configuration class to store the configuration of a [`InternS1ForConditionalGeneration`]. It is used to instantiate a
|
||||
InternS1 model according to the specified arguments, defining the model architecture.
|
||||
|
||||
Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
|
||||
documentation from [`PretrainedConfig`] for more information.
|
||||
|
||||
|
||||
Args:
|
||||
vision_config (`Union[AutoConfig, dict]`, *optional*, defaults to `InternVisonConfig`):
|
||||
The config object or dictionary of the vision backbone.
|
||||
text_config (`Union[AutoConfig, dict]`, *optional*, defaults to `Qwen2Config`):
|
||||
The config object or dictionary of the text backbone.
|
||||
image_token_id (`int`, *optional*, defaults to 151667):
|
||||
The image token index to encode the image prompt.
|
||||
image_seq_length (`int`, *optional*, defaults to 256):
|
||||
Number of image tokens to use per image patch.
|
||||
downsample_ratio (`float`, *optional*, defaults to 0.5):
|
||||
Factor by which to downsample the image.
|
||||
projector_hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`):
|
||||
The non-linear activation function (function or string) in the projector.
|
||||
vision_feature_layer (`int`, *optional*, defaults to -1):
|
||||
The index of the layer to use as the image features.
|
||||
vision_feature_select_strategy (`str`, *optional*, defaults to `"default"`):
|
||||
The feature selection strategy used to select the vision feature from the vision backbone.
|
||||
Can be one of `"default"` or `"full"`.
|
||||
|
||||
```python
|
||||
>>> from transformers import InternS1ForConditionalGeneration, InternS1Config
|
||||
|
||||
>>> # Initializing a InternS1 style configuration
|
||||
>>> configuration = InternS1Config()
|
||||
|
||||
>>> # Initializing a model (with random weights) from configuration
|
||||
>>> model = InternS1ForConditionalGeneration(configuration)
|
||||
|
||||
>>> # Accessing the model configuration
|
||||
>>> configuration = model.config
|
||||
```"""
|
||||
|
||||
model_type = "interns1"
|
||||
sub_configs = {"text_config": AutoConfig, "vision_config": InternS1VisionConfig}
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
vision_config=None,
|
||||
text_config=None,
|
||||
image_token_id=151667,
|
||||
image_seq_length=256,
|
||||
downsample_ratio=0.5,
|
||||
projector_hidden_act="gelu",
|
||||
vision_feature_layer=-1,
|
||||
vision_feature_select_strategy="default",
|
||||
**kwargs,
|
||||
):
|
||||
from transformers import CONFIG_MAPPING
|
||||
|
||||
self.image_token_id = image_token_id
|
||||
self.image_seq_length = image_seq_length
|
||||
self.downsample_ratio = downsample_ratio
|
||||
self.projector_hidden_act = projector_hidden_act
|
||||
self.vision_feature_layer = vision_feature_layer
|
||||
self.vision_feature_select_strategy = vision_feature_select_strategy
|
||||
|
||||
if isinstance(vision_config, dict):
|
||||
self.vision_config = InternS1VisionConfig(**vision_config)
|
||||
elif isinstance(vision_config, InternS1VisionConfig):
|
||||
self.vision_config = vision_config
|
||||
elif vision_config is None:
|
||||
self.vision_config = InternS1VisionConfig()
|
||||
|
||||
if isinstance(text_config, dict):
|
||||
text_config["model_type"] = text_config["model_type"] if "model_type" in text_config else "qwen3"
|
||||
text_config = CONFIG_MAPPING[text_config["model_type"]](**text_config)
|
||||
elif text_config is None:
|
||||
text_config = CONFIG_MAPPING["qwen3"]()
|
||||
|
||||
self.text_config = text_config
|
||||
|
||||
super().__init__(**kwargs)
|
||||
|
||||
|
||||
__all__ = ["InternS1VisionConfig", "InternS1Config"]
|
||||
6
generation_config.json
Normal file
6
generation_config.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": 151645,
|
||||
"transformers_version": "4.53.0"
|
||||
}
|
||||
151388
merges.txt
Normal file
151388
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00004.safetensors
Normal file
3
model-00001-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ae0c9b37f256e0f3d636d89c0f3b1f15d5accd900935aa246b1edb26bf114c8b
|
||||
size 4916843808
|
||||
3
model-00002-of-00004.safetensors
Normal file
3
model-00002-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:493e945a856cdd4d8ee40d90aff4144ab417d9d75484d8414ddd779c9f5351c6
|
||||
size 4915962480
|
||||
3
model-00003-of-00004.safetensors
Normal file
3
model-00003-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e23506f9d41df413781d363027506c2342a83386b84efbc9ef577f8109a8ebf3
|
||||
size 4915962496
|
||||
3
model-00004-of-00004.safetensors
Normal file
3
model-00004-of-00004.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:19162a9cf951ab491f9294f858a3a070eff107e1d3f6354464ad37914088f889
|
||||
size 2328949432
|
||||
849
model.safetensors.index.json
Normal file
849
model.safetensors.index.json
Normal file
@@ -0,0 +1,849 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_parameters": 8538804224,
|
||||
"total_size": 17077608448
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.embed_tokens.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.20.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.20.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.20.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.20.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.input_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.32.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.33.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.33.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.33.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.33.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.33.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.33.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.33.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.33.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.33.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.33.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.33.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
|
||||
"model.language_model.layers.34.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.34.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.input_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.35.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
|
||||
"model.language_model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.7.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.language_model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.8.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
|
||||
"model.language_model.norm.weight": "model-00004-of-00004.safetensors",
|
||||
"model.multi_modal_projector.layer_norm.bias": "model-00001-of-00004.safetensors",
|
||||
"model.multi_modal_projector.layer_norm.weight": "model-00001-of-00004.safetensors",
|
||||
"model.multi_modal_projector.linear_1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.multi_modal_projector.linear_1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.multi_modal_projector.linear_2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.multi_modal_projector.linear_2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.embeddings.cls_token": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.embeddings.patch_embeddings.projection.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.embeddings.patch_embeddings.projection.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.embeddings.position_embeddings": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.0.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.1.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.10.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.11.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.12.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.13.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.14.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.15.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.16.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.17.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.18.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.19.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.2.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.20.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.21.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.22.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.23.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.3.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.4.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.5.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.6.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.7.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.8.mlp.fc2.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.k_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.k_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.projection_layer.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.projection_layer.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.q_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.q_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.v_proj.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.attention.v_proj.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.lambda_1": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.lambda_2": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.layernorm_after.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.layernorm_after.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.layernorm_before.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.layernorm_before.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.mlp.fc1.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.mlp.fc1.weight": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.mlp.fc2.bias": "model-00001-of-00004.safetensors",
|
||||
"model.vision_tower.encoder.layer.9.mlp.fc2.weight": "model-00001-of-00004.safetensors"
|
||||
}
|
||||
}
|
||||
1200
modeling_interns1.py
Normal file
1200
modeling_interns1.py
Normal file
File diff suppressed because it is too large
Load Diff
35
preprocessor_config.json
Normal file
35
preprocessor_config.json
Normal file
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"crop_size": null,
|
||||
"crop_to_patches": false,
|
||||
"data_format": "channels_first",
|
||||
"default_to_square": true,
|
||||
"device": null,
|
||||
"disable_grouping": null,
|
||||
"do_center_crop": null,
|
||||
"do_convert_rgb": true,
|
||||
"do_normalize": true,
|
||||
"do_rescale": true,
|
||||
"do_resize": true,
|
||||
"image_mean": [
|
||||
0.485,
|
||||
0.456,
|
||||
0.406
|
||||
],
|
||||
"image_processor_type": "GotOcr2ImageProcessorFast",
|
||||
"image_std": [
|
||||
0.229,
|
||||
0.224,
|
||||
0.225
|
||||
],
|
||||
"input_data_format": null,
|
||||
"max_patches": 12,
|
||||
"min_patches": 1,
|
||||
"processor_class": "InternS1Processor",
|
||||
"resample": 3,
|
||||
"rescale_factor": 0.00392156862745098,
|
||||
"return_tensors": null,
|
||||
"size": {
|
||||
"height": 448,
|
||||
"width": 448
|
||||
}
|
||||
}
|
||||
317
processing_interns1.py
Normal file
317
processing_interns1.py
Normal file
@@ -0,0 +1,317 @@
|
||||
# coding=utf-8
|
||||
# Copyright 2025 HuggingFace Inc. team. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from typing import Optional, Union
|
||||
|
||||
import numpy as np
|
||||
|
||||
from transformers.image_processing_utils import BatchFeature
|
||||
from transformers.image_utils import ImageInput, concatenate_list, make_flat_list_of_images
|
||||
from transformers.processing_utils import ImagesKwargs, MultiModalData, ProcessingKwargs, ProcessorMixin, Unpack
|
||||
from transformers.tokenization_utils_base import PreTokenizedInput, TextInput
|
||||
from transformers.video_utils import VideoInput, make_batched_videos
|
||||
|
||||
|
||||
class InternS1ImagesKwargs(ImagesKwargs, total=False):
|
||||
crop_to_patches: Optional[bool]
|
||||
min_patches: Optional[int]
|
||||
max_patches: Optional[int]
|
||||
|
||||
|
||||
class InternS1ProcessorKwargs(ProcessingKwargs, total=False):
|
||||
images_kwargs: InternS1ImagesKwargs
|
||||
_defaults = {
|
||||
"text_kwargs": {
|
||||
"padding_side": "left",
|
||||
"return_mm_token_type_ids": False,
|
||||
},
|
||||
"images_kwargs": {
|
||||
"crop_to_patches": True,
|
||||
},
|
||||
"videos_kwargs": {},
|
||||
}
|
||||
|
||||
|
||||
class InternS1Processor(ProcessorMixin):
|
||||
r"""
|
||||
Constructs a InternS1 processor which wraps a [`AutoImageProcessor`] and
|
||||
[`PretrainedTokenizerFast`] tokenizer into a single processor that inherits both the image processor and
|
||||
tokenizer functionalities. See the [`~InternS1Processor.__call__`] and [`~InternS1Processor.decode`] for more information.
|
||||
Args:
|
||||
image_processor ([`AutoImageProcessor`], *optional*):
|
||||
The image processor is a required input.
|
||||
tokenizer ([`PreTrainedTokenizer`, `PreTrainedTokenizerFast`], *optional*):
|
||||
The tokenizer is a required input.
|
||||
video_processor ([`AutoVideoProcessor`], *optional*):
|
||||
The video processor is a required input.
|
||||
image_seq_length (`int`, *optional*, defaults to 256):
|
||||
The number of image token to use per image patch. it should be set so that:
|
||||
image_seq_length = (config.image_size // config.patch_size) ** 2 * (config.scale_factor**2)
|
||||
chat_template (`str`, *optional*): A Jinja template which will be used to convert lists of messages
|
||||
in a chat into a tokenizable string.
|
||||
"""
|
||||
|
||||
attributes = ["image_processor", "tokenizer", "video_processor"]
|
||||
image_processor_class = "AutoImageProcessor"
|
||||
video_processor_class = "AutoVideoProcessor"
|
||||
tokenizer_class = "AutoTokenizer"
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
image_processor=None,
|
||||
tokenizer=None,
|
||||
video_processor=None,
|
||||
image_seq_length: int = 256,
|
||||
chat_template=None,
|
||||
**kwargs,
|
||||
):
|
||||
self.image_seq_length = image_seq_length
|
||||
self.start_image_token = tokenizer.start_image_token
|
||||
self.end_image_token = tokenizer.end_image_token
|
||||
self.start_image_token_id = tokenizer.start_image_token_id
|
||||
self.end_image_token_id = tokenizer.end_image_token_id
|
||||
self.image_token = tokenizer.context_image_token
|
||||
self.video_token = tokenizer.video_token
|
||||
self.image_token_id = tokenizer.context_image_token_id
|
||||
self.image_ids = [self.image_token_id, self.start_image_token_id, self.end_image_token_id]
|
||||
|
||||
super().__init__(image_processor, tokenizer, video_processor, chat_template=chat_template, **kwargs)
|
||||
|
||||
def _insert_media_placeholders(
|
||||
self,
|
||||
text: list[str],
|
||||
image_pixel_values,
|
||||
video_pixel_values,
|
||||
image_num_patches: list[int],
|
||||
video_num_patches: list[int],
|
||||
image_num_patches_indices: np.ndarray,
|
||||
video_num_patches_indices: np.ndarray,
|
||||
video_patch_indices: np.ndarray,
|
||||
):
|
||||
"""
|
||||
Processes interleaved text with <image> and <video> placeholders, replacing them with appropriate
|
||||
image and video tokens while keeping track of the patches used.
|
||||
"""
|
||||
image_index = 0
|
||||
video_index = 0
|
||||
processed_text = []
|
||||
image_video_patches = []
|
||||
replace_strings = []
|
||||
# Support interleaved image and video in prompts:
|
||||
# Processed patches of images and videos are inserted in `image_video_patches` in the order they appear in the prompts
|
||||
for prompt in text:
|
||||
new_prompt = prompt
|
||||
while self.image_token in new_prompt or self.video_token in new_prompt:
|
||||
if self.image_token in new_prompt and (
|
||||
self.video_token not in new_prompt
|
||||
or new_prompt.index(self.image_token) < new_prompt.index(self.video_token)
|
||||
):
|
||||
# Get the slice of patches corresponding to the current image
|
||||
start_index = image_num_patches_indices[image_index - 1] if image_index > 0 else 0
|
||||
end_index = image_num_patches_indices[image_index]
|
||||
image_video_patches.append(image_pixel_values[start_index:end_index])
|
||||
# Replace the corresponding image placeholder with the correct number of image tokens
|
||||
new_prompt = new_prompt.replace(self.image_token, "<placeholder>", 1)
|
||||
replace_strings.append(
|
||||
f"{self.start_image_token}{self.image_token * self.image_seq_length * image_num_patches[image_index]}{self.end_image_token}"
|
||||
)
|
||||
image_index += 1
|
||||
else:
|
||||
# Get the slice of patches corresponding to the current video
|
||||
# Here we need to account for both the multiple video frames and the potential multiple patches per frame
|
||||
# As of now, InternS1 only supports one patch per frame, but we keep the code flexible for future updates
|
||||
current_patch_index = video_patch_indices[video_index - 1] if video_index > 0 else 0
|
||||
end_patch_index = video_patch_indices[video_index]
|
||||
start_index = video_num_patches_indices[current_patch_index] if video_index > 0 else 0
|
||||
end_index = video_num_patches_indices[end_patch_index - 1]
|
||||
image_video_patches.append(video_pixel_values[start_index:end_index])
|
||||
# Get the number of patches per frame and replace the video placeholder with the correct number of image tokens
|
||||
num_patches = list(video_num_patches[current_patch_index:end_patch_index])
|
||||
video_prompt = "\n".join(
|
||||
f"Frame{i + 1}: {self.start_image_token}{self.image_token * self.image_seq_length * num_patches[i]}{self.end_image_token}"
|
||||
for i in range(len(num_patches))
|
||||
)
|
||||
replace_strings.append(video_prompt)
|
||||
new_prompt = new_prompt.replace(self.video_token, "<placeholder>", 1)
|
||||
video_index += 1
|
||||
while "<placeholder>" in new_prompt:
|
||||
replace_str = replace_strings.pop(0)
|
||||
new_prompt = new_prompt.replace("<placeholder>", replace_str, 1)
|
||||
processed_text.append(new_prompt)
|
||||
|
||||
return processed_text, image_video_patches, image_index, video_index
|
||||
|
||||
def __call__(
|
||||
self,
|
||||
images: Optional[ImageInput] = None,
|
||||
text: Optional[Union[TextInput, PreTokenizedInput, list[TextInput], list[PreTokenizedInput]]] = None,
|
||||
audio=None,
|
||||
videos: Optional[VideoInput] = None,
|
||||
**kwargs: Unpack[InternS1ProcessorKwargs],
|
||||
) -> BatchFeature:
|
||||
"""
|
||||
Main method to prepare for the model one or several sequences(s) and image(s). This method forwards the `text`
|
||||
and `kwargs` arguments to PreTrainedTokenizerFast's [`~PreTrainedTokenizerFast.__call__`] to encode the text if `text`
|
||||
is not `None`, otherwise encode default OCR queries which depends on the `format`, `box`, `color`, `multi_page` and
|
||||
`crop_to_patches` arguments. To prepare the vision inputs, this method forwards the `images` and `kwrags` arguments to
|
||||
GotOcr2ImageProcessor's [`~GotOcr2ImageProcessor.__call__`] if `images` is not `None`.
|
||||
|
||||
Args:
|
||||
images (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `list[PIL.Image.Image]`, `list[np.ndarray]`, `list[torch.Tensor]`):
|
||||
The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch
|
||||
tensor. Both channels-first and channels-last formats are supported.
|
||||
text (`str`, `list[str]`, `list[list[str]]`):
|
||||
The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings
|
||||
(pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set
|
||||
`is_split_into_words=True` (to lift the ambiguity with a batch of sequences).
|
||||
videos (`np.ndarray`, `torch.Tensor`, `list[np.ndarray]`, `list[torch.Tensor]`):
|
||||
The image or batch of videos to be prepared. Each video can be a 4D NumPy array or PyTorch
|
||||
return_tensors (`str` or [`~utils.TensorType`], *optional*):
|
||||
If set, will return tensors of a particular framework. Acceptable values are:
|
||||
- `'tf'`: Return TensorFlow `tf.constant` objects.
|
||||
- `'pt'`: Return PyTorch `torch.Tensor` objects.
|
||||
- `'np'`: Return NumPy `np.ndarray` objects.
|
||||
- `'jax'`: Return JAX `jnp.ndarray` objects.
|
||||
|
||||
Returns:
|
||||
[`BatchFeature`]: A [`BatchFeature`] with the following fields:
|
||||
|
||||
- **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`.
|
||||
- **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when
|
||||
`return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not
|
||||
`None`).
|
||||
- **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`.
|
||||
"""
|
||||
if text is None:
|
||||
raise ValueError("You have to specify text.")
|
||||
|
||||
output_kwargs = self._merge_kwargs(
|
||||
InternS1ProcessorKwargs,
|
||||
tokenizer_init_kwargs=self.tokenizer.init_kwargs,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
if not isinstance(text, (list, tuple)):
|
||||
text = [text]
|
||||
|
||||
# Process images and videos separately, as videos don't support crop_to_patches
|
||||
image_num_patches = []
|
||||
video_num_patches = []
|
||||
image_videos_inputs = {}
|
||||
image_pixel_values = None
|
||||
video_pixel_values = None
|
||||
image_num_patches_indices = np.array([0])
|
||||
video_patch_indices = np.array([0])
|
||||
video_num_patches_indices = np.array([0])
|
||||
if images is not None:
|
||||
images = make_flat_list_of_images(images)
|
||||
image_inputs = self.image_processor(images=images, **output_kwargs["images_kwargs"])
|
||||
image_num_patches = image_inputs.pop("num_patches")
|
||||
image_pixel_values = image_inputs.pop("pixel_values")
|
||||
image_num_patches_indices = np.cumsum(image_num_patches)
|
||||
if videos is not None:
|
||||
videos = make_batched_videos(videos)
|
||||
video_inputs = self.video_processor(videos=videos, **output_kwargs["videos_kwargs"])
|
||||
video_pixel_values = video_inputs.pop("pixel_values_videos")
|
||||
|
||||
# Obtain per frame information first and then flatten to (BS * T, ...)
|
||||
num_frames_per_video = [len(video) for video in video_pixel_values]
|
||||
video_num_patches = [1 for frames in num_frames_per_video for _ in range(frames)]
|
||||
video_patch_indices = np.cumsum(num_frames_per_video)
|
||||
video_num_patches_indices = np.cumsum(video_num_patches)
|
||||
video_pixel_values = video_pixel_values.flatten(0, 1)
|
||||
|
||||
if images is not None or videos is not None:
|
||||
text, image_video_patches, image_index, video_index = self._insert_media_placeholders(
|
||||
text,
|
||||
image_pixel_values,
|
||||
video_pixel_values,
|
||||
image_num_patches,
|
||||
video_num_patches,
|
||||
image_num_patches_indices,
|
||||
video_num_patches_indices,
|
||||
video_patch_indices,
|
||||
)
|
||||
if images is not None and image_index != len(images):
|
||||
raise ValueError("Number of image placeholders in the prompt does not match the number of images.")
|
||||
if videos is not None and video_index != len(videos):
|
||||
raise ValueError("Number of video placeholders in the prompt does not match the number of videos.")
|
||||
|
||||
# Concatenate the interleaved image and video patches (function agnostic to the patches type (list, numpy array, torch tensor))
|
||||
image_videos_inputs = {"pixel_values": concatenate_list(image_video_patches)}
|
||||
|
||||
return_tensors = output_kwargs["text_kwargs"].pop("return_tensors", None)
|
||||
return_mm_token_type_ids = output_kwargs["text_kwargs"].pop("return_mm_token_type_ids", None)
|
||||
text_inputs = self.tokenizer(text, **output_kwargs["text_kwargs"])
|
||||
self._check_special_mm_tokens(text, text_inputs, modalities=["image"])
|
||||
|
||||
if return_mm_token_type_ids:
|
||||
array_ids = np.array(text_inputs["input_ids"])
|
||||
mm_token_type_ids = np.zeros_like(text_inputs["input_ids"])
|
||||
mm_token_type_ids[np.isin(array_ids, self.image_ids)] = 1
|
||||
text_inputs["mm_token_type_ids"] = mm_token_type_ids.tolist()
|
||||
|
||||
return BatchFeature(data={**text_inputs, **image_videos_inputs}, tensor_type=return_tensors)
|
||||
|
||||
def _get_num_multimodal_tokens(self, image_sizes=None, **kwargs):
|
||||
"""
|
||||
Computes the number of placeholder tokens needed for multimodal inputs with the given sizes.
|
||||
|
||||
Args:
|
||||
image_sizes (`list[list[int]]`, *optional*):
|
||||
The input sizes formatted as (height, width) per each image.
|
||||
|
||||
Returns:
|
||||
`MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided
|
||||
input modalities, along with other useful data.
|
||||
"""
|
||||
|
||||
vision_data = {}
|
||||
if image_sizes is not None:
|
||||
images_kwargs = InternS1ProcessorKwargs._defaults.get("images_kwargs", {})
|
||||
images_kwargs.update(kwargs)
|
||||
|
||||
num_image_patches = [
|
||||
self.image_processor.get_number_of_image_tokens(*image_size, images_kwargs)
|
||||
for image_size in image_sizes
|
||||
]
|
||||
# Add 2 for BOI and EOI tokens
|
||||
num_image_tokens = [2 + (self.image_seq_length * num_patches) for num_patches in num_image_patches]
|
||||
vision_data.update({"num_image_tokens": num_image_tokens, "num_image_patches": num_image_patches})
|
||||
|
||||
return MultiModalData(**vision_data)
|
||||
|
||||
def batch_decode(self, *args, **kwargs):
|
||||
"""
|
||||
This method forwards all its arguments to PreTrainedTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please
|
||||
refer to the docstring of this method for more information.
|
||||
"""
|
||||
return self.tokenizer.batch_decode(*args, **kwargs)
|
||||
|
||||
def decode(self, *args, **kwargs):
|
||||
"""
|
||||
This method forwards all its arguments to PreTrainedTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to
|
||||
the docstring of this method for more information.
|
||||
"""
|
||||
return self.tokenizer.decode(*args, **kwargs)
|
||||
|
||||
@property
|
||||
def model_input_names(self):
|
||||
tokenizer_input_names = self.tokenizer.model_input_names
|
||||
image_processor_input_names = self.image_processor.model_input_names
|
||||
return list(tokenizer_input_names) + list(image_processor_input_names)
|
||||
|
||||
|
||||
__all__ = ["InternS1Processor"]
|
||||
7
processor_config.json
Normal file
7
processor_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"image_seq_length": 256,
|
||||
"processor_class": "InternS1Processor",
|
||||
"auto_map": {
|
||||
"AutoProcessor": "processing_interns1.InternS1Processor"
|
||||
}
|
||||
}
|
||||
35
special_tokens_map.json
Normal file
35
special_tokens_map.json
Normal file
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"context_image_token": "<IMG_CONTEXT>",
|
||||
"end_image_token": "</img>",
|
||||
"eos_token": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"start_image_token": "<img>",
|
||||
"video_token": "<video>"
|
||||
}
|
||||
1119
tokenization_interns1.py
Normal file
1119
tokenization_interns1.py
Normal file
File diff suppressed because it is too large
Load Diff
3
tokenizer_FASTA.model
Normal file
3
tokenizer_FASTA.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6e719023a50767e2da1165925feb3afe77d63702f08d0cd39c4ddadba7cdaaca
|
||||
size 5899
|
||||
3
tokenizer_IUPAC.model
Normal file
3
tokenizer_IUPAC.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6e719023a50767e2da1165925feb3afe77d63702f08d0cd39c4ddadba7cdaaca
|
||||
size 5899
|
||||
3
tokenizer_SMILES.model
Normal file
3
tokenizer_SMILES.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d8dd3252680ab2fedacab7e71b75a48f08d6fbae70a9cc38d355c65ec42fbd0d
|
||||
size 3290
|
||||
432
tokenizer_config.json
Normal file
432
tokenizer_config.json
Normal file
@@ -0,0 +1,432 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_eos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151925": {
|
||||
"content": "<SMILES>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151926": {
|
||||
"content": "</SMILES>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151927": {
|
||||
"content": "<SELFIES>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151928": {
|
||||
"content": "</SELFIES>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152441": {
|
||||
"content": "<IUPAC>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152442": {
|
||||
"content": "</IUPAC>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152955": {
|
||||
"content": "<FASTA>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152956": {
|
||||
"content": "</FASTA>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152957": {
|
||||
"content": "<IMG_CONTEXT>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152958": {
|
||||
"content": "<img>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152959": {
|
||||
"content": "</img>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152960": {
|
||||
"content": "<quad>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152961": {
|
||||
"content": "</quad>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152962": {
|
||||
"content": "<ref>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152963": {
|
||||
"content": "</ref>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152964": {
|
||||
"content": "<box>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152965": {
|
||||
"content": "</box>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152966": {
|
||||
"content": "<|action_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152967": {
|
||||
"content": "<|action_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152968": {
|
||||
"content": "<|interpreter|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152969": {
|
||||
"content": "<|plugin|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"152970": {
|
||||
"content": "<video>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"context_image_token": "<IMG_CONTEXT>",
|
||||
"end_image_token": "</img>",
|
||||
"eos_token": "<|im_end|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {
|
||||
"context_image_token": "<IMG_CONTEXT>",
|
||||
"end_image_token": "</img>",
|
||||
"start_image_token": "<img>",
|
||||
"video_token": "<video>"
|
||||
},
|
||||
"model_max_length": 32768,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"processor_class": "InternS1Processor",
|
||||
"split_special_tokens": false,
|
||||
"start_image_token": "<img>",
|
||||
"tokenizer_class": "InternS1Tokenizer",
|
||||
"unk_token": null,
|
||||
"video_token": "<video>",
|
||||
"auto_map": {
|
||||
"AutoTokenizer": [
|
||||
"tokenization_interns1.InternS1Tokenizer",
|
||||
"tokenization_interns1.InternS1Tokenizer"
|
||||
]
|
||||
}
|
||||
}
|
||||
46
video_preprocessor_config.json
Normal file
46
video_preprocessor_config.json
Normal file
@@ -0,0 +1,46 @@
|
||||
{
|
||||
"crop_size": null,
|
||||
"crop_to_patches": false,
|
||||
"data_format": "channels_first",
|
||||
"default_to_square": true,
|
||||
"device": null,
|
||||
"disable_grouping": null,
|
||||
"do_center_crop": null,
|
||||
"do_convert_rgb": true,
|
||||
"do_normalize": true,
|
||||
"do_pad": null,
|
||||
"do_rescale": true,
|
||||
"do_resize": true,
|
||||
"do_sample_frames": false,
|
||||
"fps": null,
|
||||
"image_mean": [
|
||||
0.485,
|
||||
0.456,
|
||||
0.406
|
||||
],
|
||||
"image_processor_type": "GotOcr2ImageProcessorFast",
|
||||
"image_std": [
|
||||
0.229,
|
||||
0.224,
|
||||
0.225
|
||||
],
|
||||
"initial_shift": true,
|
||||
"input_data_format": null,
|
||||
"max_patches": 12,
|
||||
"min_patches": 1,
|
||||
"num_frames": null,
|
||||
"processor_class": "InternS1Processor",
|
||||
"resample": 3,
|
||||
"rescale_factor": 0.00392156862745098,
|
||||
"return_tensors": null,
|
||||
"size": {
|
||||
"height": 448,
|
||||
"width": 448
|
||||
},
|
||||
"size_divisor": null,
|
||||
"video_metadata": null,
|
||||
"video_processor_type": "InternS1VideoProcessor",
|
||||
"auto_map": {
|
||||
"AutoVideoProcessor": "video_processing_interns1.InternS1VideoProcessor"
|
||||
}
|
||||
}
|
||||
197
video_processing_interns1.py
Normal file
197
video_processing_interns1.py
Normal file
@@ -0,0 +1,197 @@
|
||||
# coding=utf-8
|
||||
# Copyright 2025 The HuggingFace Inc. team. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
"""Fast Video processor class for InternS1."""
|
||||
|
||||
from typing import Optional, Union
|
||||
|
||||
from transformers.image_processing_utils import BatchFeature
|
||||
from transformers.image_utils import (
|
||||
OPENAI_CLIP_MEAN,
|
||||
OPENAI_CLIP_STD,
|
||||
SizeDict,
|
||||
)
|
||||
from transformers.processing_utils import Unpack, VideosKwargs
|
||||
from transformers.utils import (
|
||||
TensorType,
|
||||
is_torch_available,
|
||||
is_torchvision_available,
|
||||
is_torchvision_v2_available,
|
||||
is_vision_available,
|
||||
)
|
||||
from transformers.utils.import_utils import requires
|
||||
from transformers.video_processing_utils import BaseVideoProcessor
|
||||
from transformers.video_utils import VideoMetadata, group_videos_by_shape, reorder_videos
|
||||
|
||||
|
||||
if is_torchvision_available():
|
||||
if is_torchvision_v2_available():
|
||||
from torchvision.transforms.v2 import functional as F
|
||||
else:
|
||||
from torchvision.transforms import functional as F
|
||||
|
||||
|
||||
if is_torch_available():
|
||||
import torch
|
||||
|
||||
if is_vision_available():
|
||||
from transformers.image_utils import PILImageResampling
|
||||
|
||||
|
||||
class InternS1VideoProcessorInitKwargs(VideosKwargs):
|
||||
initial_shift: Union[bool, float, int]
|
||||
|
||||
|
||||
@requires(backends=("torchvision",))
|
||||
class InternS1VideoProcessor(BaseVideoProcessor):
|
||||
resample = PILImageResampling.BICUBIC
|
||||
image_mean = OPENAI_CLIP_MEAN
|
||||
image_std = OPENAI_CLIP_STD
|
||||
size = {"height": 384, "width": 384}
|
||||
do_resize = True
|
||||
do_rescale = True
|
||||
do_normalize = True
|
||||
do_convert_rgb = True
|
||||
initial_shift = True
|
||||
do_sample_frames = False # Set to False for BC, recommended to set `True` in new models
|
||||
valid_kwargs = InternS1VideoProcessorInitKwargs
|
||||
model_input_names = ["pixel_values_videos"]
|
||||
|
||||
def __init__(self, **kwargs: Unpack[InternS1VideoProcessorInitKwargs]):
|
||||
super().__init__(**kwargs)
|
||||
|
||||
def sample_frames(
|
||||
self,
|
||||
video: "torch.Tensor",
|
||||
metadata: Optional[Union[VideoMetadata, dict]] = None,
|
||||
num_frames: Optional[int] = None,
|
||||
fps: Optional[Union[int, float]] = None,
|
||||
initial_shift: Optional[Union[bool, float, int]] = None,
|
||||
):
|
||||
"""
|
||||
Default sampling function which uniformly samples the desired number of frames between 0 and total number of frames.
|
||||
If `fps` is passed along with metadata, `fps` frames per second are sampled uniformty. Arguments `num_frames`
|
||||
and `fps` are mutually exclusive.
|
||||
|
||||
Args:
|
||||
video (`torch.Tensor`):
|
||||
Video that need to be sampled.
|
||||
metadata (`VideoMetadata`, *optional*):
|
||||
Metadata of the video containing information about total duration, fps and total number of frames.
|
||||
num_frames (`int`, *optional*):
|
||||
Maximum number of frames to sample. Defaults to `self.num_frames`.
|
||||
fps (`int` or `float`, *optional*):
|
||||
Target frames to sample per second. Defaults to `self.fps`.
|
||||
initial_shift (`bool`, `float` or `int`, defaults to `self.initial_shift`):
|
||||
The initial shift to apply when sampling frames. If `True`, the shift is set so that frames are sampled from the middle of the video.
|
||||
|
||||
Returns:
|
||||
torch.Tensor:
|
||||
Sampled video frames.
|
||||
"""
|
||||
num_frames = num_frames if num_frames is not None else self.num_frames
|
||||
initial_shift = initial_shift if initial_shift is not None else self.initial_shift
|
||||
total_num_frames = video.shape[0]
|
||||
|
||||
# If num_frames is not given but fps is, calculate num_frames from fps
|
||||
if num_frames is None and fps is not None:
|
||||
if metadata is None:
|
||||
raise ValueError(
|
||||
"Asked to sample `fps` frames per second but no video metadata was provided which is required when sampling with `fps`. "
|
||||
"Please pass in `VideoMetadata` object or use a fixed `num_frames` per input video"
|
||||
)
|
||||
num_frames = int(total_num_frames / metadata["fps"] * fps)
|
||||
|
||||
if initial_shift is True:
|
||||
initial_shift = total_num_frames / num_frames / 2
|
||||
|
||||
if num_frames > total_num_frames:
|
||||
raise ValueError(
|
||||
f"Video can't be sampled. The `num_frames={num_frames}` exceeds `total_num_frames={total_num_frames}`. "
|
||||
)
|
||||
|
||||
indices = torch.arange(initial_shift, total_num_frames, total_num_frames / num_frames).int()
|
||||
video = video[indices].contiguous()
|
||||
return video
|
||||
|
||||
def _preprocess(
|
||||
self,
|
||||
videos: list["torch.Tensor"],
|
||||
video_metadata: Union[list[VideoMetadata], list[dict]],
|
||||
do_convert_rgb: bool,
|
||||
do_resize: bool,
|
||||
size: SizeDict,
|
||||
size_divisor: Optional[int],
|
||||
interpolation: Optional["F.InterpolationMode"],
|
||||
do_center_crop: bool,
|
||||
crop_size: SizeDict,
|
||||
do_rescale: bool,
|
||||
do_pad: bool,
|
||||
rescale_factor: float,
|
||||
do_normalize: bool,
|
||||
image_mean: Optional[Union[float, list[float]]],
|
||||
image_std: Optional[Union[float, list[float]]],
|
||||
do_sample_frames: Optional[bool] = None,
|
||||
fps: Optional[Union[int, float]] = None,
|
||||
num_frames: Optional[int] = None,
|
||||
initial_shift: Optional[Union[bool, float, int]] = None,
|
||||
return_tensors: Optional[Union[str, TensorType]] = None,
|
||||
device: Optional["torch.Tensor"] = None,
|
||||
**kwargs
|
||||
) -> BatchFeature:
|
||||
if do_sample_frames:
|
||||
# Sample video frames
|
||||
videos = [
|
||||
self.sample_frames(video, metadata, fps=fps, num_frames=num_frames, initial_shift=initial_shift)
|
||||
for video, metadata in zip(videos, video_metadata)
|
||||
]
|
||||
|
||||
# We need to sample frames first before moving to device, if `do_sample_frames=True`. Otherwise
|
||||
# moving the whole video incurs high GPU mem usage for long videos
|
||||
if device is not None:
|
||||
videos = [video.to(device) for video in videos]
|
||||
|
||||
# Group videos by size for batched resizing
|
||||
grouped_videos, grouped_videos_index = group_videos_by_shape(videos)
|
||||
resized_videos_grouped = {}
|
||||
for shape, stacked_videos in grouped_videos.items():
|
||||
if do_convert_rgb:
|
||||
stacked_videos = self.convert_to_rgb(stacked_videos)
|
||||
if do_resize:
|
||||
stacked_videos = self.resize(
|
||||
stacked_videos, size=size, size_divisor=size_divisor, interpolation=interpolation
|
||||
)
|
||||
resized_videos_grouped[shape] = stacked_videos
|
||||
resized_videos = reorder_videos(resized_videos_grouped, grouped_videos_index)
|
||||
|
||||
# Group videos by size for further processing
|
||||
# Needed in case do_resize is False, or resize returns videos with different sizes
|
||||
grouped_videos, grouped_videos_index = group_videos_by_shape(resized_videos)
|
||||
processed_videos_grouped = {}
|
||||
for shape, stacked_videos in grouped_videos.items():
|
||||
if do_center_crop:
|
||||
stacked_videos = self.center_crop(stacked_videos, crop_size)
|
||||
# Fused rescale and normalize
|
||||
stacked_videos = self.rescale_and_normalize(
|
||||
stacked_videos, do_rescale, rescale_factor, do_normalize, image_mean, image_std
|
||||
)
|
||||
processed_videos_grouped[shape] = stacked_videos
|
||||
|
||||
processed_videos = reorder_videos(processed_videos_grouped, grouped_videos_index)
|
||||
processed_videos = torch.stack(processed_videos, dim=0) if return_tensors else processed_videos
|
||||
|
||||
return BatchFeature(data={"pixel_values_videos": processed_videos}, tensor_type=return_tensors)
|
||||
|
||||
|
||||
__all__ = ["InternS1VideoProcessor"]
|
||||
151645
vocab.json
Normal file
151645
vocab.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user