ibm-granite/granite-speech-3.3-8b
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: ibm-granite/granite-speech-3.3-8b
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-10-29 07:25:18 +08:00
commit ffada56896
26 changed files with 295842 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

99
.eval_results/open_asr_leaderboard.yaml Normal file
View File

@@ -0,0 +1,99 @@
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: mean_wer
value: 5.74
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: rtfx
value: 145.42
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: ami_wer
value: 8.98
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: earnings22_wer
value: 9.42
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: gigaspeech_wer
value: 10.19
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: librispeech_clean_wer
value: 1.43
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: librispeech_other_wer
value: 2.86
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: spgispeech_wer
value: 3.91
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: tedlium_wer
value: 3.4
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio
- dataset:
id: hf-audio/open-asr-leaderboard
task_id: voxpopuli_wer
value: 5.72
date: '2025-04-14'
source:
url: https://huggingface.co/hf-audio
name: open-asr-leaderboard
user: hf-audio

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
10226_10111_000000.wav filter=lfs diff=lfs merge=lfs -text

3
10226_10111_000000.wav Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6ee3e432f4ce88415747f0549628bfef0df742365813d38b18bf28067a40cd51
size 539884

351
README.md Normal file
View File

@@ -0,0 +1,351 @@
---
license: apache-2.0
language:
- multilingual
- en
- fr
- de
- es
- pt
base_model:
- ibm-granite/granite-3.3-8b-instruct
library_name: transformers
---
# Granite-speech-3.3-8b (revision 3.3.2)
**Model Summary:**
Granite-speech-3.3-8b is a compact and efficient speech-language model, specifically designed for automatic speech recognition (ASR) and automatic speech translation (AST). Granite-speech-3.3-8b uses a two-pass design, unlike integrated models that combine speech and language into a single pass. Initial calls to granite-speech-3.3-8b will transcribe audio files into text. To process the transcribed text using the underlying Granite language model, users must make a second call as each step must be explicitly initiated.
The model was trained on a collection of public corpora comprising diverse datasets for ASR and AST as well as synthetic datasets tailored to support the speech translation task. Granite-speech-3.3-8b was trained by modality aligning granite-3.3-8b-instruct (https://huggingface.co/ibm-granite/granite-3.3-8b-instruct) to speech on publicly available open source corpora containing audio inputs and text targets.
* Compared to revision 3.3.1, revision 3.3.2 supports multilingual speech inputs in English, French, German, Spanish and Portuguese and provides additional accuracy improvements for English ASR.
* Compared to the initial release, revision 3.3.2 is also trained on additional data and uses a deeper acoustic encoder for improved transcription accuracy.
**Evaluations:**
We evaluated granite-speech-3.3-8b revision 3.3.2 alongside other speech-language models in the less than 8b parameter range as well as dedicated ASR and AST systems on standard benchmarks. The evaluation spanned multiple public benchmarks, with particular emphasis on English ASR tasks while also including multilingual ASR and AST for X-En and En-X translations.
<br>
![image/png](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/R43NGvyl2HzWKnElo5cBv.png)
<br>
![image/png](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/GXBqn5V1QPcsguSnlNqbi.png)
<br>
![image/png](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/Ham13Oq2hBuIb8gyFUR4u.png)
<br>
![image/png](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/VNNCu1Ztb1jA8mTt6bFoC.png)
<br>
![image/png](https://cdn-uploads.huggingface.co/production/uploads/666ec38102791b3b49f453e8/anLzlE_AlIw_c6R_S-dCI.png)
<br>
**Release Date**: June 19, 2025
**License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
**Supported Languages:**
English, French, German, Spanish, Portuguese
**Intended Use:**
The model is intended to be used in enterprise applications that involve processing of speech inputs. In particular, the model is well-suited for English, French, German, Spanish and Portuguese speech-to-text and speech translations to and from English for the same languages plus English-to-Japanese and English-to-Mandarin. The model can also be used for tasks that involve text-only input since it calls the underlying granite-3.3-8b-instruct when the user specifies a prompt that does not contain audio.
## Generation:
Granite Speech model is supported natively in `transformers` from the `main` branch. Below is a simple example of how to use the `granite-speech-3.3-8b` revision 3.3.2 model.
### Usage with `transformers`
First, make sure to install a recent version of transformers:
```shell
pip install transformers>=4.52.4 torchaudio peft soundfile
```
Then run the code:
```python
import torch
import torchaudio
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
from huggingface_hub import hf_hub_download
device = "cuda" if torch.cuda.is_available() else "cpu"
model_name = "ibm-granite/granite-speech-3.3-8b"
processor = AutoProcessor.from_pretrained(model_name)
tokenizer = processor.tokenizer
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_name, device_map=device, torch_dtype=torch.bfloat16
)
# load audio
audio_path = hf_hub_download(repo_id=model_name, filename="10226_10111_000000.wav")
wav, sr = torchaudio.load(audio_path, normalize=True)
assert wav.shape[0] == 1 and sr == 16000 # mono, 16khz
# create text prompt
system_prompt = "Knowledge Cutoff Date: April 2024.\nToday's Date: April 9, 2025.\nYou are Granite, developed by IBM. You are a helpful AI assistant"
user_prompt = "<|audio|>can you transcribe the speech into a written format?"
chat = [
dict(role="system", content=system_prompt),
dict(role="user", content=user_prompt),
]
prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
# run the processor+model
model_inputs = processor(prompt, wav, device=device, return_tensors="pt").to(device)
model_outputs = model.generate(**model_inputs, max_new_tokens=200, do_sample=False, num_beams=1)
# Transformers includes the input IDs in the response.
num_input_tokens = model_inputs["input_ids"].shape[-1]
new_tokens = torch.unsqueeze(model_outputs[0, num_input_tokens:], dim=0)
output_text = tokenizer.batch_decode(
new_tokens, add_special_tokens=False, skip_special_tokens=True
)
print(f"STT output = {output_text[0].upper()}")
```
### Usage with `vLLM`
First, make sure to install the latest version of vLLM:
```shell
pip install vllm --upgrade
```
* Code for offline mode:
```python
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
from vllm.assets.audio import AudioAsset
from vllm.lora.request import LoRARequest
model_id = "ibm-granite/granite-speech-3.3-8b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
def get_prompt(question: str, has_audio: bool):
"""Build the input prompt to send to vLLM."""
if has_audio:
question = f"<|audio|>{question}"
chat = [
{
"role": "user",
"content": question
}
]
return tokenizer.apply_chat_template(chat, tokenize=False)
# NOTE - you may see warnings about multimodal lora layers being ignored;
# this is okay as the lora in this model is only applied to the LLM.
model = LLM(
model=model_id,
enable_lora=True,
max_lora_rank=64,
max_model_len=2048, # This may be needed for lower resource devices.
limit_mm_per_prompt={"audio": 1},
)
### 1. Example with Audio [make sure to use the lora]
question = "can you transcribe the speech into a written format?"
prompt_with_audio = get_prompt(
question=question,
has_audio=True,
)
audio = AudioAsset("mary_had_lamb").audio_and_sample_rate
inputs = {
"prompt": prompt_with_audio,
"multi_modal_data": {
"audio": audio,
}
}
outputs = model.generate(
inputs,
sampling_params=SamplingParams(
temperature=0.2,
max_tokens=64,
),
lora_request=[LoRARequest("speech", 1, model_id)]
)
print(f"Audio Example - Question: {question}")
print(f"Generated text: {outputs[0].outputs[0].text}")
### 2. Example without Audio [do NOT use the lora]
question = "What is the capital of Brazil?"
prompt = get_prompt(
question=question,
has_audio=False,
)
outputs = model.generate(
{"prompt": prompt},
sampling_params=SamplingParams(
temperature=0.2,
max_tokens=12,
),
)
print(f"Text Only Example - Question: {question}")
print(f"Generated text: {outputs[0].outputs[0].text}")
```
* Code for online mode:
```python
"""
Launch the vLLM server with the following command:
vllm serve ibm-granite/granite-speech-3.3-8b \
--api-key token-abc123 \
--max-model-len 2048 \
--enable-lora \
--lora-modules speech=ibm-granite/granite-speech-3.3-8b \
--max-lora-rank 64
"""
import base64
import requests
from openai import OpenAI
from vllm.assets.audio import AudioAsset
# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "token-abc123"
openai_api_base = "http://localhost:8000/v1"
client = OpenAI(
# defaults to os.environ.get("OPENAI_API_KEY")
api_key=openai_api_key,
base_url=openai_api_base,
)
base_model_name = "ibm-granite/granite-speech-3.3-8b"
lora_model_name = "speech"
# Any format supported by librosa is supported
audio_url = AudioAsset("mary_had_lamb").url
# Use base64 encoded audio in the payload
def encode_audio_base64_from_url(audio_url: str) -> str:
"""Encode an audio retrieved from a remote url to base64 format."""
with requests.get(audio_url) as response:
response.raise_for_status()
result = base64.b64encode(response.content).decode('utf-8')
return result
audio_base64 = encode_audio_base64_from_url(audio_url=audio_url)
### 1. Example with Audio
# NOTE: we pass the name of the lora model (`speech`) here because we have audio.
question = "can you transcribe the speech into a written format?"
chat_completion_with_audio = client.chat.completions.create(
messages=[{
"role": "user",
"content": [
{
"type": "text",
"text": question
},
{
"type": "audio_url",
"audio_url": {
# Any format supported by librosa is supported
"url": f"data:audio/ogg;base64,{audio_base64}"
},
},
],
}],
temperature=0.2,
max_tokens=64,
model=lora_model_name,
)
print(f"Audio Example - Question: {question}")
print(f"Generated text: {chat_completion_with_audio.choices[0].message.content}")
### 2. Example without Audio
# NOTE: we pass the name of the base model here because we do not have audio.
question = "What is the capital of Brazil?"
chat_completion_with_audio = client.chat.completions.create(
messages=[{
"role": "user",
"content": [
{
"type": "text",
"text": question
},
],
}],
temperature=0.2,
max_tokens=12,
model=base_model_name,
)
print(f"Text Only Example - Question: {question}")
print(f"Generated text: {chat_completion_with_audio.choices[0].message.content}")
```
**Model Architecture:**
The architecture of granite-speech-3.3-8b revision 3.3.2 consists of the following components:
(1) Speech encoder: 16 conformer blocks trained with Connectionist Temporal Classification (CTC) on character-level targets on the subset containing
only ASR corpora (see configuration below). In addition, our CTC encoder uses block-attention with 4-seconds audio blocks and self-conditioned CTC
from the middle layer.
| Configuration parameter | Value |
|-----------------|----------------------|
| Input dimension | 160 (80 logmels x 2) |
| Nb. of layers | 16 |
| Hidden dimension | 1024 |
| Nb. of attention heads | 8 |
| Attention head size | 128 |
| Convolution kernel size | 15 |
| Output dimension | 256 |
(2) Speech projector and temporal downsampler (speech-text modality adapter): we use a 2-layer window query transformer (q-former) operating on
blocks of 15 1024-dimensional acoustic embeddings coming out of the last conformer block of the speech encoder that get downsampled by a factor of 5
using 3 trainable queries per block and per layer. The total temporal downsampling factor is 10 (2x from the encoder and 5x from the projector)
resulting in a 10Hz acoustic embeddings rate for the LLM. The encoder, projector and LoRA adapters were fine-tuned/trained jointly on all the
corpora mentioned under **Training Data**.
(3) Large language model: granite-3.3-8b-instruct with 128k context length (https://huggingface.co/ibm-granite/granite-3.3-8b-instruct).
(4) LoRA adapters: rank=64 applied to the query, value projection matrices
**Training Data:**
Overall, our training data is largely comprised of two key sources: (1) publicly available datasets (2) Synthetic data created from publicly
available datasets specifically targeting the speech translation task. A detailed description of the training datasets can be found in the table
below:
| Name | Task | Nb. hours | Source |
|-----------|--------------|----------------|--------------|
| CommonVoice-17 En,De,Es,Fr,Pt | ASR | 5600 | https://huggingface.co/datasets/mozilla-foundation/common_voice_17_0 |
| MLS En,De,Es,Fr,Pt | ASR | 48000 | https://huggingface.co/datasets/facebook/multilingual_librispeech |
| Librispeech English | ASR | 1000 | https://huggingface.co/datasets/openslr/librispeech_asr |
| VoxPopuli En,De,Fr,Es | ASR | 1100 | https://huggingface.co/datasets/facebook/voxpopuli |
| AMI English | ASR | 100 | https://huggingface.co/datasets/edinburghcstr/ami |
| YODAS English | ASR | 10000 | https://huggingface.co/datasets/espnet/yodas |
| Earnings-22 English | ASR | 105 | https://huggingface.co/datasets/esb/datasets |
| Switchboard English | ASR | 260 | https://catalog.ldc.upenn.edu/LDC97S62 |
| CallHome English | ASR | 18 | https://catalog.ldc.upenn.edu/LDC97T14 |
| Fisher English | ASR | 2000 | https://catalog.ldc.upenn.edu/LDC2004S13 |
| Voicemail part I English | ASR | 40 | https://catalog.ldc.upenn.edu/LDC98S77 |
| Voicemail part II English | ASR | 40 | https://catalog.ldc.upenn.edu/LDC2002S35 |
| CommonVoice-17 De,Es,Fr,Pt->En | AST | 3000 | Translations with Granite-3 and Phi-4 |
| CommonVoice-17 En->De,Es,Fr,It,Ja,Pt,Zh | AST | 18000 | Translations with Phi-4 and MADLAD |
**Infrastructure:**
We train Granite Speech using IBM's super computing cluster, Blue Vela, which is outfitted with NVIDIA H100 GPUs. This cluster provides a scalable
and efficient infrastructure for training our models over thousands of GPUs. The training of this particular model was completed in 13 days on 32
H100 GPUs.
**Ethical Considerations and Limitations:**
The use of Large Speech and Language Models can trigger certain risks and ethical considerations. Although our alignment processes include safety considerations, the model may in some cases produce inaccurate, biased, offensive or unwanted responses to user prompts. Additionally, whether smaller models may exhibit increased susceptibility to hallucination in generation scenarios due to their reduced sizes, which could limit their ability to generate coherent and contextually accurate responses, remains uncertain. This aspect is currently an active area of research, and we anticipate more rigorous exploration, comprehension, and mitigations in this domain.
IBM recommends using this model for automatic speech recognition and translation tasks. The model's modular design improves safety by limiting how audio inputs can influence the system. If an unfamiliar or malformed prompt is received, the model simply echoes it with its transcription. This minimizes the risk of adversarial inputs, unlike integrated models that directly interpret audio and may be more exposed to such attacks. Note that more general speech tasks may pose higher inherent risks of triggering unwanted outputs.
To enhance safety, we recommend using granite-speech-3.3-8b alongside Granite Guardian. Granite Guardian is a fine-tuned instruct model designed to detect and flag risks in prompts and responses across key dimensions outlined in the IBM AI Risk Atlas.
**Resources**
- 📄 Read the full technical report: https://arxiv.org/abs/2505.08699 (covers initial release only)
- 🔧 Notebooks: [Finetune on custom data](https://github.com/ibm-granite/granite-speech-models/blob/main/notebooks/fine_tuning_granite_speech.ipynb), [two-pass spoken question answering](https://github.com/ibm-granite/granite-speech-models/blob/main/notebooks/two_pass_spoken_qa.ipynb)
- ⭐️ Learn about the latest updates with Granite: https://www.ibm.com/granite
- 🚀 Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/
- 💡 Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources

29
adapter_config.json Normal file
View File

@@ -0,0 +1,29 @@
{
"alpha_pattern": {},
"auto_mapping": null,
"base_model_name_or_path": "ibm-granite/granite-speech-3.3-8b",
"bias": "none",
"fan_in_fan_out": false,
"inference_mode": true,
"init_lora_weights": true,
"layer_replication": null,
"layers_pattern": null,
"layers_to_transform": null,
"loftq_config": {},
"lora_alpha": 32,
"lora_dropout": 0.0,
"megatron_config": null,
"megatron_core": "megatron.core",
"modules_to_save": null,
"peft_type": "LORA",
"r": 64,
"rank_pattern": {},
"revision": null,
"target_modules": [
"q_proj",
"v_proj"
],
"task_type": "CAUSAL_LM",
"use_dora": false,
"use_rslora": false
}

3
adapter_model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b6b83dd573f82ddd3221c9634c155abe36e27f9db2004c570b9e68ea338fbab
size 136336192

10
added_tokens.json Normal file
View File

@@ -0,0 +1,10 @@
{
"<|audio|>": 49159,
"<|end_of_cite|>": 49156,
"<|end_of_plugin|>": 49158,
"<|end_of_role|>": 49153,
"<|start_of_cite|>": 49155,
"<|start_of_plugin|>": 49157,
"<|start_of_role|>": 49152,
"<|tool_call|>": 49154
}

62
chat_template.jinja Normal file
View File

@@ -0,0 +1,62 @@
{%- if messages[0]['role'] == 'system' %}
{%- set system_message = messages[0]['content'] %}
{%- set loop_messages = messages[1:] %}
{%- else %}
{%- set system_message = " Knowledge Cutoff Date: April 2024.
Today's Date: " + strftime_now('%B %d, %Y') + ". You are Granite, developed by IBM." %}
{%- if tools and documents %}
{%- set system_message = system_message + " You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.
Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data." %}
{%- elif tools %}
{%- set system_message = system_message + " You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request." %}
{%- elif documents %}
{%- set system_message = system_message + " Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data." %}
{%- elif thinking %}
{%- set system_message = system_message + " You are a helpful AI assistant.
Respond to every user query in a comprehensive and detailed way. You can write down your thoughts and reasoning process before responding. In the thought process, engage in a comprehensive cycle of analysis, summarization, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. In the response section, based on various attempts, explorations, and reflections from the thoughts section, systematically present the final solution that you deem correct. The response should summarize the thought process. Write your thoughts between <think></think> and write your response between <response></response> for each user query." %}
{%- else %}
{%- set system_message = system_message + " You are a helpful AI assistant." %}
{%- endif %}
{%- if 'citations' in controls and documents %}
{%- set system_message = system_message + '
Use the symbols <|start_of_cite|> and <|end_of_cite|> to indicate when a fact comes from a document in the search result, e.g <|start_of_cite|> {document_id: 1}my fact <|end_of_cite|> for a fact from document 1. Afterwards, list all the citations with their corresponding documents in an ordered list.' %}
{%- endif %}
{%- if 'hallucinations' in controls and documents %}
{%- set system_message = system_message + '
Finally, after the response is written, include a numbered list of sentences from the response with a corresponding risk value that are hallucinated and not based in the documents.' %}
{%- endif %}
{%- set loop_messages = messages %}
{%- endif %}
{{- '<|start_of_role|>system<|end_of_role|>' + system_message + '<|end_of_text|>
' }}
{%- if available_tools %}
{{- '<|start_of_role|>available_tools<|end_of_role|>' }}
{{- available_tools | tojson(indent=4) }}
{{- '<|end_of_text|>
' }}
{%- endif %}
{%- if documents %}
{%- for document in documents %}
{{- '<|start_of_role|>document {"document_id" :"' + document['doc_id'] | string + '"}<|end_of_role|>
' }}
{{- document['text'] }}
{{- '<|end_of_text|>
' }}
{%- if not loop.last %}
{{- '
'}}
{%- endif %}
{%- endfor %}
{%- endif %}
{%- for message in loop_messages %}
{{- '<|start_of_role|>' + message['role'] + '<|end_of_role|>' + message['content'] + '<|end_of_text|>
' }}
{%- if loop.last and add_generation_prompt %}
{{- '<|start_of_role|>assistant' }}
{%- if controls %}
{{- ' ' + controls | tojson()}}
{%- endif %}
{{- '<|end_of_role|>' }}
{%- endif %}
{%- endfor %}

79
config.json Normal file
View File

@@ -0,0 +1,79 @@
{
"architectures": [
"GraniteSpeechForConditionalGeneration"
],
"audio_token_index": 49159,
"downsample_rate": 5,
"encoder_config": {
"context_size": 200,
"conv_expansion_factor": 2,
"conv_kernel_size": 15,
"dim_head": 128,
"dropout": 0.1,
"feedforward_mult": 4,
"hidden_dim": 1024,
"input_dim": 160,
"max_pos_emb": 512,
"model_type": "granite_speech_encoder",
"num_heads": 8,
"num_layers": 16,
"output_dim": 256
},
"has_lora_adapter": true,
"initializer_range": 0.02,
"model_type": "granite_speech",
"projector_config": {
"_attn_implementation_autoset": true,
"attention_probs_dropout_prob": 0.1,
"cross_attention_frequency": 1,
"encoder_hidden_size": 1024,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 1024,
"initializer_range": 0.02,
"intermediate_size": 4096,
"layer_norm_eps": 1e-12,
"max_position_embeddings": 2048,
"model_type": "blip_2_qformer",
"num_attention_heads": 16,
"num_hidden_layers": 2,
"position_embedding_type": "absolute",
"use_qformer_text_input": false,
"vocab_size": 30522
},
"text_config": {
"_name_or_path": "/proj/speech/data/granite-3.3-8b-instruct/r250409a",
"architectures": [
"GraniteForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"attention_multiplier": 0.0078125,
"bos_token_id": 0,
"embedding_multiplier": 12.0,
"eos_token_id": 0,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 12800,
"logits_scaling": 16.0,
"max_position_embeddings": 131072,
"mlp_bias": false,
"model_type": "granite",
"num_attention_heads": 32,
"num_hidden_layers": 40,
"num_key_value_heads": 8,
"pad_token_id": 0,
"residual_multiplier": 0.22,
"rms_norm_eps": 1e-05,
"rope_scaling": null,
"rope_theta": 10000000.0,
"tie_word_embeddings": true,
"torch_dtype": "bfloat16",
"use_cache": true,
"vocab_size": 49160
},
"torch_dtype": "bfloat16",
"transformers_version": "4.52.4",
"window_size": 15
}

9
generation_config.json Normal file
View File

@@ -0,0 +1,9 @@
{
"_from_model_config": true,
"bos_token_id": 0,
"eos_token_id": 0,
"pad_token_id": 0,
"transformers_version": "4.52.4",
"use_cache": true,
"suppress_tokens": [49159]
}

48892
merges.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
model-00001-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1276f820dab1cc88a244cd1a93483f5756a60d3bb6df1ae3b7ec16e7f6487815
size 1996624616

3
model-00002-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd092699e0c2af5f0c0c8fb01116273b7a78b6646e15981710bd3b69fe789da4
size 1992382192

3
model-00003-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8b2850cb530272425b09bee47e4e37d4b611c1bd8b3646e64e1103f69b997b60
size 1992382224

3
model-00004-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7e75f3e50455db95d0458beeccdc0fc9cfe833123f60be5685d39d30eb6f3c24
size 1992382232

3
model-00005-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:70797e51dfe00e67e67a571931fb947423a25e847d8a183ef6548218c9ebc190
size 1992382232

3
model-00006-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fe96bdc622693137d757441ed071255443ba3798700cb67bc0f2965409e5c60e
size 1992382232

3
model-00007-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:af874ec068bff1fd53ebd19424955a5793f04c0951ac234740df2444a1f6b4ae
size 1992382232

3
model-00008-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bbc716fbbff32e943a982c4f46ac1221325dc1aa8d3d5d9c7f270d5b8cdc48b3
size 1992382232

3
model-00009-of-00009.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6fcdf24f4c53d42a9c562db81d58b2d4c9cddef74e405bed6f9cf622a2e23451
size 1354236592

960
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,960 @@
{
"metadata": {
"total_size": 17297414784
},
"weight_map": {
"encoder.input_linear.bias": "model-00009-of-00009.safetensors",
"encoder.input_linear.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.0.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.0.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.1.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.1.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.10.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.10.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.11.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.11.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.12.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.12.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.13.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.13.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.14.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.14.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.15.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.15.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.2.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.2.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.3.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.3.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.4.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.4.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.5.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.5.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.6.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.6.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.7.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.7.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.8.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.8.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.attn.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.attn.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.attn.rel_pos_emb.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.attn.to_kv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.attn.to_out.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.attn.to_out.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.attn.to_q.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.batch_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.batch_norm.num_batches_tracked": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.batch_norm.running_mean": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.batch_norm.running_var": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.batch_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.depth_conv.conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.down_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.down_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.up_conv.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.conv.up_conv.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff1.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff1.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff1.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff1.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff1.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff1.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff2.down_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff2.down_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff2.pre_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff2.pre_norm.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff2.up_proj.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.ff2.up_proj.weight": "model-00009-of-00009.safetensors",
"encoder.layers.9.post_norm.bias": "model-00009-of-00009.safetensors",
"encoder.layers.9.post_norm.weight": "model-00009-of-00009.safetensors",
"encoder.out.bias": "model-00009-of-00009.safetensors",
"encoder.out.weight": "model-00009-of-00009.safetensors",
"encoder.out_mid.bias": "model-00009-of-00009.safetensors",
"encoder.out_mid.weight": "model-00009-of-00009.safetensors",
"language_model.model.embed_tokens.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.0.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.1.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.10.input_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.10.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.input_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.11.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.input_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.12.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.input_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.13.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.14.input_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.14.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.input_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.15.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.input_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.16.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.input_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.17.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.input_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.18.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
"language_model.model.layers.19.input_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.19.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.2.input_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.2.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.20.input_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.20.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.input_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.21.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.input_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.22.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.input_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.23.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
"language_model.model.layers.24.input_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.24.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.input_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.25.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.input_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.26.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.input_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.27.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.input_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.28.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
"language_model.model.layers.29.input_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.29.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.3.input_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.3.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
"language_model.model.layers.30.input_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.30.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.input_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.31.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.input_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.32.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.input_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.33.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
"language_model.model.layers.34.input_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.34.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.input_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.35.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.input_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.36.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.input_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.37.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.input_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.38.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
"language_model.model.layers.39.input_layernorm.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.39.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
"language_model.model.layers.4.input_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.4.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.input_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.5.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.input_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.6.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.input_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.7.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.input_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.8.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
"language_model.model.layers.9.input_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.layers.9.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
"language_model.model.norm.weight": "model-00009-of-00009.safetensors",
"projector.linear.bias": "model-00009-of-00009.safetensors",
"projector.linear.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.attention.key.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.attention.key.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.attention.query.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.attention.query.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.attention.value.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.attention.value.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.output.LayerNorm.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.output.LayerNorm.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.output.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.attention.output.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.attention.key.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.attention.key.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.attention.query.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.attention.query.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.attention.value.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.attention.value.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.output.LayerNorm.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.output.LayerNorm.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.output.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.crossattention.output.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.intermediate_query.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.intermediate_query.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.output_query.LayerNorm.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.output_query.LayerNorm.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.output_query.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.0.output_query.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.attention.key.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.attention.key.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.attention.query.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.attention.query.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.attention.value.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.attention.value.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.output.LayerNorm.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.output.LayerNorm.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.output.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.attention.output.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.attention.key.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.attention.key.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.attention.query.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.attention.query.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.attention.value.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.attention.value.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.output.LayerNorm.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.output.LayerNorm.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.output.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.crossattention.output.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.intermediate_query.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.intermediate_query.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.output_query.LayerNorm.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.output_query.LayerNorm.weight": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.output_query.dense.bias": "model-00009-of-00009.safetensors",
"projector.qformer.encoder.layer.1.output_query.dense.weight": "model-00009-of-00009.safetensors",
"projector.qformer.layernorm.bias": "model-00009-of-00009.safetensors",
"projector.qformer.layernorm.weight": "model-00009-of-00009.safetensors",
"projector.query": "model-00009-of-00009.safetensors"
}
}

1
preprocessor_config.json Normal file
View File

@@ -0,0 +1 @@
{}

39
special_tokens_map.json Normal file
View File

@@ -0,0 +1,39 @@
{
"additional_special_tokens": [
"<|start_of_role|>",
"<|end_of_role|>",
"<|tool_call|>",
"<|start_of_cite|>",
"<|end_of_cite|>",
"<|start_of_plugin|>",
"<|end_of_plugin|>"
],
"bos_token": {
"content": "<|end_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|end_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|end_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<|end_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

244999
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

242
tokenizer_config.json Normal file
View File

@@ -0,0 +1,242 @@
{
"add_bos_token": false,
"add_prefix_space": false,
"added_tokens_decoder": {
"0": {
"content": "<|end_of_text|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<fim_prefix>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "<fim_middle>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"3": {
"content": "<fim_suffix>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"4": {
"content": "<fim_pad>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"5": {
"content": "<filename>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"6": {
"content": "<gh_stars>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"7": {
"content": "<issue_start>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"8": {
"content": "<issue_comment>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"9": {
"content": "<issue_closed>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"10": {
"content": "<jupyter_start>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"11": {
"content": "<jupyter_text>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"12": {
"content": "<jupyter_code>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"13": {
"content": "<jupyter_output>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"14": {
"content": "<empty_output>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"15": {
"content": "<commit_before>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"16": {
"content": "<commit_msg>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"17": {
"content": "<commit_after>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"18": {
"content": "<reponame>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49152": {
"content": "<|start_of_role|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49153": {
"content": "<|end_of_role|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49154": {
"content": "<|tool_call|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49155": {
"content": "<|start_of_cite|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49156": {
"content": "<|end_of_cite|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49157": {
"content": "<|start_of_plugin|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49158": {
"content": "<|end_of_plugin|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49159": {
"content": "<|audio|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [
"<|start_of_role|>",
"<|end_of_role|>",
"<|tool_call|>",
"<|start_of_cite|>",
"<|end_of_cite|>",
"<|start_of_plugin|>",
"<|end_of_plugin|>"
],
"bos_token": "<|end_of_text|>",
"clean_up_tokenization_spaces": true,
"eos_token": "<|end_of_text|>",
"errors": "replace",
"extra_special_tokens": {},
"model_max_length": 9223372036854775807,
"pad_token": "<|end_of_text|>",
"padding_side": "left",
"tokenizer_class": "GPT2Tokenizer",
"unk_token": "<|end_of_text|>",
"vocab_size": 49152
}

1
vocab.json Normal file
View File

File diff suppressed because one or more lines are too long