初始化项目，由ModelHub XC社区提供模型

Model: adept/fuyu-8b
Source: Original Platform

.gitattributes

@@ -0,0 +1,39 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.model filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,129 @@
+---
+license: cc-by-nc-4.0
+---
+# Fuyu-8B Model Card
+
+We’re releasing Fuyu-8B, a small version of the multimodal model that powers our product. The model is available on HuggingFace. We think Fuyu-8B is exciting because:
+1. It has a much simpler architecture and training procedure than other multi-modal models, which makes it easier to understand, scale, and deploy.
+2. It’s designed from the ground up for digital agents, so it can support arbitrary image resolutions, answer questions about graphs and diagrams, answer UI-based questions, and do fine-grained localization on screen images.
+3. It’s fast - we can get responses for large images in less than 100 milliseconds.
+4. Despite being optimized for our use-case, it performs well at standard image understanding benchmarks such as visual question-answering and natural-image-captioning.
+
+Please note that **the model we have released is a base model. We expect you to need to finetune the model for specific use cases like verbose captioning or multimodal chat.** In our experience, the model responds well to few-shotting and fine-tuning for a variety of use-cases. 
+
+## Model
+
+[Fuyu-8B](https://www.adept.ai/blog/fuyu-8b) is a multi-modal text and image transformer trained by [Adept AI](https://www.adept.ai/).
+
+Architecturally, Fuyu is a vanilla decoder-only transformer - there is no image encoder. 
+Image patches are instead linearly projected into the first layer of the transformer, bypassing the embedding lookup. 
+We simply treat the transformer decoder like an image transformer (albeit with no pooling and causal attention).
+See the below diagram for more details.
+
+![architecture](architecture.png)
+
+This simplification allows us to support arbitrary image resolutions. 
+To accomplish this, we treat the sequence of image tokens like the sequence of text tokens. 
+We remove image-specific position embeddings and feed in as many image tokens as necessary in raster-scan order. 
+To tell the model when a line has broken, we simply use a special image-newline character. 
+The model can use its existing position embeddings to reason about different image sizes, and we can use images of arbitrary size at training time, removing the need for separate high and low-resolution training stages.
+
+### Model Description
+
+- **Developed by:** Adept-AI
+- **Model type:** Decoder-only multi-modal transformer model 
+- **License:** [CC-BY-NC](https://creativecommons.org/licenses/by-nc/4.0/deed.en)
+- **Model Description:** This is a multi-modal model that can consume images and text and produce text. 
+- **Resources for more information:** Check out our [blog post](https://www.adept.ai/blog/fuyu-8b).
+
+## Evaluation
+Though not the focus of this model, we did evaluate it on standard image understanding benchmarks:
+
+| Eval Task           | Fuyu-8B | Fuyu-Medium       | LLaVA 1.5 (13.5B) | QWEN-VL (10B) | PALI-X (55B) | PALM-e-12B | PALM-e-562B |
+| ------------------- | ------- | ----------------- | ----------------- | ------------- | ------------ | ---------- | ----------- |
+| VQAv2               | 74.2    |     77.4          | 80                | 79.5          | 86.1         | 76.2       | 80.0        |
+| OKVQA               | 60.6    |     63.1          | n/a               | 58.6          | 66.1         | 55.5       | 66.1        |
+| COCO Captions       | 141     |     138           | n/a               | n/a           | 149          | 135        | 138         |
+| AI2D                | 64.5    |     73.7          | n/a               | 62.3          | 81.2         | n/a        | n/a         |
+
+## How to Use
+
+You can load the model and perform inference as follows:
+```python
+from transformers import FuyuProcessor, FuyuForCausalLM
+from PIL import Image
+import requests
+
+# load model and processor
+model_id = "adept/fuyu-8b"
+processor = FuyuProcessor.from_pretrained(model_id)
+model = FuyuForCausalLM.from_pretrained(model_id, device_map="cuda:0")
+
+# prepare inputs for the model
+text_prompt = "Generate a coco-style caption.\n"
+url = "https://huggingface.co/adept/fuyu-8b/resolve/main/bus.png"
+image = Image.open(requests.get(url, stream=True).raw)
+
+inputs = processor(text=text_prompt, images=image, return_tensors="pt").to("cuda:0")
+
+# autoregressively generate text
+generation_output = model.generate(**inputs, max_new_tokens=7)
+generation_text = processor.batch_decode(generation_output[:, -7:], skip_special_tokens=True)
+assert generation_text == ['A blue bus parked on the side of a road.']
+```
+
+N.B.: The token `|SPEAKER|` is a placeholder token for image patch embeddings, so it will show up in the model context (e.g., in the portion of `generation_output` representing the model context).
+`|NEWLINE|` is the "image newline" token, denoting new rows in the raster scan order input of the image patches.
+`\x04` is the "beginning of answer" token.
+
+Fuyu can also perform some question answering on natural images and charts/diagrams (thought fine-tuning may be required for good performance):
+```python
+text_prompt = "What color is the bus?\n"
+url = "https://huggingface.co/adept/fuyu-8b/resolve/main/bus.png"
+image = Image.open(requests.get(url, stream=True).raw)
+
+inputs = processor(text=text_prompt, images=image, return_tensors="pt").to("cuda:0")
+
+generation_output = model.generate(**inputs, max_new_tokens=6)
+generation_text = processor.batch_decode(generation_output[:, -6:], skip_special_tokens=True)
+assert generation_text == ["The bus is blue.\n"]
+
+
+text_prompt = "What is the highest life expectancy at birth of male?\n"
+url = "https://huggingface.co/adept/fuyu-8b/resolve/main/chart.png"
+image = Image.open(requests.get(url, stream=True).raw)
+
+model_inputs = processor(text=text_prompt, images=image, return_tensors="pt").to("cuda:0")
+
+generation_output = model.generate(**model_inputs, max_new_tokens=16)
+generation_text = processor.batch_decode(generation_output[:, -16:], skip_special_tokens=True)
+assert generation_text == ["The life expectancy at birth of males in 2018 is 80.7.\n"]
+```
+For best performance, it's recommended to end questions with `\n`, as shown above!
+
+## Uses
+
+### Direct Use
+
+The model is intended for research purposes only. 
+**Because this is a raw model release, we have not added further finetuning, postprocessing or sampling strategies to control for undesirable outputs. You should expect to have to fine-tune the model for your use-case.**
+
+Possible research areas and tasks include
+
+- Applications in computer control or digital agents.
+- Research on multi-modal models generally.
+
+Excluded uses are described below.
+
+### Out-of-Scope Use
+
+The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
+
+## Limitations and Bias
+
+### Limitations
+
+- Faces and people in general may not be generated properly.
+
+### Bias
+While the capabilities of these models are impressive, they can also reinforce or exacerbate social biases.

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "|ENDOFTEXT|": 71013
+}

config.json

@@ -0,0 +1,33 @@
+{
+  "architectures": [
+    "FuyuForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 71013,
+  "hidden_act": "relu2",
+  "hidden_dropout": 0.0,
+  "hidden_size": 4096,
+  "image_size": 300,
+  "initializer_range": 0.02,
+  "intermediate_size": 16384,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 16384,
+  "model_type": "fuyu",
+  "num_attention_heads": 64,
+  "num_channels": 3,
+  "num_hidden_layers": 36,
+  "partial_rotary_factor": 0.5,
+  "patch_size": 30,
+  "qk_layernorm": true,
+  "rope_scaling": null,
+  "rope_theta": 25000.0,
+  "text_config": {
+    "model_type": "persimmon"
+  },
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.35.0.dev0",
+  "use_cache": true,
+  "vocab_size": 262144
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 71013,
+  "transformers_version": "4.35.0.dev0"
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0360c23415254b77cf052e5dc5b57ecd98f28d65e1e4cb7a222c9fa35e97c0e
+size 9934219312

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb67849dfa2f360d8bf653386a3cbc36002b8921de62f9916f70872ab929e3eb
+size 8882334152

model.safetensors.index.json

@@ -0,0 +1,589 @@
+{
+  "metadata": {
+    "total_size": 18816477184
+  },
+  "weight_map": {
+    "language_model.lm_head.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.final_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.final_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.0.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.0.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.1.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.10.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.11.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.12.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.13.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.14.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.15.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.16.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.17.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.18.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.19.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.2.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.20.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.20.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.21.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.22.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.23.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.24.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.25.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.26.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.27.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.28.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.29.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.3.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.3.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.30.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.30.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.31.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.32.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.33.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.34.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.input_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.mlp.dense_4h_to_h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.mlp.dense_4h_to_h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.mlp.dense_h_to_4h.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.mlp.dense_h_to_4h.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.post_attention_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.dense.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.dense.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.k_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.k_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.q_layernorm.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.q_layernorm.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.query_key_value.bias": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.35.self_attn.query_key_value.weight": "model-00002-of-00002.safetensors",
+    "language_model.model.layers.4.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.4.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.5.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.6.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.7.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.8.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.input_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.mlp.dense_4h_to_h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.mlp.dense_4h_to_h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.mlp.dense_h_to_4h.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.mlp.dense_h_to_4h.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.post_attention_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.dense.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.dense.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.k_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.k_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.q_layernorm.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.q_layernorm.weight": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.query_key_value.bias": "model-00001-of-00002.safetensors",
+    "language_model.model.layers.9.self_attn.query_key_value.weight": "model-00001-of-00002.safetensors",
+    "vision_embed_tokens.bias": "model-00002-of-00002.safetensors",
+    "vision_embed_tokens.weight": "model-00002-of-00002.safetensors"
+  }
+}

preprocessor_config.json

@@ -0,0 +1,8 @@
+{
+  "image_processor_type": "FuyuImageProcessor",
+  "padding_mode": "constant",
+  "padding_value": 1.0,
+  "processor_class": "FuyuProcessor",
+  "target_height": 1080,
+  "target_width": 1920
+}

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "|ENDOFTEXT|"
+  ],
+  "bos_token": "|ENDOFTEXT|",
+  "eos_token": "|ENDOFTEXT|",
+  "unk_token": "<unk>"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20119416a890c3fefc86210999fb8189dde194c12f7f861dbb3bc7a922baaabb
+size 16454569

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:407d001d18582959881b30ff39b660d4bd5ee50957d03f57daeeb3059e9db54c
+size 6128796

tokenizer_config.json

@@ -0,0 +1,36 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "71013": {
+      "content": "|ENDOFTEXT|",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "|ENDOFTEXT|"
+  ],
+  "bos_token": "|ENDOFTEXT|",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "|ENDOFTEXT|",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": null,
+  "processor_class": "FuyuProcessor",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}