初始化项目，由ModelHub XC社区提供模型

Model: HuggingFaceH4/starchat2-15b-v0.1
Source: Original Platform

.gitattributes

@@ -0,0 +1,43 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+model_logo.png filter=lfs diff=lfs merge=lfs -text
+model-00001-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00002-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00003-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00004-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00005-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00006-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
+model-00007-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,159 @@
+---
+base_model: HuggingFaceH4/starchat2-15b-sft-v0.1
+tags:
+- alignment-handbook
+- generated_from_trainer
+datasets:
+- HuggingFaceH4/ultrafeedback_binarized
+- HuggingFaceH4/orca_dpo_pairs
+model-index:
+- name: starchat2-15b-v0.1
+  results: []
+---
+
+<img src="https://huggingface.co/HuggingFaceH4/starchat2-15b-v0.1/resolve/main/model_logo.png" alt="StarChat2 15B Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
+
+# Model Card for StarChat2 15B
+
+StarChat is a series of language models that are trained to act as helpful coding assistants. StarChat2 is the latest model in the series, and is a fine-tuned version of [StarCoder2](https://huggingface.co/bigcode/starcoder2-15b) that was trained with SFT and DPO on a mix of synthetic datasets.
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+- **Model type:** A 16B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
+- **Language(s) (NLP):** Primarily English and 600+ programming languages.
+- **License:** BigCode Open RAIL-M v1
+- **Finetuned from model:** [bigcode/starcoder2-15b](https://huggingface.co/bigcode/starcoder2-15b)
+
+### Model Sources
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** https://github.com/huggingface/alignment-handbook
+- **Demo:** https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
+
+## Performance
+
+StarChat2 15B was trained to balance chat and programming capabilities. It achieves strong performance on chat benchmarks like [MT Bench](https://huggingface.co/spaces/lmsys/mt-bench) and [IFEval](https://arxiv.org/abs/2311.07911), as well as the canonical HumanEval benchmark for Python code completion. The scores reported below were obtained using the [LightEval](https://github.com/huggingface/lighteval) evaluation suite (commit `988959cb905df4baa050f82b4d499d46e8b537f2`) and each prompt has been formatted with the model's corresponding chat template to simulate real-world usage. This is why some scores may differ from those reported in technical reports or on the Open LLM Leaderboard. 
+
+| Model                                                                                           | MT Bench | IFEval | HumanEval |
+|-------------------------------------------------------------------------------------------------|---------:|-------:|----------:|
+| [starchat2-15b-v0.1](https://huggingface.co/HuggingFaceH4/starchat2-15b-v0.1)                   |     7.66 |  35.12 |     71.34 |
+| [deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) |     4.17 |  14.23 |     80.48 |
+| [CodeLlama-13b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-13b-Instruct-hf)         |     6.80 |  43.44 |     50.60 |
+
+
+## Intended uses & limitations
+
+The model was fine-tuned on a blend of chat, code, math, and reasoning datasets. As a result, the model can be used for chat and you can check out our [demo](https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground) to test its coding capabilities. 
+
+Here's how you can run the model using the `pipeline()` function from 🤗 Transformers:
+
+```python
+# pip install 'transformers @ git+https://github.com/huggingface/transformers.git@831bc25d8fdb85768402f772cf65cc3d7872b211'
+# pip install accelerate
+
+import torch
+from transformers import pipeline
+
+pipe = pipeline(
+    "text-generation",
+    model="HuggingFaceH4/starchat2-15b-v0.1",
+    device_map="auto",
+    torch_dtype=torch.bfloat16,
+)
+messages = [
+    {
+        "role": "system",
+        "content": "You are StarChat2, an expert programming assistant",
+    },
+    {"role": "user", "content": "Write a simple website in HTML. When a user clicks the button, it shows a random Chuck Norris joke."},
+]
+outputs = pipe(
+    messages,
+    max_new_tokens=512,
+    do_sample=True,
+    temperature=0.7,
+    top_k=50,
+    top_p=0.95,
+    stop_sequence="<|im_end|>",
+)
+print(outputs[0]["generated_text"][-1]["content"])
+```
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+StarChat2 15B has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). 
+Models trained primarily on code data will also have a more skewed demographic bias commensurate with the demographics of the GitHub community, for more on this see the [StarCoder2 dataset](https://huggingface.co/datasets/bigcode/the-stack-v2)
+
+Since the base model was pretrained on a large corpus of code, it may produce code snippets that are syntactically valid but semantically incorrect. 
+For example, it may produce code that does not compile or that produces incorrect results.  
+It may also produce code that is vulnerable to security exploits.  
+We have observed the model also has a tendency to produce false URLs which should be carefully inspected before clicking.
+
+StarChat2 15B was fine-tuned from the base model [StarCoder2](https://huggingface.co/bigcode/starcoder2-15b), please refer to its model card's [Limitations Section](https://huggingface.co/bigcode/starcoder2-15b#limitations) for relevant information. 
+In particular, the model was evaluated on some categories of gender biases, propensity for toxicity, and risk of suggesting code completions with known security flaws; these evaluations are reported in its [technical report](https://huggingface.co/papers/2402.19173).
+
+
+## Training details
+
+This model is a fine-tuned version of [starchat2-15b-sft-v0.1](https://huggingface.co/HuggingFaceH4/starchat2-15b-sft-v0.1) on the HuggingFaceH4/ultrafeedback_binarized and the HuggingFaceH4/orca_dpo_pairs datasets. Check out the recipe in the [Alignment Handbook](https://github.com/huggingface/alignment-handbook) for more details.
+
+It achieves the following results on the evaluation set:
+- Loss: 0.4347
+- Rewards/chosen: -0.9461
+- Rewards/rejected: -2.7745
+- Rewards/accuracies: 0.7658
+- Rewards/margins: 1.8284
+- Logps/rejected: -322.1934
+- Logps/chosen: -316.1898
+- Logits/rejected: -2.3817
+- Logits/chosen: -2.3005
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-07
+- train_batch_size: 2
+- eval_batch_size: 4
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 8
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 128
+- total_eval_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 2
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.717         | 0.17  | 100  | 0.6006          | -0.0924        | -0.2899          | 0.6329             | 0.1975          | -272.5022      | -299.1165    | -2.5313         | -2.4191       |
+| 0.6273        | 0.35  | 200  | 0.5160          | -0.3994        | -0.9461          | 0.6930             | 0.5467          | -285.6261      | -305.2568    | -2.5281         | -2.4278       |
+| 0.5538        | 0.52  | 300  | 0.4781          | -0.6589        | -1.5892          | 0.7247             | 0.9302          | -298.4870      | -310.4470    | -2.4996         | -2.4110       |
+| 0.5056        | 0.7   | 400  | 0.4594          | -0.8283        | -2.1332          | 0.7437             | 1.3050          | -309.3687      | -313.8344    | -2.4472         | -2.3644       |
+| 0.4983        | 0.87  | 500  | 0.4512          | -0.7758        | -2.2806          | 0.7468             | 1.5049          | -312.3167      | -312.7843    | -2.4223         | -2.3404       |
+| 0.4662        | 1.04  | 600  | 0.4431          | -0.7839        | -2.4016          | 0.7658             | 1.6177          | -314.7355      | -312.9465    | -2.4049         | -2.3215       |
+| 0.4411        | 1.22  | 700  | 0.4415          | -1.0090        | -2.7582          | 0.7690             | 1.7492          | -321.8679      | -317.4481    | -2.3840         | -2.3016       |
+| 0.471         | 1.39  | 800  | 0.4368          | -0.9617        | -2.7445          | 0.7690             | 1.7828          | -321.5930      | -316.5019    | -2.3809         | -2.2991       |
+| 0.4485        | 1.57  | 900  | 0.4351          | -0.9490        | -2.7594          | 0.7722             | 1.8103          | -321.8916      | -316.2497    | -2.3815         | -2.3004       |
+| 0.4411        | 1.74  | 1000 | 0.4348          | -0.9293        | -2.7469          | 0.7658             | 1.8176          | -321.6409      | -315.8547    | -2.3823         | -2.3011       |
+| 0.4499        | 1.92  | 1100 | 0.4348          | -0.9482        | -2.7767          | 0.7658             | 1.8285          | -322.2369      | -316.2320    | -2.3828         | -2.3012       |
+
+
+### Framework versions
+
+- Transformers 4.39.0.dev0
+- Pytorch 2.1.2+cu121
+- Datasets 2.16.1
+- Tokenizers 0.15.1

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<|im_end|>": 49153,
+  "<|im_start|>": 49152
+}

all_results.json

@@ -0,0 +1,21 @@
+{
+    "epoch": 2.0,
+    "eval_logits/chosen": -2.3005340099334717,
+    "eval_logits/rejected": -2.3817031383514404,
+    "eval_logps/chosen": -316.1898498535156,
+    "eval_logps/rejected": -322.1933898925781,
+    "eval_loss": 0.4346597194671631,
+    "eval_rewards/accuracies": 0.7658227682113647,
+    "eval_rewards/chosen": -0.9460535049438477,
+    "eval_rewards/margins": 1.8284220695495605,
+    "eval_rewards/rejected": -2.7744758129119873,
+    "eval_runtime": 117.6177,
+    "eval_samples": 2500,
+    "eval_samples_per_second": 21.255,
+    "eval_steps_per_second": 0.672,
+    "train_loss": 0.5184940074794384,
+    "train_runtime": 19743.6623,
+    "train_samples": 73494,
+    "train_samples_per_second": 7.445,
+    "train_steps_per_second": 0.058
+}

config.json

@@ -0,0 +1,31 @@
+{
+  "_name_or_path": "HuggingFaceH4/starcoder2-15b-ift",
+  "architectures": [
+    "Starcoder2ForCausalLM"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": 0,
+  "embedding_dropout": 0.1,
+  "eos_token_id": 0,
+  "hidden_act": "gelu_pytorch_tanh",
+  "hidden_size": 6144,
+  "initializer_range": 0.01275,
+  "intermediate_size": 24576,
+  "max_position_embeddings": 16384,
+  "mlp_type": "default",
+  "model_type": "starcoder2",
+  "norm_epsilon": 1e-05,
+  "norm_type": "layer_norm",
+  "num_attention_heads": 48,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 4,
+  "residual_dropout": 0.1,
+  "rope_theta": 100000,
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.39.0.dev0",
+  "use_bias": true,
+  "use_cache": true,
+  "vocab_size": 49154
+}

configuration.json

@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

eval_results.json

@@ -0,0 +1,16 @@
+{
+    "epoch": 2.0,
+    "eval_logits/chosen": -2.3005340099334717,
+    "eval_logits/rejected": -2.3817031383514404,
+    "eval_logps/chosen": -316.1898498535156,
+    "eval_logps/rejected": -322.1933898925781,
+    "eval_loss": 0.4346597194671631,
+    "eval_rewards/accuracies": 0.7658227682113647,
+    "eval_rewards/chosen": -0.9460535049438477,
+    "eval_rewards/margins": 1.8284220695495605,
+    "eval_rewards/rejected": -2.7744758129119873,
+    "eval_runtime": 117.6177,
+    "eval_samples": 2500,
+    "eval_samples_per_second": 21.255,
+    "eval_steps_per_second": 0.672
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "4.39.0.dev0"
+}

model-00001-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6c48426c7c3775db521715b86af7f9e707ee3463d3ea2a2d7fab62566228879
+size 4908132312

model-00002-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fceeb5864469f8ccab6be64005fcb25ef2b603167ec30de940faf877e106cdc
+size 4996327240

model-00003-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce13a732d413eff9378b505a814716fdbd2113fc6a18a988d7ff3a08dafa410f
+size 4983729152

model-00004-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30b986168df98421a5f475b5754bbc43c085b59dc987eebd7b6caf414dc48c72
+size 4996327312

model-00005-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e418fb172f02e39c64231be87bd548f70a6669e995a47202add964548521b576
+size 4983729152

model-00006-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90df89c6c5e2f34e2964ff7fc3f195cd5440b6ebe509ff78df20532cb5a1956a
+size 4996327312

model-00007-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d63cc490c3ae8a3091850ad146166725fc92eb4cb7561f72305fd9e73483eb60
+size 2051327128

model.safetensors.index.json

@@ -0,0 +1,651 @@
+{
+  "metadata": {
+    "total_size": 31915827200
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00007-of-00007.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.input_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.input_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.10.input_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.10.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
+    "model.layers.10.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.10.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.input_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.12.input_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.12.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
+    "model.layers.12.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
+    "model.layers.12.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.12.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.12.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.12.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.12.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.12.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.12.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.13.input_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.13.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
+    "model.layers.13.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.13.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.input_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.input_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.input_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.input_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.18.input_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.18.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
+    "model.layers.18.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
+    "model.layers.18.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.18.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.18.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.18.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.18.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.18.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.18.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
+    "model.layers.19.input_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.19.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
+    "model.layers.19.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.19.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.2.input_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.2.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
+    "model.layers.2.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.2.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.20.input_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.20.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
+    "model.layers.20.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.20.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.input_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.input_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.input_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.input_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.25.input_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.25.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
+    "model.layers.25.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
+    "model.layers.25.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.25.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.25.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.25.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.25.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.25.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.25.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
+    "model.layers.26.input_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.26.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
+    "model.layers.26.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.26.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.input_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.input_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.input_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.3.input_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.3.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
+    "model.layers.3.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.3.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.30.input_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.30.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
+    "model.layers.30.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.30.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.31.input_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.31.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
+    "model.layers.31.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
+    "model.layers.31.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.31.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.31.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.31.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.31.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.31.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.31.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
+    "model.layers.32.input_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.32.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
+    "model.layers.32.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.32.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.input_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.input_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.input_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.input_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.36.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.input_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.37.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.38.input_layernorm.bias": "model-00007-of-00007.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00007-of-00007.safetensors",
+    "model.layers.38.mlp.c_fc.bias": "model-00007-of-00007.safetensors",
+    "model.layers.38.mlp.c_fc.weight": "model-00007-of-00007.safetensors",
+    "model.layers.38.mlp.c_proj.bias": "model-00007-of-00007.safetensors",
+    "model.layers.38.mlp.c_proj.weight": "model-00007-of-00007.safetensors",
+    "model.layers.38.post_attention_layernorm.bias": "model-00007-of-00007.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00007-of-00007.safetensors",
+    "model.layers.38.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.38.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.38.self_attn.o_proj.bias": "model-00007-of-00007.safetensors",
+    "model.layers.38.self_attn.o_proj.weight": "model-00007-of-00007.safetensors",
+    "model.layers.38.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.38.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.38.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
+    "model.layers.38.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
+    "model.layers.39.input_layernorm.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00007-of-00007.safetensors",
+    "model.layers.39.mlp.c_fc.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.mlp.c_fc.weight": "model-00007-of-00007.safetensors",
+    "model.layers.39.mlp.c_proj.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.mlp.c_proj.weight": "model-00007-of-00007.safetensors",
+    "model.layers.39.post_attention_layernorm.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.k_proj.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.k_proj.weight": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.o_proj.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.o_proj.weight": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.q_proj.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.q_proj.weight": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.v_proj.bias": "model-00007-of-00007.safetensors",
+    "model.layers.39.self_attn.v_proj.weight": "model-00007-of-00007.safetensors",
+    "model.layers.4.input_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.4.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
+    "model.layers.4.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.4.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.5.input_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.5.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
+    "model.layers.5.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
+    "model.layers.5.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.5.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.5.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.5.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.5.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.5.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.5.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
+    "model.layers.6.input_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.6.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
+    "model.layers.6.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.6.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.input_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.input_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.input_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
+    "model.norm.bias": "model-00007-of-00007.safetensors",
+    "model.norm.weight": "model-00007-of-00007.safetensors"
+  }
+}

model_logo.png

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85603fd9cdff6d10e73147bfc3ed2c07cd7e8054a44d295bc2bebd6c51837948
+size 3137166

runs/Mar09_21-24-05_ip-26-0-161-138/events.out.tfevents.1710019788.ip-26-0-161-138.3654361.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7f80ab9898f9f9563229fd2f3670229119bf4fe2b847e15f944944cd3106b1e
+size 92659

runs/Mar09_21-24-05_ip-26-0-161-138/events.out.tfevents.1710039650.ip-26-0-161-138.3654361.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bd62f7cba2a3a1f0a8641d1442db4fc7c8c38bef615a36ae2d793f24b52e162
+size 828

special_tokens_map.json

@@ -0,0 +1,34 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": {
+    "content": "<|im_start|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,338 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<fim_prefix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<fim_middle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<fim_suffix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<fim_pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<repo_name>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<file_sep>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<jupyter_script>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<code_to_intermediate>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "<intermediate_to_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "<pr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "19": {
+      "content": "<pr_status>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "20": {
+      "content": "<pr_is_merged>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "21": {
+      "content": "<pr_base>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "22": {
+      "content": "<pr_file>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "23": {
+      "content": "<pr_base_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "24": {
+      "content": "<pr_diff>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "25": {
+      "content": "<pr_diff_hunk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "26": {
+      "content": "<pr_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "27": {
+      "content": "<pr_event_id>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "28": {
+      "content": "<pr_review>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "29": {
+      "content": "<pr_review_state>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30": {
+      "content": "<pr_review_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "31": {
+      "content": "<pr_in_reply_to_review_id>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32": {
+      "content": "<pr_in_reply_to_comment_id>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "33": {
+      "content": "<pr_diff_hunk_comment_line>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "34": {
+      "content": "<NAME>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "35": {
+      "content": "<EMAIL>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "36": {
+      "content": "<KEY>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "37": {
+      "content": "<PASSWORD>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49152": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49153": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": "<|im_start|>",
+  "chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|im_end|>",
+  "model_max_length": 2048,
+  "pad_token": "<|im_end|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 2.0,
+    "train_loss": 0.5184940074794384,
+    "train_runtime": 19743.6623,
+    "train_samples": 73494,
+    "train_samples_per_second": 7.445,
+    "train_steps_per_second": 0.058
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:146d3e8d29927d670f4fbefc63112a9a715f1d6848b08ec05dfa4b65d7c3b689
+size 6520