初始化项目，由ModelHub XC社区提供模型

Model: EleutherAI/pythia-12b
Source: Original Platform

.gitattributes

@@ -0,0 +1,49 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*.tfevents* filter=lfs diff=lfs merge=lfs -text
+*.db* filter=lfs diff=lfs merge=lfs -text
+*.ark* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
+**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.gguf* filter=lfs diff=lfs merge=lfs -text
+*.ggml filter=lfs diff=lfs merge=lfs -text
+*.llamafile* filter=lfs diff=lfs merge=lfs -text
+*.pt2 filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,280 @@
+---
+language:
+- en
+tags:
+- pytorch
+- causal-lm
+- pythia
+license: apache-2.0
+datasets:
+- EleutherAI/pile
+---
+
+The *Pythia Scaling Suite* is a collection of models developed to facilitate 
+interpretability research [(see paper)](https://arxiv.org/pdf/2304.01373.pdf). 
+It contains two sets of eight models of sizes 
+70M, 160M, 410M, 1B, 1.4B, 2.8B, 6.9B, and 12B. For each size, there are two 
+models: one trained on the Pile, and one trained on the Pile after the dataset 
+has been globally deduplicated. All 8 model sizes are trained on the exact 
+same data, in the exact same order. We also provide 154 intermediate 
+checkpoints per model, hosted on Hugging Face as branches.
+
+The Pythia model suite was deliberately designed to promote scientific 
+research on large language models, especially interpretability research. 
+Despite not centering downstream performance as a design goal, we find the 
+models <a href="#evaluations">match or exceed</a> the performance of 
+similar and same-sized models, such as those in the OPT and GPT-Neo suites.
+
+<details>
+  <summary style="font-weight: 600">Past early release and naming convention.</summary>
+
+Previously, we released an early version of the Pythia suite to the public. 
+However, we decided to retrain the model suite to address a few hyperparameter 
+discrepancies. This model card <a href="#changelog">lists the changes</a>; 
+see appendix B in the Pythia paper for further discussion. We found no 
+difference in benchmark performance between the two Pythia versions. 
+The old models are 
+[still available](https://huggingface.co/models?other=pythia_v0), but we 
+suggest the retrained suite if you are just starting to use Pythia.<br>
+**This is the current release.**
+
+Please note that all models in the *Pythia* suite were renamed in January 
+2023. For clarity, a <a href="#naming-convention-and-parameter-count">table 
+comparing the old and new names</a> is provided in this model card, together 
+with exact parameter counts.
+</details>
+<br>
+
+# Pythia-12B
+
+## Model Details
+
+- Developed by: [EleutherAI](http://eleuther.ai)
+- Model type: Transformer-based Language Model
+- Language: English
+- Learn more: [Pythia's GitHub repository](https://github.com/EleutherAI/pythia)
+ for training procedure, config files, and details on how to use.
+ [See paper](https://arxiv.org/pdf/2304.01373.pdf) for more evals and implementation
+ details.
+- Library: [GPT-NeoX](https://github.com/EleutherAI/gpt-neox)
+- License: Apache 2.0
+- Contact: to ask questions about this model, join the [EleutherAI 
+Discord](https://discord.gg/zBGx3azzUn), and post them in `#release-discussion`.
+ Please read the existing *Pythia* documentation before asking about it in the 
+ EleutherAI Discord. For general correspondence: [contact@eleuther.
+ ai](mailto:contact@eleuther.ai).
+
+<figure>
+
+| Pythia model | Non-Embedding Params | Layers | Model Dim | Heads | Batch Size | Learning Rate         | Equivalent Models      |
+| -----------: | -------------------: | :----: | :-------: | :---: | :--------: | :-------------------: | :--------------------: |
+| 70M          | 18,915,328           | 6      | 512       | 8     | 2M         | 1.0 x 10<sup>-3</sup> | —                      |
+| 160M         | 85,056,000           | 12     | 768       | 12    | 2M         | 6.0 x 10<sup>-4</sup> | GPT-Neo 125M, OPT-125M |
+| 410M         | 302,311,424          | 24     | 1024      | 16    | 2M         | 3.0 x 10<sup>-4</sup> | OPT-350M               |
+| 1.0B         | 805,736,448          | 16     | 2048      | 8     | 2M         | 3.0 x 10<sup>-4</sup> | —                      |
+| 1.4B         | 1,208,602,624        | 24     | 2048      | 16    | 2M         | 2.0 x 10<sup>-4</sup> | GPT-Neo 1.3B, OPT-1.3B |
+| 2.8B         | 2,517,652,480        | 32     | 2560      | 32    | 2M         | 1.6 x 10<sup>-4</sup> | GPT-Neo 2.7B, OPT-2.7B |
+| 6.9B         | 6,444,163,072        | 32     | 4096      | 32    | 2M         | 1.2 x 10<sup>-4</sup> | OPT-6.7B               |
+| 12B          | 11,327,027,200       | 36     | 5120      | 40    | 2M         | 1.2 x 10<sup>-4</sup> | —                      |
+<figcaption>Engineering details for the <i>Pythia Suite</i>. Deduped and 
+non-deduped models of a given size have the same hyperparameters. “Equivalent” 
+models have <b>exactly</b> the same architecture, and the same number of 
+non-embedding parameters.</figcaption>
+</figure>
+
+## Uses and Limitations
+
+### Intended Use
+
+The primary intended use of Pythia is research on the behavior, functionality, 
+and limitations of large language models. This suite is intended to provide 
+a controlled setting for performing scientific experiments. We also provide 
+154 checkpoints per model: initial `step0`, 10 log-spaced checkpoints 
+`step{1,2,4...512}`, and 143 evenly-spaced checkpoints from `step1000` to 
+`step143000`. These checkpoints are hosted on Hugging Face as branches. Note 
+that branch `143000` corresponds exactly to the model checkpoint on the `main` 
+branch of each model.
+
+You may also further fine-tune and adapt Pythia-12B for deployment, 
+as long as your use is in accordance with the Apache 2.0 license. Pythia 
+models work with the Hugging Face [Transformers 
+Library](https://huggingface.co/docs/transformers/index). If you decide to use 
+pre-trained Pythia-12B as a basis for your fine-tuned model, please 
+conduct your own risk and bias assessment. 
+
+### Out-of-scope use
+
+The Pythia Suite is **not** intended for deployment. It is not a in itself 
+a product and cannot be used for human-facing interactions. For example, 
+the model may generate harmful or offensive text. Please evaluate the risks
+associated with your particular use case.
+
+Pythia models are English-language only, and are not suitable for translation 
+or generating text in other languages.
+
+Pythia-12B has not been fine-tuned for downstream contexts in which 
+language models are commonly deployed, such as writing genre prose, 
+or commercial chatbots. This means Pythia-12B will **not** 
+respond to a given prompt the way a product like ChatGPT does. This is because,
+ unlike this model, ChatGPT was fine-tuned using methods such as Reinforcement 
+Learning from Human Feedback (RLHF) to better “follow” human instructions.
+
+### Limitations and biases
+
+The core functionality of a large language model is to take a string of text 
+and predict the next token. The token used by the model need not produce the 
+most “accurate” text. Never rely on Pythia-12B to produce factually accurate 
+output.
+
+This model was trained on [the Pile](https://pile.eleuther.ai/), a dataset 
+known to contain profanity and texts that are lewd or otherwise offensive. 
+See [Section 6 of the Pile paper](https://arxiv.org/abs/2101.00027) for a 
+discussion of documented biases with regards to gender, religion, and race. 
+Pythia-12B may produce socially unacceptable or undesirable text, *even if* 
+the prompt itself does not include anything explicitly offensive. 
+
+If you plan on using text generated through, for example, the Hosted Inference 
+API, we recommend having a human curate the outputs of this language model 
+before presenting it to other people. Please inform your audience that the 
+text was generated by Pythia-12B.
+
+### Quickstart
+
+Pythia models can be loaded and used via the following code, demonstrated here 
+for the third `pythia-70m-deduped` checkpoint:
+
+```python
+from transformers import GPTNeoXForCausalLM, AutoTokenizer
+
+model = GPTNeoXForCausalLM.from_pretrained(
+  "EleutherAI/pythia-70m-deduped",
+  revision="step3000",
+  cache_dir="./pythia-70m-deduped/step3000",
+)
+
+tokenizer = AutoTokenizer.from_pretrained(
+  "EleutherAI/pythia-70m-deduped",
+  revision="step3000",
+  cache_dir="./pythia-70m-deduped/step3000",
+)
+
+inputs = tokenizer("Hello, I am", return_tensors="pt")
+tokens = model.generate(**inputs)
+tokenizer.decode(tokens[0])
+```
+
+Revision/branch `step143000` corresponds exactly to the model checkpoint on 
+the `main` branch of each model.<br>
+For more information on how to use all Pythia models, see [documentation on 
+GitHub](https://github.com/EleutherAI/pythia).
+
+## Training
+
+### Training data
+
+[The Pile](https://pile.eleuther.ai/) is a 825GiB general-purpose dataset in 
+English. It was created by EleutherAI specifically for training large language 
+models. It contains texts from 22 diverse sources, roughly broken down into 
+five categories: academic writing (e.g. arXiv), internet (e.g. CommonCrawl), 
+prose (e.g. Project Gutenberg), dialogue (e.g. YouTube subtitles), and 
+miscellaneous (e.g. GitHub, Enron Emails). See [the Pile 
+paper](https://arxiv.org/abs/2101.00027) for a breakdown of all data sources, 
+methodology, and a discussion of ethical implications. Consult [the 
+datasheet](https://arxiv.org/abs/2201.07311) for more detailed documentation 
+about the Pile and its component datasets. The Pile can be downloaded from 
+the [official website](https://pile.eleuther.ai/), or from a [community 
+mirror](https://the-eye.eu/public/AI/pile/).<br>
+The Pile was **not** deduplicated before being used to train Pythia-12B.
+
+### Training procedure
+
+All models were trained on the exact same data, in the exact same order. Each 
+model saw 299,892,736,000 tokens during training, and 143 checkpoints for each 
+model are saved every 2,097,152,000 tokens, spaced evenly throughout training, 
+from `step1000` to `step143000` (which is the same as `main`). In addition, we 
+also provide frequent early checkpoints: `step0` and `step{1,2,4...512}`.
+This corresponds to training for just under 1 epoch on the Pile for 
+non-deduplicated models, and about 1.5 epochs on the deduplicated Pile.
+
+All *Pythia* models trained for 143000 steps at a batch size 
+of 2M (2,097,152 tokens).<br>
+See [GitHub](https://github.com/EleutherAI/pythia) for more details on training
+ procedure, including [how to reproduce 
+ it](https://github.com/EleutherAI/pythia/blob/main/README.md#reproducing-training).<br>
+Pythia uses the same tokenizer as [GPT-NeoX-
+20B](https://huggingface.co/EleutherAI/gpt-neox-20b).
+
+## Evaluations
+
+All 16 *Pythia* models were evaluated using the [LM Evaluation 
+Harness](https://github.com/EleutherAI/lm-evaluation-harness). You can access 
+the results by model and step at `results/json/*` in the [GitHub 
+repository](https://github.com/EleutherAI/pythia/tree/main/results/json/).<br>
+Expand the sections below to see plots of evaluation results for all 
+Pythia and Pythia-deduped models compared with OPT and BLOOM.
+
+<details>
+  <summary>LAMBADA – OpenAI</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/lambada_openai_v1.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>Physical Interaction: Question Answering (PIQA)</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/piqa_v1.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>WinoGrande</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/winogrande_v1.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>AI2 Reasoning Challenge—Easy Set</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/arc_easy_v1.png" style="width:auto"/>
+</details>
+
+<details>
+  <summary>SciQ</summary>
+  <img src="/EleutherAI/pythia-12b/resolve/main/eval_plots/sciq_v1.png" style="width:auto"/>
+</details>
+
+## Changelog
+
+This section compares differences between previously released 
+[Pythia v0](https://huggingface.co/models?other=pythia_v0) and the current 
+models. See Appendix B of the Pythia paper for further discussion of these 
+changes and the motivation behind them. We found that retraining Pythia had no 
+impact on benchmark performance.
+
+- All model sizes are now trained with uniform batch size of 2M tokens. 
+Previously, the models of size 160M, 410M, and 1.4B parameters were trained 
+with batch sizes of 4M tokens.
+- We added checkpoints at initialization (step 0) and steps {1,2,4,8,16,32,64,
+128,256,512} in addition to every 1000 training steps.
+- Flash Attention was used in the new retrained suite.
+- We remedied a minor inconsistency that existed in the original suite: all 
+models of size 2.8B parameters or smaller had a learning rate (LR) schedule 
+which decayed to a minimum LR of 10% the starting LR rate, but the 6.9B and 
+12B models all used an LR schedule which decayed to a minimum LR of 0. In 
+the redone training runs, we rectified this inconsistency: all models now were 
+trained with LR decaying to a minimum of 0.1× their maximum LR.
+
+### Naming convention and parameter count
+
+*Pythia* models were renamed in January 2023. It is possible that the old 
+naming convention still persists in some documentation by accident. The 
+current naming convention (70M, 160M, etc.) is based on total parameter count. 
+
+<figure style="width:32em">
+  
+| current Pythia suffix | old suffix | total params   | non-embedding params |
+| --------------------: | ---------: | -------------: | -------------------: |
+| 70M                   | 19M        | 70,426,624     | 18,915,328           |
+| 160M                  | 125M       | 162,322,944    | 85,056,000           |
+| 410M                  | 350M       | 405,334,016    | 302,311,424          |
+| 1B                    | 800M       | 1,011,781,632  | 805,736,448          |
+| 1.4B                  | 1.3B       | 1,414,647,808  | 1,208,602,624        |
+| 2.8B                  | 2.7B       | 2,775,208,960  | 2,517,652,480        |
+| 6.9B                  | 6.7B       | 6,857,302,016  | 6,444,163,072        |
+| 12B                   | 13B        | 11,846,072,320 | 11,327,027,200       |
+</figure>

config.json

@@ -0,0 +1,24 @@
+{
+  "architectures": [
+    "GPTNeoXForCausalLM"
+  ],
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "hidden_act": "gelu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 20480,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 2048,
+  "model_type": "gpt_neox",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 36,
+  "rotary_emb_base": 10000,
+  "rotary_pct": 0.25,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.24.0",
+  "use_cache": true,
+  "use_parallel_residual": true,
+  "vocab_size": 50688
+}

configuration.json

@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:962f068c6b1c898d6bafec43a394ea64f0d5fc5f8215da651606955d418dfbff
+size 9811441814

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1af79279e8946de577ae27d7d71136e9dde72581048023b5073014e91ef9e810
+size 9925840824

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78e64bf62882c9fa1d853a30f47bdff919012a27e433a80ffd25606a1a784d51
+size 4105922122

model.safetensors.index.json

@@ -0,0 +1,551 @@
+{
+    "metadata": {
+        "total_size": 23843140808
+    },
+    "weight_map": {
+        "embed_out.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.embed_in.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.final_layer_norm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.final_layer_norm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.0.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.10.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.11.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.12.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.13.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.14.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.14.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.14.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.15.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.2.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.20.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.22.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.23.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.24.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.25.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.26.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.27.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.28.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.mlp.dense_4h_to_h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.mlp.dense_4h_to_h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.mlp.dense_h_to_4h.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.mlp.dense_h_to_4h.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.29.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.3.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.30.attention.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.attention.dense.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.attention.dense.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.attention.masked_bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.attention.query_key_value.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.attention.query_key_value.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.attention.rotary_emb.inv_freq": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.input_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.input_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.mlp.dense_4h_to_h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.30.mlp.dense_4h_to_h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.30.mlp.dense_h_to_4h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.30.mlp.dense_h_to_4h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.30.post_attention_layernorm.bias": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.30.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+        "gpt_neox.layers.31.attention.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.attention.dense.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.attention.dense.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.attention.masked_bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.attention.query_key_value.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.attention.query_key_value.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.attention.rotary_emb.inv_freq": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.input_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.mlp.dense_4h_to_h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.mlp.dense_4h_to_h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.mlp.dense_h_to_4h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.mlp.dense_h_to_4h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.post_attention_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.attention.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.attention.dense.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.attention.dense.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.attention.masked_bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.attention.query_key_value.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.attention.query_key_value.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.attention.rotary_emb.inv_freq": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.input_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.mlp.dense_4h_to_h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.mlp.dense_4h_to_h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.mlp.dense_h_to_4h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.mlp.dense_h_to_4h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.post_attention_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.32.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.attention.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.attention.dense.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.attention.dense.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.attention.masked_bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.attention.query_key_value.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.attention.query_key_value.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.attention.rotary_emb.inv_freq": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.input_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.mlp.dense_4h_to_h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.mlp.dense_4h_to_h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.mlp.dense_h_to_4h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.mlp.dense_h_to_4h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.post_attention_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.33.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.attention.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.attention.dense.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.attention.dense.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.attention.masked_bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.attention.query_key_value.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.attention.query_key_value.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.attention.rotary_emb.inv_freq": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.input_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.mlp.dense_4h_to_h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.mlp.dense_4h_to_h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.mlp.dense_h_to_4h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.mlp.dense_h_to_4h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.post_attention_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.34.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.attention.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.attention.dense.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.attention.dense.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.attention.masked_bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.attention.query_key_value.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.attention.query_key_value.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.attention.rotary_emb.inv_freq": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.input_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.input_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.mlp.dense_4h_to_h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.mlp.dense_4h_to_h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.mlp.dense_h_to_4h.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.mlp.dense_h_to_4h.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.post_attention_layernorm.bias": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.35.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+        "gpt_neox.layers.4.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.attention.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.attention.dense.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.attention.dense.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.attention.masked_bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.attention.query_key_value.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.attention.query_key_value.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.attention.rotary_emb.inv_freq": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.input_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.mlp.dense_4h_to_h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.mlp.dense_4h_to_h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.mlp.dense_h_to_4h.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.mlp.dense_h_to_4h.weight": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.post_attention_layernorm.bias": "model-00001-of-00003.safetensors",
+        "gpt_neox.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors"
+    }
+}

pytorch_model-00001-of-00003.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d21715462588073675eca9c65ba55a47d911da460a6083938d04acd4589f542
+size 9811493183

pytorch_model-00002-of-00003.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ce3319dbdf23e96b36ab4be6517bab2cac1b0578c5bde6005547b8bb657c11f
+size 9925895489

pytorch_model-00003-of-00003.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb82155af2c813a9aac8bcbe12266af8a1b0be084a3d6c0d244a745e0993cca4
+size 4105939923

pytorch_model.bin.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5e8ab741160f5bb497faa342ff52403855477f01a32b2f878664873698a944d
+size 47256

special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c24618a1b3e6a38167beff1c72cffd126c3a66254347304b50547d12c5f25624
+size 2113710

tokenizer_config.json

@@ -0,0 +1,9 @@
+{
+  "add_prefix_space": false,
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "name_or_path": "EleutherAI/gpt-neox-20b",
+  "special_tokens_map_file": "/admin/home-hailey/.cache/huggingface/hub/models--EleutherAI--gpt-neox-20b/snapshots/4e49eadb5d14bd22f314ec3f45b69a87b88c7691/special_tokens_map.json",
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "unk_token": "<|endoftext|>"
+}