HuggingFaceH4/starchat2-15b-v0.1
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: HuggingFaceH4/starchat2-15b-v0.1
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-18 23:28:12 +08:00
commit 3df476fd64
27 changed files with 150577 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
.gitattributes vendored Normal file
View File

@@ -0,0 +1,43 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
model_logo.png filter=lfs diff=lfs merge=lfs -text
model-00001-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
model-00002-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
model-00003-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
model-00004-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
model-00005-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
model-00006-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text
model-00007-of-00007.safetensors filter=lfs diff=lfs merge=lfs -text

159
README.md Normal file
View File

@@ -0,0 +1,159 @@
---
base_model: HuggingFaceH4/starchat2-15b-sft-v0.1
tags:
- alignment-handbook
- generated_from_trainer
datasets:
- HuggingFaceH4/ultrafeedback_binarized
- HuggingFaceH4/orca_dpo_pairs
model-index:
- name: starchat2-15b-v0.1
results: []
---
<img src="https://huggingface.co/HuggingFaceH4/starchat2-15b-v0.1/resolve/main/model_logo.png" alt="StarChat2 15B Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
# Model Card for StarChat2 15B
StarChat is a series of language models that are trained to act as helpful coding assistants. StarChat2 is the latest model in the series, and is a fine-tuned version of [StarCoder2](https://huggingface.co/bigcode/starcoder2-15b) that was trained with SFT and DPO on a mix of synthetic datasets.
## Model Details
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Model type:** A 16B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
- **Language(s) (NLP):** Primarily English and 600+ programming languages.
- **License:** BigCode Open RAIL-M v1
- **Finetuned from model:** [bigcode/starcoder2-15b](https://huggingface.co/bigcode/starcoder2-15b)
### Model Sources
<!-- Provide the basic links for the model. -->
- **Repository:** https://github.com/huggingface/alignment-handbook
- **Demo:** https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
## Performance
StarChat2 15B was trained to balance chat and programming capabilities. It achieves strong performance on chat benchmarks like [MT Bench](https://huggingface.co/spaces/lmsys/mt-bench) and [IFEval](https://arxiv.org/abs/2311.07911), as well as the canonical HumanEval benchmark for Python code completion. The scores reported below were obtained using the [LightEval](https://github.com/huggingface/lighteval) evaluation suite (commit `988959cb905df4baa050f82b4d499d46e8b537f2`) and each prompt has been formatted with the model's corresponding chat template to simulate real-world usage. This is why some scores may differ from those reported in technical reports or on the Open LLM Leaderboard.
| Model | MT Bench | IFEval | HumanEval |
|-------------------------------------------------------------------------------------------------|---------:|-------:|----------:|
| [starchat2-15b-v0.1](https://huggingface.co/HuggingFaceH4/starchat2-15b-v0.1) | 7.66 | 35.12 | 71.34 |
| [deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) | 4.17 | 14.23 | 80.48 |
| [CodeLlama-13b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-13b-Instruct-hf) | 6.80 | 43.44 | 50.60 |
## Intended uses & limitations
The model was fine-tuned on a blend of chat, code, math, and reasoning datasets. As a result, the model can be used for chat and you can check out our [demo](https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground) to test its coding capabilities.
Here's how you can run the model using the `pipeline()` function from 🤗 Transformers:
```python
# pip install 'transformers @ git+https://github.com/huggingface/transformers.git@831bc25d8fdb85768402f772cf65cc3d7872b211'
# pip install accelerate
import torch
from transformers import pipeline
pipe = pipeline(
"text-generation",
model="HuggingFaceH4/starchat2-15b-v0.1",
device_map="auto",
torch_dtype=torch.bfloat16,
)
messages = [
{
"role": "system",
"content": "You are StarChat2, an expert programming assistant",
},
{"role": "user", "content": "Write a simple website in HTML. When a user clicks the button, it shows a random Chuck Norris joke."},
]
outputs = pipe(
messages,
max_new_tokens=512,
do_sample=True,
temperature=0.7,
top_k=50,
top_p=0.95,
stop_sequence="<|im_end|>",
)
print(outputs[0]["generated_text"][-1]["content"])
```
## Bias, Risks, and Limitations
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
StarChat2 15B has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so).
Models trained primarily on code data will also have a more skewed demographic bias commensurate with the demographics of the GitHub community, for more on this see the [StarCoder2 dataset](https://huggingface.co/datasets/bigcode/the-stack-v2)
Since the base model was pretrained on a large corpus of code, it may produce code snippets that are syntactically valid but semantically incorrect.
For example, it may produce code that does not compile or that produces incorrect results.
It may also produce code that is vulnerable to security exploits.
We have observed the model also has a tendency to produce false URLs which should be carefully inspected before clicking.
StarChat2 15B was fine-tuned from the base model [StarCoder2](https://huggingface.co/bigcode/starcoder2-15b), please refer to its model card's [Limitations Section](https://huggingface.co/bigcode/starcoder2-15b#limitations) for relevant information.
In particular, the model was evaluated on some categories of gender biases, propensity for toxicity, and risk of suggesting code completions with known security flaws; these evaluations are reported in its [technical report](https://huggingface.co/papers/2402.19173).
## Training details
This model is a fine-tuned version of [starchat2-15b-sft-v0.1](https://huggingface.co/HuggingFaceH4/starchat2-15b-sft-v0.1) on the HuggingFaceH4/ultrafeedback_binarized and the HuggingFaceH4/orca_dpo_pairs datasets. Check out the recipe in the [Alignment Handbook](https://github.com/huggingface/alignment-handbook) for more details.
It achieves the following results on the evaluation set:
- Loss: 0.4347
- Rewards/chosen: -0.9461
- Rewards/rejected: -2.7745
- Rewards/accuracies: 0.7658
- Rewards/margins: 1.8284
- Logps/rejected: -322.1934
- Logps/chosen: -316.1898
- Logits/rejected: -2.3817
- Logits/chosen: -2.3005
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-07
- train_batch_size: 2
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 8
- total_train_batch_size: 128
- total_eval_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 2
### Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
| 0.717 | 0.17 | 100 | 0.6006 | -0.0924 | -0.2899 | 0.6329 | 0.1975 | -272.5022 | -299.1165 | -2.5313 | -2.4191 |
| 0.6273 | 0.35 | 200 | 0.5160 | -0.3994 | -0.9461 | 0.6930 | 0.5467 | -285.6261 | -305.2568 | -2.5281 | -2.4278 |
| 0.5538 | 0.52 | 300 | 0.4781 | -0.6589 | -1.5892 | 0.7247 | 0.9302 | -298.4870 | -310.4470 | -2.4996 | -2.4110 |
| 0.5056 | 0.7 | 400 | 0.4594 | -0.8283 | -2.1332 | 0.7437 | 1.3050 | -309.3687 | -313.8344 | -2.4472 | -2.3644 |
| 0.4983 | 0.87 | 500 | 0.4512 | -0.7758 | -2.2806 | 0.7468 | 1.5049 | -312.3167 | -312.7843 | -2.4223 | -2.3404 |
| 0.4662 | 1.04 | 600 | 0.4431 | -0.7839 | -2.4016 | 0.7658 | 1.6177 | -314.7355 | -312.9465 | -2.4049 | -2.3215 |
| 0.4411 | 1.22 | 700 | 0.4415 | -1.0090 | -2.7582 | 0.7690 | 1.7492 | -321.8679 | -317.4481 | -2.3840 | -2.3016 |
| 0.471 | 1.39 | 800 | 0.4368 | -0.9617 | -2.7445 | 0.7690 | 1.7828 | -321.5930 | -316.5019 | -2.3809 | -2.2991 |
| 0.4485 | 1.57 | 900 | 0.4351 | -0.9490 | -2.7594 | 0.7722 | 1.8103 | -321.8916 | -316.2497 | -2.3815 | -2.3004 |
| 0.4411 | 1.74 | 1000 | 0.4348 | -0.9293 | -2.7469 | 0.7658 | 1.8176 | -321.6409 | -315.8547 | -2.3823 | -2.3011 |
| 0.4499 | 1.92 | 1100 | 0.4348 | -0.9482 | -2.7767 | 0.7658 | 1.8285 | -322.2369 | -316.2320 | -2.3828 | -2.3012 |
### Framework versions
- Transformers 4.39.0.dev0
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1

4
added_tokens.json Normal file
View File

@@ -0,0 +1,4 @@
{
"<|im_end|>": 49153,
"<|im_start|>": 49152
}

21
all_results.json Normal file
View File

@@ -0,0 +1,21 @@
{
"epoch": 2.0,
"eval_logits/chosen": -2.3005340099334717,
"eval_logits/rejected": -2.3817031383514404,
"eval_logps/chosen": -316.1898498535156,
"eval_logps/rejected": -322.1933898925781,
"eval_loss": 0.4346597194671631,
"eval_rewards/accuracies": 0.7658227682113647,
"eval_rewards/chosen": -0.9460535049438477,
"eval_rewards/margins": 1.8284220695495605,
"eval_rewards/rejected": -2.7744758129119873,
"eval_runtime": 117.6177,
"eval_samples": 2500,
"eval_samples_per_second": 21.255,
"eval_steps_per_second": 0.672,
"train_loss": 0.5184940074794384,
"train_runtime": 19743.6623,
"train_samples": 73494,
"train_samples_per_second": 7.445,
"train_steps_per_second": 0.058
}

31
config.json Normal file
View File

@@ -0,0 +1,31 @@
{
"_name_or_path": "HuggingFaceH4/starcoder2-15b-ift",
"architectures": [
"Starcoder2ForCausalLM"
],
"attention_dropout": 0.1,
"bos_token_id": 0,
"embedding_dropout": 0.1,
"eos_token_id": 0,
"hidden_act": "gelu_pytorch_tanh",
"hidden_size": 6144,
"initializer_range": 0.01275,
"intermediate_size": 24576,
"max_position_embeddings": 16384,
"mlp_type": "default",
"model_type": "starcoder2",
"norm_epsilon": 1e-05,
"norm_type": "layer_norm",
"num_attention_heads": 48,
"num_hidden_layers": 40,
"num_key_value_heads": 4,
"residual_dropout": 0.1,
"rope_theta": 100000,
"sliding_window": 4096,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.39.0.dev0",
"use_bias": true,
"use_cache": true,
"vocab_size": 49154
}

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

16
eval_results.json Normal file
View File

@@ -0,0 +1,16 @@
{
"epoch": 2.0,
"eval_logits/chosen": -2.3005340099334717,
"eval_logits/rejected": -2.3817031383514404,
"eval_logps/chosen": -316.1898498535156,
"eval_logps/rejected": -322.1933898925781,
"eval_loss": 0.4346597194671631,
"eval_rewards/accuracies": 0.7658227682113647,
"eval_rewards/chosen": -0.9460535049438477,
"eval_rewards/margins": 1.8284220695495605,
"eval_rewards/rejected": -2.7744758129119873,
"eval_runtime": 117.6177,
"eval_samples": 2500,
"eval_samples_per_second": 21.255,
"eval_steps_per_second": 0.672
}

6
generation_config.json Normal file
View File

@@ -0,0 +1,6 @@
{
"_from_model_config": true,
"bos_token_id": 50256,
"eos_token_id": 50256,
"transformers_version": "4.39.0.dev0"
}

48873
merges.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
model-00001-of-00007.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d6c48426c7c3775db521715b86af7f9e707ee3463d3ea2a2d7fab62566228879
size 4908132312

3
model-00002-of-00007.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9fceeb5864469f8ccab6be64005fcb25ef2b603167ec30de940faf877e106cdc
size 4996327240

3
model-00003-of-00007.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ce13a732d413eff9378b505a814716fdbd2113fc6a18a988d7ff3a08dafa410f
size 4983729152

3
model-00004-of-00007.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:30b986168df98421a5f475b5754bbc43c085b59dc987eebd7b6caf414dc48c72
size 4996327312

3
model-00005-of-00007.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e418fb172f02e39c64231be87bd548f70a6669e995a47202add964548521b576
size 4983729152

3
model-00006-of-00007.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:90df89c6c5e2f34e2964ff7fc3f195cd5440b6ebe509ff78df20532cb5a1956a
size 4996327312

3
model-00007-of-00007.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d63cc490c3ae8a3091850ad146166725fc92eb4cb7561f72305fd9e73483eb60
size 2051327128

651
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,651 @@
{
"metadata": {
"total_size": 31915827200
},
"weight_map": {
"lm_head.weight": "model-00007-of-00007.safetensors",
"model.embed_tokens.weight": "model-00001-of-00007.safetensors",
"model.layers.0.input_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.0.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
"model.layers.0.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
"model.layers.0.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.0.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.0.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.1.input_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.1.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
"model.layers.1.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
"model.layers.1.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.1.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.1.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.10.input_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.10.input_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.10.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
"model.layers.10.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
"model.layers.10.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.10.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.10.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.11.input_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.11.input_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.11.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
"model.layers.11.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
"model.layers.11.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.11.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.11.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.12.input_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.12.input_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.12.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
"model.layers.12.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
"model.layers.12.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.12.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.12.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.12.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.12.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.12.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.12.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.13.input_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.13.input_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.13.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
"model.layers.13.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
"model.layers.13.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.13.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.13.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.14.input_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.14.input_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.14.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
"model.layers.14.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
"model.layers.14.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.14.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.14.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.15.input_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.15.input_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.15.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
"model.layers.15.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
"model.layers.15.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.15.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.15.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.16.input_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.16.input_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.16.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
"model.layers.16.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
"model.layers.16.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.16.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.16.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.17.input_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.17.input_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.17.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
"model.layers.17.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
"model.layers.17.mlp.c_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.17.mlp.c_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.17.post_attention_layernorm.bias": "model-00003-of-00007.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.18.input_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.18.input_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.18.mlp.c_fc.bias": "model-00003-of-00007.safetensors",
"model.layers.18.mlp.c_fc.weight": "model-00003-of-00007.safetensors",
"model.layers.18.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.18.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.18.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.18.self_attn.k_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.18.self_attn.o_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.18.self_attn.q_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.18.self_attn.v_proj.bias": "model-00003-of-00007.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00003-of-00007.safetensors",
"model.layers.19.input_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.19.input_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.19.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
"model.layers.19.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
"model.layers.19.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.19.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.19.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.2.input_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.2.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
"model.layers.2.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
"model.layers.2.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.2.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.2.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.20.input_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.20.input_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.20.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
"model.layers.20.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
"model.layers.20.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.20.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.20.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.21.input_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.21.input_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.21.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
"model.layers.21.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
"model.layers.21.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.21.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.21.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.22.input_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.22.input_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.22.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
"model.layers.22.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
"model.layers.22.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.22.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.22.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.23.input_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.23.input_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.23.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
"model.layers.23.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
"model.layers.23.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.23.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.23.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.24.input_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.24.input_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.24.mlp.c_fc.bias": "model-00004-of-00007.safetensors",
"model.layers.24.mlp.c_fc.weight": "model-00004-of-00007.safetensors",
"model.layers.24.mlp.c_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.24.mlp.c_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.24.post_attention_layernorm.bias": "model-00004-of-00007.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.o_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.25.input_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.25.input_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.25.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
"model.layers.25.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
"model.layers.25.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.25.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.25.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.25.self_attn.k_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.25.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.25.self_attn.q_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.25.self_attn.v_proj.bias": "model-00004-of-00007.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00004-of-00007.safetensors",
"model.layers.26.input_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.26.input_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.26.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
"model.layers.26.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
"model.layers.26.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.26.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.26.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.27.input_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.27.input_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.27.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
"model.layers.27.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
"model.layers.27.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.27.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.27.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.28.input_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.28.input_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.28.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
"model.layers.28.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
"model.layers.28.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.28.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.28.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.29.input_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.29.input_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.29.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
"model.layers.29.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
"model.layers.29.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.29.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.29.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.3.input_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.3.input_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.3.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
"model.layers.3.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
"model.layers.3.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.3.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.3.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.30.input_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.30.input_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.30.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
"model.layers.30.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
"model.layers.30.mlp.c_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.30.mlp.c_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.30.post_attention_layernorm.bias": "model-00005-of-00007.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.31.input_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.31.input_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.31.mlp.c_fc.bias": "model-00005-of-00007.safetensors",
"model.layers.31.mlp.c_fc.weight": "model-00005-of-00007.safetensors",
"model.layers.31.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.31.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.31.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.31.self_attn.k_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.31.self_attn.o_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.31.self_attn.q_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.31.self_attn.v_proj.bias": "model-00005-of-00007.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00005-of-00007.safetensors",
"model.layers.32.input_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.32.input_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.32.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
"model.layers.32.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
"model.layers.32.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.32.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.32.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.32.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.32.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.33.input_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.33.input_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.33.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
"model.layers.33.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
"model.layers.33.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.33.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.33.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.33.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.33.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.34.input_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.34.input_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.34.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
"model.layers.34.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
"model.layers.34.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.34.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.34.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.34.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.34.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.35.input_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.35.input_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.35.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
"model.layers.35.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
"model.layers.35.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.35.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.35.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.35.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.35.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.36.input_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.36.input_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.36.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
"model.layers.36.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
"model.layers.36.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.36.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.36.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.36.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.36.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.37.input_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.37.input_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.37.mlp.c_fc.bias": "model-00006-of-00007.safetensors",
"model.layers.37.mlp.c_fc.weight": "model-00006-of-00007.safetensors",
"model.layers.37.mlp.c_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.37.mlp.c_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.37.post_attention_layernorm.bias": "model-00006-of-00007.safetensors",
"model.layers.37.post_attention_layernorm.weight": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.o_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.o_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.37.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.38.input_layernorm.bias": "model-00007-of-00007.safetensors",
"model.layers.38.input_layernorm.weight": "model-00007-of-00007.safetensors",
"model.layers.38.mlp.c_fc.bias": "model-00007-of-00007.safetensors",
"model.layers.38.mlp.c_fc.weight": "model-00007-of-00007.safetensors",
"model.layers.38.mlp.c_proj.bias": "model-00007-of-00007.safetensors",
"model.layers.38.mlp.c_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.38.post_attention_layernorm.bias": "model-00007-of-00007.safetensors",
"model.layers.38.post_attention_layernorm.weight": "model-00007-of-00007.safetensors",
"model.layers.38.self_attn.k_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.38.self_attn.k_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.38.self_attn.o_proj.bias": "model-00007-of-00007.safetensors",
"model.layers.38.self_attn.o_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.38.self_attn.q_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.38.self_attn.q_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.38.self_attn.v_proj.bias": "model-00006-of-00007.safetensors",
"model.layers.38.self_attn.v_proj.weight": "model-00006-of-00007.safetensors",
"model.layers.39.input_layernorm.bias": "model-00007-of-00007.safetensors",
"model.layers.39.input_layernorm.weight": "model-00007-of-00007.safetensors",
"model.layers.39.mlp.c_fc.bias": "model-00007-of-00007.safetensors",
"model.layers.39.mlp.c_fc.weight": "model-00007-of-00007.safetensors",
"model.layers.39.mlp.c_proj.bias": "model-00007-of-00007.safetensors",
"model.layers.39.mlp.c_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.39.post_attention_layernorm.bias": "model-00007-of-00007.safetensors",
"model.layers.39.post_attention_layernorm.weight": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.k_proj.bias": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.k_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.o_proj.bias": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.o_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.q_proj.bias": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.q_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.v_proj.bias": "model-00007-of-00007.safetensors",
"model.layers.39.self_attn.v_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.4.input_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.4.input_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.4.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
"model.layers.4.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
"model.layers.4.mlp.c_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.4.mlp.c_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.4.post_attention_layernorm.bias": "model-00001-of-00007.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.5.input_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.5.input_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.5.mlp.c_fc.bias": "model-00001-of-00007.safetensors",
"model.layers.5.mlp.c_fc.weight": "model-00001-of-00007.safetensors",
"model.layers.5.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.5.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.5.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.5.self_attn.k_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.5.self_attn.o_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.5.self_attn.q_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.5.self_attn.v_proj.bias": "model-00001-of-00007.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00007.safetensors",
"model.layers.6.input_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.6.input_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.6.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
"model.layers.6.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
"model.layers.6.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.6.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.6.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.7.input_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.7.input_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.7.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
"model.layers.7.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
"model.layers.7.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.7.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.7.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.8.input_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.8.input_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.8.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
"model.layers.8.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
"model.layers.8.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.8.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.8.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.9.input_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.9.input_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.9.mlp.c_fc.bias": "model-00002-of-00007.safetensors",
"model.layers.9.mlp.c_fc.weight": "model-00002-of-00007.safetensors",
"model.layers.9.mlp.c_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.9.mlp.c_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.9.post_attention_layernorm.bias": "model-00002-of-00007.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.k_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.o_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.q_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.v_proj.bias": "model-00002-of-00007.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00002-of-00007.safetensors",
"model.norm.bias": "model-00007-of-00007.safetensors",
"model.norm.weight": "model-00007-of-00007.safetensors"
}
}

3
model_logo.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:85603fd9cdff6d10e73147bfc3ed2c07cd7e8054a44d295bc2bebd6c51837948
size 3137166

3
runs/Mar09_21-24-05_ip-26-0-161-138/events.out.tfevents.1710019788.ip-26-0-161-138.3654361.0 Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f7f80ab9898f9f9563229fd2f3670229119bf4fe2b847e15f944944cd3106b1e
size 92659

3
runs/Mar09_21-24-05_ip-26-0-161-138/events.out.tfevents.1710039650.ip-26-0-161-138.3654361.1 Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8bd62f7cba2a3a1f0a8641d1442db4fc7c8c38bef615a36ae2d793f24b52e162
size 828

34
special_tokens_map.json Normal file
View File

@@ -0,0 +1,34 @@
{
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>"
],
"bos_token": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

98427
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

338
tokenizer_config.json Normal file
View File

@@ -0,0 +1,338 @@
{
"add_prefix_space": false,
"added_tokens_decoder": {
"0": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<fim_prefix>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "<fim_middle>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"3": {
"content": "<fim_suffix>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"4": {
"content": "<fim_pad>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"5": {
"content": "<repo_name>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"6": {
"content": "<file_sep>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"7": {
"content": "<issue_start>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"8": {
"content": "<issue_comment>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"9": {
"content": "<issue_closed>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"10": {
"content": "<jupyter_start>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"11": {
"content": "<jupyter_text>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"12": {
"content": "<jupyter_code>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"13": {
"content": "<jupyter_output>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"14": {
"content": "<jupyter_script>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"15": {
"content": "<empty_output>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"16": {
"content": "<code_to_intermediate>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"17": {
"content": "<intermediate_to_code>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"18": {
"content": "<pr>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"19": {
"content": "<pr_status>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"20": {
"content": "<pr_is_merged>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"21": {
"content": "<pr_base>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"22": {
"content": "<pr_file>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"23": {
"content": "<pr_base_code>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"24": {
"content": "<pr_diff>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"25": {
"content": "<pr_diff_hunk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"26": {
"content": "<pr_comment>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"27": {
"content": "<pr_event_id>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"28": {
"content": "<pr_review>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"29": {
"content": "<pr_review_state>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"30": {
"content": "<pr_review_comment>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"31": {
"content": "<pr_in_reply_to_review_id>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"32": {
"content": "<pr_in_reply_to_comment_id>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"33": {
"content": "<pr_diff_hunk_comment_line>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"34": {
"content": "<NAME>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"35": {
"content": "<EMAIL>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"36": {
"content": "<KEY>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"37": {
"content": "<PASSWORD>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49152": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"49153": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>"
],
"bos_token": "<|im_start|>",
"chat_template": "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
"clean_up_tokenization_spaces": true,
"eos_token": "<|im_end|>",
"model_max_length": 2048,
"pad_token": "<|im_end|>",
"tokenizer_class": "GPT2Tokenizer",
"unk_token": "<|endoftext|>",
"vocab_size": 49152
}

8
train_results.json Normal file
View File

@@ -0,0 +1,8 @@
{
"epoch": 2.0,
"train_loss": 0.5184940074794384,
"train_runtime": 19743.6623,
"train_samples": 73494,
"train_samples_per_second": 7.445,
"train_steps_per_second": 0.058
}

1931
trainer_state.json Normal file
View File

File diff suppressed because it is too large Load Diff

3
training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:146d3e8d29927d670f4fbefc63112a9a715f1d6848b08ec05dfa4b65d7c3b689
size 6520

1
vocab.json Normal file
View File

File diff suppressed because one or more lines are too long