beomi/kollama-7b
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: beomi/kollama-7b
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-31 11:36:18 +08:00
commit f05670bd37
40 changed files with 104770 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
.gitattributes vendored Normal file
View File

@@ -0,0 +1,34 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

1
.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
streaming_params

127
README.md Normal file
View File

@@ -0,0 +1,127 @@
---
license: mit
language:
- ko
- en
metrics:
- perplexity
- accuracy
pipeline_tag: text-generation
tags:
- llama
- KoLLAMA
- KoreanGPT
---
> 🚧 Note: this repo is under construction 🚧
## Todo
✅ - finish
⏳ - currently working on it
- ✅ Train new BBPE Tokenizer
- ✅ Test train code on TPUv4 Pods (with model parallel)
- ✅ Converting test (jax to PyTorch)
- ✅ LM train validation on minimal dataset (1 sentence 1000 step)
- ⏳ Build Data Shuffler (curriculum learning)
- ⏳ Train 7B Model
- Train 13B Model
- Train 33B Model
- Train 65B Model
# KoLLaMA Model Card
KoLLaMA (7B) trained on Korean/English/Code dataset with LLaMA Architecture via JAX,
with the warm support from [Google TPU Research Cloud program](https://sites.research.google/trc/about/) for providing part of the computation resources.
## Model details
**Researcher developing the model**
Junbum Lee (aka Beomi)
**Model date**
KoLLaMA was trained between 2022.04~
**Model version**
This is alpha version of the model.
**Model type**
LLaMA is an auto-regressive language model, based on the transformer architecture. The model comes in different sizes: 7B, 13B, 33B and 65B parameters.
(This repo contains 7B model!)
**Paper or resources for more information**
More information can be found in the paper “LLaMA, Open and Efficient Foundation Language Models”, available at https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models/.
More info for KoAlpaca:
[TBD]
**Citations details**
KoLLAMA: [TBD]
LLAMA: https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models/
**License**
MIT
**Where to send questions or comments about the model**
Questions and comments about KoLLaMA can be sent via the [GitHub repository](https://github.com/beomi/KoLLAMA) of the project , by opening an issue.
## Intended use
**Primary intended uses**
The primary use of KoLLaMA is research on Korean Opensource large language models
**Primary intended users**
The primary intended users of the model are researchers in natural language processing, machine learning and artificial intelligence.
**Out-of-scope use cases**
LLaMA is a base, or foundational, model. As such, it should not be used on downstream applications without further risk evaluation and mitigation. In particular, our model has not been trained with human feedback, and can thus generate toxic or offensive content, incorrect information or generally unhelpful answers.
## Factors
**Relevant factors**
One of the most relevant factors for which model performance may vary is which language is used. Although we included 20 languages in the training data, most of our dataset is made of English text, and we thus expect the model to perform better for English than other languages. Relatedly, it has been shown in previous studies that performance might vary for different dialects, and we expect that it will be the case for our model.
## Evaluation datasets
[TBD]
## Training dataset
[TBD]
## Ethical considerations
**Data**
The data used to train the model is collected from various sources, mostly from the Web. As such, it contains offensive, harmful and biased content. We thus expect the model to exhibit such biases from the training data.
**Human life**
The model is not intended to inform decisions about matters central to human life, and should not be used in such a way.
**Risks and harms**
Risks and harms of large language models include the generation of harmful, offensive or biased content. These models are often prone to generating incorrect information, sometimes referred to as hallucinations. We do not expect our model to be an exception in this regard.
**Use cases**
LLaMA is a foundational model, and as such, it should not be used for downstream applications without further investigation and mitigations of risks. These risks and potential fraught use cases include, but are not limited to: generation of misinformation and generation of harmful, biased or offensive content.

22
config.json Normal file
View File

@@ -0,0 +1,22 @@
{
"architectures": [
"LlamaForCausalLM"
],
"bos_token_id": 1,
"eos_token_id": 0,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 11008,
"max_position_embeddings": 2048,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"pad_token_id": 0,
"rms_norm_eps": 1e-06,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.28.0.dev0",
"use_cache": true,
"vocab_size": 52000
}

7
generation_config.json Normal file
View File

@@ -0,0 +1,7 @@
{
"_from_model_config": true,
"bos_token_id": 1,
"eos_token_id": 0,
"pad_token_id": 0,
"transformers_version": "4.28.0.dev0"
}

3
model-00001-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9c324368fecbb8ec75bac57de96b05cacd47e7492c8a2a4fdd45935e23d4b558
size 964970768

3
model-00002-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d0f286fb61518c66527d8068edbbb99e3b70874433b6d28e3e9d8cb24297089f
size 989891504

3
model-00003-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c2be7fce5c3215b82277384f639d52d932cca044c28c375d5272e1e16e9cc08d
size 966839552

3
model-00004-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:97acecd8da92c85e920163511cdc06d7c3ed1594202f2b48fa55e1fd32625aae
size 966823304

3
model-00005-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f41ec261fbe33a860733c9ee3f615c335519f53a9e686435cf8be720e4d154e7
size 989908120

3
model-00006-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f81849e2d164c2b2fe17bc32cbfcdf500fd9074fec07c8cf5d221faa94314f9c
size 943754776

3
model-00007-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bc2e9a851f2c5a7f19533c002f4d453bf49cb62cca622b30a02c2d6a67cee1ca
size 989891528

3
model-00008-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:97a8dd5c54867f12483d76c7e052dbe0144cd5f9b6af92942edd28f3baeea3ac
size 966839576

3
model-00009-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d43e731cd5b6e04a0b17a7d373e689b70042e0d691a4d68bc255ae46ef08ae9f
size 966823328

3
model-00010-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4392bcf2ddc866771ff2da9a3b10871f94b4b5c6c4a25f64d7127493832cb30e
size 989908136

3
model-00011-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7112312110a30baa0d8897e9e32d1094361f852f56c6dc15d29bf9daec20bfc7
size 943754776

3
model-00012-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3b65acb12a2d0c972f94953b4e47a5e43190253e56a0b5707a8b4a776113fc48
size 989891528

3
model-00013-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:15617a4ca6aa5f616fda13a7c4f42dbf39088c421ee39d42f57bad0b0ed0b62e
size 966839576

3
model-00014-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:158150c19ac05c7574667e7e3426d1d128a5b19332d55896c237d4788272d0ec
size 742435416

3
model-00015-of-00015.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8c44cb7aab1718ded5eeed32ab07df9f06ebc84114eb50beb76e12306b291b3b
size 425984128

330
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,330 @@
{
"metadata": {
"total_size": 13804519424
},
"weight_map": {
"lm_head.weight": "model-00015-of-00015.safetensors",
"model.embed_tokens.weight": "model-00001-of-00015.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00015.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00015.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.0.self_attn.rotary_emb.inv_freq": "model-00001-of-00015.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.1.input_layernorm.weight": "model-00002-of-00015.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00002-of-00015.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.1.self_attn.rotary_emb.inv_freq": "model-00001-of-00015.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00015.safetensors",
"model.layers.10.input_layernorm.weight": "model-00005-of-00015.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00005-of-00015.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.10.self_attn.rotary_emb.inv_freq": "model-00005-of-00015.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.11.input_layernorm.weight": "model-00006-of-00015.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00006-of-00015.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.11.self_attn.rotary_emb.inv_freq": "model-00006-of-00015.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.12.input_layernorm.weight": "model-00006-of-00015.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00006-of-00015.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.12.self_attn.rotary_emb.inv_freq": "model-00006-of-00015.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.13.input_layernorm.weight": "model-00007-of-00015.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00007-of-00015.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.13.self_attn.rotary_emb.inv_freq": "model-00006-of-00015.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00006-of-00015.safetensors",
"model.layers.14.input_layernorm.weight": "model-00007-of-00015.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00007-of-00015.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.14.self_attn.rotary_emb.inv_freq": "model-00007-of-00015.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.15.input_layernorm.weight": "model-00008-of-00015.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00008-of-00015.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.15.self_attn.rotary_emb.inv_freq": "model-00007-of-00015.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00007-of-00015.safetensors",
"model.layers.16.input_layernorm.weight": "model-00008-of-00015.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00008-of-00015.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.16.self_attn.rotary_emb.inv_freq": "model-00008-of-00015.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.17.input_layernorm.weight": "model-00008-of-00015.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00008-of-00015.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.17.self_attn.rotary_emb.inv_freq": "model-00008-of-00015.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.18.input_layernorm.weight": "model-00009-of-00015.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00009-of-00015.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00008-of-00015.safetensors",
"model.layers.18.self_attn.rotary_emb.inv_freq": "model-00009-of-00015.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.19.input_layernorm.weight": "model-00009-of-00015.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00009-of-00015.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.19.self_attn.rotary_emb.inv_freq": "model-00009-of-00015.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.2.input_layernorm.weight": "model-00002-of-00015.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00002-of-00015.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.2.self_attn.rotary_emb.inv_freq": "model-00002-of-00015.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.20.input_layernorm.weight": "model-00010-of-00015.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00010-of-00015.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.20.self_attn.rotary_emb.inv_freq": "model-00009-of-00015.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00009-of-00015.safetensors",
"model.layers.21.input_layernorm.weight": "model-00010-of-00015.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00010-of-00015.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.21.self_attn.rotary_emb.inv_freq": "model-00010-of-00015.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.22.input_layernorm.weight": "model-00010-of-00015.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00010-of-00015.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.22.self_attn.rotary_emb.inv_freq": "model-00010-of-00015.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00010-of-00015.safetensors",
"model.layers.23.input_layernorm.weight": "model-00011-of-00015.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00011-of-00015.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.23.self_attn.rotary_emb.inv_freq": "model-00011-of-00015.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.24.input_layernorm.weight": "model-00011-of-00015.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00011-of-00015.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.24.self_attn.rotary_emb.inv_freq": "model-00011-of-00015.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.25.input_layernorm.weight": "model-00012-of-00015.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00012-of-00015.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.25.self_attn.rotary_emb.inv_freq": "model-00011-of-00015.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00011-of-00015.safetensors",
"model.layers.26.input_layernorm.weight": "model-00012-of-00015.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00012-of-00015.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.26.self_attn.rotary_emb.inv_freq": "model-00012-of-00015.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.27.input_layernorm.weight": "model-00013-of-00015.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00013-of-00015.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.27.self_attn.rotary_emb.inv_freq": "model-00012-of-00015.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00012-of-00015.safetensors",
"model.layers.28.input_layernorm.weight": "model-00013-of-00015.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00013-of-00015.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.28.self_attn.rotary_emb.inv_freq": "model-00013-of-00015.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.29.input_layernorm.weight": "model-00013-of-00015.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00013-of-00015.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.29.self_attn.rotary_emb.inv_freq": "model-00013-of-00015.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.3.input_layernorm.weight": "model-00003-of-00015.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00003-of-00015.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.3.self_attn.rotary_emb.inv_freq": "model-00002-of-00015.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00002-of-00015.safetensors",
"model.layers.30.input_layernorm.weight": "model-00014-of-00015.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00014-of-00015.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00013-of-00015.safetensors",
"model.layers.30.self_attn.rotary_emb.inv_freq": "model-00014-of-00015.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.31.input_layernorm.weight": "model-00014-of-00015.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00014-of-00015.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.31.self_attn.rotary_emb.inv_freq": "model-00014-of-00015.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00014-of-00015.safetensors",
"model.layers.4.input_layernorm.weight": "model-00003-of-00015.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00003-of-00015.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.4.self_attn.rotary_emb.inv_freq": "model-00003-of-00015.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.5.input_layernorm.weight": "model-00003-of-00015.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00003-of-00015.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.5.self_attn.rotary_emb.inv_freq": "model-00003-of-00015.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.6.input_layernorm.weight": "model-00004-of-00015.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00004-of-00015.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00003-of-00015.safetensors",
"model.layers.6.self_attn.rotary_emb.inv_freq": "model-00004-of-00015.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.7.input_layernorm.weight": "model-00004-of-00015.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00004-of-00015.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.7.self_attn.rotary_emb.inv_freq": "model-00004-of-00015.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.8.input_layernorm.weight": "model-00005-of-00015.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00005-of-00015.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.8.self_attn.rotary_emb.inv_freq": "model-00004-of-00015.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00004-of-00015.safetensors",
"model.layers.9.input_layernorm.weight": "model-00005-of-00015.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00005-of-00015.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00005-of-00015.safetensors",
"model.layers.9.self_attn.rotary_emb.inv_freq": "model-00005-of-00015.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00005-of-00015.safetensors",
"model.norm.weight": "model-00014-of-00015.safetensors"
}
}

3
pytorch_model-00001-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bd2842ba2f0182ecdbbb3ac64448a8b58daedc86f8ce9899d13172cda71c9104
size 964974575

3
pytorch_model-00002-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:96252c7fa665d5ffc551410525479bf96564023b3aa3838d6728d2b72164b170
size 989896539

3
pytorch_model-00003-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:54135af3fb38031555cc2ad4f7f67f5124cbd69ba88b4d8b80b8ecc0bb46c991
size 966845201

3
pytorch_model-00004-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bb5f0753c75403ece83be131c7a0676a7bec10063a2489fa1bbd91de19925dad
size 966828735

3
pytorch_model-00005-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ac2e16455a953f318a7c454319484e2d24013d9735f5f62ed324938ac9782f82
size 989913535

3
pytorch_model-00006-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:79a3d2a13356a9eaeecdc9acd29bd64bca6fa7c5bb7a7a5c4b46e39f302341d2
size 943760401

3
pytorch_model-00007-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a6dc0516a346e35d447d64fa135f518ef49ae98bb1b167524bef500ac9bc86de
size 989896539

3
pytorch_model-00008-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:946dbf3548d36854c276c01b32ac4cb0f083b87b3d0b5c1a11bbee9a264527b6
size 966845201

3
pytorch_model-00009-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4379105508847b46b95b708879342760525494dc89cfd6a5e00d56a01ebe2529
size 966828799

3
pytorch_model-00010-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d239c10644cc8fa8dd99db51ec9777c6138397127173b93a2b7a148d650fed5b
size 989913535

3
pytorch_model-00011-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cca04247a20f5ecce9a068662065dcdc51fb9a1a7bbc8721b167028f76019ec7
size 943760401

3
pytorch_model-00012-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:34603afa3693bab6ffa75a60319954a9200903cff8c918f5c945d40a1e9988f8
size 989896539

3
pytorch_model-00013-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7dbd03cc6a6e3a206d70f62814547dd131e76b6cbcbcc20df34c715afcc9d395
size 966845201

3
pytorch_model-00014-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:041601ffc908cdd96f9d73d83b79a65ec7c51e608c74309c07be934fe4c21ba8
size 742439845

3
pytorch_model-00015-of-00015.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:048efece930f0a5cda75e4c7897e8a3a6de29f1eab92587f87fe48776cc6ccf8
size 425984938

330
pytorch_model.bin.index.json Normal file
View File

@@ -0,0 +1,330 @@
{
"metadata": {
"total_size": 13804519424
},
"weight_map": {
"lm_head.weight": "pytorch_model-00015-of-00015.bin",
"model.embed_tokens.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00015.bin",
"model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.1.input_layernorm.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.1.mlp.down_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.1.mlp.gate_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.1.mlp.up_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.1.post_attention_layernorm.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00015.bin",
"model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00015.bin",
"model.layers.10.input_layernorm.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.mlp.down_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.mlp.gate_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.mlp.up_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.post_attention_layernorm.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.self_attn.k_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.self_attn.o_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.self_attn.q_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00015.bin",
"model.layers.10.self_attn.v_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.11.input_layernorm.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.mlp.down_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.mlp.gate_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.mlp.up_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.post_attention_layernorm.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.self_attn.k_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.self_attn.o_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.self_attn.q_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00015.bin",
"model.layers.11.self_attn.v_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.input_layernorm.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.mlp.down_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.mlp.gate_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.mlp.up_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.post_attention_layernorm.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.self_attn.k_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.self_attn.o_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.self_attn.q_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00015.bin",
"model.layers.12.self_attn.v_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.13.input_layernorm.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.13.mlp.down_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.13.mlp.gate_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.13.mlp.up_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.13.post_attention_layernorm.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.13.self_attn.k_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.13.self_attn.o_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.13.self_attn.q_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00015.bin",
"model.layers.13.self_attn.v_proj.weight": "pytorch_model-00006-of-00015.bin",
"model.layers.14.input_layernorm.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.mlp.down_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.mlp.gate_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.mlp.up_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.post_attention_layernorm.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.self_attn.k_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.self_attn.o_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.self_attn.q_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00015.bin",
"model.layers.14.self_attn.v_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.15.input_layernorm.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.15.mlp.down_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.15.mlp.gate_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.15.mlp.up_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.15.post_attention_layernorm.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.15.self_attn.k_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.15.self_attn.o_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.15.self_attn.q_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00015.bin",
"model.layers.15.self_attn.v_proj.weight": "pytorch_model-00007-of-00015.bin",
"model.layers.16.input_layernorm.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.mlp.down_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.mlp.gate_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.mlp.up_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.post_attention_layernorm.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.self_attn.k_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.self_attn.o_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.self_attn.q_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00015.bin",
"model.layers.16.self_attn.v_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.input_layernorm.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.mlp.down_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.mlp.gate_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.mlp.up_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.post_attention_layernorm.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.self_attn.k_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.self_attn.o_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.self_attn.q_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00015.bin",
"model.layers.17.self_attn.v_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.18.input_layernorm.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.18.mlp.down_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.18.mlp.gate_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.18.mlp.up_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.18.post_attention_layernorm.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.18.self_attn.k_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.18.self_attn.o_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.18.self_attn.q_proj.weight": "pytorch_model-00008-of-00015.bin",
"model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00015.bin",
"model.layers.18.self_attn.v_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.input_layernorm.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.mlp.down_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.mlp.gate_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.mlp.up_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.post_attention_layernorm.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.self_attn.k_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.self_attn.o_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.self_attn.q_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00015.bin",
"model.layers.19.self_attn.v_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.2.input_layernorm.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.mlp.down_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.mlp.gate_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.mlp.up_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.post_attention_layernorm.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.self_attn.k_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.self_attn.o_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.self_attn.q_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00015.bin",
"model.layers.2.self_attn.v_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.20.input_layernorm.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.20.mlp.down_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.20.mlp.gate_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.20.mlp.up_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.20.post_attention_layernorm.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.20.self_attn.k_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.20.self_attn.o_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.20.self_attn.q_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00015.bin",
"model.layers.20.self_attn.v_proj.weight": "pytorch_model-00009-of-00015.bin",
"model.layers.21.input_layernorm.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.mlp.down_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.mlp.gate_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.mlp.up_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.post_attention_layernorm.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.self_attn.k_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.self_attn.o_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.self_attn.q_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00010-of-00015.bin",
"model.layers.21.self_attn.v_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.input_layernorm.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.mlp.down_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.mlp.gate_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.mlp.up_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.post_attention_layernorm.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.self_attn.k_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.self_attn.o_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.self_attn.q_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00010-of-00015.bin",
"model.layers.22.self_attn.v_proj.weight": "pytorch_model-00010-of-00015.bin",
"model.layers.23.input_layernorm.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.mlp.down_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.mlp.gate_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.mlp.up_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.post_attention_layernorm.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.self_attn.k_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.self_attn.o_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.self_attn.q_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00011-of-00015.bin",
"model.layers.23.self_attn.v_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.input_layernorm.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.mlp.down_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.mlp.gate_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.mlp.up_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.post_attention_layernorm.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.self_attn.k_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.self_attn.o_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.self_attn.q_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00011-of-00015.bin",
"model.layers.24.self_attn.v_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.25.input_layernorm.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.25.mlp.down_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.25.mlp.gate_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.25.mlp.up_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.25.post_attention_layernorm.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.25.self_attn.k_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.25.self_attn.o_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.25.self_attn.q_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00011-of-00015.bin",
"model.layers.25.self_attn.v_proj.weight": "pytorch_model-00011-of-00015.bin",
"model.layers.26.input_layernorm.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.mlp.down_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.mlp.gate_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.mlp.up_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.post_attention_layernorm.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.self_attn.k_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.self_attn.o_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.self_attn.q_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00012-of-00015.bin",
"model.layers.26.self_attn.v_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.27.input_layernorm.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.27.mlp.down_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.27.mlp.gate_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.27.mlp.up_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.27.post_attention_layernorm.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.27.self_attn.k_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.27.self_attn.o_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.27.self_attn.q_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00012-of-00015.bin",
"model.layers.27.self_attn.v_proj.weight": "pytorch_model-00012-of-00015.bin",
"model.layers.28.input_layernorm.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.mlp.down_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.mlp.gate_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.mlp.up_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.post_attention_layernorm.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.self_attn.k_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.self_attn.o_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.self_attn.q_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00013-of-00015.bin",
"model.layers.28.self_attn.v_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.input_layernorm.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.mlp.down_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.mlp.gate_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.mlp.up_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.post_attention_layernorm.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.self_attn.k_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.self_attn.o_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.self_attn.q_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00013-of-00015.bin",
"model.layers.29.self_attn.v_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.3.input_layernorm.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.3.mlp.down_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.3.mlp.up_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.3.post_attention_layernorm.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.3.self_attn.k_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.3.self_attn.o_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.3.self_attn.q_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00015.bin",
"model.layers.3.self_attn.v_proj.weight": "pytorch_model-00002-of-00015.bin",
"model.layers.30.input_layernorm.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.30.mlp.down_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.30.mlp.gate_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.30.mlp.up_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.30.post_attention_layernorm.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.30.self_attn.k_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.30.self_attn.o_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.30.self_attn.q_proj.weight": "pytorch_model-00013-of-00015.bin",
"model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00014-of-00015.bin",
"model.layers.30.self_attn.v_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.input_layernorm.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.mlp.down_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.mlp.gate_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.mlp.up_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.post_attention_layernorm.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.self_attn.k_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.self_attn.o_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.self_attn.q_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00014-of-00015.bin",
"model.layers.31.self_attn.v_proj.weight": "pytorch_model-00014-of-00015.bin",
"model.layers.4.input_layernorm.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.mlp.down_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.mlp.gate_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.mlp.up_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.post_attention_layernorm.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.self_attn.k_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.self_attn.o_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.self_attn.q_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00015.bin",
"model.layers.4.self_attn.v_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.input_layernorm.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.mlp.down_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.mlp.gate_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.mlp.up_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.post_attention_layernorm.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.self_attn.k_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.self_attn.o_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.self_attn.q_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00015.bin",
"model.layers.5.self_attn.v_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.6.input_layernorm.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.6.mlp.down_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.6.mlp.gate_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.6.mlp.up_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.6.post_attention_layernorm.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.6.self_attn.k_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.6.self_attn.o_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.6.self_attn.q_proj.weight": "pytorch_model-00003-of-00015.bin",
"model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00015.bin",
"model.layers.6.self_attn.v_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.input_layernorm.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.mlp.down_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.mlp.gate_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.mlp.up_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.post_attention_layernorm.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.self_attn.k_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.self_attn.o_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.self_attn.q_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00015.bin",
"model.layers.7.self_attn.v_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.8.input_layernorm.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.8.mlp.down_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.8.mlp.gate_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.8.mlp.up_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.8.post_attention_layernorm.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.8.self_attn.k_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.8.self_attn.o_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.8.self_attn.q_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00015.bin",
"model.layers.8.self_attn.v_proj.weight": "pytorch_model-00004-of-00015.bin",
"model.layers.9.input_layernorm.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.mlp.down_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.mlp.gate_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.mlp.up_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.post_attention_layernorm.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.self_attn.k_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.self_attn.o_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.self_attn.q_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00015.bin",
"model.layers.9.self_attn.v_proj.weight": "pytorch_model-00005-of-00015.bin",
"model.norm.weight": "pytorch_model-00014-of-00015.bin"
}
}

5
special_tokens_map.json Normal file
View File

@@ -0,0 +1,5 @@
{
"bos_token": "<|sep|>",
"eos_token": "<|endoftext|>",
"pad_token": "<|endoftext|>"
}

103816
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

8
tokenizer_config.json Normal file
View File

@@ -0,0 +1,8 @@
{
"name_or_path": "beomi/KoLLAMA",
"eos_token": "<|endoftext|>",
"bos_token": "<|sep|>",
"pad_token": "<|endoftext|>",
"model_max_length": 1000000000000000019884624838656,
"tokenizer_class": "PreTrainedTokenizerFast"
}