初始化项目，由ModelHub XC社区提供模型
Model: beomi/kollama-13b Source: Original Platform
This commit is contained in:
ModelHub XC
34
.gitattributes
vendored
Normal file
34
.gitattributes
vendored
Normal file
@@ -0,0 +1,34 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
128
README.md
Normal file
128
README.md
Normal file
@@ -0,0 +1,128 @@
|
||||
---
|
||||
license: mit
|
||||
language:
|
||||
- ko
|
||||
- en
|
||||
metrics:
|
||||
- perplexity
|
||||
- accuracy
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- llama
|
||||
- KoLLAMA
|
||||
- KoreanGPT
|
||||
---
|
||||
|
||||
> 🚧 Note: this repo is under construction 🚧
|
||||
|
||||
## Todo
|
||||
|
||||
✅ - finish
|
||||
|
||||
⏳ - currently working on it
|
||||
|
||||
|
||||
- ✅ Train new BBPE Tokenizer
|
||||
- ✅ Test train code on TPUv4 Pods (with model parallel)
|
||||
- ✅ Converting test (jax to PyTorch)
|
||||
- ✅ LM train validation on minimal dataset (1 sentence 1000 step)
|
||||
- ⏳ Build Data Shuffler (curriculum learning)
|
||||
- ⏳ Train 7B Model
|
||||
- ⏳ Train 13B Model
|
||||
- Train 33B Model
|
||||
- Train 65B Model
|
||||
|
||||
|
||||
|
||||
# KoLLaMA-13B Model Card
|
||||
|
||||
KoLLaMA (13B) trained on Korean/English/Code dataset with LLaMA Architecture via JAX,
|
||||
with the warm support from [Google TPU Research Cloud program](https://sites.research.google/trc/about/) for providing part of the computation resources.
|
||||
|
||||
|
||||
## Model details
|
||||
|
||||
**Researcher developing the model**
|
||||
|
||||
Junbum Lee (aka Beomi)
|
||||
|
||||
**Model date**
|
||||
|
||||
KoLLaMA was trained between 2022.04~
|
||||
|
||||
**Model version**
|
||||
|
||||
This is alpha version of the model.
|
||||
|
||||
**Model type**
|
||||
|
||||
LLaMA is an auto-regressive language model, based on the transformer architecture. The model comes in different sizes: 7B, 13B, 33B and 65B parameters.
|
||||
|
||||
(This repo contains 13B model!)
|
||||
|
||||
**Paper or resources for more information**
|
||||
|
||||
More information can be found in the paper “LLaMA, Open and Efficient Foundation Language Models”, available at https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models/.
|
||||
|
||||
More info for KoAlpaca:
|
||||
|
||||
[TBD]
|
||||
|
||||
**Citations details**
|
||||
|
||||
KoLLAMA: [TBD]
|
||||
LLAMA: https://research.facebook.com/publications/llama-open-and-efficient-foundation-language-models/
|
||||
|
||||
**License**
|
||||
|
||||
MIT
|
||||
|
||||
**Where to send questions or comments about the model**
|
||||
|
||||
Questions and comments about KoLLaMA can be sent via the [GitHub repository](https://github.com/beomi/KoLLAMA) of the project , by opening an issue.
|
||||
|
||||
## Intended use
|
||||
**Primary intended uses**
|
||||
|
||||
The primary use of KoLLaMA is research on Korean Opensource large language models
|
||||
|
||||
**Primary intended users**
|
||||
|
||||
The primary intended users of the model are researchers in natural language processing, machine learning and artificial intelligence.
|
||||
|
||||
**Out-of-scope use cases**
|
||||
|
||||
LLaMA is a base, or foundational, model. As such, it should not be used on downstream applications without further risk evaluation and mitigation. In particular, our model has not been trained with human feedback, and can thus generate toxic or offensive content, incorrect information or generally unhelpful answers.
|
||||
|
||||
## Factors
|
||||
|
||||
**Relevant factors**
|
||||
|
||||
One of the most relevant factors for which model performance may vary is which language is used. Although we included 20 languages in the training data, most of our dataset is made of English text, and we thus expect the model to perform better for English than other languages. Relatedly, it has been shown in previous studies that performance might vary for different dialects, and we expect that it will be the case for our model.
|
||||
|
||||
## Evaluation datasets
|
||||
|
||||
[TBD]
|
||||
|
||||
|
||||
## Training dataset
|
||||
|
||||
[TBD]
|
||||
|
||||
## Ethical considerations
|
||||
|
||||
**Data**
|
||||
|
||||
The data used to train the model is collected from various sources, mostly from the Web. As such, it contains offensive, harmful and biased content. We thus expect the model to exhibit such biases from the training data.
|
||||
|
||||
**Human life**
|
||||
|
||||
The model is not intended to inform decisions about matters central to human life, and should not be used in such a way.
|
||||
|
||||
**Risks and harms**
|
||||
|
||||
Risks and harms of large language models include the generation of harmful, offensive or biased content. These models are often prone to generating incorrect information, sometimes referred to as hallucinations. We do not expect our model to be an exception in this regard.
|
||||
|
||||
**Use cases**
|
||||
|
||||
LLaMA is a foundational model, and as such, it should not be used for downstream applications without further investigation and mitigations of risks. These risks and potential fraught use cases include, but are not limited to: generation of misinformation and generation of harmful, biased or offensive content.
|
||||
22
config.json
Normal file
22
config.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 0,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 5120,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 13824,
|
||||
"max_position_embeddings": 2048,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 40,
|
||||
"num_hidden_layers": 40,
|
||||
"pad_token_id": 0,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"tie_word_embeddings": false,
|
||||
"torch_dtype": "float16",
|
||||
"transformers_version": "4.28.0.dev0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 52000
|
||||
}
|
||||
7
generation_config.json
Normal file
7
generation_config.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 1,
|
||||
"eos_token_id": 0,
|
||||
"pad_token_id": 0,
|
||||
"transformers_version": "4.28.0.dev0"
|
||||
}
|
||||
3
model-00001-of-00028.safetensors
Normal file
3
model-00001-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:29912762ced57be1cd94520a508b4e558ddc89428229911f4bfdf3d3fd9d84a3
|
||||
size 883754024
|
||||
3
model-00002-of-00028.safetensors
Normal file
3
model-00002-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:75a1e380fbdd663b9103f64d93f2822e831d9f65699d52ecdff31cf0bb5d8377
|
||||
size 969975720
|
||||
3
model-00003-of-00028.safetensors
Normal file
3
model-00003-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7b950cc4292324494598074e277f95c73d81a5220d696fb00751d9ab123fb63e
|
||||
size 933255328
|
||||
3
model-00004-of-00028.safetensors
Normal file
3
model-00004-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:70ba7f03dd01cef8c6e2057624c93b6c6f61f86686402a02c83a1e60d4c205fa
|
||||
size 969975720
|
||||
3
model-00005-of-00028.safetensors
Normal file
3
model-00005-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a430a3f6b68ca4756e40bf93ee10137a7b233bb8bead624b748d762bd8249a84
|
||||
size 933255328
|
||||
3
model-00006-of-00028.safetensors
Normal file
3
model-00006-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:18eae0463af72bd26b948311363edf733b4a28732c10e564d24c0f3f8d6c1a3a
|
||||
size 969975720
|
||||
3
model-00007-of-00028.safetensors
Normal file
3
model-00007-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2bd326f182800d954ea04ec7ce2c0a7a24767eb9e008b4eb855de7eed9e6e8f9
|
||||
size 933255328
|
||||
3
model-00008-of-00028.safetensors
Normal file
3
model-00008-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b1d97007d2ad725fe9ef865b7c8374909625d82667058a8a264af8dd3d3a3f34
|
||||
size 969975728
|
||||
3
model-00009-of-00028.safetensors
Normal file
3
model-00009-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a5e7f8e7672e4b6f49eefaf77fec35df719ec4a5802a8c1d4276b330681d7290
|
||||
size 933255344
|
||||
3
model-00010-of-00028.safetensors
Normal file
3
model-00010-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3689c4534e6c3523743a6e57af4502074c189eed6623472dc14952fa6aad37db
|
||||
size 969975736
|
||||
3
model-00011-of-00028.safetensors
Normal file
3
model-00011-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:46f8f2fd4658c684b07f70d56088208ee4deff34f1183e03d4e42b92a0b7e554
|
||||
size 933255344
|
||||
3
model-00012-of-00028.safetensors
Normal file
3
model-00012-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:cee95a5e98b7b4c6e66b03db737b7fd85da1cfc1a00bbd538e39d11c8db934f6
|
||||
size 969975736
|
||||
3
model-00013-of-00028.safetensors
Normal file
3
model-00013-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9996ed42ea245dbab2e0ef086f11d18ebdff9b412e433f1d7bba0368bb759269
|
||||
size 933255344
|
||||
3
model-00014-of-00028.safetensors
Normal file
3
model-00014-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:321022996d2a18e0763c042f60c199b047d989cde2dcd67d0d2fbb68df9856fc
|
||||
size 969975736
|
||||
3
model-00015-of-00028.safetensors
Normal file
3
model-00015-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:649781dcd3c63e1879e7d9c35b1a87b05c4c1b6997fab224c2cac281e0e242a0
|
||||
size 933255344
|
||||
3
model-00016-of-00028.safetensors
Normal file
3
model-00016-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:184dcabf1f2cc50583cdbc66b4b81b241ae62deeff4e5e5b7e078e1719867c9a
|
||||
size 969975736
|
||||
3
model-00017-of-00028.safetensors
Normal file
3
model-00017-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:988b2b18c0c0afa608bc5a3e139326482e786218572351fd456281b7afd25b1c
|
||||
size 933255344
|
||||
3
model-00018-of-00028.safetensors
Normal file
3
model-00018-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:704f5490455b66b1b44202d0a0f7066c8fe56b9d9f80740ca8ec2da6b160a55a
|
||||
size 969975736
|
||||
3
model-00019-of-00028.safetensors
Normal file
3
model-00019-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1d85ea989eb7e2ce7f5d373b7248fb7e55c6780e44d9a3cd04a322de9038a186
|
||||
size 933255344
|
||||
3
model-00020-of-00028.safetensors
Normal file
3
model-00020-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c5b0861f77ad745ec6ebcc3834ef862dd7de526591ef9463c7c1df6c3922fe7a
|
||||
size 969975736
|
||||
3
model-00021-of-00028.safetensors
Normal file
3
model-00021-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:2ab0f6556424ca995db043de2372562c969ca055b9d44967c931cf469494093d
|
||||
size 933255344
|
||||
3
model-00022-of-00028.safetensors
Normal file
3
model-00022-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:21e42f4978cf3f113b229dee2c38ac80b4073e05008b0b1e4c0239ec59f53549
|
||||
size 969975736
|
||||
3
model-00023-of-00028.safetensors
Normal file
3
model-00023-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:301fe277844c725cfc04664d4354895e8c004e60874409e5d50b85e1923c3db0
|
||||
size 933255344
|
||||
3
model-00024-of-00028.safetensors
Normal file
3
model-00024-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9ac89771a8b714312c481788a2b8a04099d8e44d95829dd1d71eb95c3be54eca
|
||||
size 969975736
|
||||
3
model-00025-of-00028.safetensors
Normal file
3
model-00025-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c0973973b16a934b88ae75777963619cad81224d35b05fecbd19f90f543d8f65
|
||||
size 933255344
|
||||
3
model-00026-of-00028.safetensors
Normal file
3
model-00026-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d085d4c8415c98efe47d57ddf8b9ea892f9a70f6e06ecf7f1ad3d34050cb4e3b
|
||||
size 969975736
|
||||
3
model-00027-of-00028.safetensors
Normal file
3
model-00027-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:eeb81571ccfbc70d8c99f3330bc9295956b450f750467c7889ceef286959e775
|
||||
size 933255344
|
||||
3
model-00028-of-00028.safetensors
Normal file
3
model-00028-of-00028.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7af40a9d84101d8ce0452e46ae05ab6b5d9cea00e0f6e3d2711b89d241ac824e
|
||||
size 815626904
|
||||
410
model.safetensors.index.json
Normal file
410
model.safetensors.index.json
Normal file
@@ -0,0 +1,410 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 26441338880
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00028-of-00028.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00028.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00028.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00028.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00028.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00028.safetensors",
|
||||
"model.layers.0.self_attn.rotary_emb.inv_freq": "model-00001-of-00028.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00028.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.self_attn.rotary_emb.inv_freq": "model-00002-of-00028.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.self_attn.rotary_emb.inv_freq": "model-00008-of-00028.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.11.self_attn.rotary_emb.inv_freq": "model-00009-of-00028.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.12.self_attn.rotary_emb.inv_freq": "model-00009-of-00028.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00009-of-00028.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.self_attn.rotary_emb.inv_freq": "model-00010-of-00028.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00010-of-00028.safetensors",
|
||||
"model.layers.14.self_attn.rotary_emb.inv_freq": "model-00011-of-00028.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.15.self_attn.rotary_emb.inv_freq": "model-00011-of-00028.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00011-of-00028.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.self_attn.rotary_emb.inv_freq": "model-00012-of-00028.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00012-of-00028.safetensors",
|
||||
"model.layers.17.self_attn.rotary_emb.inv_freq": "model-00013-of-00028.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.18.self_attn.rotary_emb.inv_freq": "model-00013-of-00028.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00013-of-00028.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.self_attn.rotary_emb.inv_freq": "model-00014-of-00028.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00002-of-00028.safetensors",
|
||||
"model.layers.2.self_attn.rotary_emb.inv_freq": "model-00003-of-00028.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00014-of-00028.safetensors",
|
||||
"model.layers.20.self_attn.rotary_emb.inv_freq": "model-00015-of-00028.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.21.self_attn.rotary_emb.inv_freq": "model-00015-of-00028.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00015-of-00028.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.self_attn.rotary_emb.inv_freq": "model-00016-of-00028.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00016-of-00028.safetensors",
|
||||
"model.layers.23.self_attn.rotary_emb.inv_freq": "model-00017-of-00028.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.24.self_attn.rotary_emb.inv_freq": "model-00017-of-00028.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00017-of-00028.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.self_attn.rotary_emb.inv_freq": "model-00018-of-00028.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00018-of-00028.safetensors",
|
||||
"model.layers.26.self_attn.rotary_emb.inv_freq": "model-00019-of-00028.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.27.self_attn.rotary_emb.inv_freq": "model-00019-of-00028.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00019-of-00028.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.self_attn.rotary_emb.inv_freq": "model-00020-of-00028.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00020-of-00028.safetensors",
|
||||
"model.layers.29.self_attn.rotary_emb.inv_freq": "model-00021-of-00028.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.3.self_attn.rotary_emb.inv_freq": "model-00003-of-00028.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00003-of-00028.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.30.self_attn.rotary_emb.inv_freq": "model-00021-of-00028.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00021-of-00028.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.self_attn.rotary_emb.inv_freq": "model-00022-of-00028.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00022-of-00028.safetensors",
|
||||
"model.layers.32.self_attn.rotary_emb.inv_freq": "model-00023-of-00028.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.33.self_attn.rotary_emb.inv_freq": "model-00023-of-00028.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00023-of-00028.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.self_attn.rotary_emb.inv_freq": "model-00024-of-00028.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00024-of-00028.safetensors",
|
||||
"model.layers.35.self_attn.rotary_emb.inv_freq": "model-00025-of-00028.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.36.input_layernorm.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.36.mlp.down_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.36.mlp.gate_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.36.mlp.up_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.36.post_attention_layernorm.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.36.self_attn.k_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.36.self_attn.o_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.36.self_attn.q_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.36.self_attn.rotary_emb.inv_freq": "model-00025-of-00028.safetensors",
|
||||
"model.layers.36.self_attn.v_proj.weight": "model-00025-of-00028.safetensors",
|
||||
"model.layers.37.input_layernorm.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.mlp.down_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.mlp.gate_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.mlp.up_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.post_attention_layernorm.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.self_attn.k_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.self_attn.o_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.self_attn.q_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.self_attn.rotary_emb.inv_freq": "model-00026-of-00028.safetensors",
|
||||
"model.layers.37.self_attn.v_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.38.input_layernorm.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.mlp.down_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.mlp.gate_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.mlp.up_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.post_attention_layernorm.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.self_attn.k_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.self_attn.o_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.self_attn.q_proj.weight": "model-00026-of-00028.safetensors",
|
||||
"model.layers.38.self_attn.rotary_emb.inv_freq": "model-00027-of-00028.safetensors",
|
||||
"model.layers.38.self_attn.v_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.39.input_layernorm.weight": "model-00028-of-00028.safetensors",
|
||||
"model.layers.39.mlp.down_proj.weight": "model-00028-of-00028.safetensors",
|
||||
"model.layers.39.mlp.gate_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.39.mlp.up_proj.weight": "model-00028-of-00028.safetensors",
|
||||
"model.layers.39.post_attention_layernorm.weight": "model-00028-of-00028.safetensors",
|
||||
"model.layers.39.self_attn.k_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.39.self_attn.o_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.39.self_attn.q_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.39.self_attn.rotary_emb.inv_freq": "model-00027-of-00028.safetensors",
|
||||
"model.layers.39.self_attn.v_proj.weight": "model-00027-of-00028.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.self_attn.rotary_emb.inv_freq": "model-00004-of-00028.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00004-of-00028.safetensors",
|
||||
"model.layers.5.self_attn.rotary_emb.inv_freq": "model-00005-of-00028.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.6.self_attn.rotary_emb.inv_freq": "model-00005-of-00028.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00005-of-00028.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.self_attn.rotary_emb.inv_freq": "model-00006-of-00028.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00006-of-00028.safetensors",
|
||||
"model.layers.8.self_attn.rotary_emb.inv_freq": "model-00007-of-00028.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00008-of-00028.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.layers.9.self_attn.rotary_emb.inv_freq": "model-00007-of-00028.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00007-of-00028.safetensors",
|
||||
"model.norm.weight": "model-00028-of-00028.safetensors"
|
||||
}
|
||||
}
|
||||
3
pytorch_model-00001-of-00028.bin
Normal file
3
pytorch_model-00001-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7f080290e479e5303e06a8a69bd4ec6835191e40022f90e7f8c5fe3a0ad3059a
|
||||
size 883756048
|
||||
3
pytorch_model-00002-of-00028.bin
Normal file
3
pytorch_model-00002-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:54e3f8fcea4aeeca003b0b228907aa61aa8d8c912ea98f7a99d1d1337348f007
|
||||
size 969979357
|
||||
3
pytorch_model-00003-of-00028.bin
Normal file
3
pytorch_model-00003-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c7856dee5e481e21e6279460c912118afbf3861a58c6593379fb2c7227537dc1
|
||||
size 933258973
|
||||
3
pytorch_model-00004-of-00028.bin
Normal file
3
pytorch_model-00004-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:8d2af83e24ab7ec3642c9716cae975224e01728cc4c47fb711c2e0b223342c62
|
||||
size 969979357
|
||||
3
pytorch_model-00005-of-00028.bin
Normal file
3
pytorch_model-00005-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9bdb317c89a0e899152d72c2bad92a9c86d858470518d4eedddd3fba3bfcd432
|
||||
size 933258973
|
||||
3
pytorch_model-00006-of-00028.bin
Normal file
3
pytorch_model-00006-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:26faa44b322949cf4c4178fe081a6ed0e6a3d4f3631e90207f6e353d9089e5a0
|
||||
size 969979357
|
||||
3
pytorch_model-00007-of-00028.bin
Normal file
3
pytorch_model-00007-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a5db0bfb4162a58425622f134faf10ce652b8ce3d140c39c7bad664e3c4ba33d
|
||||
size 933258973
|
||||
3
pytorch_model-00008-of-00028.bin
Normal file
3
pytorch_model-00008-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:dca421cc47467d35463bc9f24a79d4943c0a79e76fc22853f7e606019e0d3755
|
||||
size 969979357
|
||||
3
pytorch_model-00009-of-00028.bin
Normal file
3
pytorch_model-00009-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:cb042e180e8c3e919d9229893d835c218686a5d75b94e2b38a2b4aa1fda7335e
|
||||
size 933258973
|
||||
3
pytorch_model-00010-of-00028.bin
Normal file
3
pytorch_model-00010-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0d4074fc82c1bf6eb9bcda2b54d674c1c78b883e8dfbd53087894a1e8b3f0c2d
|
||||
size 969979357
|
||||
3
pytorch_model-00011-of-00028.bin
Normal file
3
pytorch_model-00011-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:eecbb52e0cc739c8e703da53c4eb7d0c6fda6a42a7857e9ddda208155c87ad01
|
||||
size 933258973
|
||||
3
pytorch_model-00012-of-00028.bin
Normal file
3
pytorch_model-00012-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d87ef24cd81803688bf360749ee5e18bd89685ff93ca9e68fa0aac5557a9a7b1
|
||||
size 969979357
|
||||
3
pytorch_model-00013-of-00028.bin
Normal file
3
pytorch_model-00013-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:138102a2c3a30335b2b62b989f08715cb2b972724cd22aa25396849e6d3a8789
|
||||
size 933258973
|
||||
3
pytorch_model-00014-of-00028.bin
Normal file
3
pytorch_model-00014-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4b0a556a74c60d9cb20ef26fda6f915bc7a3a0cd4465f7fa43c1b6b5f1bef127
|
||||
size 969979357
|
||||
3
pytorch_model-00015-of-00028.bin
Normal file
3
pytorch_model-00015-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0db7c4c907df33969791673f52e60517647388735897549f703fe13c94fc7da7
|
||||
size 933258973
|
||||
3
pytorch_model-00016-of-00028.bin
Normal file
3
pytorch_model-00016-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:16f977ca91f477e744fb9421cb0b252c87ff7be17d76fe3e52b9fd83ce7e6705
|
||||
size 969979357
|
||||
3
pytorch_model-00017-of-00028.bin
Normal file
3
pytorch_model-00017-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9b06f96ca0776e21ec4fa3fcb6e791c61bbab70da108e12c3bca27be72bb8800
|
||||
size 933258973
|
||||
3
pytorch_model-00018-of-00028.bin
Normal file
3
pytorch_model-00018-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a676d42cdc8268f07e3c838e59790d4657e5030ef93f60612aa991fc6e837ed8
|
||||
size 969979357
|
||||
3
pytorch_model-00019-of-00028.bin
Normal file
3
pytorch_model-00019-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9ac69f5e9e55dc3833aa97bb44a3c672d27f6c08cecd312cd48ccd68f162b189
|
||||
size 933258973
|
||||
3
pytorch_model-00020-of-00028.bin
Normal file
3
pytorch_model-00020-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:90ae03c9629fc11ec66453e749f64f8562d0d7da85cc913c405cec5d151f4609
|
||||
size 969979357
|
||||
3
pytorch_model-00021-of-00028.bin
Normal file
3
pytorch_model-00021-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4b89969e3b83c1bb381bfc11f047a0d9e566b1140bdd2c28e970a095574dfbc7
|
||||
size 933258973
|
||||
3
pytorch_model-00022-of-00028.bin
Normal file
3
pytorch_model-00022-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:86ca57e1743afbc20875ac0969550f0ecc9399b043b1bca38e4f7d2aaa55cecc
|
||||
size 969979357
|
||||
3
pytorch_model-00023-of-00028.bin
Normal file
3
pytorch_model-00023-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7e5f12f4af7eebb6b15bb7d719987cb71c741189e49e12917aa6e4a5fa517c95
|
||||
size 933258973
|
||||
3
pytorch_model-00024-of-00028.bin
Normal file
3
pytorch_model-00024-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:80a736b9e84a9b61c498c5ff3db3daa8259df7480bf3d7deed5d670776da44d7
|
||||
size 969979357
|
||||
3
pytorch_model-00025-of-00028.bin
Normal file
3
pytorch_model-00025-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e020510af416faa55316229d98a014bae0d141fd9123ea7f68dd91a3fde4b6d8
|
||||
size 933258973
|
||||
3
pytorch_model-00026-of-00028.bin
Normal file
3
pytorch_model-00026-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:36ef634f1b95af318689594906236ab02fe1b1c50c86e274b1cb1d3376c7fa4e
|
||||
size 969979357
|
||||
3
pytorch_model-00027-of-00028.bin
Normal file
3
pytorch_model-00027-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a1727a61c26d6ed8beca461ae62076efb47b34b6cb2a429ec20302f1301ed32e
|
||||
size 933258973
|
||||
3
pytorch_model-00028-of-00028.bin
Normal file
3
pytorch_model-00028-of-00028.bin
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6af23940413238812345cb771ee7453768da17cdf3d1c8c57bdde19296a19512
|
||||
size 815628671
|
||||
410
pytorch_model.bin.index.json
Normal file
410
pytorch_model.bin.index.json
Normal file
@@ -0,0 +1,410 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_size": 26441338880
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "pytorch_model-00028-of-00028.bin",
|
||||
"model.embed_tokens.weight": "pytorch_model-00001-of-00028.bin",
|
||||
"model.layers.0.input_layernorm.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.0.mlp.down_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00028.bin",
|
||||
"model.layers.0.mlp.up_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.0.post_attention_layernorm.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00028.bin",
|
||||
"model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00028.bin",
|
||||
"model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00028.bin",
|
||||
"model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00028.bin",
|
||||
"model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00028.bin",
|
||||
"model.layers.1.input_layernorm.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.mlp.down_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.mlp.gate_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.mlp.up_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.post_attention_layernorm.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.self_attn.k_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.self_attn.o_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.self_attn.q_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.1.self_attn.v_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.10.input_layernorm.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.mlp.down_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.mlp.gate_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.mlp.up_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.post_attention_layernorm.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.self_attn.k_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.self_attn.o_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.self_attn.q_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.10.self_attn.v_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.11.input_layernorm.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.mlp.down_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.mlp.gate_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.mlp.up_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.post_attention_layernorm.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.self_attn.k_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.self_attn.o_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.self_attn.q_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.11.self_attn.v_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.12.input_layernorm.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.12.mlp.down_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.12.mlp.gate_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.12.mlp.up_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.12.post_attention_layernorm.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.12.self_attn.k_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.12.self_attn.o_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.12.self_attn.q_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.12.self_attn.v_proj.weight": "pytorch_model-00009-of-00028.bin",
|
||||
"model.layers.13.input_layernorm.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.mlp.down_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.mlp.gate_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.mlp.up_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.post_attention_layernorm.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.self_attn.k_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.self_attn.o_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.self_attn.q_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.13.self_attn.v_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.14.input_layernorm.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.mlp.down_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.mlp.gate_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.mlp.up_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.post_attention_layernorm.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.self_attn.k_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.self_attn.o_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.self_attn.q_proj.weight": "pytorch_model-00010-of-00028.bin",
|
||||
"model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.14.self_attn.v_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.15.input_layernorm.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.15.mlp.down_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.15.mlp.gate_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.15.mlp.up_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.15.post_attention_layernorm.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.15.self_attn.k_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.15.self_attn.o_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.15.self_attn.q_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.15.self_attn.v_proj.weight": "pytorch_model-00011-of-00028.bin",
|
||||
"model.layers.16.input_layernorm.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.mlp.down_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.mlp.gate_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.mlp.up_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.post_attention_layernorm.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.self_attn.k_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.self_attn.o_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.self_attn.q_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.16.self_attn.v_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.17.input_layernorm.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.mlp.down_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.mlp.gate_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.mlp.up_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.post_attention_layernorm.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.self_attn.k_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.self_attn.o_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.self_attn.q_proj.weight": "pytorch_model-00012-of-00028.bin",
|
||||
"model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.17.self_attn.v_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.18.input_layernorm.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.18.mlp.down_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.18.mlp.gate_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.18.mlp.up_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.18.post_attention_layernorm.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.18.self_attn.k_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.18.self_attn.o_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.18.self_attn.q_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.18.self_attn.v_proj.weight": "pytorch_model-00013-of-00028.bin",
|
||||
"model.layers.19.input_layernorm.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.mlp.down_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.mlp.gate_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.mlp.up_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.post_attention_layernorm.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.self_attn.k_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.self_attn.o_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.self_attn.q_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.19.self_attn.v_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.2.input_layernorm.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.mlp.down_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.mlp.gate_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.mlp.up_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.post_attention_layernorm.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.self_attn.k_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.self_attn.o_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.self_attn.q_proj.weight": "pytorch_model-00002-of-00028.bin",
|
||||
"model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.2.self_attn.v_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.20.input_layernorm.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.mlp.down_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.mlp.gate_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.mlp.up_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.post_attention_layernorm.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.self_attn.k_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.self_attn.o_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.self_attn.q_proj.weight": "pytorch_model-00014-of-00028.bin",
|
||||
"model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.20.self_attn.v_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.21.input_layernorm.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.21.mlp.down_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.21.mlp.gate_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.21.mlp.up_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.21.post_attention_layernorm.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.21.self_attn.k_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.21.self_attn.o_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.21.self_attn.q_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.21.self_attn.v_proj.weight": "pytorch_model-00015-of-00028.bin",
|
||||
"model.layers.22.input_layernorm.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.mlp.down_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.mlp.gate_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.mlp.up_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.post_attention_layernorm.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.self_attn.k_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.self_attn.o_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.self_attn.q_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.22.self_attn.v_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.23.input_layernorm.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.mlp.down_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.mlp.gate_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.mlp.up_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.post_attention_layernorm.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.self_attn.k_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.self_attn.o_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.self_attn.q_proj.weight": "pytorch_model-00016-of-00028.bin",
|
||||
"model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.23.self_attn.v_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.24.input_layernorm.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.24.mlp.down_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.24.mlp.gate_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.24.mlp.up_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.24.post_attention_layernorm.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.24.self_attn.k_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.24.self_attn.o_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.24.self_attn.q_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.24.self_attn.v_proj.weight": "pytorch_model-00017-of-00028.bin",
|
||||
"model.layers.25.input_layernorm.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.mlp.down_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.mlp.gate_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.mlp.up_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.post_attention_layernorm.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.self_attn.k_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.self_attn.o_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.self_attn.q_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.25.self_attn.v_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.26.input_layernorm.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.mlp.down_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.mlp.gate_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.mlp.up_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.post_attention_layernorm.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.self_attn.k_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.self_attn.o_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.self_attn.q_proj.weight": "pytorch_model-00018-of-00028.bin",
|
||||
"model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.26.self_attn.v_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.27.input_layernorm.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.27.mlp.down_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.27.mlp.gate_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.27.mlp.up_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.27.post_attention_layernorm.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.27.self_attn.k_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.27.self_attn.o_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.27.self_attn.q_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.27.self_attn.v_proj.weight": "pytorch_model-00019-of-00028.bin",
|
||||
"model.layers.28.input_layernorm.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.mlp.down_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.mlp.gate_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.mlp.up_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.post_attention_layernorm.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.self_attn.k_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.self_attn.o_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.self_attn.q_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.28.self_attn.v_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.29.input_layernorm.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.mlp.down_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.mlp.gate_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.mlp.up_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.post_attention_layernorm.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.self_attn.k_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.self_attn.o_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.self_attn.q_proj.weight": "pytorch_model-00020-of-00028.bin",
|
||||
"model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.29.self_attn.v_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.3.input_layernorm.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.3.mlp.down_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.3.mlp.up_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.3.post_attention_layernorm.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.3.self_attn.k_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.3.self_attn.o_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.3.self_attn.q_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.3.self_attn.v_proj.weight": "pytorch_model-00003-of-00028.bin",
|
||||
"model.layers.30.input_layernorm.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.30.mlp.down_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.30.mlp.gate_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.30.mlp.up_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.30.post_attention_layernorm.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.30.self_attn.k_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.30.self_attn.o_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.30.self_attn.q_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.30.self_attn.v_proj.weight": "pytorch_model-00021-of-00028.bin",
|
||||
"model.layers.31.input_layernorm.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.mlp.down_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.mlp.gate_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.mlp.up_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.post_attention_layernorm.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.self_attn.k_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.self_attn.o_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.self_attn.q_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.31.self_attn.v_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.32.input_layernorm.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.mlp.down_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.mlp.gate_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.mlp.up_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.post_attention_layernorm.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.self_attn.k_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.self_attn.o_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.self_attn.q_proj.weight": "pytorch_model-00022-of-00028.bin",
|
||||
"model.layers.32.self_attn.rotary_emb.inv_freq": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.32.self_attn.v_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.33.input_layernorm.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.33.mlp.down_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.33.mlp.gate_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.33.mlp.up_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.33.post_attention_layernorm.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.33.self_attn.k_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.33.self_attn.o_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.33.self_attn.q_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.33.self_attn.rotary_emb.inv_freq": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.33.self_attn.v_proj.weight": "pytorch_model-00023-of-00028.bin",
|
||||
"model.layers.34.input_layernorm.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.mlp.down_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.mlp.gate_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.mlp.up_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.post_attention_layernorm.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.self_attn.k_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.self_attn.o_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.self_attn.q_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.self_attn.rotary_emb.inv_freq": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.34.self_attn.v_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.35.input_layernorm.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.mlp.down_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.mlp.gate_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.mlp.up_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.post_attention_layernorm.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.self_attn.k_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.self_attn.o_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.self_attn.q_proj.weight": "pytorch_model-00024-of-00028.bin",
|
||||
"model.layers.35.self_attn.rotary_emb.inv_freq": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.35.self_attn.v_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.36.input_layernorm.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.36.mlp.down_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.36.mlp.gate_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.36.mlp.up_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.36.post_attention_layernorm.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.36.self_attn.k_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.36.self_attn.o_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.36.self_attn.q_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.36.self_attn.rotary_emb.inv_freq": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.36.self_attn.v_proj.weight": "pytorch_model-00025-of-00028.bin",
|
||||
"model.layers.37.input_layernorm.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.mlp.down_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.mlp.gate_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.mlp.up_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.post_attention_layernorm.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.self_attn.k_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.self_attn.o_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.self_attn.q_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.self_attn.rotary_emb.inv_freq": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.37.self_attn.v_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.38.input_layernorm.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.mlp.down_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.mlp.gate_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.mlp.up_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.post_attention_layernorm.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.self_attn.k_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.self_attn.o_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.self_attn.q_proj.weight": "pytorch_model-00026-of-00028.bin",
|
||||
"model.layers.38.self_attn.rotary_emb.inv_freq": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.38.self_attn.v_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.39.input_layernorm.weight": "pytorch_model-00028-of-00028.bin",
|
||||
"model.layers.39.mlp.down_proj.weight": "pytorch_model-00028-of-00028.bin",
|
||||
"model.layers.39.mlp.gate_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.39.mlp.up_proj.weight": "pytorch_model-00028-of-00028.bin",
|
||||
"model.layers.39.post_attention_layernorm.weight": "pytorch_model-00028-of-00028.bin",
|
||||
"model.layers.39.self_attn.k_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.39.self_attn.o_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.39.self_attn.q_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.39.self_attn.rotary_emb.inv_freq": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.39.self_attn.v_proj.weight": "pytorch_model-00027-of-00028.bin",
|
||||
"model.layers.4.input_layernorm.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.mlp.down_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.mlp.gate_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.mlp.up_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.post_attention_layernorm.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.self_attn.k_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.self_attn.o_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.self_attn.q_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.4.self_attn.v_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.5.input_layernorm.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.mlp.down_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.mlp.gate_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.mlp.up_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.post_attention_layernorm.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.self_attn.k_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.self_attn.o_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.self_attn.q_proj.weight": "pytorch_model-00004-of-00028.bin",
|
||||
"model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.5.self_attn.v_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.6.input_layernorm.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.6.mlp.down_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.6.mlp.gate_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.6.mlp.up_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.6.post_attention_layernorm.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.6.self_attn.k_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.6.self_attn.o_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.6.self_attn.q_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.6.self_attn.v_proj.weight": "pytorch_model-00005-of-00028.bin",
|
||||
"model.layers.7.input_layernorm.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.mlp.down_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.mlp.gate_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.mlp.up_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.post_attention_layernorm.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.self_attn.k_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.self_attn.o_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.self_attn.q_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.7.self_attn.v_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.8.input_layernorm.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.mlp.down_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.mlp.gate_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.mlp.up_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.post_attention_layernorm.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.self_attn.k_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.self_attn.o_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.self_attn.q_proj.weight": "pytorch_model-00006-of-00028.bin",
|
||||
"model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.8.self_attn.v_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.9.input_layernorm.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.9.mlp.down_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.9.mlp.gate_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.9.mlp.up_proj.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.9.post_attention_layernorm.weight": "pytorch_model-00008-of-00028.bin",
|
||||
"model.layers.9.self_attn.k_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.9.self_attn.o_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.9.self_attn.q_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00028.bin",
|
||||
"model.layers.9.self_attn.v_proj.weight": "pytorch_model-00007-of-00028.bin",
|
||||
"model.norm.weight": "pytorch_model-00028-of-00028.bin"
|
||||
}
|
||||
}
|
||||
103816
tokenizer.json
Normal file
103816
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
8
tokenizer_config.json
Normal file
8
tokenizer_config.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"name_or_path": "beomi/KoLLAMA",
|
||||
"eos_token": "<|endoftext|>",
|
||||
"bos_token": "<|sep|>",
|
||||
"pad_token": "<|endoftext|>",
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"tokenizer_class": "PreTrainedTokenizerFast"
|
||||
}
|
||||
Reference in New Issue
Block a user