IDEA-CCNL/Ziya-Writing-LLaMa-13B-v1
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: IDEA-CCNL/Ziya-Writing-LLaMa-13B-v1
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-02 17:19:16 +08:00
commit d2f6a28f87
27 changed files with 766 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

169
README.md Normal file
View File

@@ -0,0 +1,169 @@
---
license: gpl-3.0
language:
- zh
- en
library_name: transformers
pipeline_tag: text-generation
---
# Ziya-LLaMA-13B-v1
- Main Page:[Fengshenbang](https://fengshenbang-lm.com/)
- Github: [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM)
# 姜子牙系列模型
- [Ziya-LLaMA-13B-v1.1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-v1.1)
- [Ziya-LLaMA-13B-v1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-v1)
- [Ziya-LLaMA-7B-Reward](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-7B-Reward)
- [Ziya-LLaMA-13B-Pretrain-v1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-Pretrain-v1)
- [Ziya-BLIP2-14B-Visual-v1](https://huggingface.co/IDEA-CCNL/Ziya-BLIP2-14B-Visual-v1)
## 简介 Brief Introduction
姜子牙写作大模型V1是基于LLaMa的130亿参数的指令微调模型，在写作任务上进行了能力增强，是专注于写作的大模型。姜子牙写作模型可以完成公文报告、讲稿书信、创意文案等多类的写作任务。
Ziya-Writing-LLaMa-13B-v1 is a 13-billion parameter instruction fine-tuned model based on LLaMa, which has been enhanced for better performance in writing tasks. It is a large model that focuses on writing. Ziya-Writing-LLaMa-13B-v1 can handle several types of writing tasks, including official reports, speeches, creative copywriting, and more.
更多细节可以参考我们的公众号文章：
[姜子牙大模型系列 | 写作模型ziya-writing开源！开箱即用，快来认领专属你的写作小助手吧](https://mp.weixin.qq.com/s/3Z7i4uIxx_u-Peu8iUX7EQ)
## 软件依赖
```
pip install torch==1.12.1 tokenizers==0.13.3 git+https://github.com/huggingface/transformers
```
## 模型分类 Model Taxonomy
| 需求 Demand | 任务 Task | 系列 Series | 模型 Model | 参数 Parameter | 额外 Extra |
| :----: | :----: | :----: | :----: | :----: | :----: |
| 写作 Writing | AGI模型 | 姜子牙 Ziya | LLaMA | 13B | English&Chinese |
## 模型信息 Model Information
### 有监督微调 Supervised finetuning
我们从网络中收集并清洗了大量真实的真人写作数据，利用GPT-3.5生成对应的写作指令，并进行了极为严格的人工校验。
在此基础上，我们利用奖励模型和一定的清洗逻辑，精心挑选了难度更高的写作指令，剔除了简单的数据，并保证了指令的多样性。
我们利用evol-instruct的方法，生成了约30万条高质量的通用指令数据。我们混合了通用指令数据和写作指令数据，这使得ziya-writing不仅拥有良好的意图理解能力，也能够生成优秀的回答。
We collected and cleaned a large amount of real human writing data from the internet and used GPT-3.5 to generate corresponding writing instructions which have undergone extremely strict manual verification.
Based on this, we used a reward model and certain cleaning logic to carefully select more challenging writing instructions, eliminating simple data, and ensuring the diversity of instructions.
We used the evol-instruct method to generate about 300,000 high-quality general instruction data. We mixed general instruction data and writing instruction data, which made ziya-writing not only have good intention understanding ability, but also can generate excellent responses.
### 人类反馈学习 Human-Feedback training
我们在实验中发现，利用少量人类标注的高质量的写作排序数据，使用强化学习训练模型，就能对进一步拔高模型的写作效果。
为了进一步提升模型的表现，使其能够充分理解人类意图、减少“幻觉”和不安全的输出，基于指令微调后的模型，进行了人类反馈训练（Human-Feedback Training，HFT）。在训练中，我们采用了以人类反馈强化学习（RM、PPO）为主。
我们在内部自研的框架上实现了HFT的训练流程，该框架可以利用最少8张40G的A100显卡完成Ziya-Writing-LLaMA-13B-v1的全参数训练。在PPO训练中，我们没有限制生成样本的长度，以确保长文本任务的奖励准确性。每次训练的总经验池尺寸超过100k样本，确保了训练的充分性。
In our experiment, we found that by using a small amount of high-quality human-annotated writing ranking data and training the model with reinforcement learning, we could effectively improve the writing performance of the model.
To further improve the performance of the model, enabling it to fully understand human intentions, reduce "hallucinations" and unsafe outputs, we conducted Human-Feedback Training (HFT) based on the model fine-tuned with instructions. In the training process, we used human feedback reinforcement learning (RM, PPO).
We implemented the HFT training process on an internally developed framework, which can use a minimum of 8 40GB A100 GPUs to complete the full parameter training of Ziya-Writing-LLaMA-13B-v1. In the PPO training, we did not limit the length of the generated samples to ensure the accuracy of rewards for long-text tasks. The total experience pool size for each training exceeded 100k samples, ensuring the sufficiency of the training.
### 效果评估 Performance
写作文案的优劣评价是一个较为主观的评判，很难用一个准确率或者满意度的打分来衡量。因此，我们使用了匿名模型多人Side-by-Side评估的机制，收集了100条不同难度的写作指令数据进行评估，我们后续也会公开这个评测集。
我们以胜出率作为评价模型好坏的指标，一个模型的胜出率计算公式为：
胜出率=（该模型的胜出数量+打平数量/2）/总标注数
一般而言，由于语言模型大多基于采样来生成回答，因此胜出率大于55%表示该模型显著胜出于另外一个模型，胜出率小于45%表示该模型明显落后，胜出率在45%至55%之间表示两个模型基本持平。
The evaluation of the quality of a writing task is quite subjective, making it difficult to measure with precise accuracy or satisfaction score. Therefore, we've used an anonymous multi-person Side-by-Side evaluation mechanism, and have collected 100 pieces of writing instruction data of different difficulties for evaluation. We will also make this evaluation set public in the future.
We use the win rate as an indicator of the quality of a model. The formula to calculate a model's win rate is as follows:
Win Rate = (Number of wins for the model + Number of draws / 2) / Total number of annotations
Generally, since most language models generate responses based on sampling, hence, a win rate greater than 55% indicates that the model significantly outperforms another model, a win rate less than 45% shows that the model clearly lags behind, and a win rate between 45% and 55% signifies that the two models are essentially on par.
| Ziya-Writing-LLaMa-13B-v1 | 平均胜出率 | 最大胜出率 | 最小胜出率 |
| :----: | :----: | :----: | :----: |
| vs Ziya-LLaMa-13B-v1.1 | 70.7 | 73.5 | 69 |
| vs baichuan-vicuna-7b | 69.6 | 73.5 | 68 |
| vs Moss-16B | 65.1 | 69 | 62 |
| vs ChatGLM2-6B | 58.3 | 61.5 | 56 |
| vs Minimax-abab5 | 52.3 | 53 | 50.5 |
| vs GPT-3.5-turbo | 44.7 | 49.5 | 38 |
（注：最大胜出率和最小胜出率，是对每一个标注人员的标注结果进行单独统计，计算出最大和最小的得分；平均胜出率是对所有标注人员的标注结果进行汇总统计，计算出平均的得分。）
## <span id="jump"> 使用 Usage </span>
由于LLaMA权重的许可限制，该模型不能用于商业用途，请严格遵守LLaMA的使用政策。
```python3
from transformers import AutoTokenizer
from transformers import LlamaForCausalLM
import torch
device = torch.device("cuda")
query="帮我写一份去西安的旅游计划"
model = LlamaForCausalLM.from_pretrained("IDEA-CCNL/Ziya-Writing-LLaMa-13B-v1", torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("IDEA-CCNL/Ziya-Writing-LLaMa-13B-v1", use_fast=False)
inputs = '<human>:' + query.strip() + '\n<bot>:'
input_ids = tokenizer(inputs, return_tensors="pt").input_ids.to(device)
generate_ids = model.generate(
input_ids,
max_new_tokens=2048,
do_sample = True,
top_p = 0.85,
temperature = 0.85,
repetition_penalty=1.,
eos_token_id=2,
bos_token_id=1,
pad_token_id=0)
output = tokenizer.batch_decode(generate_ids)[0]
print(output)
```
## 微调示例 Finetune Example
Refer to [ziya_finetune](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen/examples/ziya_llama)
## 推理量化示例 Inference & Quantization Example
Refer to [ziya_inference](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen/examples/ziya_inference)
## 引用 Citation
如果您在您的工作中使用了我们的模型，可以引用我们的[论文](https://arxiv.org/abs/2210.08590)：
If you are using the resource for your work, please cite the our [paper](https://arxiv.org/abs/2210.08590):
```text
@article{fengshenbang,
author = {Jiaxing Zhang and Ruyi Gan and Junjie Wang and Yuxiang Zhang and Lin Zhang and Ping Yang and Xinyu Gao and Ziwei Wu and Xiaoqun Dong and Junqing He and Jianheng Zhuo and Qi Yang and Yongfeng Huang and Xiayu Li and Yanghan Wu and Junyu Lu and Xinyu Zhu and Weifeng Chen and Ting Han and Kunhao Pan and Rui Wang and Hao Wang and Xiaojun Wu and Zhongshen Zeng and Chongpei Chen},
title = {Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence},
journal = {CoRR},
volume = {abs/2209.02970},
year = {2022}
}
```
You can also cite our [website](https://github.com/IDEA-CCNL/Fengshenbang-LM/):
欢迎引用我们的[网站](https://github.com/IDEA-CCNL/Fengshenbang-LM/):
```text
@misc{Fengshenbang-LM,
title={Fengshenbang-LM},
author={IDEA-CCNL},
year={2021},
howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
}
```

27
config.json Normal file
View File

@@ -0,0 +1,27 @@
{
"_name_or_path": "Ziya-Writing-13B-v1",
"architectures": [
"LlamaForCausalLM"
],
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 5120,
"initializer_range": 0.02,
"intermediate_size": 13824,
"layer_norm_eps": 1e-05,
"max_position_embeddings": 2048,
"model_type": "llama",
"num_attention_heads": 40,
"num_hidden_layers": 40,
"pad_token_id": 0,
"rms_norm_eps": 1e-06,
"rotary_emb_base": 10000,
"rotary_pct": 1,
"tie_word_embeddings": false,
"torch_dtype": "float32",
"transformers_version": "4.30.2",
"use_cache": false,
"use_parallel_residual": false,
"vocab_size": 39424
}

7
generation_config.json Normal file
View File

@@ -0,0 +1,7 @@
{
"_from_model_config": true,
"bos_token_id": 1,
"eos_token_id": 2,
"pad_token_id": 0,
"transformers_version": "4.30.2"
}

3
pytorch_model-00001-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fec33896da438c959c1678f34d52aa74ee4880b1f096a4f1fa1fd29758dbc56f
size 2778773825

3
pytorch_model-00002-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:835797c5b52ffacdd8301b60c2144d5e0d94ecd54e2a90e5cb6be0ee5a9f3358
size 2820759049

3
pytorch_model-00003-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:538a7f3384ca5d5089c199d70b04b78e41c26a9463e48bac90399edd3351f0eb
size 2925658559

3
pytorch_model-00004-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f48b21a9d5b7a2e485aa75e0cc0cb02b03f4f7b219b574c271b892253e03bdf7
size 2852217535

3
pytorch_model-00005-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:04fd8d5d39030a6456eb803e5180705e38a86338d413c9e03713714c3b380642
size 2820759049

3
pytorch_model-00006-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3f71bc6829964fd160778d124ecffe444fcbd143a9c6b57f983a673df965be78
size 2820759049

3
pytorch_model-00007-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ced14dbc9445e1a9ad0351f3d77f5a4fcef3c9b37e287cd08d53822b107662c3
size 2925658559

3
pytorch_model-00008-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:48b3be835159714808f72adaf484c5505dde41daa9ea89044e9df215c519cf3f
size 2852217535

3
pytorch_model-00009-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:70991c6dc933260d93db64ae1354bd657e83522403a20735177e06daf8b86245
size 2820759049

3
pytorch_model-00010-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2296ade2da137db6ac680c261daa891d40b67b16c838510c20ad74674722de4d
size 2820759049

3
pytorch_model-00011-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:03c64c86a4e2308c1439f54f66c8ba869c54541284d279c291390719439e5c5a
size 2925658559

3
pytorch_model-00012-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eb1238b462ac11633cecce1c7353001a729432d3132df525367f6d30c86ddada
size 2852217535

3
pytorch_model-00013-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:587bff335a3c0f082a24d1d373a8a81aae1237e78ae08e8a8a74874091fc2cda
size 2820759049

3
pytorch_model-00014-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b0bcb7f2fe588c7da18bcf324f33bcc0be9da33f688ab2e4045d592f3966a465
size 2820759049

3
pytorch_model-00015-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:92d35a473866d7d22a2b70d6e4e3c90efbd9e414211703c9cec124fe7f964979
size 2925658559

3
pytorch_model-00016-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a23288b26590fb341e1ae858ffb7f74000a364aeab811a42e514d7528be1967f
size 2852217535

3
pytorch_model-00017-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:57a3311f2535c08ec72214b89d4fb976d4ec468553e7953f6f6a682a311e0b1e
size 2820759049

3
pytorch_model-00018-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a11a3899ebb62971fa6be301f5efe989ed46dedbc34968ecb6f10a54b110e2f4
size 2820759049

3
pytorch_model-00019-of-00019.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e4565877a6f050ac4ea04d146b9e9aeb4e83600e5323da3cb365d731b562fd43
size 1090582638

410
pytorch_model.bin.index.json Normal file
View File

@@ -0,0 +1,410 @@
{
"metadata": {
"total_size": 52367554560
},
"weight_map": {
"lm_head.weight": "pytorch_model-00019-of-00019.bin",
"model.embed_tokens.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00019.bin",
"model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.1.input_layernorm.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.1.mlp.down_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.1.mlp.up_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.1.post_attention_layernorm.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00019.bin",
"model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00019.bin",
"model.layers.10.input_layernorm.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.10.mlp.down_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.10.mlp.gate_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.10.mlp.up_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.10.post_attention_layernorm.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.10.self_attn.k_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.10.self_attn.o_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.10.self_attn.q_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00019.bin",
"model.layers.10.self_attn.v_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.11.input_layernorm.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.mlp.down_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.mlp.gate_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.mlp.up_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.post_attention_layernorm.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.self_attn.k_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.self_attn.o_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.self_attn.q_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00019.bin",
"model.layers.11.self_attn.v_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.12.input_layernorm.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.12.mlp.down_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.12.mlp.gate_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.12.mlp.up_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.12.post_attention_layernorm.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.12.self_attn.k_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.12.self_attn.o_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.12.self_attn.q_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00019.bin",
"model.layers.12.self_attn.v_proj.weight": "pytorch_model-00006-of-00019.bin",
"model.layers.13.input_layernorm.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.mlp.down_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.mlp.gate_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.mlp.up_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.post_attention_layernorm.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.self_attn.k_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.self_attn.o_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.self_attn.q_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00019.bin",
"model.layers.13.self_attn.v_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.input_layernorm.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.mlp.down_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.mlp.gate_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.mlp.up_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.post_attention_layernorm.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.self_attn.k_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.self_attn.o_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.self_attn.q_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00019.bin",
"model.layers.14.self_attn.v_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.15.input_layernorm.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.15.mlp.down_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.15.mlp.gate_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.15.mlp.up_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.15.post_attention_layernorm.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.15.self_attn.k_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.15.self_attn.o_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.15.self_attn.q_proj.weight": "pytorch_model-00007-of-00019.bin",
"model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00019.bin",
"model.layers.15.self_attn.v_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.input_layernorm.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.mlp.down_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.mlp.gate_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.mlp.up_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.post_attention_layernorm.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.self_attn.k_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.self_attn.o_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.self_attn.q_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00019.bin",
"model.layers.16.self_attn.v_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.17.input_layernorm.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.17.mlp.down_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.17.mlp.gate_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.17.mlp.up_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.17.post_attention_layernorm.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.17.self_attn.k_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.17.self_attn.o_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.17.self_attn.q_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00019.bin",
"model.layers.17.self_attn.v_proj.weight": "pytorch_model-00008-of-00019.bin",
"model.layers.18.input_layernorm.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.mlp.down_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.mlp.gate_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.mlp.up_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.post_attention_layernorm.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.self_attn.k_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.self_attn.o_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.self_attn.q_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00019.bin",
"model.layers.18.self_attn.v_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.19.input_layernorm.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.19.mlp.down_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.19.mlp.gate_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.19.mlp.up_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.19.post_attention_layernorm.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.19.self_attn.k_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.19.self_attn.o_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.19.self_attn.q_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00019.bin",
"model.layers.19.self_attn.v_proj.weight": "pytorch_model-00009-of-00019.bin",
"model.layers.2.input_layernorm.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.mlp.down_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.mlp.gate_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.mlp.up_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.post_attention_layernorm.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.self_attn.k_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.self_attn.o_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.self_attn.q_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00019.bin",
"model.layers.2.self_attn.v_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.20.input_layernorm.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.mlp.down_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.mlp.gate_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.mlp.up_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.post_attention_layernorm.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.self_attn.k_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.self_attn.o_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.self_attn.q_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00010-of-00019.bin",
"model.layers.20.self_attn.v_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.21.input_layernorm.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.21.mlp.down_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.21.mlp.gate_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.21.mlp.up_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.21.post_attention_layernorm.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.21.self_attn.k_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.21.self_attn.o_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.21.self_attn.q_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00010-of-00019.bin",
"model.layers.21.self_attn.v_proj.weight": "pytorch_model-00010-of-00019.bin",
"model.layers.22.input_layernorm.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.mlp.down_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.mlp.gate_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.mlp.up_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.post_attention_layernorm.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.self_attn.k_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.self_attn.o_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.self_attn.q_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00011-of-00019.bin",
"model.layers.22.self_attn.v_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.input_layernorm.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.mlp.down_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.mlp.gate_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.mlp.up_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.post_attention_layernorm.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.self_attn.k_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.self_attn.o_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.self_attn.q_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00011-of-00019.bin",
"model.layers.23.self_attn.v_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.24.input_layernorm.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.24.mlp.down_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.24.mlp.gate_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.24.mlp.up_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.24.post_attention_layernorm.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.24.self_attn.k_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.24.self_attn.o_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.24.self_attn.q_proj.weight": "pytorch_model-00011-of-00019.bin",
"model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00012-of-00019.bin",
"model.layers.24.self_attn.v_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.input_layernorm.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.mlp.down_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.mlp.gate_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.mlp.up_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.post_attention_layernorm.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.self_attn.k_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.self_attn.o_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.self_attn.q_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00012-of-00019.bin",
"model.layers.25.self_attn.v_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.26.input_layernorm.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.26.mlp.down_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.26.mlp.gate_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.26.mlp.up_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.26.post_attention_layernorm.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.26.self_attn.k_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.26.self_attn.o_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.26.self_attn.q_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00012-of-00019.bin",
"model.layers.26.self_attn.v_proj.weight": "pytorch_model-00012-of-00019.bin",
"model.layers.27.input_layernorm.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.mlp.down_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.mlp.gate_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.mlp.up_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.post_attention_layernorm.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.self_attn.k_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.self_attn.o_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.self_attn.q_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00013-of-00019.bin",
"model.layers.27.self_attn.v_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.28.input_layernorm.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.28.mlp.down_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.28.mlp.gate_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.28.mlp.up_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.28.post_attention_layernorm.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.28.self_attn.k_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.28.self_attn.o_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.28.self_attn.q_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00013-of-00019.bin",
"model.layers.28.self_attn.v_proj.weight": "pytorch_model-00013-of-00019.bin",
"model.layers.29.input_layernorm.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.mlp.down_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.mlp.gate_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.mlp.up_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.post_attention_layernorm.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.self_attn.k_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.self_attn.o_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.self_attn.q_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00014-of-00019.bin",
"model.layers.29.self_attn.v_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.3.input_layernorm.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.3.mlp.down_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.3.mlp.up_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.3.post_attention_layernorm.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.3.self_attn.k_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.3.self_attn.o_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.3.self_attn.q_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00019.bin",
"model.layers.3.self_attn.v_proj.weight": "pytorch_model-00002-of-00019.bin",
"model.layers.30.input_layernorm.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.30.mlp.down_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.30.mlp.gate_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.30.mlp.up_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.30.post_attention_layernorm.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.30.self_attn.k_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.30.self_attn.o_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.30.self_attn.q_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00014-of-00019.bin",
"model.layers.30.self_attn.v_proj.weight": "pytorch_model-00014-of-00019.bin",
"model.layers.31.input_layernorm.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.mlp.down_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.mlp.gate_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.mlp.up_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.post_attention_layernorm.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.self_attn.k_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.self_attn.o_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.self_attn.q_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00015-of-00019.bin",
"model.layers.31.self_attn.v_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.input_layernorm.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.mlp.down_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.mlp.gate_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.mlp.up_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.post_attention_layernorm.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.self_attn.k_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.self_attn.o_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.self_attn.q_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.32.self_attn.rotary_emb.inv_freq": "pytorch_model-00015-of-00019.bin",
"model.layers.32.self_attn.v_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.33.input_layernorm.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.33.mlp.down_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.33.mlp.gate_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.33.mlp.up_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.33.post_attention_layernorm.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.33.self_attn.k_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.33.self_attn.o_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.33.self_attn.q_proj.weight": "pytorch_model-00015-of-00019.bin",
"model.layers.33.self_attn.rotary_emb.inv_freq": "pytorch_model-00016-of-00019.bin",
"model.layers.33.self_attn.v_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.input_layernorm.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.mlp.down_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.mlp.gate_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.mlp.up_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.post_attention_layernorm.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.self_attn.k_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.self_attn.o_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.self_attn.q_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.34.self_attn.rotary_emb.inv_freq": "pytorch_model-00016-of-00019.bin",
"model.layers.34.self_attn.v_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.35.input_layernorm.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.35.mlp.down_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.35.mlp.gate_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.35.mlp.up_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.35.post_attention_layernorm.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.35.self_attn.k_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.35.self_attn.o_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.35.self_attn.q_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.35.self_attn.rotary_emb.inv_freq": "pytorch_model-00016-of-00019.bin",
"model.layers.35.self_attn.v_proj.weight": "pytorch_model-00016-of-00019.bin",
"model.layers.36.input_layernorm.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.mlp.down_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.mlp.gate_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.mlp.up_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.post_attention_layernorm.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.self_attn.k_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.self_attn.o_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.self_attn.q_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.36.self_attn.rotary_emb.inv_freq": "pytorch_model-00017-of-00019.bin",
"model.layers.36.self_attn.v_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.37.input_layernorm.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.37.mlp.down_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.37.mlp.gate_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.37.mlp.up_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.37.post_attention_layernorm.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.37.self_attn.k_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.37.self_attn.o_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.37.self_attn.q_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.37.self_attn.rotary_emb.inv_freq": "pytorch_model-00017-of-00019.bin",
"model.layers.37.self_attn.v_proj.weight": "pytorch_model-00017-of-00019.bin",
"model.layers.38.input_layernorm.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.mlp.down_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.mlp.gate_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.mlp.up_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.post_attention_layernorm.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.self_attn.k_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.self_attn.o_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.self_attn.q_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.38.self_attn.rotary_emb.inv_freq": "pytorch_model-00018-of-00019.bin",
"model.layers.38.self_attn.v_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.39.input_layernorm.weight": "pytorch_model-00019-of-00019.bin",
"model.layers.39.mlp.down_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.39.mlp.gate_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.39.mlp.up_proj.weight": "pytorch_model-00019-of-00019.bin",
"model.layers.39.post_attention_layernorm.weight": "pytorch_model-00019-of-00019.bin",
"model.layers.39.self_attn.k_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.39.self_attn.o_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.39.self_attn.q_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.39.self_attn.rotary_emb.inv_freq": "pytorch_model-00018-of-00019.bin",
"model.layers.39.self_attn.v_proj.weight": "pytorch_model-00018-of-00019.bin",
"model.layers.4.input_layernorm.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.mlp.down_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.mlp.gate_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.mlp.up_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.post_attention_layernorm.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.self_attn.k_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.self_attn.o_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.self_attn.q_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00019.bin",
"model.layers.4.self_attn.v_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.input_layernorm.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.mlp.down_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.mlp.gate_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.mlp.up_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.post_attention_layernorm.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.self_attn.k_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.self_attn.o_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.self_attn.q_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00019.bin",
"model.layers.5.self_attn.v_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.6.input_layernorm.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.6.mlp.down_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.6.mlp.gate_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.6.mlp.up_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.6.post_attention_layernorm.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.6.self_attn.k_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.6.self_attn.o_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.6.self_attn.q_proj.weight": "pytorch_model-00003-of-00019.bin",
"model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00019.bin",
"model.layers.6.self_attn.v_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.input_layernorm.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.mlp.down_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.mlp.gate_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.mlp.up_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.post_attention_layernorm.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.self_attn.k_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.self_attn.o_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.self_attn.q_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00019.bin",
"model.layers.7.self_attn.v_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.8.input_layernorm.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.8.mlp.down_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.8.mlp.gate_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.8.mlp.up_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.8.post_attention_layernorm.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.8.self_attn.k_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.8.self_attn.o_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.8.self_attn.q_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00019.bin",
"model.layers.8.self_attn.v_proj.weight": "pytorch_model-00004-of-00019.bin",
"model.layers.9.input_layernorm.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.mlp.down_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.mlp.gate_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.mlp.up_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.post_attention_layernorm.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.self_attn.k_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.self_attn.o_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.self_attn.q_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00019.bin",
"model.layers.9.self_attn.v_proj.weight": "pytorch_model-00005-of-00019.bin",
"model.norm.weight": "pytorch_model-00019-of-00019.bin"
}
}

24
special_tokens_map.json Normal file
View File

@@ -0,0 +1,24 @@
{
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"pad_token": "<unk>",
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}

3
tokenizer.model Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:147fa8ef9267d7760a39e07a15eeadb64ce6560a74fa92f4e69cf206307e876c
size 588649

34
tokenizer_config.json Normal file
View File

@@ -0,0 +1,34 @@
{
"add_bos_token": true,
"add_eos_token": false,
"bos_token": {
"__type": "AddedToken",
"content": "<s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"clean_up_tokenization_spaces": false,
"eos_token": {
"__type": "AddedToken",
"content": "</s>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
},
"model_max_length": 2048,
"pad_token": null,
"padding_side": "right",
"sp_model_kwargs": {},
"tokenizer_class": "LlamaTokenizer",
"unk_token": {
"__type": "AddedToken",
"content": "<unk>",
"lstrip": false,
"normalized": true,
"rstrip": false,
"single_word": false
}
}