openthaigpt/openthaigpt-r1-32b-instruct
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: openthaigpt/openthaigpt-r1-32b-instruct
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-16 00:40:08 +08:00
commit fe7cc25a95
24 changed files with 1649 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

0
LICENSE Normal file
View File

187
README.md Normal file
View File

@@ -0,0 +1,187 @@
---
license: other
license_name: qwen
language:
- th
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- openthaigpt
- qwen
- reasoning
model-index:
- name: openthaigpt-r1-32b-instruct
results:
- task:
type: reasoning
dataset:
name: SkyThought
type: mathematical_reasoning
metrics:
- name: AIME24-TH
type: accuracy
value: 56.67
- name: AIME24
type: accuracy
value: 63.36
source:
name: 🇹🇭 OpenThaiGPT R1 Benchmark
url: https://openthaigpt.aieat.or.th/
---
# 🇹🇭 OpenThaiGPT R1 32b
![OpenThaiGPT](https://cdn-uploads.huggingface.co/production/uploads/5fcd9c426d942eaf4d1ebd30/tByCXPW7JG3krRcTn1IlN.png)
[More Info](https://openthaigpt.aieat.or.th/)
🇹🇭 **OpenThaiGPT R1 32b** is an advanced 32-billion-parameter Thai language reasoning model that outperforms larger models like DeepSeek R1 70b and Typhoon R1 70b, while being less than half their size. This model excels at complex reasoning tasks, including mathematics, logic, and code reasoning in Thai language.
## Highlights
- **State-of-the-art Thai reasoning model**, outperforming larger models on mathematical and logical reasoning tasks
- **Explicit reasoning capabilities** with the ability to show step-by-step thought processes
- **Significantly smaller size** (32b) while outperforming 70b models
- **Specialized for Thai language reasoning** including complex mathematics and logic problems
- **High performance on code reasoning** in both Thai and English
## Benchmark Results
| **SkyThought** | **OpenThaiGPT R1 32b** | **DeepSeek R1 70b** | **Typhoon R1 Distill 70b** |
|----------------------|-----------------------------------------------------------------------|--------------------------|----------------------------|
| **AIME24-TH** | <b>56.67</b> | 33.33 | 53.33 |
| **AIME24** | <b>63.36</b> | 53.33 | 53.33 |
| **MATH500-TH** | <b>83.8</b> | 75.4 | 81 |
| **MATH500** | 89.4 | 88.88 | <b>90.2</b> |
| **LiveCodeBench-TH** | <b>62.16</b> | 53.15 | 47.75 |
| **LiveCodeBench** | <b>69.67</b> | 64.97 | 54.79 |
| **OpenThaiEval** | 76.05 | 74.17 | <b>77.59</b> |
| **AVERAGE** | <b style="color:blue">71.58</b> | 63.31 | 65.42 |
## Recommended System Prompt
```
<No system prompt>
```
## Model Technical Report
https://arxiv.org/abs/2504.01789
If OpenThaiGPT has been beneficial for your work, kindly consider citing it as follows:
```tex
@misc{yuenyong2025openthaigpt16r1thaicentric,
title={OpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Models},
author={Sumeth Yuenyong and Thodsaporn Chay-intr and Kobkrit Viriyayudhakorn},
year={2025},
eprint={2504.01789},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2504.01789},
}
```
## How to use
### Online Web Interface
https://chindax.iapp.co.th
### Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "openthaigpt/openthaigpt-r1-32b-instruct"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "จงหาพื้นที่ของวงกลมที่มีรัศมี 7 หน่วย"
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=16384,
temperature=0.6
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```
### vLLM
1. Install VLLM (https://github.com/vllm-project/vllm)
2. Run server
```bash
vllm serve openthaigpt/openthaigpt-r1-32b --tensor-parallel-size 2
```
* Note, change `--tensor-parallel-size 2` to the amount of available GPU cards.
3. Run inference (CURL example)
```bash
curl -X POST 'http://127.0.0.1:8000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-d '{
"model": "openthaigpt/openthaigpt-r1-32b-instruct",
"messages": [
{
"role": "user",
"content": "จงหาพื้นที่ของวงกลมที่มีรัศมี 7 หน่วย"
}
],
"max_tokens": 16384,
"temperature": 0.6,
"top_p": 0.95,
"top_k": 40
}'
```
### GPU Memory Requirements
| **Number of Parameters** | **FP 16 bits** | **8 bits (Quantized)** | **4 bits (Quantized)** |
|------------------|----------------|------------------------|------------------------|
| **32b** | 64 GB | 32 GB | 16 GB |
## Chat Template
```python
{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜>'}}{% endif %}
```
## Licenses
* This model is available for **Research** and **Commercial uses** under the specified terms. Please see the LICENSE file for more information.
## Supports
- Official website: https://openthaigpt.aieat.or.th
- Facebook page: https://web.facebook.com/groups/openthaigpt
- A Discord server for discussion and support [here](https://discord.gg/rUTp6dfVUF)
- E-mail: kobkrit@iapp.co.th
### OpenThaiGPT Team
<img src="https://cdn-uploads.huggingface.co/production/uploads/5fcd9c426d942eaf4d1ebd30/e8gT15eRfNbyEZhu-UzMX.png" width="200px">
* Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th / kobkrit@aieat.or.th)
* Sumeth Yuenyong (sumeth.yue@mahidol.edu)
* Thodsaporn Chay-intr (thodsaporn@iapp.co.th)
## Sponsors
<img src="https://cdn-uploads.huggingface.co/production/uploads/5fcd9c426d942eaf4d1ebd30/zSEA_n0cIOZk5pV_t2qii.png" width="400px">
* ได้รับการสนับสนุน GPU Nvidia H100 x 8 จากบริษัท บริษัท สยาม เอไอ คอร์เปอเรชั่น จำกัด: https://siam.ai/
* ได้รับทุนวิจัยสนับสนุนจากกองทุนส่งเสริมวิทยาศาสตร์ วิจัยและนวัตกรรม โดยหน่วยบริหารและจัดการทุนด้านการเพิ่มความสามารถในการแข่งขันของประเทศ (บพข.) ร่วมกับ บริษัท ไอแอพพ์เทคโนโลยี จำกัด ซึ่งมี สมาคมผู้ประกอบการปัญญาประดิษฐ์ประเทศไทย เป็นผู้ดำเนินงานโครงการ
<i>Disclaimer: Provided responses are not guaranteed.</i>

346
args.json Normal file
View File

File diff suppressed because one or more lines are too long

30
config.json Normal file
View File

@@ -0,0 +1,30 @@
{
"_name_or_path": "/workspace/deploy/modelC-2025-02-24",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151643,
"hidden_act": "silu",
"hidden_size": 5120,
"initializer_range": 0.02,
"intermediate_size": 27648,
"max_position_embeddings": 131072,
"max_window_layers": 64,
"model_type": "qwen2",
"num_attention_heads": 40,
"num_hidden_layers": 64,
"num_key_value_heads": 8,
"pad_token_id": 151643,
"rms_norm_eps": 1e-05,
"rope_scaling": null,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.49.0.dev0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 152064
}

9
generation_config.json Normal file
View File

@@ -0,0 +1,9 @@
{
"_from_model_config": true,
"bos_token_id": 151646,
"do_sample": true,
"eos_token_id": 151643,
"temperature": 0.6,
"top_p": 0.95,
"transformers_version": "4.49.0.dev0"
}

3
model-00001-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f1baa1bfdaa062b49beae7fce1344f87ff9e464b1a62568954c436479588c41e
size 4891730992

3
model-00002-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cc130e94d318959b67af423efd1fda7d63d74971a6c36cb67545de8792df5390
size 4876059352

3
model-00003-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a353f7b56249de654c6d90cf74d2ade29e35cae6ddaf39f0a5ee522c7735cbcc
size 4876059384

3
model-00004-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b7eb688ae01c68b2e32ffc6926bd8d431b61f6257e9ca9fdf77c5b7c1cceadb
size 4876059416

3
model-00005-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:01ef29836a20146dc090cb5a8d9b83733be0b811f471fc2148bbf517d0121499
size 4876059416

3
model-00006-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f01aa828876d443191d4185e7d37febc614b13f98976ec6ac41734b142acbd70
size 4876059416

3
model-00007-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7d7276e0fcfda203cd7157901660ac13067597ee72735b1426d93f54b97b1048
size 4876059416

3
model-00008-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aafd37ba4e783a39ee71dcd56b6a047589225106c73d7da630c58ef94e4668e8
size 4876059416

3
model-00009-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:80706b0b27edc1c288a15ae14d2a1f8e77ba11bc928e141e3c7e55716e26a1e6
size 4876059416

3
model-00010-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4d3bab1cc2bd9ed225ec2154b581e42f1d0ae4ee524ebc59e52a66abac5d9651
size 4876059416

3
model-00011-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:67b4a97dbd8b02c8a1a3c66cdb1d1d615edc73f3457a1900a9f13ea7f81c8897
size 4876059416

3
model-00012-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3c5afb1cbd1338b8741dc5d4d3c6d979c35f5b01e2d32a13808fcf3bd4ed370e
size 4876059416

3
model-00013-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:30b4f69ce235c87c3bad6d73d22a94ecd7bc0b02d63a72229908fb65c04f930e
size 4876059416

3
model-00014-of-00014.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e20c94591727f021cbb6ce3de33a2e118e8832713c5160ea21f3e093799bdc5b
size 2123397800

778
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,778 @@
{
"metadata": {
"total_size": 65527752704
},
"weight_map": {
"lm_head.weight": "model-00014-of-00014.safetensors",
"model.embed_tokens.weight": "model-00001-of-00014.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.10.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.13.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.14.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.input_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.18.self_attn.k_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.q_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.v_proj.bias": "model-00004-of-00014.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00004-of-00014.safetensors",
"model.layers.19.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.20.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.input_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.23.self_attn.k_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.q_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.v_proj.bias": "model-00005-of-00014.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00005-of-00014.safetensors",
"model.layers.24.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.input_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.28.self_attn.k_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.q_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.v_proj.bias": "model-00006-of-00014.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00006-of-00014.safetensors",
"model.layers.29.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.3.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.3.self_attn.k_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.q_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.v_proj.bias": "model-00001-of-00014.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00014.safetensors",
"model.layers.30.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.input_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.32.mlp.down_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.mlp.up_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.post_attention_layernorm.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.32.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.33.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.33.mlp.gate_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.33.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.33.self_attn.k_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.k_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.o_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.q_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.q_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.v_proj.bias": "model-00007-of-00014.safetensors",
"model.layers.33.self_attn.v_proj.weight": "model-00007-of-00014.safetensors",
"model.layers.34.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.34.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.34.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.35.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.35.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.36.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.36.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.input_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.37.mlp.down_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.mlp.up_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.post_attention_layernorm.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.37.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.38.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.38.mlp.gate_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.38.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.38.self_attn.k_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.k_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.o_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.q_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.q_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.v_proj.bias": "model-00008-of-00014.safetensors",
"model.layers.38.self_attn.v_proj.weight": "model-00008-of-00014.safetensors",
"model.layers.39.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.39.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.39.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.4.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.40.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.40.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.40.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.41.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.41.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.input_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.42.mlp.down_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.mlp.up_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.post_attention_layernorm.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.42.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.43.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.43.mlp.gate_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.43.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.43.self_attn.k_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.k_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.o_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.q_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.q_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.v_proj.bias": "model-00009-of-00014.safetensors",
"model.layers.43.self_attn.v_proj.weight": "model-00009-of-00014.safetensors",
"model.layers.44.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.44.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.44.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.45.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.45.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.46.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.46.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.input_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.47.mlp.down_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.mlp.up_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.post_attention_layernorm.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.47.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.48.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.48.mlp.gate_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.48.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.48.self_attn.k_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.k_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.o_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.q_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.q_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.v_proj.bias": "model-00010-of-00014.safetensors",
"model.layers.48.self_attn.v_proj.weight": "model-00010-of-00014.safetensors",
"model.layers.49.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.49.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.49.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.5.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.50.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.50.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.50.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.51.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.51.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.input_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.52.mlp.down_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.mlp.up_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.post_attention_layernorm.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.52.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.53.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.53.mlp.gate_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.53.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.53.self_attn.k_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.k_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.o_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.q_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.q_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.v_proj.bias": "model-00011-of-00014.safetensors",
"model.layers.53.self_attn.v_proj.weight": "model-00011-of-00014.safetensors",
"model.layers.54.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.54.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.54.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.55.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.55.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.56.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.56.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.input_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.57.mlp.down_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.mlp.up_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.post_attention_layernorm.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.57.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.58.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.58.mlp.gate_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.58.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.58.self_attn.k_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.k_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.o_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.q_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.q_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.v_proj.bias": "model-00012-of-00014.safetensors",
"model.layers.58.self_attn.v_proj.weight": "model-00012-of-00014.safetensors",
"model.layers.59.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.59.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.59.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.6.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.60.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.60.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.60.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.61.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.61.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.input_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.62.mlp.down_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.mlp.up_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.post_attention_layernorm.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.62.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.input_layernorm.weight": "model-00014-of-00014.safetensors",
"model.layers.63.mlp.down_proj.weight": "model-00014-of-00014.safetensors",
"model.layers.63.mlp.gate_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.mlp.up_proj.weight": "model-00014-of-00014.safetensors",
"model.layers.63.post_attention_layernorm.weight": "model-00014-of-00014.safetensors",
"model.layers.63.self_attn.k_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.k_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.o_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.q_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.q_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.v_proj.bias": "model-00013-of-00014.safetensors",
"model.layers.63.self_attn.v_proj.weight": "model-00013-of-00014.safetensors",
"model.layers.7.input_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.8.self_attn.k_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.q_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.v_proj.bias": "model-00002-of-00014.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00002-of-00014.safetensors",
"model.layers.9.input_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.k_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.q_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.v_proj.bias": "model-00003-of-00014.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00003-of-00014.safetensors",
"model.norm.weight": "model-00014-of-00014.safetensors"
}
}

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<｜begin▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<｜end▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<｜end▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e20ddafc659ba90242154b55275402edeca0715e5dbb30f56815a4ce081f4893
size 11422778

195
tokenizer_config.json Normal file
View File

@@ -0,0 +1,195 @@
{
"add_bos_token": true,
"add_eos_token": false,
"add_prefix_space": null,
"added_tokens_decoder": {
"151643": {
"content": "<｜end▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151644": {
"content": "<｜User｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151645": {
"content": "<｜Assistant｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151646": {
"content": "<｜begin▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151647": {
"content": "<|EOT|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151648": {
"content": "<think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151649": {
"content": "</think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151650": {
"content": "<|quad_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
}
},
"bos_token": "<｜begin▁of▁sentence｜>",
"chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜>'}}{% endif %}",
"clean_up_tokenization_spaces": false,
"eos_token": "<｜end▁of▁sentence｜>",
"extra_special_tokens": {},
"legacy": true,
"model_max_length": 16384,
"pad_token": "<｜end▁of▁sentence｜>",
"sp_model_kwargs": {},
"tokenizer_class": "LlamaTokenizer",
"unk_token": null,
"use_default_system_prompt": false
}