Ichsan2895/Merak-7B-v5-PROTOTYPE1
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Ichsan2895/Merak-7B-v5-PROTOTYPE1
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-17 21:40:11 +08:00
commit 95f9715b05
15 changed files with 589 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

97
README.md Normal file
View File

@@ -0,0 +1,97 @@
---
datasets:
- wikipedia
- Ichsan2895/OASST_Top1_Indonesian
- Ichsan2895/alpaca-gpt4-indonesian
language:
- id
- en
pipeline_tag: text-generation
license: cc-by-nc-sa-4.0
---
<div style="width: auto; margin-left: auto; margin-right: auto">
<img src="https://huggingface.co/Ichsan2895/Merak-7B-v4/resolve/main/FINAL_LOGO/6.png" alt="MERAK" style="width: 50%; min-width: 100px; display: block; margin: auto;">
</div>
# THIS IS 1st PROTOTYPE OF MERAK-7B-v5!
Merak-7B is the Large Language Model of Indonesian Language
This model is based on [Mistral-7B-OpenOrca](https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca) and fine tuned by some of Indonesia Wikipedia articles that I cleaned before.
Leveraging QLoRA (QLora: Efficient Finetuning of Quantized LLMs), Merak-7B is able to run with 16 GB VRAM. We also use DPO-Trainer for RLHF with TRL library..
Licensed under Creative Commons-By Attribution-Share Alike-Non Commercial (CC-BY-SA-NC 4.0) Merak-7B empowers AI enthusiasts, researchers alike.
Big thanks to all my friends and communities that help to build our first model. Thanks for Axolotl for a great fine tuning tool which designed to streamline the fine-tuning of various AI models.
Feel free, to ask me about the model and please share the news on your social media.
## CITATION
```
@software{lian2023mistralorca1
title = {MistralOrca: Mistral-7B Model Instruct-tuned on Filtered OpenOrcaV1 GPT-4 Dataset},
author = {Wing Lian and Bleys Goodson and Guan Wang and Eugene Pentland and Austin Cook and Chanvichet Vong and "Teknium"},
year = {2023},
publisher = {HuggingFace},
journal = {HuggingFace repository},
howpublished = {\url{https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca},
}
@misc{mukherjee2023orca,
title={Orca: Progressive Learning from Complex Explanation Traces of GPT-4},
author={Subhabrata Mukherjee and Arindam Mitra and Ganesh Jawahar and Sahaj Agarwal and Hamid Palangi and Ahmed Awadallah},
year={2023},
eprint={2306.02707},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
@ONLINE{wikidump,
author = "Wikimedia Foundation",
title = "Wikimedia Downloads",
url = "https://dumps.wikimedia.org"
}
@inproceedings{wolf-etal-2020-transformers,
title = "Transformers: State-of-the-Art Natural Language Processing",
author = "Thomas Wolf and Lysandre Debut and Victor Sanh and Julien Chaumond and Clement Delangue and Anthony Moi and Pierric Cistac and Tim Rault and Rémi Louf and Morgan Funtowicz and Joe Davison and Sam Shleifer and Patrick von Platen and Clara Ma and Yacine Jernite and Julien Plu and Canwen Xu and Teven Le Scao and Sylvain Gugger and Mariama Drame and Quentin Lhoest and Alexander M. Rush",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
month = oct,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.emnlp-demos.6",
pages = "38--45"
}
@misc{vonwerra2022trl,
author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang},
title = {TRL: Transformer Reinforcement Learning},
year = {2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/huggingface/trl}}
}
@article{dettmers2023qlora,
title = {QLoRA: Efficient Finetuning of Quantized LLMs},
author = {Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
journal = {arXiv preprint arXiv:2305.14314},
year = {2023}
}
```
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
## HOW TO CITE THIS PROJECT
If you use the Merak-7B model in your research or project, please cite it as:
```
@article{Merak,
title={Merak-7B: The LLM for Bahasa Indonesia},
author={Muhammad Ichsan},
publisher={Hugging Face}
journal={Hugging Face Repository},
year={2023}
}
```

4
added_tokens.json Normal file
View File

@@ -0,0 +1,4 @@
{
"<|im_end|>": 32000,
"<|im_start|>": 32001
}

25
config.json Normal file
View File

@@ -0,0 +1,25 @@
{
"_name_or_path": "Ichsan2895/Merak-7B-v4",
"architectures": [
"MistralForCausalLM"
],
"bos_token_id": 1,
"eos_token_id": 2,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 14336,
"max_position_embeddings": 32768,
"model_type": "mistral",
"num_attention_heads": 32,
"num_hidden_layers": 32,
"num_key_value_heads": 8,
"rms_norm_eps": 1e-05,
"rope_theta": 10000.0,
"sliding_window": 4096,
"tie_word_embeddings": false,
"torch_dtype": "float32",
"transformers_version": "4.34.1",
"use_cache": false,
"vocab_size": 32002
}

7
generation_config.json Normal file
View File

@@ -0,0 +1,7 @@
{
"_from_model_config": true,
"bos_token_id": 1,
"eos_token_id": 2,
"transformers_version": "4.34.1",
"use_cache": false
}

3
pytorch_model-00001-of-00006.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b4e5be4251e2bd1f2bac58169006213e5e56e9e8183a4f5056c83c44e4534de4
size 4987241157

3
pytorch_model-00002-of-00006.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:632c33573ec94ef60f171e073653b30edc8eaa0b39049daccf90834cc3f24a04
size 4899127691

3
pytorch_model-00003-of-00006.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:09ab8d559ed98a9ab7f6dbb746afb44edd8d45d8762c08dcb758a99baa9bbfbe
size 4999825537

3
pytorch_model-00004-of-00006.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9fe5a882402b23cfabcffcadd1980564e80652e830541471161aaf8d0045c647
size 4999825597

3
pytorch_model-00005-of-00006.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:36966b1083ba34cb12844dd996deb482c9627ac0e3569f240612017c39365c14
size 4832018477

3
pytorch_model-00006-of-00006.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:315b961e2e113ca147f42da257963b7a0590f455b364dc1d4fa4e39a448f17b8
size 4249057137

298
pytorch_model.bin.index.json Normal file
View File

@@ -0,0 +1,298 @@
{
"metadata": {
"total_size": 28966993920
},
"weight_map": {
"lm_head.weight": "pytorch_model-00006-of-00006.bin",
"model.embed_tokens.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.10.input_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.10.mlp.down_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.10.mlp.gate_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.10.mlp.up_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.10.post_attention_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.10.self_attn.k_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.10.self_attn.o_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.10.self_attn.q_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.10.self_attn.v_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.11.input_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.mlp.down_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.mlp.gate_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.mlp.up_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.post_attention_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.self_attn.k_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.self_attn.o_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.self_attn.q_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.11.self_attn.v_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.input_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.mlp.down_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.mlp.gate_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.mlp.up_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.post_attention_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.self_attn.k_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.self_attn.o_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.self_attn.q_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.12.self_attn.v_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.input_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.mlp.down_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.mlp.gate_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.mlp.up_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.post_attention_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.self_attn.k_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.self_attn.o_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.self_attn.q_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.13.self_attn.v_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.input_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.mlp.down_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.mlp.gate_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.mlp.up_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.post_attention_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.self_attn.k_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.self_attn.o_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.self_attn.q_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.14.self_attn.v_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.input_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.mlp.down_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.mlp.gate_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.mlp.up_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.post_attention_layernorm.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.self_attn.k_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.self_attn.o_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.self_attn.q_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.15.self_attn.v_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.16.input_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.16.mlp.down_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.16.mlp.gate_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.16.mlp.up_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.16.post_attention_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.16.self_attn.k_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.16.self_attn.o_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.16.self_attn.q_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.16.self_attn.v_proj.weight": "pytorch_model-00003-of-00006.bin",
"model.layers.17.input_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.mlp.down_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.mlp.gate_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.mlp.up_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.post_attention_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.self_attn.k_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.self_attn.o_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.self_attn.q_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.17.self_attn.v_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.input_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.mlp.down_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.mlp.gate_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.mlp.up_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.post_attention_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.self_attn.k_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.self_attn.o_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.self_attn.q_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.18.self_attn.v_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.input_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.mlp.down_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.mlp.gate_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.mlp.up_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.post_attention_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.self_attn.k_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.self_attn.o_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.self_attn.q_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.19.self_attn.v_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.20.input_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.mlp.down_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.mlp.gate_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.mlp.up_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.post_attention_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.self_attn.k_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.self_attn.o_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.self_attn.q_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.20.self_attn.v_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.input_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.mlp.down_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.mlp.gate_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.mlp.up_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.post_attention_layernorm.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.self_attn.k_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.self_attn.o_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.self_attn.q_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.21.self_attn.v_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.22.input_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.22.mlp.down_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.22.mlp.gate_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.22.mlp.up_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.22.post_attention_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.22.self_attn.k_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.22.self_attn.o_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.22.self_attn.q_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.22.self_attn.v_proj.weight": "pytorch_model-00004-of-00006.bin",
"model.layers.23.input_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.mlp.down_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.mlp.gate_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.mlp.up_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.post_attention_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.self_attn.k_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.self_attn.o_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.self_attn.q_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.23.self_attn.v_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.input_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.mlp.down_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.mlp.gate_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.mlp.up_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.post_attention_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.self_attn.k_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.self_attn.o_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.self_attn.q_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.24.self_attn.v_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.input_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.mlp.down_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.mlp.gate_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.mlp.up_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.post_attention_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.self_attn.k_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.self_attn.o_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.self_attn.q_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.25.self_attn.v_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.input_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.mlp.down_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.mlp.gate_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.mlp.up_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.post_attention_layernorm.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.self_attn.k_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.self_attn.o_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.self_attn.q_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.26.self_attn.v_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.27.input_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.27.mlp.down_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.27.mlp.gate_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.27.mlp.up_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.27.post_attention_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.27.self_attn.k_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.27.self_attn.o_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.27.self_attn.q_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.27.self_attn.v_proj.weight": "pytorch_model-00005-of-00006.bin",
"model.layers.28.input_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.mlp.down_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.mlp.gate_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.mlp.up_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.post_attention_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.self_attn.k_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.self_attn.o_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.self_attn.q_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.28.self_attn.v_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.input_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.mlp.down_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.mlp.gate_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.mlp.up_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.post_attention_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.self_attn.k_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.self_attn.o_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.self_attn.q_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.29.self_attn.v_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.30.input_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.mlp.down_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.mlp.gate_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.mlp.up_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.post_attention_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.self_attn.k_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.self_attn.o_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.self_attn.q_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.30.self_attn.v_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.input_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.mlp.down_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.mlp.gate_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.mlp.up_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.post_attention_layernorm.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.self_attn.k_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.self_attn.o_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.self_attn.q_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.31.self_attn.v_proj.weight": "pytorch_model-00006-of-00006.bin",
"model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.5.input_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.5.mlp.down_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.5.mlp.gate_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.5.mlp.up_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.5.post_attention_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.5.self_attn.o_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00006.bin",
"model.layers.6.input_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.mlp.down_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.mlp.gate_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.mlp.up_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.post_attention_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.self_attn.k_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.self_attn.o_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.self_attn.q_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.6.self_attn.v_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.input_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.mlp.down_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.mlp.gate_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.mlp.up_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.post_attention_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.self_attn.k_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.self_attn.o_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.self_attn.q_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.7.self_attn.v_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.input_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.mlp.down_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.mlp.gate_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.mlp.up_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.post_attention_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.self_attn.k_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.self_attn.o_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.self_attn.q_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.8.self_attn.v_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.input_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.mlp.down_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.mlp.gate_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.mlp.up_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.post_attention_layernorm.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.self_attn.k_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.self_attn.o_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.self_attn.q_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.layers.9.self_attn.v_proj.weight": "pytorch_model-00002-of-00006.bin",
"model.norm.weight": "pytorch_model-00006-of-00006.bin"
}
}

37
special_tokens_map.json Normal file
View File

@@ -0,0 +1,37 @@
{
"additional_special_tokens": [
"<unk>",
"<s>",
"</s>",
"<|im_end|>",
"<|im_start|>"
],
"bos_token": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

BIN
tokenizer.model (Stored with Git LFS) Normal file
View File

Binary file not shown.

65
tokenizer_config.json Normal file
View File

@@ -0,0 +1,65 @@
{
"add_bos_token": true,
"add_eos_token": false,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "</s>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"32000": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"32001": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"additional_special_tokens": [
"<unk>",
"<s>",
"</s>",
"<|im_end|>",
"<|im_start|>"
],
"bos_token": "<s>",
"chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
"clean_up_tokenization_spaces": false,
"eos_token": "</s>",
"legacy": true,
"model_max_length": 1000000000000000019884624838656,
"pad_token": "</s>",
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": true
}