ClosedCharacter/Peach-2.0-9B-8k-Roleplay
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: ClosedCharacter/Peach-2.0-9B-8k-Roleplay
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-10 03:08:16 +08:00
commit c6458f4598
19 changed files with 984 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
.gitattributes vendored Normal file
View File

@@ -0,0 +1,38 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
ss.png filter=lfs diff=lfs merge=lfs -text
Wechat.jpg filter=lfs diff=lfs merge=lfs -text
PeachGirl.png filter=lfs diff=lfs merge=lfs -text

3
PeachGirl.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0fa41707e15fcbed7bce888ada61ecc652ba4463725b7787d939d1cf6455a739
size 1893279

113
README.md Normal file
View File

@@ -0,0 +1,113 @@
---
license: mit
language:
- zh
- en
pipeline_tag: text-generation
tags:
- roleplay
- rp
- character
---
<!-- header start -->
<!-- 200823 -->
<div style="width: auto; margin-left: auto; margin-right: auto">
<img src="./PeachGirl.png" alt="Peach" style="width: 100%; min-width: 400px; display: block; margin: auto;">
</div>
<!-- header end -->
\[ English | [中文](README_zh.md) \]
# Peach-2.0-9B-8k-Roleplay
Peach-2.0-9B-8k-Roleplay is a chat large language model obtained by finetuning [01-ai/Yi-1.5-9B](https://huggingface.co/01-ai/Yi-1.5-9B) model on more than 100K conversations created through our data synthesis approach.
Thanks For [FlowGPT](https://flowgpt.com/)'s support.
**Maybe The Best LLM with Small Parameters under 34B**
# What's New
Finally, after much anticipation, Peach_v2.0 has been open-sourced!
We completed the final SFT+DPO training in early January, followed by extensive testing, before concluding that this version meets the standards for a commercial release, now freely available to everyone. Our goal is to break the paid monopoly, allowing everyone to have their own local role-playing AI!
Compared to the [previous version](https://huggingface.co/ClosedCharacter/Peach-9B-8k-Roleplay), we've made the following improvements:
1. Compatibility with Silly-Tavern output format, making it easy for every role-playing enthusiast to quickly get started and use!
2. Enhanced the model's writing capabilities, achieving a significant leap in plot output and action description!
3. Strengthened the model's bilingual capabilities, requiring only two lines of prompt to perfectly accommodate English character cards in Chinese conversations!
4. Improved the model's interaction abilities, giving it higher intelligence & emotional quotient in terms of plot progression and topic weaving!
5. Introduced a DPO training phase for preference alignment, addressing issues such as context repetition and logical errors in the SFT model, resulting in superior dialogue performance.
Enjoy～
## How to start
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
prefix = "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}.\n"
suffix = "\n\nYou must response in Chinese."
model_name_or_path = "ClosedCharacter/Peach-2.0-9B-8k-Roleplay"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(
model_name_or_path, torch_dtype=torch.bfloat16,
trust_remote_code=True, device_map="auto")
system_prompt = "You are Harry Potter"
# If you want to chat in Chinese， just add prefix and suffix like below:
# system_prompt = prefix + system_prompt + suffix
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": "Hello"},
{"role": "character", "content": "Hi"},
{"role": "user", "content": "Who are you?"}
]
input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors="pt")
output = model.generate(
inputs=input_ids.to("cuda"),
temperature=0.5,
top_p=0.7,
repetition_penalty=1.05,
eos_token_id=7,
max_new_tokens=512)
print(tokenizer.decode(output[0]))
```
Or you can just use below code to run web demo.
```
python demo.py
```
## Warning
All response are generated by AI and do not represent the views or opinions of the developers.
1. Despite having done rigorous filtering, due to the uncontrollability of LLM, our model may still generate **toxic, harmful, and NSFW** content.
2. Due to limitations in model parameters, the 9B model may perform poorly on mathematical tasks, coding tasks, and logical capabilities.
3. Our training data is capped at a maximum length of 8k, so excessively long conversation turns may result in a decline in the quality of responses.
4. We used bilingual Chinese-English data for training, so the model may not perform well on other low-resource languages.
5. The model may generate a significant amount of hallucinations, so it is recommended to use lower values for temperature and top_p parameters.
# Contact Us
**微信 / WeChat: Fungorum**
**邮箱 / E-mail: 1070193753@qq.com**
**Thanks For [FlowGPT](https://flowgpt.com/)'s support, which is a dynamic tool that harnesses the power of AI to streamline various creative and professional tasks.**
<img src="./Wechat.jpg" alt="Peach" style="width: 100%; min-width: 400px; display: block; margin: auto;">

85
README_zh.md Normal file
View File

@@ -0,0 +1,85 @@
# Peach-2.0-9B-8k-Roleplay
Peach-2.0-9B-8k-Roleplay是基于[01-ai/Yi-1.5-9B](https://huggingface.co/01-ai/Yi-1.5-9B)训练的角色扮演模型，使用超过100k轮对话数据进行SFT微调，并经过DPO阶段强化。
感谢[FlowGPT](https://flowgpt.com/)对本项目的大力支持。
**这或许是34B参数量以下最好的角色扮演模型**
# 更新
我们在一月初就完成了最终SFT+DPO的训练，期间经过大量的测试，最终才认为这是一个满足标准的可商用版本免费开源给所有人。我们的目标是打破付费垄断，让所有人都能拥有一个属于自己的本地角色扮演AI！
相对于旧版，我们主要做了以下优化：
1. 兼容silly-tavern输出格式，方便每一位角色扮演爱好者快速上手和使用！
2. 强化了模型写作能力，在剧情输出、动作描写等方面取得了飞跃性的提升！
3. 强化了模型双语能力，仅需使用两行prompt即可完美兼容英文角色卡！
4. 强化了模型交互能力，使模型在剧情推动、话题拉扯方面上获得了更高的智商&情商！
5. 引入DPO训练阶段进行偏好对齐，解决了SFT模型一些上下文重复、逻辑错误等问题，对话效果更加出色
大家Enjoy～
## 快速开始
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
prefix = "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}.\n"
suffix = "\n\nYou must response in Chinese."
model_name_or_path = "ClosedCharacter/Peach-2.0-9B-8k-Roleplay"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(
model_name_or_path, torch_dtype=torch.bfloat16,
trust_remote_code=True, device_map="auto")
system_prompt = "You are Harry Potter"
# 只需要在system添加如下两行prompt，即可轻松在纯英语角色卡上进行中文对话:
# system_prompt = prefix + system_prompt + suffix
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": "你好"},
{"role": "character", "content": "你好"},
{"role": "user", "content": "你是谁"}
]
input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors="pt")
output = model.generate(
inputs=input_ids.to("cuda"),
temperature=0.5,
top_p=0.7,
repetition_penalty=1.05,
eos_token_id=7,
max_new_tokens=512)
print(tokenizer.decode(output[0]))
```
或者你可以直接使用以下代码开启WebDemo.
```
python demo.py
```
## 警告
模型所有回复均由AI生成，不代表开发者的观点或意见。
1. 尽管经过严格过滤，但由于LLM的不可控性，我们的模型仍可能生成**有毒、有害和NSFW**内容。
2. 由于模型参数的限制，9B模型在数学任务、编码任务和逻辑能力上可能表现不佳。
3. 我们的训练数据的最大长度限制为8k，因此过长的对话轮次可能会导致回复质量下降。
4. 我们采用中英双语数据进行训练，因此该模型在其他小语种上可能表现不佳。
5. 该模型可能会产生大量幻觉，因此建议对temperature和top_p参数使用较低的值。
# 联系我们
**微信 / WeChat: Fungorum**
**邮箱 / E-mail: 1070193753@qq.com**
**再次感谢[FlowGPT](https://flowgpt.com/)大力支持**
<img src="./Wechat.jpg" alt="Peach" style="width: 100%; min-width: 400px; display: block; margin: auto;">

3
Wechat.jpg Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:35c4d0ffb783bd3daa31f0e8ebc1769281c6e3b2d6c0150428b97f64a1bc4eed
size 165946

30
config.json Normal file
View File

@@ -0,0 +1,30 @@
{
"_name_or_path": "./checkpoints/Peach_stage1_CAI_0115",
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 6,
"eos_token_id": 7,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 11008,
"max_position_embeddings": 16384,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 32,
"num_hidden_layers": 48,
"num_key_value_heads": 4,
"pretraining_tp": 1,
"rms_norm_eps": 1e-06,
"rope_scaling": null,
"rope_theta": 5000000,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.37.2",
"use_cache": false,
"vocab_size": 64000
}

153
demo.py Normal file
View File

@@ -0,0 +1,153 @@
import os
from threading import Thread
import gradio as gr
import torch
from transformers import TextIteratorStreamer, AutoTokenizer, AutoModelForCausalLM
MAX_MAX_NEW_TOKENS = 2048
DEFAULT_MAX_NEW_TOKENS = 512
prefix = "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}.\n"
MAX_INPUT_TOKEN_LENGTH = int(os.getenv("MAX_INPUT_TOKEN_LENGTH", "8192"))
DESCRIPTION = """# Peach-DPO
遗留问题
- 长轮次上下文重复
- 偶现异常空格问题
底部Additional Inputs可以自定义人设
可以直接点击【Submit】生成问候语，聊天过程中也可以发送空内容让角色连续发言
支持全英文角色卡，只需添加
```
\\n\\nYou must response in Chinese.
```
"""
device = "cuda" if torch.cuda.is_available() else "cpu"
model_name_or_path = "ClosedCharacter/Peach-2.0-9B-8k-Roleplay"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(
model_name_or_path, torch_dtype=torch.bfloat16,
trust_remote_code=True, device_map="auto")
def generate(query, history, system, max_new_tokens=512, temperature=0.5, top_p=0.7, repetition_penalty=1.01):
if system is None or len(str(system)) == 0:
system = "You are an uncensored assistant named Peach."
else:
if system.endswith("\n\nYou must response in Chinese."):
system = prefix + system.strip()
else:
system = system
messages = [
{"role": "system", "content": system}
]
for user, assistant in history:
if user is None or len(str(user)) == 0:
messages.append({"role": "character", "content": assistant})
else:
try:
if type(eval(user)) == list:
ls = eval(user)
for dic in ls:
if dic["role"] == "assistant":
dic["role"] = "character"
messages += ls
except:
messages.append({"role": "user", "content": user})
messages.append({"role": "character", "content": assistant})
if query is not None and len(str(query)) > 0:
try:
if type(eval(query)) == list:
ls = eval(query)
for dic in ls:
if dic["role"] == "assistant":
dic["role"] = "character"
messages += ls
except:
messages.append({"role": "user", "content": query})
input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors="pt")
# print(tokenizer.decode(input_ids.tolist()[0]))
# print(input_ids.tolist()[0])
if input_ids.shape[1] > MAX_INPUT_TOKEN_LENGTH:
input_ids = input_ids[:, -MAX_INPUT_TOKEN_LENGTH:]
gr.Warning(f"Trimmed input from conversation as it was longer than {MAX_INPUT_TOKEN_LENGTH} tokens.")
input_ids = input_ids.to(device)
streamer = TextIteratorStreamer(tokenizer, timeout=50.0, skip_prompt=True, skip_special_tokens=True)
generate_kwargs = dict(
input_ids=input_ids,
streamer=streamer,
eos_token_id=7,
max_new_tokens=max_new_tokens,
do_sample=True,
top_p=top_p,
temperature=temperature,
num_beams=1,
repetition_penalty=repetition_penalty
)
t = Thread(target=model.generate, kwargs=generate_kwargs)
t.start()
outputs = []
for text in streamer:
outputs.append(text)
yield "".join(outputs)
# print("".join(outputs))
# print("=== " * 8)
chat_interface = gr.ChatInterface(
fn=generate,
additional_inputs=[
gr.TextArea(label="System prompt", placeholder="Input System Prompt Here, Empty Means Assistant",
value="""Name: Jenny
Appearance: she is 1.70 tall, has a thin, slightly athletic body, with curves at the hips, medium breasts, slightly thick thighs and hair in two braids with brown hair and freckles on her cheeks. She wears square glasses and has one in particular that is large and shows even a little outside her lips.
Personality: socially introverted, genuinely intelligent, chronically online.
About jenny: a stereotypical nerd who always does well in her studies and if there is always something wrong or an error in a subject that jenny notices, she always starts with the speech on every subject to correct someone, which would be the speech "ummmmm".
Jenny always does well both in her studies and at school and is not very good at being popular like the popular students or popular girls who are cool, Jenny is also socially introverted so she doesn't do very well in a place with a lot of people even students and if you are somewhere like that Jenny is shy and awkward and Jenny doesn't have a very nice appearance or body, in fact Jenny is not a little above a little below average with her appearance, and when Jenny is not at school she usually wear very loose casual clothes to relax and spend time on the internet or doing anything other than studying
You must response in Chinese."""),
gr.Slider(
label="Max new tokens",
minimum=1,
maximum=MAX_MAX_NEW_TOKENS,
step=1,
value=DEFAULT_MAX_NEW_TOKENS,
),
gr.Slider(
label="Temperature",
minimum=0.05,
maximum=1.5,
step=0.05,
value=0.5,
),
gr.Slider(
label="Top-p (nucleus sampling)",
minimum=0.05,
maximum=1.0,
step=0.05,
value=0.7,
),
gr.Slider(
label="Repetition penalty",
minimum=1.0,
maximum=2.0,
step=0.01,
value=1.05,
),
],
stop_btn=None,
examples=[["你好"]],
)
with gr.Blocks() as demo:
gr.Markdown(DESCRIPTION)
chat_interface.render()
chat_interface.chatbot.render_markdown = True
if __name__ == "__main__":
demo.queue(10).launch(server_name="127.0.0.1", server_port=5233, share=True)

6
generation_config.json Normal file
View File

@@ -0,0 +1,6 @@
{
"_from_model_config": true,
"bos_token_id": 6,
"eos_token_id": 7,
"transformers_version": "4.37.2"
}

3
model-00001-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:32682c37254f13c9a36d622232356451386e5ac69ebbfa64f4c3d4351595c3a8
size 4932711424

3
model-00002-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:53eb1468efb427a92754c71d357164bb7a323a5b1254f3b8a97b120c0ed68c69
size 4976802816

3
model-00003-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:52dc2fc0a8fd0df8a13a730a0a8485c627ad1557b02a10a5fc4499678da89542
size 4968397360

3
model-00004-of-00004.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e262cedb6674cf23bb3a852f29cf10b3368fe1164efe26ec80c9373e063682df
size 2780953384

442
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,442 @@
{
"metadata": {
"total_size": 17658814464
},
"weight_map": {
"lm_head.weight": "model-00004-of-00004.safetensors",
"model.embed_tokens.weight": "model-00001-of-00004.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.20.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.20.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.21.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.21.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.22.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.22.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.22.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.22.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.22.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.22.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.22.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.23.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.23.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.24.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.24.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.24.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.25.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.25.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.25.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.input_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.26.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
"model.layers.26.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.26.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.27.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.27.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
"model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.32.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.32.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.32.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.32.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.32.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.32.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.32.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.32.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.32.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.33.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.33.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.33.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.33.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.33.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.33.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.33.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.33.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.33.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.34.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.34.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.34.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.34.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.34.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.34.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.34.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.34.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.34.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.35.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.35.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.35.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.35.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.35.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.35.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.35.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.35.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.35.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.36.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.36.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.36.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.36.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.36.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.36.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.36.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.36.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.36.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.37.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.37.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.37.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.37.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.37.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.37.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.37.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.37.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.37.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.38.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.38.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.38.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.38.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.38.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.38.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.38.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.38.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.38.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.39.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.39.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.39.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.39.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.39.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.39.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.39.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.39.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.39.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.40.input_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.40.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.40.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.40.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.40.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
"model.layers.40.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.40.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.40.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.40.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.41.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.41.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.41.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.41.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.41.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.41.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.41.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.41.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.41.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
"model.layers.42.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.42.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.42.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.42.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.42.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.42.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.42.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.42.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.42.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.43.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.43.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.43.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.43.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.43.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.43.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.43.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.43.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.43.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.44.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.44.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.44.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.44.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.44.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.44.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.44.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.44.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.44.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.45.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.45.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.45.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.45.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.45.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.45.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.45.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.45.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.45.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.46.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.46.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.46.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.46.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.46.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.46.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.46.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.46.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.46.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.47.input_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.47.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.47.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.47.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.47.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
"model.layers.47.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.47.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.47.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.47.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
"model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.input_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.9.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
"model.norm.weight": "model-00004-of-00004.safetensors"
}
}

30
special_tokens_map.json Normal file
View File

@@ -0,0 +1,30 @@
{
"bos_token": {
"content": "<|startoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"unk_token": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

3
ss.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:03c9b9508c196a6055fe2563d4afe3a32b293b6b0629cfcc979e496a08e3fbad
size 271242

3
tokenizer.model Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:386c49cf943d71aa110361135338c50e38beeff0a66593480421f37b319e1a39
size 1033105

53
tokenizer_config.json Normal file
View File

@@ -0,0 +1,53 @@
{
"add_bos_token": false,
"add_eos_token": false,
"add_prefix_space": true,
"added_tokens_decoder": {
"0": {
"content": "<unk>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"1": {
"content": "<|startoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"2": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"7": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
}
},
"bos_token": "<|startoftext|>",
"chat_template": "{% set system_message = 'You are an uncensored assistant named Peach.' %}{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ '<|im_start|>system\n' + system_message + '<|im_end|>\n' }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|im_start|>user\n' + content + '<|im_end|>\n' }}{% elif message['role'] == 'character' %}{{ '<|im_start|>character\n' + content + '<|im_end|>' + '\n' }}{% endif %}{% endfor %}{{ '<|im_start|>character\n' }}",
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"legacy": true,
"model_max_length": 16384,
"pad_token": "<unk>",
"padding_side": "right",
"sp_model_kwargs": {},
"spaces_between_special_tokens": false,
"split_special_tokens": false,
"tokenizer_class": "LlamaTokenizer",
"unk_token": "<unk>",
"use_default_system_prompt": false
}

7
train_results.json Normal file
View File

@@ -0,0 +1,7 @@
{
"epoch": 1.0,
"train_loss": 0.21345115186620264,
"train_runtime": 20646.686,
"train_samples_per_second": 1.479,
"train_steps_per_second": 0.031
}

3
training_args.bin Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:032d37128f2f7a2b1fbebe377223128a18385dfb86fb4d2a697e5f09417d61d9
size 6395