113 lines
4.4 KiB
Markdown
113 lines
4.4 KiB
Markdown
|
---
|
|||
|
|
license: mit
|
|||
|
|
language:
|
|||
|
|
- zh
|
|||
|
|
- en
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
tags:
|
|||
|
|
- roleplay
|
|||
|
|
- rp
|
|||
|
|
- character
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- header start -->
|
|||
|
|
<!-- 200823 -->
|
|||
|
|
<div style="width: auto; margin-left: auto; margin-right: auto">
|
|||
|
|
<img src="./PeachGirl.png" alt="Peach" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
|||
|
|
</div>
|
|||
|
|
<!-- header end -->
|
|||
|
|
|
|||
|
|
\[ English | [中文](README_zh.md) \]
|
|||
|
|
|
|||
|
|
# Peach-2.0-9B-8k-Roleplay
|
|||
|
|
|
|||
|
|
Peach-2.0-9B-8k-Roleplay is a chat large language model obtained by finetuning [01-ai/Yi-1.5-9B](https://huggingface.co/01-ai/Yi-1.5-9B) model on more than 100K conversations created through our data synthesis approach.
|
|||
|
|
|
|||
|
|
Thanks For [FlowGPT](https://flowgpt.com/)'s support.
|
|||
|
|
|
|||
|
|
**Maybe The Best LLM with Small Parameters under 34B**
|
|||
|
|
|
|||
|
|
|
|||
|
|
# What's New
|
|||
|
|
Finally, after much anticipation, Peach_v2.0 has been open-sourced!
|
|||
|
|
|
|||
|
|
We completed the final SFT+DPO training in early January, followed by extensive testing, before concluding that this version meets the standards for a commercial release, now freely available to everyone. Our goal is to break the paid monopoly, allowing everyone to have their own local role-playing AI!
|
|||
|
|
|
|||
|
|
Compared to the [previous version](https://huggingface.co/ClosedCharacter/Peach-9B-8k-Roleplay), we've made the following improvements:
|
|||
|
|
|
|||
|
|
1. Compatibility with Silly-Tavern output format, making it easy for every role-playing enthusiast to quickly get started and use!
|
|||
|
|
2. Enhanced the model's writing capabilities, achieving a significant leap in plot output and action description!
|
|||
|
|
3. Strengthened the model's bilingual capabilities, requiring only two lines of prompt to perfectly accommodate English character cards in Chinese conversations!
|
|||
|
|
4. Improved the model's interaction abilities, giving it higher intelligence & emotional quotient in terms of plot progression and topic weaving!
|
|||
|
|
5. Introduced a DPO training phase for preference alignment, addressing issues such as context repetition and logical errors in the SFT model, resulting in superior dialogue performance.
|
|||
|
|
|
|||
|
|
Enjoy～
|
|||
|
|
|
|||
|
|
|
|||
|
|
## How to start
|
|||
|
|
```python
|
|||
|
|
import torch
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
|
|||
|
|
|
|||
|
|
prefix = "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}.\n"
|
|||
|
|
suffix = "\n\nYou must response in Chinese."
|
|||
|
|
|
|||
|
|
model_name_or_path = "ClosedCharacter/Peach-2.0-9B-8k-Roleplay"
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=False)
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|||
|
|
model_name_or_path, torch_dtype=torch.bfloat16,
|
|||
|
|
trust_remote_code=True, device_map="auto")
|
|||
|
|
|
|||
|
|
system_prompt = "You are Harry Potter"
|
|||
|
|
# If you want to chat in Chinese， just add prefix and suffix like below:
|
|||
|
|
# system_prompt = prefix + system_prompt + suffix
|
|||
|
|
|
|||
|
|
messages = [
|
|||
|
|
{"role": "system", "content": system_prompt},
|
|||
|
|
{"role": "user", "content": "Hello"},
|
|||
|
|
{"role": "character", "content": "Hi"},
|
|||
|
|
{"role": "user", "content": "Who are you?"}
|
|||
|
|
]
|
|||
|
|
|
|||
|
|
input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors="pt")
|
|||
|
|
output = model.generate(
|
|||
|
|
inputs=input_ids.to("cuda"),
|
|||
|
|
temperature=0.5,
|
|||
|
|
top_p=0.7,
|
|||
|
|
repetition_penalty=1.05,
|
|||
|
|
eos_token_id=7,
|
|||
|
|
max_new_tokens=512)
|
|||
|
|
print(tokenizer.decode(output[0]))
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Or you can just use below code to run web demo.
|
|||
|
|
```
|
|||
|
|
python demo.py
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Warning
|
|||
|
|
All response are generated by AI and do not represent the views or opinions of the developers.
|
|||
|
|
|
|||
|
|
1. Despite having done rigorous filtering, due to the uncontrollability of LLM, our model may still generate **toxic, harmful, and NSFW** content.
|
|||
|
|
|
|||
|
|
2. Due to limitations in model parameters, the 9B model may perform poorly on mathematical tasks, coding tasks, and logical capabilities.
|
|||
|
|
|
|||
|
|
3. Our training data is capped at a maximum length of 8k, so excessively long conversation turns may result in a decline in the quality of responses.
|
|||
|
|
|
|||
|
|
4. We used bilingual Chinese-English data for training, so the model may not perform well on other low-resource languages.
|
|||
|
|
|
|||
|
|
5. The model may generate a significant amount of hallucinations, so it is recommended to use lower values for temperature and top_p parameters.
|
|||
|
|
|
|||
|
|
|
|||
|
|
# Contact Us
|
|||
|
|
|
|||
|
|
**微信 / WeChat: Fungorum**
|
|||
|
|
|
|||
|
|
**邮箱 / E-mail: 1070193753@qq.com**
|
|||
|
|
|
|||
|
|
**Thanks For [FlowGPT](https://flowgpt.com/)'s support, which is a dynamic tool that harnesses the power of AI to streamline various creative and professional tasks.**
|
|||
|
|
|
|||
|
|
<img src="./Wechat.jpg" alt="Peach" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|