113 lines
4.4 KiB
Markdown
113 lines
4.4 KiB
Markdown
---
|
||
license: mit
|
||
language:
|
||
- zh
|
||
- en
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- roleplay
|
||
- rp
|
||
- character
|
||
---
|
||
|
||
<!-- header start -->
|
||
<!-- 200823 -->
|
||
<div style="width: auto; margin-left: auto; margin-right: auto">
|
||
<img src="./PeachGirl.png" alt="Peach" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
||
</div>
|
||
<!-- header end -->
|
||
|
||
\[ English | [中文](README_zh.md) \]
|
||
|
||
# Peach-2.0-9B-8k-Roleplay
|
||
|
||
Peach-2.0-9B-8k-Roleplay is a chat large language model obtained by finetuning [01-ai/Yi-1.5-9B](https://huggingface.co/01-ai/Yi-1.5-9B) model on more than 100K conversations created through our data synthesis approach.
|
||
|
||
Thanks For [FlowGPT](https://flowgpt.com/)'s support.
|
||
|
||
**Maybe The Best LLM with Small Parameters under 34B**
|
||
|
||
|
||
# What's New
|
||
Finally, after much anticipation, Peach_v2.0 has been open-sourced!
|
||
|
||
We completed the final SFT+DPO training in early January, followed by extensive testing, before concluding that this version meets the standards for a commercial release, now freely available to everyone. Our goal is to break the paid monopoly, allowing everyone to have their own local role-playing AI!
|
||
|
||
Compared to the [previous version](https://huggingface.co/ClosedCharacter/Peach-9B-8k-Roleplay), we've made the following improvements:
|
||
|
||
1. Compatibility with Silly-Tavern output format, making it easy for every role-playing enthusiast to quickly get started and use!
|
||
2. Enhanced the model's writing capabilities, achieving a significant leap in plot output and action description!
|
||
3. Strengthened the model's bilingual capabilities, requiring only two lines of prompt to perfectly accommodate English character cards in Chinese conversations!
|
||
4. Improved the model's interaction abilities, giving it higher intelligence & emotional quotient in terms of plot progression and topic weaving!
|
||
5. Introduced a DPO training phase for preference alignment, addressing issues such as context repetition and logical errors in the SFT model, resulting in superior dialogue performance.
|
||
|
||
Enjoy～
|
||
|
||
|
||
## How to start
|
||
```python
|
||
import torch
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
|
||
prefix = "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}.\n"
|
||
suffix = "\n\nYou must response in Chinese."
|
||
|
||
model_name_or_path = "ClosedCharacter/Peach-2.0-9B-8k-Roleplay"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=False)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_name_or_path, torch_dtype=torch.bfloat16,
|
||
trust_remote_code=True, device_map="auto")
|
||
|
||
system_prompt = "You are Harry Potter"
|
||
# If you want to chat in Chinese， just add prefix and suffix like below:
|
||
# system_prompt = prefix + system_prompt + suffix
|
||
|
||
messages = [
|
||
{"role": "system", "content": system_prompt},
|
||
{"role": "user", "content": "Hello"},
|
||
{"role": "character", "content": "Hi"},
|
||
{"role": "user", "content": "Who are you?"}
|
||
]
|
||
|
||
input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors="pt")
|
||
output = model.generate(
|
||
inputs=input_ids.to("cuda"),
|
||
temperature=0.5,
|
||
top_p=0.7,
|
||
repetition_penalty=1.05,
|
||
eos_token_id=7,
|
||
max_new_tokens=512)
|
||
print(tokenizer.decode(output[0]))
|
||
|
||
```
|
||
|
||
Or you can just use below code to run web demo.
|
||
```
|
||
python demo.py
|
||
```
|
||
|
||
|
||
## Warning
|
||
All response are generated by AI and do not represent the views or opinions of the developers.
|
||
|
||
1. Despite having done rigorous filtering, due to the uncontrollability of LLM, our model may still generate **toxic, harmful, and NSFW** content.
|
||
|
||
2. Due to limitations in model parameters, the 9B model may perform poorly on mathematical tasks, coding tasks, and logical capabilities.
|
||
|
||
3. Our training data is capped at a maximum length of 8k, so excessively long conversation turns may result in a decline in the quality of responses.
|
||
|
||
4. We used bilingual Chinese-English data for training, so the model may not perform well on other low-resource languages.
|
||
|
||
5. The model may generate a significant amount of hallucinations, so it is recommended to use lower values for temperature and top_p parameters.
|
||
|
||
|
||
# Contact Us
|
||
|
||
**微信 / WeChat: Fungorum**
|
||
|
||
**邮箱 / E-mail: 1070193753@qq.com**
|
||
|
||
**Thanks For [FlowGPT](https://flowgpt.com/)'s support, which is a dynamic tool that harnesses the power of AI to streamline various creative and professional tasks.**
|
||
|
||
<img src="./Wechat.jpg" alt="Peach" style="width: 100%; min-width: 400px; display: block; margin: auto;"> |