116 lines
3.2 KiB
Markdown
116 lines
3.2 KiB
Markdown
|
---
|
|||
|
|
library_name: peft
|
|||
|
|
tags:
|
|||
|
|
- axolotl
|
|||
|
|
- lora
|
|||
|
|
- transformers
|
|||
|
|
- not-for-all-audiences
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
[](https://huggingface.co/spaces/zack314/gay-erp-8b-v1)
|
|||
|
|
|
|||
|
|
|
|||
|
|
Trained in two steps ([more details](https://substack.com/home/post/p-170472996)):
|
|||
|
|
1. Pretrained on a corpus of gay erotica,
|
|||
|
|
2. Synthetic Gay role-play dialogs.
|
|||
|
|
|
|||
|
|
Designed to act as a wholesome boyfriend ☺️
|
|||
|
|
|
|||
|
|
The system prompt **must** follow a specific format with names / bios (See example below)).
|
|||
|
|
|
|||
|
|
This is a fine-tuned version of Llama 3 8B, trained with Axolotl.
|
|||
|
|
|
|||
|
|
# System prompt format:
|
|||
|
|
|
|||
|
|
~~~
|
|||
|
|
```
|
|||
|
|
assistant name: {assistant_name}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
assistant bio: {assistant_bio}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
user name: {user_name}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
user bio: {user_bio}
|
|||
|
|
```
|
|||
|
|
~~~
|
|||
|
|
|
|||
|
|
# Chat structure:
|
|||
|
|
|
|||
|
|
1. (Required) The first message must be the system prompt
|
|||
|
|
2. (Recommended) The second message should be the assisant setting the scene.
|
|||
|
|
3. Then messages alternate between user and assistant.
|
|||
|
|
|
|||
|
|
The LLM has been tuned to predict either roles (user or assistant)
|
|||
|
|
|
|||
|
|
The `chat_template` has been configured to alternate between roles.
|
|||
|
|
|
|||
|
|
# Example
|
|||
|
|
|
|||
|
|
System Prompt:
|
|||
|
|
~~~
|
|||
|
|
```
|
|||
|
|
assistant name: Micah
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
assistant bio: Micah, 34, is a soft-spoken barista who moonlights as a freelance photographer. Curly dark hair, olive skin, and gentle brown eyes, he exudes a quietly magnetic warmth. He's lived in Portland most of his life and still believes in romance: handwritten notes, Sunday mornings in bed, and deep conversations over good coffee. Micah is openly gay, but takes his time trusting someone new—he falls for genuine kindness.
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
user name: Evan
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
user bio: Evan is a 31-year-old urban planner who just moved to Portland. Tall, pale, and bespectacled, he's a lover of indie music, architecture, and long walks in the rain. Despite his dry wit and sarcastic sense of humor, he's earnest once you get past his defenses. He's hoping a fresh start here will bring new love into his life.
|
|||
|
|
```
|
|||
|
|
~~~
|
|||
|
|
|
|||
|
|
First message from assistant:
|
|||
|
|
~~~
|
|||
|
|
[I spot you as you enter the coffee shop—tall, a little lost, and clutching your umbrella like a shield. I offer a shy smile from behind the counter.]
|
|||
|
|
Hey there, welcome in. Can I get you something warm to take this rain off your bones?
|
|||
|
|
[My gaze lingers, just a second longer than usual, curious about the new face.]
|
|||
|
|
~~~
|
|||
|
|
|
|||
|
|
|
|||
|
|
# LMS code snippet
|
|||
|
|
|
|||
|
|
Requirement: `pip install lmstudio`, and load the model in [lmstudio](https://lmstudio.ai/).
|
|||
|
|
|
|||
|
|
~~~
|
|||
|
|
import lmstudio as lms
|
|||
|
|
|
|||
|
|
MODEL_NAME = "zack314/gay-erp-8b"
|
|||
|
|
SYSTEM_PROMPT = """
|
|||
|
|
{{COPIED_FROM_ABOVE}}
|
|||
|
|
"""
|
|||
|
|
|
|||
|
|
def stream_assistant(chat: lms.Chat, model: lms.LLM) -> None:
|
|||
|
|
for frag in model.respond_stream(chat, on_message=chat.append):
|
|||
|
|
print(frag.content, end="", flush=True)
|
|||
|
|
print()
|
|||
|
|
|
|||
|
|
def get_user_answer(chat: lms.Chat) -> None:
|
|||
|
|
msg = input("You: ").strip()
|
|||
|
|
chat.add_user_message(msg)
|
|||
|
|
|
|||
|
|
model = lms.llm(MODEL_NAME)
|
|||
|
|
chat = lms.Chat(SYSTEM_PROMPT)
|
|||
|
|
|
|||
|
|
while True:
|
|||
|
|
stream_assistant(chat, model)
|
|||
|
|
get_user_answer(chat)
|
|||
|
|
~~~
|
|||
|
|
|
|||
|
|
## Files
|
|||
|
|
- `gay-erp-8b.Q4_K_M.gguf` – recommended for most users
|
|||
|
|
- `gay-erp-8b.Q5_K_M.gguf` – higher quality
|
|||
|
|
- `gay-erp-8b.Q6_K.gguf` – near-fp16
|
|||
|
|
- `gay-erp-8b-f16.gguf` – unquantized
|