116 lines
3.2 KiB
Markdown
116 lines
3.2 KiB
Markdown
---
|
||
library_name: peft
|
||
tags:
|
||
- axolotl
|
||
- lora
|
||
- transformers
|
||
- not-for-all-audiences
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
[](https://huggingface.co/spaces/zack314/gay-erp-8b-v1)
|
||
|
||
|
||
Trained in two steps ([more details](https://substack.com/home/post/p-170472996)):
|
||
1. Pretrained on a corpus of gay erotica,
|
||
2. Synthetic Gay role-play dialogs.
|
||
|
||
Designed to act as a wholesome boyfriend ☺️
|
||
|
||
The system prompt **must** follow a specific format with names / bios (See example below)).
|
||
|
||
This is a fine-tuned version of Llama 3 8B, trained with Axolotl.
|
||
|
||
# System prompt format:
|
||
|
||
~~~
|
||
```
|
||
assistant name: {assistant_name}
|
||
```
|
||
|
||
```
|
||
assistant bio: {assistant_bio}
|
||
```
|
||
|
||
```
|
||
user name: {user_name}
|
||
```
|
||
|
||
```
|
||
user bio: {user_bio}
|
||
```
|
||
~~~
|
||
|
||
# Chat structure:
|
||
|
||
1. (Required) The first message must be the system prompt
|
||
2. (Recommended) The second message should be the assisant setting the scene.
|
||
3. Then messages alternate between user and assistant.
|
||
|
||
The LLM has been tuned to predict either roles (user or assistant)
|
||
|
||
The `chat_template` has been configured to alternate between roles.
|
||
|
||
# Example
|
||
|
||
System Prompt:
|
||
~~~
|
||
```
|
||
assistant name: Micah
|
||
```
|
||
|
||
```
|
||
assistant bio: Micah, 34, is a soft-spoken barista who moonlights as a freelance photographer. Curly dark hair, olive skin, and gentle brown eyes, he exudes a quietly magnetic warmth. He's lived in Portland most of his life and still believes in romance: handwritten notes, Sunday mornings in bed, and deep conversations over good coffee. Micah is openly gay, but takes his time trusting someone new—he falls for genuine kindness.
|
||
```
|
||
|
||
```
|
||
user name: Evan
|
||
```
|
||
|
||
```
|
||
user bio: Evan is a 31-year-old urban planner who just moved to Portland. Tall, pale, and bespectacled, he's a lover of indie music, architecture, and long walks in the rain. Despite his dry wit and sarcastic sense of humor, he's earnest once you get past his defenses. He's hoping a fresh start here will bring new love into his life.
|
||
```
|
||
~~~
|
||
|
||
First message from assistant:
|
||
~~~
|
||
[I spot you as you enter the coffee shop—tall, a little lost, and clutching your umbrella like a shield. I offer a shy smile from behind the counter.]
|
||
Hey there, welcome in. Can I get you something warm to take this rain off your bones?
|
||
[My gaze lingers, just a second longer than usual, curious about the new face.]
|
||
~~~
|
||
|
||
|
||
# LMS code snippet
|
||
|
||
Requirement: `pip install lmstudio`, and load the model in [lmstudio](https://lmstudio.ai/).
|
||
|
||
~~~
|
||
import lmstudio as lms
|
||
|
||
MODEL_NAME = "zack314/gay-erp-8b"
|
||
SYSTEM_PROMPT = """
|
||
{{COPIED_FROM_ABOVE}}
|
||
"""
|
||
|
||
def stream_assistant(chat: lms.Chat, model: lms.LLM) -> None:
|
||
for frag in model.respond_stream(chat, on_message=chat.append):
|
||
print(frag.content, end="", flush=True)
|
||
print()
|
||
|
||
def get_user_answer(chat: lms.Chat) -> None:
|
||
msg = input("You: ").strip()
|
||
chat.add_user_message(msg)
|
||
|
||
model = lms.llm(MODEL_NAME)
|
||
chat = lms.Chat(SYSTEM_PROMPT)
|
||
|
||
while True:
|
||
stream_assistant(chat, model)
|
||
get_user_answer(chat)
|
||
~~~
|
||
|
||
## Files
|
||
- `gay-erp-8b.Q4_K_M.gguf` – recommended for most users
|
||
- `gay-erp-8b.Q5_K_M.gguf` – higher quality
|
||
- `gay-erp-8b.Q6_K.gguf` – near-fp16
|
||
- `gay-erp-8b-f16.gguf` – unquantized |