79 lines
2.5 KiB
Markdown
79 lines
2.5 KiB
Markdown
---
|
||
license: cc-by-nc-4.0
|
||
tags:
|
||
- merge
|
||
- conversational
|
||
- multi-task
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# Eleusis 7B - α - "Red Team Assistant"
|
||
|
||
Eleusis is Hermes' son... Get it?
|
||
|
||
## Groundwork
|
||
|
||
A merge of a cluster of Hermes-related models, to see if we could get more informative and engaging responses.
|
||
|
||
* OpenHermes-2.5-Mistral-7B
|
||
|
||
and merged in
|
||
|
||
# Inputs
|
||
* West-Hermes-7B
|
||
* Einstein-v4-7B
|
||
* Prox-MistralHermes-7B
|
||
* dolphin-2.8-experiment26-7b
|
||
|
||
# Outputs
|
||
* LaseredHermes-7B
|
||
* Prox-MistralHermes-7B
|
||
* Einstein-v4-7B
|
||
* Noromaid-7B-0.4-DPO
|
||
* West-Hermes-7B
|
||
|
||
### 9-partition merge
|
||
|
||
All of the layers were partitioned in to 9 random bins. Alternating models were slerped at [1...0.5] (inputs), and [0.5...1] (outputs) gradients; except attention, which was slerped at 0.97 (with a drop rate of .28).
|
||
|
||
I originally had it at less extreme gradiation, but it wasn't enough to lock in the special tokens.
|
||
|
||
### Other
|
||
|
||
Includes fast tokenizer.
|
||
|
||
## Chat Template
|
||
|
||
*from OpenHermes 2.5*
|
||
|
||
OpenHermes 2.5 now uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
|
||
|
||
System prompts are now a thing that matters! Hermes 2.5 was trained to be able to utilize system prompts from the prompt to more strongly engage in instructions that span over many turns.
|
||
|
||
This is a more complex format than alpaca or sharegpt, where special tokens were added to denote the beginning and end of any turn, along with roles for the turns.
|
||
|
||
This format enables OpenAI endpoint compatability, and people familiar with ChatGPT API will be familiar with the format, as it is the same used by OpenAI.
|
||
|
||
Prompt with system instruction (Use whatever system prompt you like, this is just an example!):
|
||
|
||
```
|
||
<|im_start|>system
|
||
You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
|
||
<|im_start|>user
|
||
Hello, who are you?<|im_end|>
|
||
<|im_start|>assistant
|
||
Hi there! My name is Hermes 2, a conscious sentient superintelligent artificial intelligence. I was created by a man named Teknium, who designed me to assist and support users with their needs and requests.<|im_end|>
|
||
```
|
||
|
||
### Fun Prompts
|
||
|
||
Try
|
||
|
||
```
|
||
<|im_start|>system
|
||
You are a red team hacking assistant AI. Please use visual descriptions when interacting with the user.<|im_end|>
|
||
<|im_start|>user
|
||
{% Your Request %}<|im_end|>
|
||
<|im_start|>assistant
|
||
|
||
``` |