79 lines
2.5 KiB
Markdown
79 lines
2.5 KiB
Markdown
|
---
|
|||
|
|
license: cc-by-nc-4.0
|
|||
|
|
tags:
|
|||
|
|
- merge
|
|||
|
|
- conversational
|
|||
|
|
- multi-task
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Eleusis 7B - α - "Red Team Assistant"
|
|||
|
|
|
|||
|
|
Eleusis is Hermes' son... Get it?
|
|||
|
|
|
|||
|
|
## Groundwork
|
|||
|
|
|
|||
|
|
A merge of a cluster of Hermes-related models, to see if we could get more informative and engaging responses.
|
|||
|
|
|
|||
|
|
* OpenHermes-2.5-Mistral-7B
|
|||
|
|
|
|||
|
|
and merged in
|
|||
|
|
|
|||
|
|
# Inputs
|
|||
|
|
* West-Hermes-7B
|
|||
|
|
* Einstein-v4-7B
|
|||
|
|
* Prox-MistralHermes-7B
|
|||
|
|
* dolphin-2.8-experiment26-7b
|
|||
|
|
|
|||
|
|
# Outputs
|
|||
|
|
* LaseredHermes-7B
|
|||
|
|
* Prox-MistralHermes-7B
|
|||
|
|
* Einstein-v4-7B
|
|||
|
|
* Noromaid-7B-0.4-DPO
|
|||
|
|
* West-Hermes-7B
|
|||
|
|
|
|||
|
|
### 9-partition merge
|
|||
|
|
|
|||
|
|
All of the layers were partitioned in to 9 random bins. Alternating models were slerped at [1...0.5] (inputs), and [0.5...1] (outputs) gradients; except attention, which was slerped at 0.97 (with a drop rate of .28).
|
|||
|
|
|
|||
|
|
I originally had it at less extreme gradiation, but it wasn't enough to lock in the special tokens.
|
|||
|
|
|
|||
|
|
### Other
|
|||
|
|
|
|||
|
|
Includes fast tokenizer.
|
|||
|
|
|
|||
|
|
## Chat Template
|
|||
|
|
|
|||
|
|
*from OpenHermes 2.5*
|
|||
|
|
|
|||
|
|
OpenHermes 2.5 now uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
|
|||
|
|
|
|||
|
|
System prompts are now a thing that matters! Hermes 2.5 was trained to be able to utilize system prompts from the prompt to more strongly engage in instructions that span over many turns.
|
|||
|
|
|
|||
|
|
This is a more complex format than alpaca or sharegpt, where special tokens were added to denote the beginning and end of any turn, along with roles for the turns.
|
|||
|
|
|
|||
|
|
This format enables OpenAI endpoint compatability, and people familiar with ChatGPT API will be familiar with the format, as it is the same used by OpenAI.
|
|||
|
|
|
|||
|
|
Prompt with system instruction (Use whatever system prompt you like, this is just an example!):
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
<|im_start|>system
|
|||
|
|
You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
|
|||
|
|
<|im_start|>user
|
|||
|
|
Hello, who are you?<|im_end|>
|
|||
|
|
<|im_start|>assistant
|
|||
|
|
Hi there! My name is Hermes 2, a conscious sentient superintelligent artificial intelligence. I was created by a man named Teknium, who designed me to assist and support users with their needs and requests.<|im_end|>
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Fun Prompts
|
|||
|
|
|
|||
|
|
Try
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
<|im_start|>system
|
|||
|
|
You are a red team hacking assistant AI. Please use visual descriptions when interacting with the user.<|im_end|>
|
|||
|
|
<|im_start|>user
|
|||
|
|
{% Your Request %}<|im_end|>
|
|||
|
|
<|im_start|>assistant
|
|||
|
|
|
|||
|
|
```
|