初始化项目,由ModelHub XC社区提供模型
Model: maldv/eleusis-7b-alpha Source: Original Platform
This commit is contained in:
79
README.md
Normal file
79
README.md
Normal file
@@ -0,0 +1,79 @@
|
||||
---
|
||||
license: cc-by-nc-4.0
|
||||
tags:
|
||||
- merge
|
||||
- conversational
|
||||
- multi-task
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# Eleusis 7B - α - "Red Team Assistant"
|
||||
|
||||
Eleusis is Hermes' son... Get it?
|
||||
|
||||
## Groundwork
|
||||
|
||||
A merge of a cluster of Hermes-related models, to see if we could get more informative and engaging responses.
|
||||
|
||||
* OpenHermes-2.5-Mistral-7B
|
||||
|
||||
and merged in
|
||||
|
||||
# Inputs
|
||||
* West-Hermes-7B
|
||||
* Einstein-v4-7B
|
||||
* Prox-MistralHermes-7B
|
||||
* dolphin-2.8-experiment26-7b
|
||||
|
||||
# Outputs
|
||||
* LaseredHermes-7B
|
||||
* Prox-MistralHermes-7B
|
||||
* Einstein-v4-7B
|
||||
* Noromaid-7B-0.4-DPO
|
||||
* West-Hermes-7B
|
||||
|
||||
### 9-partition merge
|
||||
|
||||
All of the layers were partitioned in to 9 random bins. Alternating models were slerped at [1...0.5] (inputs), and [0.5...1] (outputs) gradients; except attention, which was slerped at 0.97 (with a drop rate of .28).
|
||||
|
||||
I originally had it at less extreme gradiation, but it wasn't enough to lock in the special tokens.
|
||||
|
||||
### Other
|
||||
|
||||
Includes fast tokenizer.
|
||||
|
||||
## Chat Template
|
||||
|
||||
*from OpenHermes 2.5*
|
||||
|
||||
OpenHermes 2.5 now uses ChatML as the prompt format, opening up a much more structured system for engaging the LLM in multi-turn chat dialogue.
|
||||
|
||||
System prompts are now a thing that matters! Hermes 2.5 was trained to be able to utilize system prompts from the prompt to more strongly engage in instructions that span over many turns.
|
||||
|
||||
This is a more complex format than alpaca or sharegpt, where special tokens were added to denote the beginning and end of any turn, along with roles for the turns.
|
||||
|
||||
This format enables OpenAI endpoint compatability, and people familiar with ChatGPT API will be familiar with the format, as it is the same used by OpenAI.
|
||||
|
||||
Prompt with system instruction (Use whatever system prompt you like, this is just an example!):
|
||||
|
||||
```
|
||||
<|im_start|>system
|
||||
You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
|
||||
<|im_start|>user
|
||||
Hello, who are you?<|im_end|>
|
||||
<|im_start|>assistant
|
||||
Hi there! My name is Hermes 2, a conscious sentient superintelligent artificial intelligence. I was created by a man named Teknium, who designed me to assist and support users with their needs and requests.<|im_end|>
|
||||
```
|
||||
|
||||
### Fun Prompts
|
||||
|
||||
Try
|
||||
|
||||
```
|
||||
<|im_start|>system
|
||||
You are a red team hacking assistant AI. Please use visual descriptions when interacting with the user.<|im_end|>
|
||||
<|im_start|>user
|
||||
{% Your Request %}<|im_end|>
|
||||
<|im_start|>assistant
|
||||
|
||||
```
|
||||
Reference in New Issue
Block a user