88 lines
3.3 KiB
Markdown
88 lines
3.3 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- moe
|
||
|
|
- frankenmoe
|
||
|
|
- merge
|
||
|
|
- mergekit
|
||
|
|
- lazymergekit
|
||
|
|
- jambroz/sixtyoneeighty-7b-chat
|
||
|
|
base_model:
|
||
|
|
- jambroz/sixtyoneeighty-7b-chat
|
||
|
|
- NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
|
||
|
|
- S-miguel/The-Trinity-Coder-7B
|
||
|
|
- chihoonlee10/T3Q-Mistral-Orca-Math-DPO
|
||
|
|
---
|
||
|
|
|
||
|
|
# sixtyoneeighty-7b-MOE
|
||
|
|
|
||
|
|
sixtyoneeighty-7b-MOE is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
|
||
|
|
* [jambroz/sixtyoneeighty-7b-chat](https://huggingface.co/jambroz/sixtyoneeighty-7b-chat)
|
||
|
|
* [NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story](https://huggingface.co/NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story)
|
||
|
|
* [S-miguel/The-Trinity-Coder-7B](https://huggingface.co/S-miguel/The-Trinity-Coder-7B)
|
||
|
|
* [chihoonlee10/T3Q-Mistral-Orca-Math-DPO](https://huggingface.co/chihoonlee10/T3Q-Mistral-Orca-Math-DPO)
|
||
|
|
|
||
|
|
## 🧩 Configuration
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
base_model: cognitivecomputations/dolphin-2.8-mistral-7b-v02
|
||
|
|
gate_mode: hidden
|
||
|
|
dtype: bfloat16
|
||
|
|
experts_per_token: 2
|
||
|
|
experts:
|
||
|
|
- source_model: jambroz/sixtyoneeighty-7b-chat
|
||
|
|
positive_prompts:
|
||
|
|
- "What are some fun activities to do in Seattle?"
|
||
|
|
- "What are some fun historical facts about New York City?"
|
||
|
|
negative_prompts:
|
||
|
|
- "Write a Python script to scrape data from a website."
|
||
|
|
- "Explain the key differences between Bayesian and frequentist statistics."
|
||
|
|
|
||
|
|
- source_model: NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
|
||
|
|
positive_prompts:
|
||
|
|
- "Write me a fictional story about dragons and wizards?"
|
||
|
|
- "From now on take on the role of Dwayne Johnson"
|
||
|
|
negative_prompts:
|
||
|
|
- "When is the next solar eclipse."
|
||
|
|
- "What year did World War II end?"
|
||
|
|
|
||
|
|
- source_model: S-miguel/The-Trinity-Coder-7B
|
||
|
|
positive_prompts:
|
||
|
|
- "Can you review my JavaScript code and suggest ways to optimize it for better performance?"
|
||
|
|
- "I'm getting an 'undefined variable' error in my Python script. Here's the code: [code snippet]"
|
||
|
|
negative_prompts:
|
||
|
|
- "What are some effective strategies for managing stress and anxiety?"
|
||
|
|
- "Compare and contrast the themes in 'The Great Gatsby' and 'The Catcher in the Rye'."
|
||
|
|
|
||
|
|
- source_model: chihoonlee10/T3Q-Mistral-Orca-Math-DPO
|
||
|
|
positive_prompts:
|
||
|
|
- "What's a square root of 1337?"
|
||
|
|
- "Find the midpoint of the line segment with the given end points (-5,7) and (-2,1)"
|
||
|
|
negative_prompts:
|
||
|
|
- "What are some effective strategies for managing stress and anxiety?"
|
||
|
|
- "Compare and contrast the themes in 'The Great Gatsby' and 'The Catcher in the Rye'."
|
||
|
|
```
|
||
|
|
|
||
|
|
## 💻 Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
!pip install -qU transformers bitsandbytes accelerate
|
||
|
|
|
||
|
|
from transformers import AutoTokenizer
|
||
|
|
import transformers
|
||
|
|
import torch
|
||
|
|
|
||
|
|
model = "jambroz/sixtyoneeighty-7b-MOE"
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model)
|
||
|
|
pipeline = transformers.pipeline(
|
||
|
|
"text-generation",
|
||
|
|
model=model,
|
||
|
|
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
|
||
|
|
)
|
||
|
|
|
||
|
|
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
|
||
|
|
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
|
|
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
||
|
|
print(outputs[0]["generated_text"])
|
||
|
|
```
|