80 lines
3.2 KiB
Markdown
80 lines
3.2 KiB
Markdown
---
|
|
license: apache-2.0
|
|
tags:
|
|
- moe
|
|
- frankenmoe
|
|
- merge
|
|
- mergekit
|
|
- lazymergekit
|
|
- cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
|
|
base_model:
|
|
- cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
- cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
- cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
---
|
|
|
|
# TinyDolphin-3x-MoE
|
|
|
|
TinyDolphin-3x-MoE is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
|
|
* [cognitivecomputations/TinyDolphin-2.8.1-1.1b](https://huggingface.co/cognitivecomputations/TinyDolphin-2.8.1-1.1b)
|
|
* [cognitivecomputations/TinyDolphin-2.8.1-1.1b](https://huggingface.co/cognitivecomputations/TinyDolphin-2.8.1-1.1b)
|
|
* [cognitivecomputations/TinyDolphin-2.8.1-1.1b](https://huggingface.co/cognitivecomputations/TinyDolphin-2.8.1-1.1b)
|
|
|
|
## 🧩 Configuration
|
|
|
|
```yaml
|
|
base_model: cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
gate_mode: hidden
|
|
dtype: float16
|
|
experts:
|
|
- source_model: cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
positive_prompts:
|
|
- "think step-by-step and follow these instructions"
|
|
- "read the following passage, and summarize it in less than 30 words."
|
|
- "please answer this question, consider the options carefully, and return the most likely answer."
|
|
- source_model: cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
positive_prompts: ["produce python code"]
|
|
- source_model: cognitivecomputations/TinyDolphin-2.8.1-1.1b
|
|
positive_prompts: ["What is 2 x 22?"]
|
|
```
|
|
|
|
## 💻 Usage
|
|
|
|
```python
|
|
!pip install -qU transformers bitsandbytes accelerate
|
|
|
|
from transformers import AutoTokenizer
|
|
import transformers
|
|
import torch
|
|
|
|
model = "jtatman/TinyDolphin-3x-MoE"
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model)
|
|
pipeline = transformers.pipeline(
|
|
"text-generation",
|
|
model=model,
|
|
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
|
|
)
|
|
|
|
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
|
|
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
|
print(outputs[0]["generated_text"])
|
|
```
|
|
|
|
Eval:
|
|
|
|
hf ({'pretrained': 'jtatman/TinyDolphin-3x-MoE'}), gen_kwargs: ({}), limit: None, num_fewshot: 0, batch_size: auto (64)
|
|
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|
|
|-------------|------:|------|-----:|--------|---|-----:|---|-----:|
|
|
|arc_challenge| 1|none | 0|acc |↑ |0.3063|± |0.0135|
|
|
| | |none | 0|acc_norm|↑ |0.3285|± |0.0137|
|
|
|arc_easy | 1|none | 0|acc |↑ |0.5981|± |0.0101|
|
|
| | |none | 0|acc_norm|↑ |0.5467|± |0.0102|
|
|
|hellaswag | 1|none | 0|acc |↑ |0.4656|± |0.0050|
|
|
| | |none | 0|acc_norm|↑ |0.6004|± |0.0049|
|
|
|openbookqa | 1|none | 0|acc |↑ |0.2300|± |0.0188|
|
|
| | |none | 0|acc_norm|↑ |0.3640|± |0.0215|
|
|
|piqa | 1|none | 0|acc |↑ |0.7318|± |0.0103|
|
|
| | |none | 0|acc_norm|↑ |0.7296|± |0.0104| |