60 lines
2.0 KiB
Markdown
60 lines
2.0 KiB
Markdown
|
---
|
||
|
|
tags:
|
||
|
|
- merge
|
||
|
|
- mergekit
|
||
|
|
- lazymergekit
|
||
|
|
license: apache-2.0
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/6389d3c61e8755d777902366/-_AiKUEsY3x-N7oY52fdE.jpeg" style="border-radius:2%; width: 66%">
|
||
|
|
|
||
|
|
# pandafish-7b
|
||
|
|
|
||
|
|
pandafish-7b is an instruct model based on a [Model Stock](https://arxiv.org/abs/2403.19522) merge of the following models (via [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing)):
|
||
|
|
|
||
|
|
## 🧩 Configuration
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
models:
|
||
|
|
- model: mistralai/Mistral-7B-v0.1
|
||
|
|
- model: mistralai/Mistral-7B-Instruct-v0.2
|
||
|
|
- model: CultriX/NeuralTrix-bf16
|
||
|
|
- model: OpenPipe/mistral-ft-optimized-1227
|
||
|
|
merge_method: model_stock
|
||
|
|
base_model: mistralai/Mistral-7B-v0.1
|
||
|
|
dtype: bfloat16
|
||
|
|
```
|
||
|
|
|
||
|
|
## 🏆 Evals
|
||
|
|
|
||
|
|
| Model |Average|AGIEval|GPT4All|TruthfulQA|Bigbench|
|
||
|
|
|---------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|
||
|
|
|[pandafish-7b](https://huggingface.co/ichigoberry/pandafish-7b) [📄](https://gist.github.com/tosh/dda6a21e568d17a410ca618265f64a28)| 51.99 | **40** | **74.23** | 53.22 | 40.51 |
|
||
|
|
|[mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) [📄](https://gist.github.com/mlabonne/05d358e17dffdf9eee7c2322380c9da6) | 54.81 | 38.5 | 71.64 | **66.82** | **42.29** |
|
||
|
|
|
||
|
|
|
||
|
|
## 💻 Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
!pip install -qU transformers accelerate
|
||
|
|
|
||
|
|
from transformers import AutoTokenizer
|
||
|
|
import transformers
|
||
|
|
import torch
|
||
|
|
|
||
|
|
model = "ichigoberry/pandafish-7b"
|
||
|
|
messages = [{"role": "user", "content": "What is a large language model?"}]
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model)
|
||
|
|
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
|
|
pipeline = transformers.pipeline(
|
||
|
|
"text-generation",
|
||
|
|
model=model,
|
||
|
|
torch_dtype=torch.float16,
|
||
|
|
device_map="auto",
|
||
|
|
)
|
||
|
|
|
||
|
|
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
||
|
|
print(outputs[0]["generated_text"])
|
||
|
|
```
|