92 lines
2.5 KiB
Markdown
92 lines
2.5 KiB
Markdown
---
|
|
license: apache-2.0
|
|
tags:
|
|
- merge
|
|
- mergekit
|
|
- lazymergekit
|
|
- kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP
|
|
- VAGOsolutions/SauerkrautLM-SOLAR-Instruct
|
|
---
|
|
|
|
# SOLAR-10.7B-Instruct-ties
|
|
|
|
SOLAR-10.7B-Instruct-ties is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
|
|
* [kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP](https://huggingface.co/kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP)
|
|
* [VAGOsolutions/SauerkrautLM-SOLAR-Instruct](https://huggingface.co/VAGOsolutions/SauerkrautLM-SOLAR-Instruct)
|
|
|
|
## 🧩 Configuration
|
|
|
|
```yaml
|
|
models:
|
|
- model: upstage/SOLAR-10.7B-Instruct-v1.0
|
|
# no parameters necessary for base model
|
|
- model: kodonho/Solar-OrcaDPO-Solar-Instruct-SLERP
|
|
parameters:
|
|
density: 0.5
|
|
weight: 0.5
|
|
- model: VAGOsolutions/SauerkrautLM-SOLAR-Instruct
|
|
parameters:
|
|
density: 0.5
|
|
weight: 0.3
|
|
merge_method: ties
|
|
base_model: upstage/SOLAR-10.7B-Instruct-v1.0
|
|
parameters:
|
|
normalize: true
|
|
dtype: float16
|
|
```
|
|
## 💻 Example Python Code
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
|
|
|
|
model_name_or_path = "nfaheem/SOLAR-10.7B-Instruct-ties"
|
|
model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
|
|
device_map="auto",
|
|
revision="main")
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
|
|
|
|
prompt = "Write a story about llamas"
|
|
system_message = "You are a story writing assistant"
|
|
prompt_template=f'''{prompt}
|
|
'''
|
|
|
|
print("\n\n*** Generate:")
|
|
|
|
input_ids = tokenizer(prompt_template, return_tensors='pt').input_ids.cuda()
|
|
output = model.generate(inputs=input_ids, temperature=0.7, do_sample=True, top_p=0.95, top_k=40, max_new_tokens=512)
|
|
print(tokenizer.decode(output[0]))
|
|
|
|
# Inference can also be done using transformers' pipeline
|
|
|
|
print("*** Pipeline:")
|
|
pipe = pipeline(
|
|
"text-generation",
|
|
model=model,
|
|
tokenizer=tokenizer,
|
|
max_new_tokens=512,
|
|
do_sample=True,
|
|
temperature=0.7,
|
|
top_p=0.95,
|
|
top_k=40,
|
|
repetition_penalty=1.1
|
|
)
|
|
|
|
print(pipe(prompt_template)[0]['generated_text'])
|
|
```
|
|
## 📋 Summary Eval:
|
|
|
|
| Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
|
|
|---------|-------|-----------|--------|------------|------------|-------|
|
|
| 74.24 | 70.9 | 88.58 | 66.34 | 71.88 | 83.5 | 64.06 |
|
|
|
|
|
|
|
|
## 📈 Huggingface Leaderboard
|
|
|
|
|
|
|
|
|
|
|
|
|