230 lines
7.0 KiB
Markdown
230 lines
7.0 KiB
Markdown
---
|
|
license: apache-2.0
|
|
library_name: transformers
|
|
model-index:
|
|
- name: laser-dolphin-mixtral-4x7b-dpo
|
|
results:
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
name: AI2 Reasoning Challenge (25-Shot)
|
|
type: ai2_arc
|
|
config: ARC-Challenge
|
|
split: test
|
|
args:
|
|
num_few_shot: 25
|
|
metrics:
|
|
- type: acc_norm
|
|
value: 64.93
|
|
name: normalized accuracy
|
|
source:
|
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-dolphin-mixtral-4x7b-dpo
|
|
name: Open LLM Leaderboard
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
name: HellaSwag (10-Shot)
|
|
type: hellaswag
|
|
split: validation
|
|
args:
|
|
num_few_shot: 10
|
|
metrics:
|
|
- type: acc_norm
|
|
value: 85.81
|
|
name: normalized accuracy
|
|
source:
|
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-dolphin-mixtral-4x7b-dpo
|
|
name: Open LLM Leaderboard
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
name: MMLU (5-Shot)
|
|
type: cais/mmlu
|
|
config: all
|
|
split: test
|
|
args:
|
|
num_few_shot: 5
|
|
metrics:
|
|
- type: acc
|
|
value: 63.04
|
|
name: accuracy
|
|
source:
|
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-dolphin-mixtral-4x7b-dpo
|
|
name: Open LLM Leaderboard
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
name: TruthfulQA (0-shot)
|
|
type: truthful_qa
|
|
config: multiple_choice
|
|
split: validation
|
|
args:
|
|
num_few_shot: 0
|
|
metrics:
|
|
- type: mc2
|
|
value: 63.77
|
|
source:
|
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-dolphin-mixtral-4x7b-dpo
|
|
name: Open LLM Leaderboard
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
name: Winogrande (5-shot)
|
|
type: winogrande
|
|
config: winogrande_xl
|
|
split: validation
|
|
args:
|
|
num_few_shot: 5
|
|
metrics:
|
|
- type: acc
|
|
value: 77.82
|
|
name: accuracy
|
|
source:
|
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-dolphin-mixtral-4x7b-dpo
|
|
name: Open LLM Leaderboard
|
|
- task:
|
|
type: text-generation
|
|
name: Text Generation
|
|
dataset:
|
|
name: GSM8k (5-shot)
|
|
type: gsm8k
|
|
config: main
|
|
split: test
|
|
args:
|
|
num_few_shot: 5
|
|
metrics:
|
|
- type: acc
|
|
value: 44.88
|
|
name: accuracy
|
|
source:
|
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-dolphin-mixtral-4x7b-dpo
|
|
name: Open LLM Leaderboard
|
|
---
|
|
# Laser-Dolphin-Mixtral-4x7b-dpo
|
|
|
|
|
|
|
|
Credit to Fernando Fernandes and Eric Hartford for their project [laserRMT](https://github.com/cognitivecomputations/laserRMT)
|
|
|
|
This model is a medium-sized MoE implementation based on [cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser)
|
|
|
|
The process is outlined in this [notebook](https://github.com/cognitivecomputations/laserRMT/blob/main/examples/laser-dolphin-mixtral-2x7b.ipynb)
|
|
|
|
## Code Example
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
def generate_response(prompt):
|
|
"""
|
|
Generate a response from the model based on the input prompt.
|
|
|
|
Args:
|
|
prompt (str): Prompt for the model.
|
|
|
|
Returns:
|
|
str: The generated response from the model.
|
|
"""
|
|
# Tokenize the input prompt
|
|
inputs = tokenizer(prompt, return_tensors="pt")
|
|
|
|
# Generate output tokens
|
|
outputs = model.generate(**inputs, max_new_tokens=256, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id)
|
|
|
|
# Decode the generated tokens to a string
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
|
|
|
return response
|
|
|
|
# Load the model and tokenizer
|
|
model_id = "macadeliccc/laser-dolphin-mixtral-4x7b-dpo"
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)
|
|
|
|
prompt = "Write a quicksort algorithm in python"
|
|
|
|
# Generate and print responses for each language
|
|
print("Response:")
|
|
print(generate_response(prompt), "\n")
|
|
```
|
|
|
|
## Example output
|
|
|
|
can you write me a quicksort algorithm in python?
|
|
|
|
Sure, here's a quicksort algorithm implemented in Python:
|
|
|
|
```python
|
|
def quicksort(arr):
|
|
if len(arr) <= 1:
|
|
return arr
|
|
pivot = arr[len(arr) // 2]
|
|
left = [x for x in arr if x < pivot]
|
|
middle = [x for x in arr if x == pivot]
|
|
right = [x for x in arr if x > pivot]
|
|
return quicksort(left) + middle + quicksort(right)
|
|
```
|
|
|
|
This implementation uses the median of the array as the pivot.
|
|
It first checks if the array has one or fewer elements, in which case it is already sorted and can be returned as is.
|
|
Otherwise, it selects the pivot as the middle element of the array. Then, it partitions the array into three sub-arrays: elements less than the pivot, elements equal to the pivot, and elements greater than the pivot.
|
|
It recursively sorts the left and right sub-arrays and concatenates the results with the middle sub-array to obtain the final sorted array.
|
|
|
|
|
|
## Quantization
|
|
|
|
[4-bit AWQ](https://huggingface.co/macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ)
|
|
|
|
## Eval
|
|
|
|
**Model evaluated in 4bit**
|
|
|
|
----Benchmark Complete----
|
|
+ 2024-01-24 15:03:08
|
|
+ Time taken: 37.4 mins
|
|
+ Prompt Format: Mistral
|
|
+ Model: macadeliccc/laser-dolphin-mixtral-4x7b-dpo
|
|
+ Score (v2): 71.04
|
|
+ Parseable: 169.0
|
|
---------------
|
|
|
|
|
|
## Citations
|
|
|
|
Fernando Fernandes Neto and Eric Hartford. "Optimizing Large Language Models Using Layer-Selective Rank Reduction and Random Matrix Theory." 2024.
|
|
|
|
```bibtex
|
|
@article{sharma2023truth,
|
|
title={The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction},
|
|
author={Sharma, Pratyusha and Ash, Jordan T and Misra, Dipendra},
|
|
journal={arXiv preprint arXiv:2312.13558},
|
|
year={2023} }
|
|
```
|
|
|
|
```bibtex
|
|
@article{gao2021framework,
|
|
title={A framework for few-shot language model evaluation},
|
|
author={Gao, Leo and Tow, Jonathan and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and McDonell, Kyle and Muennighoff, Niklas and others},
|
|
journal={Version v0. 0.1. Sept},
|
|
year={2021}
|
|
}
|
|
```
|
|
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
|
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_macadeliccc__laser-dolphin-mixtral-4x7b-dpo)
|
|
|
|
| Metric |Value|
|
|
|---------------------------------|----:|
|
|
|Avg. |66.71|
|
|
|AI2 Reasoning Challenge (25-Shot)|64.93|
|
|
|HellaSwag (10-Shot) |85.81|
|
|
|MMLU (5-Shot) |63.04|
|
|
|TruthfulQA (0-shot) |63.77|
|
|
|Winogrande (5-shot) |77.82|
|
|
|GSM8k (5-shot) |44.88|
|
|
|