53 lines
1.5 KiB
Markdown
53 lines
1.5 KiB
Markdown
---
|
|
library_name: transformers
|
|
license: apache-2.0
|
|
---
|
|
|
|
# Model Card for NeuralHermes 2.5 - Mistral 7B
|
|
|
|
|
|
NeuralHermes is based on the teknium/OpenHermes-2.5-Mistral-7B model that has been further fine-tuned with Direct Preference Optimization (DPO) using the Intel/orca_dpo_pairs dataset, reformatted with the ChatML template.
|
|
|
|
It is directly inspired by the RLHF process described by Intel/neural-chat-7b-v3-1's authors to improve performance.
|
|
|
|
|
|
**IMPORTANT**
|
|
|
|
- This model was only run for 2 steps before GPU went out of memory. Hence, this is not completely fine-tuned with DPO.
|
|
- Secondly, to make it run over a small GPU, I purposefully reduced the parameters (# of LORA adapters, alpha, etc.). The values are therefore not the ideal.
|
|
|
|
|
|
|
|
## Uses
|
|
|
|
You can use the following code to use this model:
|
|
|
|
|
|
import transformers
|
|
from transformers import AutoTokenizer
|
|
|
|
# Format prompt
|
|
message = [
|
|
{"role": "system", "content": "You are a helpful assistant chatbot."},
|
|
{"role": "user", "content": "What is a Large Language Model?"}
|
|
]
|
|
tokenizer = AutoTokenizer.from_pretrained(new_model)
|
|
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)
|
|
|
|
# Create pipeline
|
|
pipeline = transformers.pipeline(
|
|
"text-generation",
|
|
model=new_model,
|
|
tokenizer=tokenizer
|
|
)
|
|
|
|
# Generate text
|
|
sequences = pipeline(
|
|
prompt,
|
|
do_sample=True,
|
|
temperature=0.7,
|
|
top_p=0.9,
|
|
num_return_sequences=1,
|
|
max_length=200,
|
|
)
|
|
print(sequences[0]['generated_text']) |