llama-2-7b-miniguanaco/README.md

license

license
apache-2.0

llama-2-7b-miniguanaco

This is my first model, with LLama-2-7b model finetuned with miniguanaco datasets.

This is a simple finetune based off a Google Colab notebook. Finetune instructions were from Labonne's first tutorial.

To run it: import torch from transformers import AutoTokenizer, AutoModelForCausalLM import math

model_path = "decruz07/llama-2-7b-miniguanaco"

tokenizer = AutoTokenizer.from_pretrained(model_path, use_default_system_prompt=False) model = AutoModelForCausalLM.from_pretrained( model_path, torch_dtype=torch.float32, device_map='auto',local_files_only=False, load_in_4bit=True ) print(model) prompt = input("please input prompt:") while len(prompt) > 0: input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to("cuda")

generation_output = model.generate( input_ids=input_ids, max_new_tokens=500,repetition_penalty=1.2 ) print(tokenizer.decode(generation_output[0])) prompt = input("please input prompt:")