Llama-160M-Chat-v1-GGUF/README.md

base_model, datasets, inference, license, model_creator, model_name, pipeline_tag, quantized_by, tags, widget

base_model	datasets	inference	license	model_creator	model_name	pipeline_tag	quantized_by	tags	widget
Felladrin/Llama-160M-Chat-v1	ehartford/wizard_vicuna_70k_unfiltered totally-not-an-llm/EverythingLM-data-V3 Open-Orca/SlimOrca-Dedup databricks/databricks-dolly-15k THUDM/webglm-qa	false	other	Felladrin	Llama-160M-Chat-v1	text-generation	afrideva	text-generation gguf ggml quantized q2_k q3_k_m q4_k_m q5_k_m q6_k q8_0	text <|im_start|>system You are a helpful assistant, who answers with empathy.<|im_end|> <|im_start|>user Got a question for you!<|im_end|> <|im_start|>assistant Sure! What's it?<|im_end|> <|im_start|>user Why do you love cats so much!? 🐈<|im_end|> <|im_start|>assistant text <|im_start|>system You are a helpful assistant who answers user's questions with empathy.<|im_end|> <|im_start|>user Who is Mona Lisa?<|im_end|> <|im_start|>assistant text <|im_start|>system You are a helpful assistant who provides concise responses.<|im_end|> <|im_start|>user Heya!<|im_end|> <|im_start|>assistant Hi! How may I help you today?<|im_end|> <|im_start|>user I need to build a simple website. Where should I start learning about web development?<|im_end|> <|im_start|>assistant text <|im_start|>user Invited some friends to come home today. Give me some ideas for games to play with them!<|im_end|> <|im_start|>assistant text <|im_start|>system You are a helpful assistant who answers user's questions with details and curiosity.<|im_end|> <|im_start|>user What are some potential applications for quantum computing?<|im_end|> <|im_start|>assistant text <|im_start|>system You are a helpful assistant who gives creative responses.<|im_end|> <|im_start|>user Write the specs of a game about mages in a fantasy world.<|im_end|> <|im_start|>assistant text <|im_start|>system You are a helpful assistant who answers user's questions with details.<|im_end|> <|im_start|>user Tell me about the pros and cons of social media.<|im_end|> <|im_start|>assistant text <|im_start|>system You are a helpful assistant who answers user's questions with confidence.<|im_end|> <|im_start|>user What is a dog?<|im_end|> <|im_start|>assistant A dog is a four-legged, domesticated animal that is a member of the class Mammalia, which includes all mammals. Dogs are known for their loyalty, playfulness, and ability to be trained for various tasks. They are also used for hunting, herding, and as service animals.<|im_end|> <|im_start|>user What is the color of an apple?<|im_end|> <|im_start|>assistant

Felladrin/Llama-160M-Chat-v1-GGUF

Quantized GGUF model files for Llama-160M-Chat-v1 from Felladrin

Name	Quant method	Size
llama-160m-chat-v1.fp16.gguf	fp16	326.58 MB
llama-160m-chat-v1.q2_k.gguf	q2_k	77.23 MB
llama-160m-chat-v1.q3_k_m.gguf	q3_k_m	87.54 MB
llama-160m-chat-v1.q4_k_m.gguf	q4_k_m	104.03 MB
llama-160m-chat-v1.q5_k_m.gguf	q5_k_m	119.04 MB
llama-160m-chat-v1.q6_k.gguf	q6_k	135.00 MB
llama-160m-chat-v1.q8_0.gguf	q8_0	174.33 MB

Original Model Card:

A Llama Chat Model of 160M Parameters

• Base model: JackFram/llama-160m
• Datasets:
  • ehartford/wizard_vicuna_70k_unfiltered
  • totally-not-an-llm/EverythingLM-data-V3
  • Open-Orca/SlimOrca-Dedup
  • databricks/databricks-dolly-15k
  • THUDM/webglm-qa
• Availability in other ML formats:
  • GGUF: Felladrin/gguf-Llama-160M-Chat-v1
  • ONNX: Felladrin/onnx-Llama-160M-Chat-v1

Recommended Prompt Format

The recommended prompt format is as follows:

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant

Recommended Inference Parameters

To get the best results, prefer using contrastive search for inference:

penalty_alpha: 0.5
top_k: 5