Llama-68M-Chat-v1-GGUF/README.md

base_model, datasets, inference, language, license, model_creator, model_name, pipeline_tag, quantized_by, tags, widget

base_model	datasets	inference	language	license	model_creator	model_name	pipeline_tag	quantized_by	tags	widget
Felladrin/Llama-68M-Chat-v1	THUDM/webglm-qa databricks/databricks-dolly-15k cognitivecomputations/wizard_vicuna_70k_unfiltered totally-not-an-llm/EverythingLM-data-V3 Amod/mental_health_counseling_conversations sablo/oasst2_curated starfishmedical/webGPT_x_dolly Open-Orca/OpenOrca mlabonne/chatml_dpo_pairs	false	en	apache-2.0	Felladrin	Llama-68M-Chat-v1	text-generation	afrideva	text-generation gguf ggml quantized q2_k q3_k_m q4_k_m q5_k_m q6_k q8_0	text <|im_start|>system You are a knowledgeable assistant. Help the user as much as you can.<|im_end|> <|im_start|>user How to become healthier?<|im_end|> <|im_start|>assistant text <|im_start|>system You are a career counselor. The user will provide you with an individual looking for guidance in their professional life, and your task is to assist them in determining what careers they are most suited for based on their skills, interests, and experience. You should also conduct research into the various options available, explain the job market trends in different industries, and advice on which qualifications would be beneficial for pursuing particular fields.<|im_end|> <|im_start|>user Heya!<|im_end|> <|im_start|>assistant Hi! How may I help you?<|im_end|> <|im_start|>user I am interested in developing a career in software engineering. What would you recommend me to do?<|im_end|> <|im_start|>assistant text <|im_start|>system You are a helpful assistant who provides concise responses.<|im_end|> <|im_start|>user Hi!<|im_end|> <|im_start|>assistant Hello there! How may I help you?<|im_end|> <|im_start|>user I need to build a simple website. Where should I start learning about web development?<|im_end|> <|im_start|>assistant text <|im_start|>system You are a very creative assistant. User will give you a task, which you should complete with all your knowledge.<|im_end|> <|im_start|>user Write the background story of an RPG game about wizards and dragons in a sci-fi world.<|im_end|> <|im_start|>assistant

Felladrin/Llama-68M-Chat-v1-GGUF

Quantized GGUF model files for Llama-68M-Chat-v1 from Felladrin

Name	Quant method	Size
llama-68m-chat-v1.fp16.gguf	fp16	136.79 MB
llama-68m-chat-v1.q2_k.gguf	q2_k	35.88 MB
llama-68m-chat-v1.q3_k_m.gguf	q3_k_m	40.66 MB
llama-68m-chat-v1.q4_k_m.gguf	q4_k_m	46.10 MB
llama-68m-chat-v1.q5_k_m.gguf	q5_k_m	51.16 MB
llama-68m-chat-v1.q6_k.gguf	q6_k	56.54 MB
llama-68m-chat-v1.q8_0.gguf	q8_0	73.02 MB

Original Model Card:

A Llama Chat Model of 68M Parameters

• Base model: JackFram/llama-68m
• Datasets:
  • THUDM/webglm-qa
  • databricks/databricks-dolly-15k
  • cognitivecomputations/wizard_vicuna_70k_unfiltered
  • totally-not-an-llm/EverythingLM-data-V3
  • Amod/mental_health_counseling_conversations
  • sablo/oasst2_curated
  • starfishmedical/webGPT_x_dolly
  • Open-Orca/OpenOrca
  • mlabonne/chatml_dpo_pairs
• Availability in other ML formats:
  • ONNX: Felladrin/onnx-Llama-68M-Chat-v1

Recommended Prompt Format

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant

Recommended Inference Parameters

penalty_alpha: 0.5
top_k: 4