Felladrin/Llama-68M-Chat-v1
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1 Commit 1 Branch 0 Tags
main
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

language, license, tags, datasets, base_model, widget, inference, model-index
language license tags datasets base_model widget inference model-index
en
apache-2.0
text-generation
THUDM/webglm-qa
databricks/databricks-dolly-15k
cognitivecomputations/wizard_vicuna_70k_unfiltered
totally-not-an-llm/EverythingLM-data-V3
Amod/mental_health_counseling_conversations
sablo/oasst2_curated
starfishmedical/webGPT_x_dolly
Open-Orca/OpenOrca
mlabonne/chatml_dpo_pairs
JackFram/llama-68m
messages
role content
system You are a career counselor. The user will provide you with an individual looking for guidance in their professional life, and your task is to assist them in determining what careers they are most suited for based on their skills, interests, and experience. You should also conduct research into the various options available, explain the job market trends in different industries, and advice on which qualifications would be beneficial for pursuing particular fields.
role content
user Heya!
role content
assistant Hi! How may I help you?
role content
user I am interested in developing a career in software engineering. What would you recommend me to do?
messages
role content
system You are a knowledgeable assistant. Help the user as much as you can.
role content
user How to become healthier?
messages
role content
system You are a helpful assistant who provides concise responses.
role content
user Hi!
role content
assistant Hello there! How may I help you?
role content
user I need to build a simple website. Where should I start learning about web development?
messages
role content
system You are a very creative assistant. User will give you a task, which you should complete with all your knowledge.
role content
user Write the background story of an RPG game about wizards and dragons in a sci-fi world.
parameters
max_new_tokens penalty_alpha top_k
64 0.5 4
name results
Llama-68M-Chat-v1
task dataset metrics source
type name
text-generation Text Generation
name type config split args
AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test
num_few_shot
25
type value name
acc_norm 23.29 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-68M-Chat-v1 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type split args
HellaSwag (10-Shot) hellaswag validation
num_few_shot
10
type value name
acc_norm 28.27 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-68M-Chat-v1 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
MMLU (5-Shot) cais/mmlu all test
num_few_shot
5
type value name
acc 25.18 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-68M-Chat-v1 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
TruthfulQA (0-shot) truthful_qa multiple_choice validation
num_few_shot
0
type value
mc2 47.27
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-68M-Chat-v1 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
Winogrande (5-shot) winogrande winogrande_xl validation
num_few_shot
5
type value name
acc 54.3 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-68M-Chat-v1 Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
GSM8k (5-shot) gsm8k main test
num_few_shot
5
type value name
acc 0.0 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-68M-Chat-v1 Open LLM Leaderboard

A Llama Chat Model of 68M Parameters

Recommended Prompt Format

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant

Recommended Inference Parameters

penalty_alpha: 0.5
top_k: 4

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 29.72
AI2 Reasoning Challenge (25-Shot) 23.29
HellaSwag (10-Shot) 28.27
MMLU (5-Shot) 25.18
TruthfulQA (0-shot) 47.27
Winogrande (5-shot) 54.30
GSM8k (5-shot) 0.00
Description
Model synced from source: Felladrin/Llama-68M-Chat-v1
Readme 580 KiB