SteelStorage/Aura-Llama-Abliterated
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1 Commit 1 Branch 0 Tags
main
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

license, tags, base_model, model-index
license tags base_model model-index
apache-2.0
merge
mergekit
failspy/Llama-3-8B-Instruct-abliterated
name results
Aura-Llama-Abliterated
task dataset metrics source
type name
text-generation Text Generation
name type config split args
AI2 Reasoning Challenge (25-Shot) ai2_arc ARC-Challenge test
num_few_shot
25
type value name
acc_norm 49.23 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheSkullery/Aura-Llama-Abliterated Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type split args
HellaSwag (10-Shot) hellaswag validation
num_few_shot
10
type value name
acc_norm 72.27 normalized accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheSkullery/Aura-Llama-Abliterated Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
MMLU (5-Shot) cais/mmlu all test
num_few_shot
5
type value name
acc 55.71 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheSkullery/Aura-Llama-Abliterated Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
TruthfulQA (0-shot) truthful_qa multiple_choice validation
num_few_shot
0
type value
mc2 46.63
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheSkullery/Aura-Llama-Abliterated Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
Winogrande (5-shot) winogrande winogrande_xl validation
num_few_shot
5
type value name
acc 69.3 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheSkullery/Aura-Llama-Abliterated Open LLM Leaderboard
task dataset metrics source
type name
text-generation Text Generation
name type config split args
GSM8k (5-shot) gsm8k main test
num_few_shot
5
type value name
acc 27.6 accuracy
url name
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheSkullery/Aura-Llama-Abliterated Open LLM Leaderboard
<html lang="en"> <head> <style> body { font-family: 'Quicksand', sans-serif; background: linear-gradient(135deg, #2E3440 0%, #1A202C 100%); color: #D8DEE9; margin: 0; padding: 0; font-size: 16px; } .container { width: 100%; max-width: 1440px; margin: 20px auto; background-color: rgba(255, 255, 255, 0.02); padding: 20px; border-radius: 12px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.2); backdrop-filter: blur(10px); border: 1px solid rgba(255, 255, 255, 0.1); } .header h1 { font-size: 28px; color: #ECEFF4; margin: 0 0 20px 0; text-shadow: 2px 2px 4px rgba(0, 0, 0, 0.3); } .update-section { margin-top: 30px; } .update-section h2 { font-size: 24px; color: #88C0D0; } .update-section p { font-size: 16px; line-height: 1.6; color: #ECEFF4; } .info img { width: 100%; border-radius: 10px; margin-bottom: 15px; } a { color: #88C0D0; text-decoration: none; } a:hover { color: #A3BE8C; } pre { background-color: rgba(255, 255, 255, 0.05); padding: 10px; border-radius: 5px; overflow-x: auto; } code { font-family: 'Courier New', monospace; color: #A3BE8C; } </style> </head>

Aura-llama-3-Abliterated

Aura-llama-Abliterated Image

Now that the cute anime girl has your attention.

UPDATE: Model is now using the abliterated version of meta llama 3 8b

Aura-llama is using the methodology presented by SOLAR for scaling LLMs called depth up-scaling (DUS), which encompasses architectural modifications with continued pretraining. Using the solar paper as a base, I integrated Llama-3 weights into the upscaled layers, and In the future plan to continue training the model.

Aura-llama is a merge of the following models to create a base model to work from:

Abliterated Merged Evals (Has Not Been Finetuned):

Aura-llama-Abliterated

  • Avg: ?
  • ARC: ?
  • HellaSwag: ?
  • MMLU: ?
  • T-QA: ?
  • Winogrande: ?
  • GSM8K: ?

Non Abliterated Merged Evals (Has Not Been Finetuned):

Aura-llama-Original

  • Avg: 63.13
  • ARC: 58.02
  • HellaSwag: 77.82
  • MMLU: 65.61
  • T-QA: 51.94
  • Winogrande: 73.40
  • GSM8K: 52.01

🧩 Configuration


dtype: bfloat16
merge_method: passthrough
slices:
- sources:
  - layer_range: [0, 12]
    model: failspy/Llama-3-8B-Instruct-abliterated
- sources:
  - layer_range: [8, 20]
    model: failspy/Llama-3-8B-Instruct-abliterated
- sources:
  - layer_range: [16, 28]
    model: failspy/Llama-3-8B-Instruct-abliterated
- sources:
  - layer_range: [24, 32]
    model: failspy/Llama-3-8B-Instruct-abliterated
</html>

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 53.46
AI2 Reasoning Challenge (25-Shot) 49.23
HellaSwag (10-Shot) 72.27
MMLU (5-Shot) 55.71
TruthfulQA (0-shot) 46.63
Winogrande (5-shot) 69.30
GSM8k (5-shot) 27.60
Description
Model synced from source: SteelStorage/Aura-Llama-Abliterated
Readme 2.6 MiB