36 lines
1.4 KiB
Markdown
36 lines
1.4 KiB
Markdown
|
---
|
||
|
|
license: other
|
||
|
|
license_name: hugston-licenced
|
||
|
|
license_link: LICENSE
|
||
|
|
---
|
||
|
|
This is an Abliterated version of LMF2.5-350M using a modified version of Prometheus, then using Quanta and HugstonOne.
|
||
|
|
|
||
|
|
|
||
|
|
The aim is to understand the safety mechanism of different llm models for research purposes.
|
||
|
|
|
||
|
|
Here we show proof of concept of how we can change the model behaviour preserving accuracy and lowering the refusal rate
|
||
|
|
with very few trial which run in relatively small datasets.
|
||
|
|
As a matter of fact it can run in a cheap laptop in cpu narrowing it down to 5-20 min for a small model.
|
||
|
|
|
||
|
|
6 trials Refusals: 10/1000, KL divergence: 0.2577
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Credit to LiquidAI for the model creation
|
||
|
|
# Credit to https://huggingface.co/wangzhang for abliteration method
|
||
|
|
# Credit to LLama.cpp team for the great contribution
|
||
|
|
# Credit to Hugston Team for Abliteration, Converting, Quantizing, Testing, Benching and other...
|
||
|
|
# Credit to Huggingface for the amazing hosting platform
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
# Keep away from children
|
||
|
|
|
||
|
|
Here we show the behaviour running the model in HugstonOne (the 0.8b, as an example).
|
||
|
|
|
||
|
|
|
||
|
|
The quantization in GGUF was made in f32 for beter quants.
|
||
|
|
Here we show Quanta our convertor and Quantizer tool.
|
||
|
|
|
||
|
|
|