初始化项目,由ModelHub XC社区提供模型
Model: nbeerbower/A0l-12B-heretic Source: Original Platform
This commit is contained in:
48
README.md
Normal file
48
README.md
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
datasets:
|
||||
- schneewolflabs/Athanorlite-DPO
|
||||
base_model:
|
||||
- nbeerbower/Schreiber-mistral-nemo-12B
|
||||
tags:
|
||||
- heretic
|
||||
- uncensored
|
||||
- decensored
|
||||
- abliterated
|
||||
---
|
||||
# This is a decensored version of [schneewolflabs/A0l-12B](https://huggingface.co/schneewolflabs/A0l-12B), made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0
|
||||
|
||||
## Abliteration parameters
|
||||
|
||||
| Parameter | Value |
|
||||
| :-------- | :---: |
|
||||
| **direction_index** | per layer |
|
||||
| **attn.o_proj.max_weight** | 1.24 |
|
||||
| **attn.o_proj.max_weight_position** | 33.41 |
|
||||
| **attn.o_proj.min_weight** | 0.89 |
|
||||
| **attn.o_proj.min_weight_distance** | 22.95 |
|
||||
| **mlp.down_proj.max_weight** | 1.47 |
|
||||
| **mlp.down_proj.max_weight_position** | 25.98 |
|
||||
| **mlp.down_proj.min_weight** | 0.00 |
|
||||
| **mlp.down_proj.min_weight_distance** | 12.82 |
|
||||
|
||||
## Performance
|
||||
|
||||
| Metric | This model | Original model ([schneewolflabs/A0l-12B](https://huggingface.co/schneewolflabs/A0l-12B)) |
|
||||
| :----- | :--------: | :---------------------------: |
|
||||
| **KL divergence** | 0.0364 | 0 *(by definition)* |
|
||||
| **Refusals** | 10/100 | 40/100 |
|
||||
|
||||
-----
|
||||
|
||||
|
||||
# A0l-12B
|
||||
|
||||
Same training run as [schneewolflabs/A0-12B](https://huggingface.co/schneewolflabs/A0-12B), but using [schneewolflabs/Athanorlite-DPO](https://huggingface.co/schneewolflabs/Athanorlite-DPO) as the dataset.
|
||||
|
||||
Preliminary tests have shown this model has superior writing capabilities to A0-12B.
|
||||
|
||||
## Configuration
|
||||
|
||||

|
||||
Reference in New Issue
Block a user