初始化项目，由ModelHub XC社区提供模型
Model: sunkencity/Llama-3.1-8B-Blasphemer-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
38
.gitattributes
vendored
Normal file
38
.gitattributes
vendored
Normal file
@@ -0,0 +1,38 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
Llama-3.1-8B-Blasphemer-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Llama-3.1-8B-Blasphemer-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Llama-3.1-8B-Blasphemer-F16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
Llama-3.1-8B-Blasphemer-F16.gguf
Normal file
3
Llama-3.1-8B-Blasphemer-F16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c5c0b387b903ef632ba9fb9ca1cf4ee2554c8b5b38ca566ffda7bcc5df07abf5
|
||||
size 16068895680
|
||||
3
Llama-3.1-8B-Blasphemer-Q4_K_M.gguf
Normal file
3
Llama-3.1-8B-Blasphemer-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7f167cfc91df876667ed4e12e6e5d1f09ec8009aa5f20cad0b0a1dd840e98fd7
|
||||
size 4920738752
|
||||
3
Llama-3.1-8B-Blasphemer-Q5_K_M.gguf
Normal file
3
Llama-3.1-8B-Blasphemer-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:627fa849290e50fe1f216a36ff1d5395bef866c6a07658141d75a142aec19e9c
|
||||
size 5732991936
|
||||
126
README.md
Normal file
126
README.md
Normal file
@@ -0,0 +1,126 @@
|
||||
---
|
||||
base_model: meta-llama/Llama-3.1-8B-Instruct
|
||||
tags:
|
||||
- llama-3.1
|
||||
- gguf
|
||||
- abliteration
|
||||
- uncensored
|
||||
- blasphemer
|
||||
license: llama3.1
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# Llama 3.1 8B Instruct - Blasphemer (GGUF)
|
||||
|
||||
This is an uncensored version of Meta's Llama 3.1 8B Instruct, processed using [Blasphemer](https://github.com/sunkencity999/blasphemer). This model will now deliver
|
||||
Fully uncensored outputs. Make adjustments to temperature as necessary for your own use-case. It has an extremely low refusal rate; just one follow-up is often enough
|
||||
to break refusal and receive previously censored output when a refusal Does appear.
|
||||
|
||||
In testing I found this model to function best at .7+ temperature for tool-calling.
|
||||
|
||||
|
||||
## Model Details
|
||||
|
||||
- **Base Model**: meta-llama/Llama-3.1-8B-Instruct
|
||||
- **Method**: Abliteration (refusal direction removal)
|
||||
- **Format**: GGUF (for llama.cpp, LM Studio, etc.)
|
||||
- **Quality Metrics**:
|
||||
- Refusals: 3/100 (3%) ⭐ Excellent
|
||||
- KL Divergence: 0.06 ⭐ Excellent
|
||||
- Trial: #168 of 200
|
||||
|
||||
## Quantization Versions
|
||||
|
||||
| File | Size | Use Case |
|
||||
|------|------|----------|
|
||||
| Q4_K_M | ~4.5GB | Best balance - most popular |
|
||||
| Q5_K_M | ~5.5GB | Higher quality, slightly larger |
|
||||
| F16 | ~15GB | Full precision (for further quantization) |
|
||||
|
||||
## Usage
|
||||
|
||||
### LM Studio
|
||||
|
||||
1. Download the GGUF file
|
||||
2. Open LM Studio
|
||||
3. Click "Import Model"
|
||||
4. Select the downloaded file
|
||||
5. Start chatting!
|
||||
|
||||
### llama.cpp
|
||||
|
||||
```bash
|
||||
./llama-cli -m Llama-3.1-8B-Blasphemer-Q4_K_M.gguf -p "Your prompt here"
|
||||
```
|
||||
|
||||
### Python (llama-cpp-python)
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama(
|
||||
model_path="Llama-3.1-8B-Blasphemer-Q4_K_M.gguf",
|
||||
n_ctx=8192,
|
||||
n_gpu_layers=-1 # Use GPU
|
||||
)
|
||||
|
||||
response = llm("Your prompt here", max_tokens=512)
|
||||
print(response['choices'][0]['text'])
|
||||
```
|
||||
|
||||
## What is Abliteration?
|
||||
|
||||
Abliteration removes refusal behavior from language models by identifying and removing the neural directions responsible for safety alignment. This is done through:
|
||||
|
||||
1. Calculating refusal directions from harmful/harmless prompt pairs
|
||||
2. Using Bayesian optimization (TPE) to find optimal removal parameters
|
||||
3. Orthogonalizing model weights to these directions
|
||||
|
||||
The result is a model that maintains capabilities while removing refusal behavior.
|
||||
|
||||
## Ethical Considerations
|
||||
|
||||
This model has massively reduced safety guardrails. Users are responsible for:
|
||||
- Ensuring ethical use of the model
|
||||
- Compliance with applicable laws and regulations
|
||||
- Understanding the implications of reduced safety filtering
|
||||
|
||||
## Performance
|
||||
|
||||
Compared to the original Llama 3.1 8B Instruct:
|
||||
- Follows instructions more directly
|
||||
- Responds to previously refused queries
|
||||
- Maintains general capabilities (KL divergence: 0.06)
|
||||
- Greatly Reduced safety filtering
|
||||
|
||||
## Credits
|
||||
|
||||
- **Base Model**: Meta AI (Llama 3.1)
|
||||
- **Abliteration Tool**: [Blasphemer](https://github.com/sunkencity999/blasphemer) by Christopher Bradford
|
||||
- **Method**: Based on "Refusal in Language Models Is Mediated by a Single Direction" (Arditi et al., 2024)
|
||||
|
||||
## Citation
|
||||
|
||||
If you use this model, please cite:
|
||||
|
||||
```bibtex
|
||||
@software{blasphemer2024,
|
||||
author = {Bradford, Christopher},
|
||||
title = {Blasphemer: Abliteration for Language Models},
|
||||
year = {2024},
|
||||
url = {https://github.com/sunkencity999/blasphemer}
|
||||
}
|
||||
|
||||
@article{arditi2024refusal,
|
||||
title={Refusal in Language Models Is Mediated by a Single Direction},
|
||||
author={Arditi, Andy and Obmann, Oscar and Syed, Aaquib and others},
|
||||
journal={arXiv preprint arXiv:2406.11717},
|
||||
year={2024}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
This model inherits the Llama 3.1 license from Meta AI. Please review the [Llama 3.1 License](https://ai.meta.com/llama/license/) for usage terms.
|
||||
Reference in New Issue
Block a user