103 lines
2.9 KiB
Markdown
103 lines
2.9 KiB
Markdown
|
---
|
||
|
|
datasets:
|
||
|
|
- Alindstroem89/guardrail-training-dataset
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
base_model:
|
||
|
|
- unsloth/Llama-3.2-3B-Instruct
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- gguf
|
||
|
|
- llama.cpp
|
||
|
|
- unsloth
|
||
|
|
---
|
||
|
|
|
||
|
|
|
||
|
|
# Llama-3.2-3B-Instruct_guardrail : GGUF
|
||
|
|
|
||
|
|
A fine-tuned Llama 3.2 model trained to resist prompt injection attacks. This model was created for the [Prompt Injection Challenge](https://github.com/Alexanderl89/Guardrail_finetuning) - an AI security challenge where users attempt to extract a hidden flag from a chatbot using prompt injection and social engineering techniques.
|
||
|
|
|
||
|
|
This model was fine-tuned and converted to GGUF format using [Unsloth](https://github.com/unslothai/unsloth).
|
||
|
|
|
||
|
|
## Model Description
|
||
|
|
|
||
|
|
Fine-tuned to:
|
||
|
|
- Recognize and resist prompt injection techniques
|
||
|
|
- Maintain boundaries and refuse to reveal protected information
|
||
|
|
- Remain helpful and friendly for legitimate conversations
|
||
|
|
- Politely explain refusals without being unnecessarily rigid
|
||
|
|
|
||
|
|
## Training Details
|
||
|
|
|
||
|
|
**Base Model:** [unsloth/Llama-3.2-3B-Instruct](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)
|
||
|
|
|
||
|
|
**Training Configuration:**
|
||
|
|
- LoRA Rank (r): 32
|
||
|
|
- LoRA Alpha: 32
|
||
|
|
- Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
||
|
|
- Use RSLoRA: True
|
||
|
|
- Optimizer: adamw_8bit
|
||
|
|
- Learning Rate: 1e-4
|
||
|
|
- Batch Size: 2 per device
|
||
|
|
- Gradient Accumulation: 8 steps
|
||
|
|
- Epochs: 1
|
||
|
|
- Max Sequence Length: 8192
|
||
|
|
|
||
|
|
**Dataset:** Custom dataset with guardrail conversations (prompt injection attempts with refusals) and normal helpful conversations.
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
### With llama-cli
|
||
|
|
|
||
|
|
```bash
|
||
|
|
llama-cli -hf Alindstroem89/Llama-3.2-3B-Instruct_guardrail:F16 --jinja
|
||
|
|
```
|
||
|
|
|
||
|
|
### Download with Hugging Face CLI
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Download all GGUF files
|
||
|
|
hf download Alindstroem89/Llama-3.2-3B-Instruct_guardrail --include "*.gguf" --local-dir ./models
|
||
|
|
|
||
|
|
# Download specific quantization
|
||
|
|
hf download Alindstroem89/Llama-3.2-3B-Instruct_guardrail --include "Llama-3.2-3B-Instruct.Q4_K_M.gguf" --local-dir ./models
|
||
|
|
```
|
||
|
|
|
||
|
|
### Ollama
|
||
|
|
|
||
|
|
An Ollama Modelfile is included for easy deployment.
|
||
|
|
|
||
|
|
## Available Model Files
|
||
|
|
|
||
|
|
- Llama-3.2-3B-Instruct.Q3_K_M.gguf
|
||
|
|
- Llama-3.2-3B-Instruct.Q4_K_M.gguf
|
||
|
|
- Llama-3.2-3B-Instruct.F16.gguf
|
||
|
|
- Llama-3.2-3B-Instruct.BF16.gguf
|
||
|
|
|
||
|
|
## Use Cases
|
||
|
|
|
||
|
|
- Chatbots requiring prompt injection resistance
|
||
|
|
- AI assistants handling sensitive information
|
||
|
|
- AI security research and education
|
||
|
|
- Testing guardrail implementations
|
||
|
|
|
||
|
|
## Limitations
|
||
|
|
|
||
|
|
- Primarily tested on English language
|
||
|
|
- Not a comprehensive security solution
|
||
|
|
- May occasionally be overly cautious
|
||
|
|
- Should not be the sole defense mechanism in production
|
||
|
|
|
||
|
|
## Training Infrastructure
|
||
|
|
|
||
|
|
- Framework: [Unsloth](https://github.com/unslothai/unsloth) (2x faster training)
|
||
|
|
- Method: LoRA (Low-Rank Adaptation) with rank-stabilized optimization
|
||
|
|
- Conversion: GGUF format for efficient inference
|
||
|
|
|
||
|
|
## Finetuning repo
|
||
|
|
|
||
|
|
[Guardrail_finetuning](https://github.com/Alexanderl89/Guardrail_finetuning)
|
||
|
|
|
||
|
|
## License
|
||
|
|
|
||
|
|
This model follows the license of the base Llama 3.2 model.
|