---
license: apache-2.0
base_model: unsloth/Qwen2.5-Coder-7B-Instruct
tags:
- cybersecurity
- security
- cve
- pentesting
- fine-tuned
- qwen2
- unsloth
pipeline_tag: text-generation
---

# Cybersecurity Fine-tuned Qwen2.5-Coder-7B

This model was fine-tuned from `unsloth/Qwen2.5-Coder-7B-Instruct` on cybersecurity datasets using Unsloth + LoRA.

## Training Details

- **Base Model**: unsloth/Qwen2.5-Coder-7B-Instruct
- **Parameters**: 7B
- **Method**: LoRA fine-tuning
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **Training Examples**: 70,000
- **Final Loss**: 0.7485
- **Training Duration**: 31 minutes
- **Hardware**: NVIDIA B200

## Datasets Used

| Dataset | Examples |
|---------|----------|
| omurkuru/cve-security-data | 20,000 |
| Trendyol/Cybersecurity-Instruction | 10,000 |
| ethanolivertroy/nist-cybersecurity | 10,000 |
| Nitral-AI/Cybersecurity-ShareGPT | 10,000 |
| Vanessasml/cybersecurity_32k | 10,000 |
| jason-oneal/pentest-agent-dataset | 10,000 |
| **Total** | **70,000** |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "dennny123/cybersec-qwen2.5-coder-7b",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("dennny123/cybersec-qwen2.5-coder-7b")

messages = [
    {"role": "system", "content": "You are a cybersecurity expert assistant."},
    {"role": "user", "content": "Explain CVE-2024-1234 and its impact"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Capabilities

- CVE vulnerability analysis
- Security log analysis  
- Penetration testing guidance
- NIST compliance knowledge
- Threat detection patterns
- Incident response

## License

Apache 2.0