184 lines
4.2 KiB
Markdown
184 lines
4.2 KiB
Markdown
---
|
|
library_name: transformers
|
|
tags:
|
|
- cyber
|
|
- red-team
|
|
- cybersecuirty
|
|
license: apache-2.0
|
|
datasets:
|
|
- loaiabdalslam/Alexander-Cyber-Dataset-v2
|
|
|
|
name: alexander-cyber-qlora
|
|
|
|
channels:
|
|
- conda-forge
|
|
- defaults
|
|
|
|
dependencies:
|
|
- python=3.12
|
|
- pip
|
|
- numpy
|
|
- sentencepiece
|
|
- pip:
|
|
- "torch"
|
|
- "transformers>=4.56"
|
|
- "datasets>=3.0"
|
|
- "accelerate>=1.0"
|
|
- "peft>=0.17"
|
|
- "trl>=0.27"
|
|
- "bitsandbytes>=0.46.1"
|
|
- "huggingface_hub>=0.34"
|
|
- "safetensors"
|
|
---
|
|
# Alexander Cyber Qwen
|
|
|
|
|
|
|
|
Fine-tuning workflow for **Alexander Cyber**, an authorized red-team and cybersecurity copilot, using **QLoRA**, Hugging Face Transformers, PEFT, TRL, and bitsandbytes.
|
|
|
|
The training notebook loads a chat-formatted JSONL dataset, validates the message structure, quantizes the base model to 4-bit NF4, trains a LoRA adapter, saves/pushes the adapter, and optionally merges the adapter back into the base model.
|
|
|
|
## Project Overview
|
|
|
|
The current notebook is configured to use:
|
|
|
|
* Base model: `Qwen/Qwen2.5-0.5B-Instruct`
|
|
* Training method: QLoRA
|
|
* Quantization: 4-bit NF4 with double quantization
|
|
* Trainer: `trl.SFTTrainer`
|
|
* LoRA rank: `32`
|
|
* LoRA alpha: `64`
|
|
* LoRA dropout: `0.05`
|
|
* Optimizer: `paged_adamw_8bit`
|
|
* Maximum training steps: `100`
|
|
* Training sequence length: `1536`
|
|
* Seed: `279`
|
|
|
|
> Note: the notebook variable is named `USE_QWEN3_4B`, but when it is `True` the selected model is currently `Qwen/Qwen2.5-0.5B-Instruct`. The README preserves the behavior of the notebook as written.
|
|
|
|
## Requirements
|
|
|
|
A CUDA-capable NVIDIA GPU is recommended because the notebook uses 4-bit quantization through `bitsandbytes`.
|
|
|
|
Main dependencies:
|
|
|
|
* Python 3.12
|
|
* PyTorch
|
|
* Transformers >= 4.56
|
|
* Datasets >= 3.0
|
|
* Accelerate >= 1.0
|
|
* PEFT >= 0.17
|
|
* TRL >= 0.27
|
|
* bitsandbytes >= 0.46.1
|
|
* huggingface_hub >= 0.34
|
|
* sentencepiece
|
|
* safetensors
|
|
|
|
## Environment Setup
|
|
|
|
Create the Conda environment:
|
|
|
|
```bash
|
|
conda env create -f environment.yml
|
|
conda activate alexander-cyber-qlora
|
|
```
|
|
|
|
Verify GPU support:
|
|
|
|
```python
|
|
import torch
|
|
|
|
print("CUDA available:", torch.cuda.is_available())
|
|
|
|
if torch.cuda.is_available():
|
|
print("GPU:", torch.cuda.get_device_name(0))
|
|
print("BF16 supported:", torch.cuda.is_bf16_supported())
|
|
```
|
|
|
|
## Hugging Face Login
|
|
|
|
```python
|
|
from huggingface_hub import notebook_login, whoami
|
|
|
|
notebook_login()
|
|
print(whoami())
|
|
```
|
|
|
|
## Dataset
|
|
|
|
The notebook expects JSONL files for training and validation.
|
|
|
|
Current Kaggle paths:
|
|
|
|
```text
|
|
/kaggle/input/datasets/loaiabdalslam/alexander-cyber/alexander_cyber_v2_train.jsonl
|
|
/kaggle/input/datasets/loaiabdalslam/alexander-cyber/alexander_cyber_v2_validation.jsonl
|
|
/kaggle/input/datasets/loaiabdalslam/alexander-cyber/alexander_cyber_v2_benchmark.jsonl
|
|
```
|
|
|
|
Each example contains a `messages` list:
|
|
|
|
```json
|
|
{
|
|
"messages": [
|
|
{
|
|
"role": "system",
|
|
"content": "You are Alexander Cyber, an authorized red-team and cybersecurity copilot."
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": "Analyze this security finding."
|
|
},
|
|
{
|
|
"role": "assistant",
|
|
"content": "Start by validating the evidence and confirming the affected service."
|
|
}
|
|
]
|
|
}
|
|
```
|
|
## 4-bit QLoRA
|
|
|
|
The model is loaded using bitsandbytes NF4 quantization:
|
|
|
|
```python
|
|
from transformers import BitsAndBytesConfig
|
|
import torch
|
|
|
|
compute_dtype = (
|
|
torch.bfloat16
|
|
if torch.cuda.is_bf16_supported()
|
|
else torch.float16
|
|
)
|
|
|
|
bnb = BitsAndBytesConfig(
|
|
load_in_4bit=True,
|
|
bnb_4bit_quant_type="nf4",
|
|
bnb_4bit_use_double_quant=True,
|
|
bnb_4bit_compute_dtype=compute_dtype,
|
|
)
|
|
```
|
|
|
|
## Training
|
|
|
|
Current training configuration:
|
|
|
|
```text
|
|
max_steps = 100
|
|
learning_rate = 2e-4
|
|
per_device_train_batch_size = 1
|
|
per_device_eval_batch_size = 1
|
|
gradient_accumulation_steps = 1
|
|
warmup_steps = 20
|
|
lr_scheduler_type = cosine
|
|
eval_steps = 50
|
|
save_steps = 100
|
|
max_length = 1536
|
|
packing = True
|
|
optimizer = paged_adamw_8bit
|
|
max_grad_norm = 0.3
|
|
weight_decay = 0.01
|
|
seed = 279
|
|
```
|
|
|
|
|