---
library_name: transformers
tags:
- cyber
- red-team
- cybersecuirty
license: apache-2.0
datasets:
- loaiabdalslam/Alexander-Cyber-Dataset-v2

name: alexander-cyber-qlora

channels:
  - conda-forge
  - defaults

dependencies:
  - python=3.12
  - pip
  - numpy
  - sentencepiece
  - pip:
      - "torch"
      - "transformers>=4.56"
      - "datasets>=3.0"
      - "accelerate>=1.0"
      - "peft>=0.17"
      - "trl>=0.27"
      - "bitsandbytes>=0.46.1"
      - "huggingface_hub>=0.34"
      - "safetensors"
---
# Alexander Cyber Qwen 

![8a04d680-5f17-4bc4-a48b-bff1a5354cc2](https://cdn-uploads.huggingface.co/production/uploads/64c4620f3e80e593568f3875/QemZjORFlqKl_4xUq539H.png)

Fine-tuning workflow for **Alexander Cyber**, an authorized red-team and cybersecurity copilot, using **QLoRA**, Hugging Face Transformers, PEFT, TRL, and bitsandbytes.

The training notebook loads a chat-formatted JSONL dataset, validates the message structure, quantizes the base model to 4-bit NF4, trains a LoRA adapter, saves/pushes the adapter, and optionally merges the adapter back into the base model.

## Project Overview

The current notebook is configured to use:

* Base model: `Qwen/Qwen2.5-0.5B-Instruct`
* Training method: QLoRA
* Quantization: 4-bit NF4 with double quantization
* Trainer: `trl.SFTTrainer`
* LoRA rank: `32`
* LoRA alpha: `64`
* LoRA dropout: `0.05`
* Optimizer: `paged_adamw_8bit`
* Maximum training steps: `100`
* Training sequence length: `1536`
* Seed: `279`

> Note: the notebook variable is named `USE_QWEN3_4B`, but when it is `True` the selected model is currently `Qwen/Qwen2.5-0.5B-Instruct`. The README preserves the behavior of the notebook as written.

## Requirements

A CUDA-capable NVIDIA GPU is recommended because the notebook uses 4-bit quantization through `bitsandbytes`.

Main dependencies:

* Python 3.12
* PyTorch
* Transformers >= 4.56
* Datasets >= 3.0
* Accelerate >= 1.0
* PEFT >= 0.17
* TRL >= 0.27
* bitsandbytes >= 0.46.1
* huggingface_hub >= 0.34
* sentencepiece
* safetensors

## Environment Setup

Create the Conda environment:

```bash
conda env create -f environment.yml
conda activate alexander-cyber-qlora
```

Verify GPU support:

```python
import torch

print("CUDA available:", torch.cuda.is_available())

if torch.cuda.is_available():
    print("GPU:", torch.cuda.get_device_name(0))
    print("BF16 supported:", torch.cuda.is_bf16_supported())
```

## Hugging Face Login

```python
from huggingface_hub import notebook_login, whoami

notebook_login()
print(whoami())
```

## Dataset

The notebook expects JSONL files for training and validation.

Current Kaggle paths:

```text
/kaggle/input/datasets/loaiabdalslam/alexander-cyber/alexander_cyber_v2_train.jsonl
/kaggle/input/datasets/loaiabdalslam/alexander-cyber/alexander_cyber_v2_validation.jsonl
/kaggle/input/datasets/loaiabdalslam/alexander-cyber/alexander_cyber_v2_benchmark.jsonl
```

Each example contains a `messages` list:

```json
{
  "messages": [
    {
      "role": "system",
      "content": "You are Alexander Cyber, an authorized red-team and cybersecurity copilot."
    },
    {
      "role": "user",
      "content": "Analyze this security finding."
    },
    {
      "role": "assistant",
      "content": "Start by validating the evidence and confirming the affected service."
    }
  ]
}
```
## 4-bit QLoRA

The model is loaded using bitsandbytes NF4 quantization:

```python
from transformers import BitsAndBytesConfig
import torch

compute_dtype = (
    torch.bfloat16
    if torch.cuda.is_bf16_supported()
    else torch.float16
)

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=compute_dtype,
)
```

## Training

Current training configuration:

```text
max_steps                   = 100
learning_rate               = 2e-4
per_device_train_batch_size = 1
per_device_eval_batch_size  = 1
gradient_accumulation_steps = 1
warmup_steps                = 20
lr_scheduler_type           = cosine
eval_steps                  = 50
save_steps                  = 100
max_length                  = 1536
packing                     = True
optimizer                   = paged_adamw_8bit
max_grad_norm               = 0.3
weight_decay                = 0.01
seed                        = 279
```