初始化项目,由ModelHub XC社区提供模型
Model: hypnonyx/traffico Source: Original Platform
This commit is contained in:
38
.gitattributes
vendored
Normal file
38
.gitattributes
vendored
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
|
gemma-3-270m-it-traffico.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
hypnonyx_traffico.png filter=lfs diff=lfs merge=lfs -text
|
||||||
202
README.md
Normal file
202
README.md
Normal file
@@ -0,0 +1,202 @@
|
|||||||
|
---
|
||||||
|
language:
|
||||||
|
- it
|
||||||
|
- en
|
||||||
|
license: other
|
||||||
|
license_name: gemma-terms-of-use-and-mitre-attack
|
||||||
|
license_link: https://ai.google.dev/gemma/terms
|
||||||
|
base_model: google/gemma-3-1b-it
|
||||||
|
tags:
|
||||||
|
- cybersecurity
|
||||||
|
- network-security
|
||||||
|
- intrusion-detection
|
||||||
|
- mitre-attack
|
||||||
|
- threat-intelligence
|
||||||
|
- conversational
|
||||||
|
- gemma3_text
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
datasets:
|
||||||
|
- CIC-IDS2017
|
||||||
|
- UNSW-NB15
|
||||||
|
library_name: transformers
|
||||||
|
model-index:
|
||||||
|
- name: traffico
|
||||||
|
results: []
|
||||||
|
---
|
||||||
|
# Traffico - Fine-tuned on ATT&CK Data
|
||||||
|

|
||||||
|
## 📋 Model Description
|
||||||
|
|
||||||
|
Traffico is a fine-tuned language model specialized in analyzing TCP/IP network traffic and detecting cyberattacks. It maps network flow patterns to the MITRE ATT&CK framework, enabling security teams to understand adversary tactics and techniques from network behavior alone.
|
||||||
|
|
||||||
|
The model is trained on synthetic datasets derived from real-world network traffic (CIC-IDS2017 + UNSW-NB15) and enriched with MITRE ATT&CK techniques. It can classify network flows as normal or malicious and provide ATT&CK-mapped threat classifications.
|
||||||
|
|
||||||
|
**Base Model**: Google Gemma 2.7B
|
||||||
|
**Training Data**: Synthetic dataset derived from ATT&CK® techniques, tactics, and procedures (TTPs)
|
||||||
|
**Fine-tuning Approach**: Supervised Fine-Tuning (SFT) using Unsloth for optimization and TRL's SFTTrainer
|
||||||
|
|
||||||
|
## 🎯 Use Cases
|
||||||
|
|
||||||
|
- **Network Intrusion Detection**: Classify network flows as benign or malicious in real-time
|
||||||
|
- **Threat Intelligence**: Map detected attacks to MITRE ATT&CK techniques and tactics
|
||||||
|
- **Security Monitoring**: Analyze TCP/IP flows from network sensors and IDS systems
|
||||||
|
- **Incident Response**: Understand adversary behavior patterns from network telemetry
|
||||||
|
- **Research**: Study attack-to-technique mappings in security datasets
|
||||||
|
|
||||||
|
## 🚀 Quick Start
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||||
|
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained("hypnonyx/Traffico")
|
||||||
|
model = AutoModelForCausalLM.from_pretrained("hypnonyx/Traffico")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Basic Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Analizza un flusso di traffico di rete
|
||||||
|
network_flow = "Protocollo: tcp | Porta dst: 80 | Byte src: 480000 | Byte dst: 40 | Pacchetti: 5200 | Durata: 0.015s"
|
||||||
|
|
||||||
|
messages = [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"content": "Analizza il seguente flusso di traffico di rete TCP/IP. Classifica se è traffico normale o un attacco e indica la tecnica MITRE ATT&CK corrispondente."
|
||||||
|
},
|
||||||
|
{"role": "user", "content": network_flow},
|
||||||
|
]
|
||||||
|
|
||||||
|
text = tokenizer.apply_chat_template(
|
||||||
|
messages,
|
||||||
|
tokenize=False,
|
||||||
|
add_generation_prompt=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
inputs = tokenizer(text, return_tensors="pt").to("cuda")
|
||||||
|
outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.3)
|
||||||
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
||||||
|
print(response)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Expected Output**: Classification of the network flow (e.g., "DoS Attack - MITRE ATT&CK: Impact/Denial of Service")
|
||||||
|
|
||||||
|
## 📊 Training Details
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|----------|-------|
|
||||||
|
| Base Model | Google Gemma 2.7B |
|
||||||
|
| Training Framework | Unsloth + TRL SFTTrainer |
|
||||||
|
| Training Dataset | Synthetic ATT&CK-derived dataset |
|
||||||
|
| Dataset Size | 10,000 examples |
|
||||||
|
| Techniques Covered | Network traffic analysis (CIC-IDS2017 + UNSW-NB15) |
|
||||||
|
| Training Duration | ~1 hour |
|
||||||
|
| Hardware | 1x NVIDIA RTX 4090 GPU |
|
||||||
|
| Learning Rate | 2e-5 |
|
||||||
|
| Batch Size | 16 (4 per device + 4 gradient accumulation steps) |
|
||||||
|
| LoRA Rank | 64 |
|
||||||
|
| Max Sequence Length | 512 tokens |
|
||||||
|
| Training Steps | 500 steps |
|
||||||
|
|
||||||
|
## 📝 Dataset Information
|
||||||
|
|
||||||
|
The training dataset was created synthetically using data derived from the MITRE ATT&CK framework and network traffic analysis datasets (CIC-IDS2017 + UNSW-NB15). It includes:
|
||||||
|
|
||||||
|
- **Network Traffic Features**: Protocol type, destination port, source/destination bytes, packet count, flow duration
|
||||||
|
- **Attack Classification**: Binary and multi-class classification of normal vs. malicious traffic
|
||||||
|
- **MITRE ATT&CK Mapping**: Techniques mapped to network-based attacks:
|
||||||
|
- **Reconnaissance**: Port scanning, network sniffing
|
||||||
|
- **Initial Access**: Brute force attacks on SSH, FTP, Telnet
|
||||||
|
- **Lateral Movement**: Data exfiltration, command & control traffic
|
||||||
|
- **Impact**: DoS/DDoS attacks, data theft
|
||||||
|
- **Attack Types Covered**: DoS, DDoS, PortScan, Brute Force, Infiltration, Botnet, Web attacks
|
||||||
|
- **Dataset Split**: 10,000 labeled examples for instruction-tuning
|
||||||
|
|
||||||
|
The synthetic data was processed to create instruction-following examples where the model learns to analyze network flows and map them to MITRE ATT&CK techniques and tactics.
|
||||||
|
|
||||||
|
## ⚠️ Limitations and Disclaimers
|
||||||
|
|
||||||
|
- **Not Exhaustive**: This model, like the underlying ATT&CK framework, does not enumerate all possible adversary behaviors. There may be undisclosed or novel techniques not covered.
|
||||||
|
- **Research Use**: While commercial use is permitted under the ATT&CK license, this model should be validated against your specific security requirements.
|
||||||
|
- **No Guarantee of Coverage**: Using this model to address or cover categories of techniques will not guarantee comprehensive defensive coverage.
|
||||||
|
- **As-Is**: This model is provided "as is" without any warranties or guarantees regarding accuracy, completeness, or fitness for a particular purpose.
|
||||||
|
|
||||||
|
## 📜 License
|
||||||
|
|
||||||
|
This model is based on **Google Gemma 2.7B** and incorporates data from the **MITRE ATT&CK framework**. Both licenses must be respected.
|
||||||
|
|
||||||
|
### Gemma License
|
||||||
|
|
||||||
|
This model is built upon Google's Gemma model, which is governed by the **Gemma Terms of Use**.
|
||||||
|
|
||||||
|
**Key Requirements:**
|
||||||
|
- This model can be used for research and commercial purposes
|
||||||
|
- You must comply with Google's Gemma Terms of Use
|
||||||
|
- You must ensure downstream usage complies with Gemma restrictions
|
||||||
|
- You acknowledge and accept Gemma's usage policies and any applicable restrictions
|
||||||
|
|
||||||
|
For full details, see: https://ai.google.dev/gemma/terms
|
||||||
|
|
||||||
|
### ATT&CK License Terms
|
||||||
|
|
||||||
|
© 2025 The MITRE Corporation. This work is reproduced and distributed with the permission of The MITRE Corporation.
|
||||||
|
|
||||||
|
The MITRE Corporation hereby grants you a non-exclusive, royalty-free license to use this model for research, development, and commercial purposes.
|
||||||
|
|
||||||
|
**Full License Text:**
|
||||||
|
|
||||||
|
```
|
||||||
|
LICENSE
|
||||||
|
The MITRE Corporation (MITRE) hereby grants you a non-exclusive, royalty-free
|
||||||
|
license to use ATT&CK® for research, development, and commercial purposes. Any
|
||||||
|
copy you make for such purposes is authorized provided that you reproduce MITRE's
|
||||||
|
copyright designation and this license in any such copy.
|
||||||
|
|
||||||
|
"© 2025 The MITRE Corporation. This work is reproduced and distributed with the
|
||||||
|
permission of The MITRE Corporation."
|
||||||
|
|
||||||
|
DISCLAIMERS
|
||||||
|
MITRE does not claim ATT&CK enumerates all possibilities for the types of actions
|
||||||
|
and behaviors documented as part of its adversary model and framework of techniques.
|
||||||
|
Using the information contained within ATT&CK to address or cover full categories
|
||||||
|
of techniques will not guarantee full defensive coverage as there may be undisclosed
|
||||||
|
techniques or variations on existing techniques not documented by ATT&CK.
|
||||||
|
|
||||||
|
ALL DOCUMENTS AND THE INFORMATION CONTAINED THEREIN ARE PROVIDED ON AN "AS IS"
|
||||||
|
BASIS AND THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY
|
||||||
|
(IF ANY), THE MITRE CORPORATION, ITS BOARD OF TRUSTEES, OFFICERS, AGENTS, AND
|
||||||
|
EMPLOYEES, DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
|
||||||
|
TO ANY WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE ANY
|
||||||
|
RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
|
||||||
|
PURPOSE.
|
||||||
|
```
|
||||||
|
|
||||||
|
### Model Modifications
|
||||||
|
|
||||||
|
This derivative work combines:
|
||||||
|
1. **Google's Gemma 2.7B** - the base language model
|
||||||
|
2. **MITRE ATT&CK** - the training dataset and knowledge domain
|
||||||
|
|
||||||
|
The model is fine-tuned on synthetic ATT&CK-derived data to specialize in threat intelligence and adversary behavior understanding. Any further use, distribution, or modification must maintain attribution and comply with both Google's Gemma Terms of Use and the MITRE ATT&CK license.
|
||||||
|
|
||||||
|
## 🔗 References
|
||||||
|
|
||||||
|
- **Google Gemma**: https://ai.google.dev/gemma/
|
||||||
|
- **Gemma Terms of Use**: https://ai.google.dev/gemma/terms
|
||||||
|
- **MITRE ATT&CK**: https://attack.mitre.org/
|
||||||
|
- **ATT&CK Documentation**: https://attack.mitre.org/docs/
|
||||||
|
|
||||||
|
## 👤 Author & Contact
|
||||||
|
|
||||||
|
**Mirko P.**
|
||||||
|
🤗 Hugging Face: [@hypnonyx](https://huggingface.co/hypnonyx)
|
||||||
|
|
||||||
|
## 🙏 Attribution
|
||||||
|
|
||||||
|
This model was created using the MITRE ATT&CK framework. We are grateful to The MITRE Corporation for making this valuable resource available to the research and security communities.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Last Updated**: March 4, 2025
|
||||||
|
**Model Version**: 1.0
|
||||||
10000
TrafficoDataset/dataset_traffico.jsonl
Normal file
10000
TrafficoDataset/dataset_traffico.jsonl
Normal file
File diff suppressed because it is too large
Load Diff
241
TrafficoDataset/train_gemma3_traffico.py
Normal file
241
TrafficoDataset/train_gemma3_traffico.py
Normal file
@@ -0,0 +1,241 @@
|
|||||||
|
# ============================================================
|
||||||
|
# Gemma-3-270M – Analisi Traffico di Rete TCP/IP
|
||||||
|
# Unsloth + LoRA + Dataset JSONL (CIC-IDS2017 + UNSW-NB15)
|
||||||
|
# ============================================================
|
||||||
|
# Struttura basata sul tuo script, adattata per il dominio
|
||||||
|
# di analisi del traffico di rete con mappatura MITRE ATT&CK.
|
||||||
|
#
|
||||||
|
# PREREQUISITI:
|
||||||
|
# Google Colab con runtime GPU (T4 basta)
|
||||||
|
# !pip install --no-deps unsloth
|
||||||
|
# !pip install transformers datasets trl peft accelerate sentencepiece
|
||||||
|
#
|
||||||
|
# FILE NECESSARI (nella stessa cartella dello script):
|
||||||
|
# dataset.jsonl ← generato dalla script apposita
|
||||||
|
# ============================================================
|
||||||
|
|
||||||
|
# ---------- INSTALL (Colab) ----------
|
||||||
|
# !pip install --no-deps unsloth
|
||||||
|
# !pip install transformers datasets trl peft accelerate sentencepiece
|
||||||
|
|
||||||
|
# ---------- IMPORT ----------
|
||||||
|
from unsloth import FastModel
|
||||||
|
from unsloth.chat_templates import get_chat_template, train_on_responses_only
|
||||||
|
import torch
|
||||||
|
from datasets import load_dataset
|
||||||
|
from trl import SFTTrainer, SFTConfig
|
||||||
|
|
||||||
|
# ---------- CONFIG ----------
|
||||||
|
MODEL_NAME = "unsloth/gemma-3-270m-it"
|
||||||
|
DATASET_PATH = "dataset_traffico.jsonl" # <== il JSONL che abbiamo generato
|
||||||
|
OUTPUT_DIR = "outputs"
|
||||||
|
MAX_SEQ_LENGTH = 512 # 512 basta per questi prompt, risparmia memoria
|
||||||
|
|
||||||
|
# ---------- LOAD MODEL ----------
|
||||||
|
model, tokenizer = FastModel.from_pretrained(
|
||||||
|
model_name = MODEL_NAME,
|
||||||
|
max_seq_length = MAX_SEQ_LENGTH,
|
||||||
|
load_in_4bit = False,
|
||||||
|
load_in_8bit = False,
|
||||||
|
full_finetuning = False,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------- LoRA ----------
|
||||||
|
# Configurazione più aggressiva sul rank (r=64) per un dominio specifico come questo.
|
||||||
|
# Target modules: tutti i proiettori del transformer.
|
||||||
|
model = FastModel.get_peft_model(
|
||||||
|
model,
|
||||||
|
r = 64,
|
||||||
|
target_modules = [
|
||||||
|
"q_proj", "k_proj", "v_proj", "o_proj",
|
||||||
|
"gate_proj", "up_proj", "down_proj",
|
||||||
|
],
|
||||||
|
lora_alpha = 64,
|
||||||
|
lora_dropout = 0,
|
||||||
|
bias = "none",
|
||||||
|
use_gradient_checkpointing = "unsloth",
|
||||||
|
random_state = 3407,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------- CHAT TEMPLATE (Gemma-3) ----------
|
||||||
|
tokenizer = get_chat_template(
|
||||||
|
tokenizer,
|
||||||
|
chat_template = "gemma3",
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------- LOAD DATASET ----------
|
||||||
|
dataset = load_dataset(
|
||||||
|
"json",
|
||||||
|
data_files = DATASET_PATH,
|
||||||
|
split = "train",
|
||||||
|
)
|
||||||
|
|
||||||
|
print(f"Dataset caricato: {len(dataset)} righe")
|
||||||
|
print(f"Campi presenti: {dataset.column_names}")
|
||||||
|
print(f"\nEsempio riga 0:")
|
||||||
|
print(dataset[0])
|
||||||
|
|
||||||
|
# ---------- CONVERT TO CHATML ----------
|
||||||
|
# Il JSONL ha campi: instruction, input, output
|
||||||
|
# Li convertiamo nel formato conversations [system, user, assistant]
|
||||||
|
# che Gemma-3 si aspetta.
|
||||||
|
def convert_to_chatml(example):
|
||||||
|
system_prompt = example["instruction"]
|
||||||
|
|
||||||
|
# Se c'è un campo 'context' lo aggiungiamo al system prompt
|
||||||
|
if "context" in example and example["context"]:
|
||||||
|
system_prompt += f"\nContesto: {example['context']}."
|
||||||
|
|
||||||
|
return {
|
||||||
|
"conversations": [
|
||||||
|
{"role": "system", "content": system_prompt},
|
||||||
|
{"role": "user", "content": example["input"]},
|
||||||
|
{"role": "assistant", "content": example["output"]},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
dataset = dataset.map(convert_to_chatml)
|
||||||
|
|
||||||
|
# ---------- APPLY GEMMA-3 TEMPLATE ----------
|
||||||
|
# Applica il template di chat di Gemma-3 a ogni esempio.
|
||||||
|
# Questo produce la stringa finale che il modello vedrà durante il training.
|
||||||
|
def formatting_prompts_func(examples):
|
||||||
|
convos = examples["conversations"]
|
||||||
|
texts = [
|
||||||
|
tokenizer.apply_chat_template(
|
||||||
|
convo,
|
||||||
|
tokenize = False,
|
||||||
|
add_generation_prompt = False,
|
||||||
|
).removeprefix("<bos>")
|
||||||
|
for convo in convos
|
||||||
|
]
|
||||||
|
return {"text": texts}
|
||||||
|
|
||||||
|
dataset = dataset.map(formatting_prompts_func, batched=True)
|
||||||
|
|
||||||
|
# Verifica come appare un prompt formattato
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print(" PROMPT FORMATTATO (esempio)")
|
||||||
|
print("=" * 60)
|
||||||
|
print(dataset[0]["text"])
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
# ---------- TRAINER ----------
|
||||||
|
trainer = SFTTrainer(
|
||||||
|
model = model,
|
||||||
|
tokenizer = tokenizer,
|
||||||
|
train_dataset = dataset,
|
||||||
|
eval_dataset = None,
|
||||||
|
args = SFTConfig(
|
||||||
|
dataset_text_field = "text",
|
||||||
|
per_device_train_batch_size = 4,
|
||||||
|
gradient_accumulation_steps = 4, # batch effettivo = 4 * 4 = 16
|
||||||
|
warmup_steps = 10,
|
||||||
|
max_steps = 500, # ~500 step su 10k righe con batch 16
|
||||||
|
learning_rate = 2e-5,
|
||||||
|
logging_steps = 25,
|
||||||
|
optim = "adamw_8bit",
|
||||||
|
weight_decay = 0.001,
|
||||||
|
lr_scheduler_type = "linear",
|
||||||
|
seed = 3407,
|
||||||
|
output_dir = OUTPUT_DIR,
|
||||||
|
report_to = "none",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------- TRAIN ONLY ON ASSISTANT ----------
|
||||||
|
# Fondamentale: il modello calcola il loss SOLO sulla risposta dell'assistant,
|
||||||
|
# non sul prompt. Così non "impara" a ripetere la domanda.
|
||||||
|
trainer = train_on_responses_only(
|
||||||
|
trainer,
|
||||||
|
instruction_part = "<start_of_turn>user\n",
|
||||||
|
response_part = "<start_of_turn>model\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------- TRAIN ----------
|
||||||
|
trainer.train()
|
||||||
|
|
||||||
|
# ---------- SAVE LoRA ----------
|
||||||
|
model.save_pretrained("gemma3-traffico-rete-lora")
|
||||||
|
tokenizer.save_pretrained("gemma3-traffico-rete-lora")
|
||||||
|
print("\n✓ Modello LoRA salvato in: gemma3-traffico-rete-lora/")
|
||||||
|
model.save_pretrained_merged(
|
||||||
|
"gemma3-traffico-rete-lora", # cartella output
|
||||||
|
tokenizer,
|
||||||
|
save_method="merged_16bit" # Float16 per GGUF
|
||||||
|
)
|
||||||
|
model.save_pretrained_gguf(
|
||||||
|
"gemma3-traffico-rete-lora",
|
||||||
|
tokenizer,
|
||||||
|
quantization_method = "BF16", # For now only Q8_0, BF16, F16 supported
|
||||||
|
)
|
||||||
|
# ---------- INFERENCE: TEST ----------
|
||||||
|
# Dopo il training, prova il modello con alcuni flussi di esempio.
|
||||||
|
from transformers import TextStreamer
|
||||||
|
|
||||||
|
test_cases = [
|
||||||
|
# Caso 1: profilo tipico DoS (masse enormi di byte src, pochissimi dst, durata minima)
|
||||||
|
"Protocollo: tcp | Porta dst: 80 | Byte src: 480000 | Byte dst: 40 | Pacchetti: 5200 | Durata: 0.015s",
|
||||||
|
# Caso 2: traffico normale HTTPS
|
||||||
|
"Protocollo: tcp | Porta dst: 443 | Byte src: 1500 | Byte dst: 6200 | Pacchetti: 9 | Durata: 3.200s",
|
||||||
|
# Caso 3: profilo PortScan (tanti dst diversi, pochi byte, durata quasi zero)
|
||||||
|
"Protocollo: tcp | Porta dst: 22 | Byte src: 60 | Byte dst: 0 | Pacchetti: 1 | Durata: 0.002s",
|
||||||
|
# Caso 4: profilo Brute Force su SSH
|
||||||
|
"Protocollo: tcp | Porta dst: 22 | Byte src: 3200 | Byte dst: 8500 | Pacchetti: 45 | Durata: 1.800s",
|
||||||
|
# Caso 5: profilo Infiltration / esfiltrazioni dati
|
||||||
|
"Protocollo: tcp | Porta dst: 443 | Byte src: 8000 | Byte dst: 120000 | Pacchetti: 200 | Durata: 25.500s",
|
||||||
|
]
|
||||||
|
|
||||||
|
streamer = TextStreamer(tokenizer, skip_prompt=True)
|
||||||
|
|
||||||
|
for i, test_input in enumerate(test_cases, 1):
|
||||||
|
messages = [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"content": (
|
||||||
|
"Analizza il seguente flusso di traffico di rete TCP/IP. "
|
||||||
|
"Classifica se è traffico normale o un attacco. "
|
||||||
|
"Se è un attacco, indica la categoria e la tecnica MITRE ATT&CK corrispondente."
|
||||||
|
),
|
||||||
|
},
|
||||||
|
{"role": "user", "content": test_input},
|
||||||
|
]
|
||||||
|
|
||||||
|
text = tokenizer.apply_chat_template(
|
||||||
|
messages,
|
||||||
|
tokenize = False,
|
||||||
|
add_generation_prompt = True,
|
||||||
|
).removeprefix("<bos>")
|
||||||
|
|
||||||
|
print(f"\n{'─' * 60}")
|
||||||
|
print(f" TEST {i}: {test_input[:80]}...")
|
||||||
|
print(f"{'─' * 60}")
|
||||||
|
print(" Risposta: ", end="")
|
||||||
|
|
||||||
|
_ = model.generate(
|
||||||
|
**tokenizer(text, return_tensors="pt").to("cuda"),
|
||||||
|
max_new_tokens = 128,
|
||||||
|
temperature = 0.3, # bassa temperatura = risposte più deterministe
|
||||||
|
top_p = 0.9,
|
||||||
|
top_k = 40,
|
||||||
|
streamer = streamer,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ---------- SAVE MERGED (opzionale) ----------
|
||||||
|
# Unisce i pesi LoRA al modello base e salva come modello completo.
|
||||||
|
# Utile per deployare senza dipendenza da PEFT.
|
||||||
|
#
|
||||||
|
model.save_pretrained_merged(
|
||||||
|
"gemma3-traffico-rete-merged",
|
||||||
|
tokenizer,
|
||||||
|
save_method = "merged_16bit",
|
||||||
|
)
|
||||||
|
#
|
||||||
|
# ---------- SAVE GGUF (opzionale) ----------
|
||||||
|
# Formato GGUF per inferenza locale con llama.cpp / Ollama.
|
||||||
|
#
|
||||||
|
model.save_pretrained_gguf(
|
||||||
|
"gemma3-traffico-rete-gguf",
|
||||||
|
tokenizer,
|
||||||
|
quantization_method = "Q8_0", # Q8_0 = buon equilibrio qualità/dimensione
|
||||||
|
)
|
||||||
3
added_tokens.json
Normal file
3
added_tokens.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"<image_soft_token>": 262144
|
||||||
|
}
|
||||||
47
chat_template.jinja
Normal file
47
chat_template.jinja
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
|
||||||
|
{%- if messages[0]['role'] == 'system' -%}
|
||||||
|
{%- if messages[0]['content'] is string -%}
|
||||||
|
{%- set first_user_prefix = messages[0]['content'] + '
|
||||||
|
|
||||||
|
' -%}
|
||||||
|
{%- else -%}
|
||||||
|
{%- set first_user_prefix = messages[0]['content'][0]['text'] + '
|
||||||
|
|
||||||
|
' -%}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- set loop_messages = messages[1:] -%}
|
||||||
|
{%- else -%}
|
||||||
|
{%- set first_user_prefix = "" -%}
|
||||||
|
{%- set loop_messages = messages -%}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- for message in loop_messages -%}
|
||||||
|
{%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
|
||||||
|
{{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if (message['role'] == 'assistant') -%}
|
||||||
|
{%- set role = "model" -%}
|
||||||
|
{%- else -%}
|
||||||
|
{%- set role = message['role'] -%}
|
||||||
|
{%- endif -%}
|
||||||
|
{{ '<start_of_turn>' + role + '
|
||||||
|
' + (first_user_prefix if loop.first else "") }}
|
||||||
|
{%- if message['content'] is string -%}
|
||||||
|
{{ message['content'] | trim }}
|
||||||
|
{%- elif message['content'] is iterable -%}
|
||||||
|
{%- for item in message['content'] -%}
|
||||||
|
{%- if item['type'] == 'image' -%}
|
||||||
|
{{ '<start_of_image>' }}
|
||||||
|
{%- elif item['type'] == 'text' -%}
|
||||||
|
{{ item['text'] | trim }}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- endfor -%}
|
||||||
|
{%- else -%}
|
||||||
|
{{ raise_exception("Invalid content type") }}
|
||||||
|
{%- endif -%}
|
||||||
|
{{ '<end_of_turn>
|
||||||
|
' }}
|
||||||
|
{%- endfor -%}
|
||||||
|
{%- if add_generation_prompt -%}
|
||||||
|
{{ '<start_of_turn>model
|
||||||
|
' }}
|
||||||
|
{%- endif -%}
|
||||||
56
config.json
Normal file
56
config.json
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
{
|
||||||
|
"_sliding_window_pattern": 6,
|
||||||
|
"architectures": [
|
||||||
|
"Gemma3ForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_bias": false,
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"attn_logit_softcapping": null,
|
||||||
|
"bos_token_id": 2,
|
||||||
|
"torch_dtype": "bfloat16",
|
||||||
|
"eos_token_id": 106,
|
||||||
|
"final_logit_softcapping": null,
|
||||||
|
"head_dim": 256,
|
||||||
|
"hidden_activation": "gelu_pytorch_tanh",
|
||||||
|
"hidden_size": 640,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 2048,
|
||||||
|
"layer_types": [
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"full_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"full_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"sliding_attention",
|
||||||
|
"full_attention"
|
||||||
|
],
|
||||||
|
"max_position_embeddings": 32768,
|
||||||
|
"model_type": "gemma3_text",
|
||||||
|
"num_attention_heads": 4,
|
||||||
|
"num_hidden_layers": 18,
|
||||||
|
"num_key_value_heads": 1,
|
||||||
|
"pad_token_id": 0,
|
||||||
|
"query_pre_attn_scalar": 256,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_local_base_freq": 10000.0,
|
||||||
|
"rope_scaling": null,
|
||||||
|
"rope_theta": 1000000.0,
|
||||||
|
"sliding_window": 512,
|
||||||
|
"transformers_version": "4.57.3",
|
||||||
|
"unsloth_fixed": true,
|
||||||
|
"unsloth_version": "2026.1.4",
|
||||||
|
"use_bidirectional_attention": false,
|
||||||
|
"use_cache": true,
|
||||||
|
"vocab_size": 262144
|
||||||
|
}
|
||||||
3
gemma-3-270m-it-traffico.Q8_0.gguf
Normal file
3
gemma-3-270m-it-traffico.Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4624765d7026923365e4df3f90e6b7046049fdd7407bd9855778fce67eaa5dcd
|
||||||
|
size 291545376
|
||||||
3
hypnonyx_traffico.png
Normal file
3
hypnonyx_traffico.png
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:cfcba0cc3c9a0a6a0817bf868e4fb522b60038e25abe15bbb1364f27d8511107
|
||||||
|
size 2326345
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1debdbdbc6a711e1abf5f0285bda7fd2a7a93805ae3c4aa986012a7bd2eac39a
|
||||||
|
size 536223056
|
||||||
33
special_tokens_map.json
Normal file
33
special_tokens_map.json
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
{
|
||||||
|
"boi_token": "<start_of_image>",
|
||||||
|
"bos_token": {
|
||||||
|
"content": "<bos>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"eoi_token": "<end_of_image>",
|
||||||
|
"eos_token": {
|
||||||
|
"content": "<end_of_turn>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"image_token": "<image_soft_token>",
|
||||||
|
"pad_token": {
|
||||||
|
"content": "<pad>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"unk_token": {
|
||||||
|
"content": "<unk>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
}
|
||||||
|
}
|
||||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
|
||||||
|
size 33384568
|
||||||
3
tokenizer.model
Normal file
3
tokenizer.model
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
|
||||||
|
size 4689074
|
||||||
51346
tokenizer_config.json
Normal file
51346
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user