初始化项目，由ModelHub XC社区提供模型
Model: stratosphere/qwen2.5-1.5b-slips-immune-summarization Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
232
README.md
Normal file
232
README.md
Normal file
@@ -0,0 +1,232 @@
|
|||||||
|
---
|
||||||
|
base_model: unsloth/Qwen2.5-1.5B-Instruct
|
||||||
|
datasets:
|
||||||
|
- stratosphere/immune-summary-sft-dataset
|
||||||
|
library_name: transformers
|
||||||
|
model_name: qwen2.5-1.5b-slips-immune-summarization
|
||||||
|
tags:
|
||||||
|
- base_model:finetune:unsloth/Qwen2.5-1.5B-Instruct
|
||||||
|
- network-security
|
||||||
|
- ids
|
||||||
|
- slips
|
||||||
|
- summarization
|
||||||
|
- cybersecurity
|
||||||
|
- lora
|
||||||
|
- sft
|
||||||
|
- transformers
|
||||||
|
- trl
|
||||||
|
- unsloth
|
||||||
|
license: apache-2.0
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
---
|
||||||
|
|
||||||
|
# Qwen2.5-1.5B — Slips IDS Security Summarization
|
||||||
|
|
||||||
|
## Model Description
|
||||||
|
|
||||||
|
A fine-tuned version of [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) specialized for translating technical network security events from [Slips IDS](https://github.com/stratosphereips/StratosphereLinuxIPS) into clear, human-readable incident summaries with severity assessments.
|
||||||
|
|
||||||
|
Slips is a network intrusion detection system that generates DAG-structured alert logs — chains of related security events per source IP per time window. Raw Slips output is highly technical and difficult to interpret quickly. This model translates those logs into structured, concise summaries grouped by event type, with per-event severity labels (CRITICAL / HIGH / MEDIUM / LOW / INFO) and an overall severity breakdown.
|
||||||
|
|
||||||
|
The model was fine-tuned using SFT (Supervised Fine-Tuning) with best-of-N response selection: for each training incident, the highest-scoring response (judged by an LLM-as-judge) among GPT-4o, GPT-4o-mini, Qwen2.5 3B, and Qwen2.5 was selected as ground truth.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### Ollama (recommended for local deployment)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ollama run stratosphere/qwen2.5-1.5b-slips-immune-summarization
|
||||||
|
# or a specific quantization:
|
||||||
|
ollama run stratosphere/qwen2.5-1.5b-slips-immune-summarization:q5_k_m
|
||||||
|
ollama run stratosphere/qwen2.5-1.5b-slips-immune-summarization:q8_0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Python (Transformers)
|
||||||
|
|
||||||
|
This model uses a **merged prompt format**: instructions and the DAG are combined into a single user message with no system prompt.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||||
|
import torch
|
||||||
|
|
||||||
|
model_id = "stratosphere/qwen2.5-1.5b-slips-immune-summarization"
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
|
||||||
|
|
||||||
|
dag_input = """
|
||||||
|
============================================================
|
||||||
|
Incident: abc123
|
||||||
|
Source IP: 192.168.1.100 | Timewindow: 5
|
||||||
|
Timeline: 2024-01-15 14:00:00 to 2024-01-15 15:00:00
|
||||||
|
Threat Level: 8.5 | Events: 42
|
||||||
|
...
|
||||||
|
""" # paste your Slips DAG analysis here
|
||||||
|
|
||||||
|
user_message = f"""You are a security analyst. Your task is to translate technical security events into clear, concise, human-readable summaries and assess their severity.
|
||||||
|
|
||||||
|
INCIDENT METADATA:
|
||||||
|
- Incident ID: abc123
|
||||||
|
- Source IP: 192.168.1.100
|
||||||
|
- Timewindow: 5
|
||||||
|
- Accumulated Threat Level: 8.5
|
||||||
|
- Time Range: 2024-01-15 14:00:00 to 2024-01-15 15:00:00
|
||||||
|
- Total Events: 42
|
||||||
|
|
||||||
|
RAW EVENTS (Time | Description):
|
||||||
|
{dag_input}
|
||||||
|
|
||||||
|
YOUR TASK:
|
||||||
|
1. Transform the technical event descriptions into clear, readable summaries using plain language
|
||||||
|
2. Group identical or very similar events (e.g., 24 identical connections → one summary line)
|
||||||
|
3. Assess the severity of each event/group based on security impact:
|
||||||
|
- CRITICAL: Active exploitation, data exfiltration, confirmed malware C2
|
||||||
|
- HIGH: Scanning, suspicious connections, potential threats
|
||||||
|
- MEDIUM: Anomalous but potentially benign behavior
|
||||||
|
- LOW: Minor issues, likely false positives
|
||||||
|
- INFO: Informational events, normal network behavior
|
||||||
|
4. Calculate the overall severity breakdown based on your assessments
|
||||||
|
|
||||||
|
OUTPUT FORMAT (match this structure exactly):
|
||||||
|
|
||||||
|
============================================================
|
||||||
|
Incident: <incident_id>
|
||||||
|
Source IP: <source_ip> | Timewindow: <timewindow>
|
||||||
|
Timeline: <start> to <end>
|
||||||
|
Threat Level: <threat_level> | Events: <count>
|
||||||
|
|
||||||
|
• HH:MM-HH:MM - [Your clear grouped summary] [YOUR_ASSESSED_SEVERITY]
|
||||||
|
• HH:MM - [Your clear summary] [YOUR_ASSESSED_SEVERITY]
|
||||||
|
|
||||||
|
Total Evidence: <count> events
|
||||||
|
Severity breakdown: [Your calculated breakdown, e.g., "High: 5, Medium: 3, Info: 2"]
|
||||||
|
|
||||||
|
RULES:
|
||||||
|
- Group identical events into ONE line
|
||||||
|
- Use time ranges (HH:MM-HH:MM) when showing grouped events
|
||||||
|
- Assess severity based on security impact, not just event type
|
||||||
|
- Keep descriptions clear and concise
|
||||||
|
- Just output the structured summary - no explanations or meta-commentary"""
|
||||||
|
|
||||||
|
messages = [{"role": "user", "content": user_message}]
|
||||||
|
|
||||||
|
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
|
||||||
|
output = model.generate(input_ids, max_new_tokens=512, do_sample=False)
|
||||||
|
print(tokenizer.decode(output[0][input_ids.shape[1]:], skip_special_tokens=True))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Training Details
|
||||||
|
|
||||||
|
### Dataset
|
||||||
|
|
||||||
|
The training data is publicly available at [stratosphere/immune-summary-sft-dataset](https://huggingface.co/datasets/stratosphere/immune-summary-sft-dataset).
|
||||||
|
|
||||||
|
- **Source**: 532 incidents from real Slips IDS network captures
|
||||||
|
- **Responses**: 4 model responses per incident (GPT-4o, GPT-4o-mini, Qwen2.5 3B, Qwen2.5 1B) used as candidate labels, scored by an LLM-as-judge
|
||||||
|
- **Selection**: Best-of-N — highest-scoring response per incident used as training target
|
||||||
|
- **Filtering**: Responses with judge score < 4 or summary token length outside [50, 400] discarded
|
||||||
|
- **Split**: 90% train / 10% eval (stratified, seed=42)
|
||||||
|
|
||||||
|
### Training Procedure
|
||||||
|
|
||||||
|
| Parameter | Value |
|
||||||
|
|-----------|-------|
|
||||||
|
| Base model | `unsloth/Qwen2.5-1.5B-Instruct` |
|
||||||
|
| Training method | SFT (Supervised Fine-Tuning) |
|
||||||
|
| Framework | Unsloth + TRL SFTTrainer |
|
||||||
|
| LoRA rank (`r`) | 64 |
|
||||||
|
| LoRA alpha | 64 |
|
||||||
|
| LoRA dropout | 0.0 |
|
||||||
|
| RSLoRA | enabled (required at r=64) |
|
||||||
|
| LoRA targets | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
|
||||||
|
| Sequence length | 4096 |
|
||||||
|
| Batch size | 1 (effective: 16 via gradient accumulation) |
|
||||||
|
| Gradient accumulation steps | 16 |
|
||||||
|
| Learning rate | 2e-5 |
|
||||||
|
| LR scheduler | cosine |
|
||||||
|
| Warmup steps | 20 |
|
||||||
|
| Weight decay | 0.01 |
|
||||||
|
| Epochs | 3 |
|
||||||
|
| Optimizer | adamw_8bit |
|
||||||
|
| Precision | BF16 |
|
||||||
|
| Quantization (training) | 4bit |
|
||||||
|
| Hardware | A100 80GB MiG 20GB slice (e-infra.cz cloud) |
|
||||||
|
| Response masking | `train_on_responses_only` — loss computed on assistant turns only |
|
||||||
|
|
||||||
|
### Framework Versions
|
||||||
|
|
||||||
|
- Unsloth: 2026.3.18
|
||||||
|
- Transformers: (auto-detected)
|
||||||
|
- PyTorch: (auto-detected)
|
||||||
|
|
||||||
|
## Evaluation
|
||||||
|
|
||||||
|
Evaluated on 47 held-out Slips IDS incidents using `gpt-oss-120b` as an independent LLM-as-judge. The judge ranked all 5 model responses per incident simultaneously on a 1–10 scale, with model labels randomized per incident to prevent position bias. Inference was performed with the **merged prompt format** (instructions + DAG combined in a single user message, no system prompt) at 4096 max input tokens.
|
||||||
|
|
||||||
|
### Overall Results
|
||||||
|
|
||||||
|
| Rank | Model | Avg Score | Avg Position | Win Rate | Wins |
|
||||||
|
|------|-------|-----------|--------------|----------|------|
|
||||||
|
| 1 | GPT-4o-mini | 6.89/10 | 1.81 | 42.6% | 20 |
|
||||||
|
| 2 | GPT-4o | 5.87/10 | 2.38 | 29.8% | 14 |
|
||||||
|
| 3 | **Qwen2.5-1.5B (finetuned)** | **4.70/10** | **3.21** | **19.1%** | **9** |
|
||||||
|
| 4 | Qwen2.5 3B (baseline) | 4.57/10 | 3.40 | 8.5% | 4 |
|
||||||
|
| 5 | Qwen2.5 1B (baseline) | 3.36/10 | 4.19 | 0.0% | 0 |
|
||||||
|
|
||||||
|
The finetuned 1.5B model beats both untuned baselines (+1.34 avg score vs Qwen2.5 1B, +0.13 vs Qwen2.5 3B) and achieves a 19.1% win rate — higher than the 3B baseline (8.5%).
|
||||||
|
|
||||||
|
### By Complexity
|
||||||
|
|
||||||
|
| Complexity | Events | Finetuned Score | GPT-4o-mini Score | GPT-4o Score |
|
||||||
|
|------------|--------|----------------|-------------------|--------------|
|
||||||
|
| Simple | < 500 (31 incidents) | 5.45/10 | 6.74/10 | 5.61/10 |
|
||||||
|
| Medium | 500–1999 (7 incidents) | 3.43/10 | 6.71/10 | 5.71/10 |
|
||||||
|
| Complex | ≥ 2000 (9 incidents) | 3.11/10 | 7.56/10 | 6.89/10 |
|
||||||
|
|
||||||
|
On simple incidents the finetuned model is competitive with GPT-4o (5.45 vs 5.61). Medium and complex incidents are the primary weakness, consistent with context length limitations at 4096 tokens.
|
||||||
|
|
||||||
|
### By Category
|
||||||
|
|
||||||
|
| Category | Finetuned Score | Finetuned Win Rate |
|
||||||
|
|----------|-----------------|--------------------|
|
||||||
|
| Malware (45 incidents) | 4.82/10 | 20.0% |
|
||||||
|
| Normal (2 incidents) | 2.00/10 | 0.0% |
|
||||||
|
|
||||||
|
### Readability
|
||||||
|
|
||||||
|
An automated readability analysis on the 47 held-out incidents shows the model achieves a compression ratio of 0.26 with 373 abstracted bullets, 256 verbatim lines, and 44 markdown fences — indicating the model learned to paraphrase and summarize rather than echo the input DAG.
|
||||||
|
|
||||||
|
### Known Limitation: Complex Incident Performance
|
||||||
|
|
||||||
|
The model struggles on medium and complex incidents (≥ 500 events), scoring 3.43/10 (medium) and 3.11/10 (complex) with 0 wins in both tiers. Large DAGs exceed the effective 4096-token context window, resulting in inference errors on the largest inputs. Reducing the input token limit to match the training sequence length mitigates but does not fully resolve this.
|
||||||
|
|
||||||
|
## Intended Use
|
||||||
|
|
||||||
|
- Automated triage of Slips IDS alerts for security analysts
|
||||||
|
- First-pass summarization of network incident logs
|
||||||
|
- Input to downstream reporting or ticketing workflows
|
||||||
|
|
||||||
|
## Out-of-Scope Use
|
||||||
|
|
||||||
|
- General-purpose chat or instruction following
|
||||||
|
- Security domains outside network IDS (malware analysis, vulnerability scanning, etc.)
|
||||||
|
- Non-English inputs
|
||||||
|
|
||||||
|
## Citation
|
||||||
|
|
||||||
|
```bibtex
|
||||||
|
@misc{qwen2.5-1.5b-slips-immune,
|
||||||
|
title = {Qwen2.5-1.5B fine-tuned for Slips IDS security summarization},
|
||||||
|
author = {Stratosphere Laboratory, CTU Prague},
|
||||||
|
year = {2026},
|
||||||
|
howpublished = {\url{https://huggingface.co/stratosphere/qwen2.5-1.5b-slips-immune-summarization}}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Model Details
|
||||||
|
|
||||||
|
- **Model size**: 1.5B params
|
||||||
|
- **Tensor type**: FP16
|
||||||
|
- **License**: Apache-2.0
|
||||||
|
- **Tags**: Text Generation, Transformers, Safetensors, Network Security, IDS, SLIPS, Summarization, Cybersecurity, LoRA, SFT, TRL, Unsloth
|
||||||
53
chat_template.jinja
Normal file
53
chat_template.jinja
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
{%- if tools %}
|
||||||
|
{{- '<|im_start|>system\n' }}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{{- messages[0]['content'] }}
|
||||||
|
{%- else %}
|
||||||
|
{{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||||
|
{%- for tool in tools %}
|
||||||
|
{{- "\n" }}
|
||||||
|
{{- tool | tojson }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||||
|
{%- else %}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
|
||||||
|
{%- else %}
|
||||||
|
{{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- for message in messages %}
|
||||||
|
{%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
|
||||||
|
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
|
||||||
|
{%- elif message.role == "assistant" %}
|
||||||
|
{{- '<|im_start|>' + message.role }}
|
||||||
|
{%- if message.content %}
|
||||||
|
{{- '\n' + message.content }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- for tool_call in message.tool_calls %}
|
||||||
|
{%- if tool_call.function is defined %}
|
||||||
|
{%- set tool_call = tool_call.function %}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '\n<tool_call>\n{"name": "' }}
|
||||||
|
{{- tool_call.name }}
|
||||||
|
{{- '", "arguments": ' }}
|
||||||
|
{{- tool_call.arguments | tojson }}
|
||||||
|
{{- '}\n</tool_call>' }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- elif message.role == "tool" %}
|
||||||
|
{%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %} {{- '<|im_start|>user' }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '\n<tool_response>\n' }}
|
||||||
|
{{- message.content }}
|
||||||
|
{{- '\n</tool_response>' }}
|
||||||
|
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||||
|
{{- '<|im_end|>\n' }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- if add_generation_prompt %}
|
||||||
|
{{- '<|im_start|>assistant\n' }}
|
||||||
|
{%- endif %}
|
||||||
62
config.json
Normal file
62
config.json
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"Qwen2ForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": null,
|
||||||
|
"torch_dtype": "bfloat16",
|
||||||
|
"eos_token_id": 151645,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 1536,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 8960,
|
||||||
|
"layer_types": [
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention"
|
||||||
|
],
|
||||||
|
"max_position_embeddings": 32768,
|
||||||
|
"max_window_layers": 21,
|
||||||
|
"model_type": "qwen2",
|
||||||
|
"num_attention_heads": 12,
|
||||||
|
"num_hidden_layers": 28,
|
||||||
|
"num_key_value_heads": 2,
|
||||||
|
"pad_token_id": 151665,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_parameters": {
|
||||||
|
"rope_theta": 1000000.0,
|
||||||
|
"rope_type": "default"
|
||||||
|
},
|
||||||
|
"sliding_window": null,
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"unsloth_fixed": true,
|
||||||
|
"unsloth_version": "2026.4.4",
|
||||||
|
"use_cache": false,
|
||||||
|
"use_sliding_window": false,
|
||||||
|
"vocab_size": 151936
|
||||||
|
}
|
||||||
125
config.yaml
Normal file
125
config.yaml
Normal file
@@ -0,0 +1,125 @@
|
|||||||
|
# Qwen Fine-tuning Configuration with Unsloth
|
||||||
|
# Optimized for 20GB VRAM — based on config.yaml with readability improvements
|
||||||
|
|
||||||
|
# Model Configuration
|
||||||
|
model:
|
||||||
|
model_name: "unsloth/Qwen2.5-1.5B-Instruct" # Options: Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen3-32B
|
||||||
|
max_seq_length: 4096 # Sequence length (avg input ~2054 tokens; 2048 truncated 38% of samples)
|
||||||
|
dtype: null # Auto-detect best dtype
|
||||||
|
load_in_4bit: true # QLoRA — 4-bit base model required for 20GB VRAM
|
||||||
|
device_map: "auto" # Automatic device mapping
|
||||||
|
|
||||||
|
# LoRA Configuration
|
||||||
|
lora_r: 64 # Increased from 16 — wider subspace helps model learn synthesis over verbatim copying
|
||||||
|
lora_alpha: 64 # Keep equal to r with RSLoRA
|
||||||
|
lora_dropout: 0.0 # No dropout — 461 samples, ~174 optimizer steps; every gradient counts
|
||||||
|
lora_targets: # Target modules for LoRA
|
||||||
|
- "q_proj"
|
||||||
|
- "k_proj"
|
||||||
|
- "v_proj"
|
||||||
|
- "o_proj"
|
||||||
|
- "gate_proj"
|
||||||
|
- "up_proj"
|
||||||
|
- "down_proj"
|
||||||
|
use_rslora: true # Mandatory at r=64 to normalize gradient scaling
|
||||||
|
random_state: 42 # Random seed for reproducibility
|
||||||
|
loftq_config: null # LoftQ configuration
|
||||||
|
|
||||||
|
# Dataset Configuration
|
||||||
|
dataset:
|
||||||
|
type: "local" # Options: "huggingface", "local"
|
||||||
|
name: "mixed_dataset" # Hugging Face dataset name (if type is huggingface)
|
||||||
|
path: "train_dataset.json" # Local dataset path (if type is local)
|
||||||
|
eval_path: "eval_dataset.json" # Enabled — allows load_best_model_at_end to select best checkpoint
|
||||||
|
split: "train" # Dataset split to use
|
||||||
|
text_column: "messages" # Column name containing conversations
|
||||||
|
use_chat_template: true # Apply chat template formatting
|
||||||
|
dpo_train_path: "dpo_train_dataset.json"
|
||||||
|
dpo_eval_path: "dpo_eval_dataset.json"
|
||||||
|
|
||||||
|
# Training Configuration
|
||||||
|
training:
|
||||||
|
mode: "sft" # Options: "sft", "dpo", "orpo"
|
||||||
|
|
||||||
|
# Batch size and accumulation
|
||||||
|
per_device_train_batch_size: 1 # 8192 seq len requires batch=1 to avoid OOM
|
||||||
|
gradient_accumulation_steps: 16 # effective batch size = 16
|
||||||
|
|
||||||
|
# Learning rate and schedule
|
||||||
|
learning_rate: 0.00002 # 2e-5 — RSLoRA stability allows higher LR than r=16 config
|
||||||
|
lr_scheduler_type: "cosine" # Learning rate scheduler
|
||||||
|
warmup_steps: 20 # 11% of ~174 steps
|
||||||
|
weight_decay: 0.01 # Weight decay
|
||||||
|
|
||||||
|
# Training duration
|
||||||
|
num_train_epochs: 3 # Number of training epochs
|
||||||
|
max_steps: -1 # Maximum training steps (-1 for full epochs)
|
||||||
|
|
||||||
|
# Precision and optimization
|
||||||
|
fp16: false # Use BF16 instead (Ampere GPU assumed)
|
||||||
|
bf16: true # BF16 — larger dynamic range than FP16, no overflow risk
|
||||||
|
optimizer: "adamw_8bit" # 8-bit optimizer — keeps optimizer states within 20GB budget
|
||||||
|
|
||||||
|
# Logging and saving
|
||||||
|
logging_steps: 1 # Logging frequency
|
||||||
|
save_steps: 50 # Model save frequency
|
||||||
|
save_total_limit: 2 # Maximum number of saved checkpoints
|
||||||
|
|
||||||
|
# Output directory
|
||||||
|
output_dir: "./qwen_finetuned_4096_20gb" # Output directory for model and checkpoints
|
||||||
|
|
||||||
|
# Data processing
|
||||||
|
dataset_num_proc: 2 # Number of processes for dataset processing
|
||||||
|
dataloader_num_workers: 0 # Number of dataloader workers
|
||||||
|
packing: false # Must be false when using train_on_responses_only
|
||||||
|
|
||||||
|
# Reporting
|
||||||
|
report_to: [] # Reporting services (wandb, tensorboard, etc.)
|
||||||
|
|
||||||
|
# Model saving format
|
||||||
|
save_method: "merged_16bit" # Options: "lora", "merged_16bit", "merged_4bit"
|
||||||
|
gguf_quantization: "q5_k_m" # Also export GGUF for Ollama. Options: q4_k_m, q5_k_m, q8_0, f16. Set to null to skip.
|
||||||
|
|
||||||
|
# Reproducibility
|
||||||
|
seed: 42 # Random seed
|
||||||
|
|
||||||
|
# DPO / ORPO Configuration
|
||||||
|
dpo:
|
||||||
|
beta: 0.1 # KL penalty coefficient for DPO (standard starting point)
|
||||||
|
orpo_lambda: 0.1 # ORPO odds-ratio weight (same scale as DPO beta)
|
||||||
|
dpo_learning_rate: 0.00005 # Lower LR than SFT — DPO is sensitive to overshooting
|
||||||
|
|
||||||
|
# Weights & Biases Configuration
|
||||||
|
use_wandb: false # Enable W&B logging
|
||||||
|
wandb:
|
||||||
|
project: "qwen-finetuning" # W&B project name
|
||||||
|
run_name: "qwen-dpo-stage2" # W&B run name
|
||||||
|
tags: ["qwen", "unsloth", "lora"] # W&B tags
|
||||||
|
|
||||||
|
# Hardware-specific configurations
|
||||||
|
hardware:
|
||||||
|
# For different GPU memory configurations
|
||||||
|
gpu_16gb:
|
||||||
|
model_name: "unsloth/Qwen1.5-3B"
|
||||||
|
per_device_train_batch_size: 2
|
||||||
|
gradient_accumulation_steps: 4
|
||||||
|
max_seq_length: 2048
|
||||||
|
|
||||||
|
gpu_24gb:
|
||||||
|
model_name: "unsloth/Qwen1.5-3B"
|
||||||
|
per_device_train_batch_size: 4
|
||||||
|
gradient_accumulation_steps: 2
|
||||||
|
max_seq_length: 4096
|
||||||
|
|
||||||
|
gpu_40gb:
|
||||||
|
model_name: "unsloth/Qwen1.5-3B"
|
||||||
|
per_device_train_batch_size: 2
|
||||||
|
gradient_accumulation_steps: 4
|
||||||
|
max_seq_length: 4096
|
||||||
|
|
||||||
|
# Evaluation Configuration (optional)
|
||||||
|
evaluation:
|
||||||
|
eval_steps: 50 # Evaluation frequency (reduced for small dataset)
|
||||||
|
metric_for_best_model: "loss" # Metric to track for best model
|
||||||
|
load_best_model_at_end: true # Load best model at end of training
|
||||||
|
save_total_limit: 2 # Keep only 2 checkpoints to save disk
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7ecb23999bffdafdd9ed89eb834305d567907f31fa4fd54f0022e075b32ffc0c
|
||||||
|
size 3087467144
|
||||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:bd5948af71b4f56cf697f7580814c7ce8b80595ef985544efcacf716126a2e31
|
||||||
|
size 11422356
|
||||||
16
tokenizer_config.json
Normal file
16
tokenizer_config.json
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
{
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"backend": "tokenizers",
|
||||||
|
"bos_token": null,
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"eos_token": "<|im_end|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"is_local": false,
|
||||||
|
"model_max_length": 32768,
|
||||||
|
"pad_token": "<|PAD_TOKEN|>",
|
||||||
|
"padding_side": "left",
|
||||||
|
"split_special_tokens": false,
|
||||||
|
"tokenizer_class": "Qwen2Tokenizer",
|
||||||
|
"unk_token": null,
|
||||||
|
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %} {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n"
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user