初始化项目,由ModelHub XC社区提供模型
Model: e12ex2/Qwen3-1.7B-SigmaRL Source: Original Platform
This commit is contained in:
72
README.md
Normal file
72
README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
---
|
||||
library_name: mlx
|
||||
license: apache-2.0
|
||||
license_link: https://huggingface.co/Qwen/Qwen3-1.7B/blob/main/LICENSE
|
||||
pipeline_tag: text-generation
|
||||
base_model: Qwen/Qwen3-1.7B
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- mlx
|
||||
- sigma
|
||||
- detection-engineering
|
||||
- cybersecurity
|
||||
- blue-team
|
||||
- security
|
||||
---
|
||||
|
||||
# Qwen3-1.7B-SigmaRL (v0.1 · prototype)
|
||||
|
||||
Threat description → **Sigma detection rule**. A small, **defensive** detection-engineering model: a LoRA fine-tune of [Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) that writes valid, backend-compilable [Sigma](https://github.com/SigmaHQ/sigma) rules from natural-language threat descriptions.
|
||||
|
||||
> **Defensive use only.** This model generates blue-team detection rules. It is not designed or intended for offensive security.
|
||||
|
||||
## Results (held-out, n=24)
|
||||
|
||||
Same 24 held-out prompts, scored by a **graded verifiable reward**: parse YAML → valid Sigma schema (pySigma) → compiles to a Splunk query (pySigma-backend-splunk).
|
||||
|
||||
| metric | base Qwen3-1.7B | **this model (SFT)** |
|
||||
|---|---|---|
|
||||
| valid + compilable rule rate | 0.0% | **45.8%** |
|
||||
| mean reward (0–0.6) | 0.10 | **0.36** |
|
||||
| tier breakdown (0.0 / 0.2 / 0.6) | 12 / 12 / 0 | 3 / 10 / 11 |
|
||||
|
||||
The base model **never once** produced a valid, compilable rule; after fine-tuning, nearly half of outputs are valid and Splunk-compilable.
|
||||
|
||||
## How it was made
|
||||
|
||||
- **Base:** `Qwen/Qwen3-1.7B` (Apache-2.0)
|
||||
- **Data:** 3,116 `(description → rule)` pairs derived from [SigmaHQ](https://github.com/SigmaHQ/sigma) (MIT)
|
||||
- **Method:** LoRA SFT via [MLX-LM](https://github.com/ml-explore/mlx-lm) — 8 layers, 600 iters, batch 1, seq 512, trained locally on an Apple M4 (peak 4.8 GB)
|
||||
- **Eval/reward:** graded verifiable reward using pySigma + the Splunk backend as the correctness oracle
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "e12ex2/Qwen3-1.7B-SigmaRL" # <-- set to your repo
|
||||
tok = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_id)
|
||||
|
||||
SYSTEM = ("You are a detection engineer. Given a threat description, output a single "
|
||||
"valid Sigma detection rule in YAML. Output only the YAML, no prose, no code fences.")
|
||||
messages = [
|
||||
{"role": "system", "content": SYSTEM},
|
||||
{"role": "user", "content": "Write a Sigma rule that detects: powershell launched with a base64 -enc command"},
|
||||
]
|
||||
text = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
|
||||
out = model.generate(**tok(text, return_tensors="pt"), max_new_tokens=512)
|
||||
print(tok.decode(out[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Prototype.** ~54% of outputs are still not fully valid Sigma (often valid YAML with an incorrect schema).
|
||||
- Trained with **512-token truncation**, so long rules were cut — a v0.2 with longer context is planned.
|
||||
- Validity is measured by **backend compilation**, not by running the rules against real logs. True-positive / false-positive evaluation against labeled telemetry is future work.
|
||||
- **Single-turn.** Generated rules should be reviewed by a detection engineer before deployment.
|
||||
|
||||
## Reward function & training code
|
||||
|
||||
The graded verifiable reward, data pipeline, and training scripts are open. Rules are scored, not hand-labeled — the same reward drives both eval and (planned) reinforcement learning.
|
||||
Reference in New Issue
Block a user