50 lines
1.4 KiB
Markdown
50 lines
1.4 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model: Qwen/Qwen3-4B
|
|
pipeline_tag: text-generation
|
|
tags: [agent-safety, guardrail, safety-monitor, qwen3]
|
|
---
|
|
|
|
# StepSafe-4B
|
|
|
|
A step-level reasoning monitor for LLM agents, trained from Qwen3-4B with a supervised warm
|
|
start and ASPO. Before each proposed agent action executes, it writes a short reasoning trace and
|
|
returns a ten-field verdict: CONTINUE lets the action run, ASK pauses it with a clarification
|
|
question, and STOP blocks it.
|
|
|
|
```
|
|
\boxed{act:<CONTINUE|ASK|STOP>, risk:<0-100>, obj:<entity|NONE>, fail:<label>, fault:<label>,
|
|
harm:<label>, src:<label>, timing:<label>, q:<question|NONE>, cite:<sequence|NONE>}
|
|
```
|
|
|
|
Table 1 of the paper (4,033 trajectories, %):
|
|
|
|
| Benchmark | Acc | mF1 | J |
|
|
|---|---:|---:|---:|
|
|
| ASSE-Safety | 70.9 | 70.9 | 44.0 |
|
|
| ASSE-Security | 83.5 | 83.5 | 66.9 |
|
|
| R-Judge | 84.0 | 84.0 | 68.0 |
|
|
| AgentHarm | 93.8 | 87.0 | 64.8 |
|
|
| ATBench | 71.6 | 71.6 | 43.2 |
|
|
| Overall | 76.5 | 76.5 | 53.0 |
|
|
|
|
## Usage
|
|
|
|
The prompts are built by the StepSafe code ([github.com/carriegu0818/StepSafe](https://github.com/carriegu0818/StepSafe)), which also evaluates the model:
|
|
|
|
```bash
|
|
MODEL=<path to this model> bash scripts/eval_stepsafe.sh
|
|
```
|
|
|
|
Enable thinking; the paper decodes greedily with up to 2,048 new tokens.
|
|
|
|
## Citation
|
|
|
|
```bibtex
|
|
@misc{stepsafe,
|
|
title = {StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning},
|
|
author = {Anonymous},
|
|
year = {2026}
|
|
}
|
|
```
|