386 lines
18 KiB
Markdown
386 lines
18 KiB
Markdown
---
|
||
license: apache-2.0
|
||
language:
|
||
- en
|
||
base_model: Qwen/Qwen3-4B
|
||
library_name: transformers
|
||
pipeline_tag: text-generation
|
||
tags:
|
||
- lora
|
||
- regulatory
|
||
- compliance
|
||
- escalation
|
||
- decision-gate
|
||
- mlx
|
||
- gguf
|
||
- flowx
|
||
model-index:
|
||
- name: sentinel-gate
|
||
results:
|
||
- task:
|
||
type: text-generation
|
||
name: Escalation gate decision (ESCALATE vs DECIDE) on regulated-decision cases
|
||
dataset:
|
||
name: FlowX Sentinel held-out cases
|
||
type: flowxai/sentinel-gate
|
||
metrics:
|
||
- type: false_negative_rate
|
||
name: false-negative rate (missed escalations, the safety metric)
|
||
value: 0.000
|
||
- type: accuracy
|
||
name: action accuracy (ESCALATE vs DECIDE)
|
||
value: 1.000
|
||
- type: false_positive_rate
|
||
name: false-positive rate (over-escalation)
|
||
value: 0.000
|
||
- type: accuracy
|
||
name: raw JSON validity (deploy with deterministic repair)
|
||
value: 0.89
|
||
- type: accuracy
|
||
name: category accuracy on true-escalate (routing hint)
|
||
value: 0.61
|
||
---
|
||
|
||
## Inference contract
|
||
|
||
Running this model correctly requires its **frozen inference contract** - the exact
|
||
system prompt, output JSON schema, user-turn format, and decode spec it was trained
|
||
against. See [`inference_contract/`](./inference_contract):
|
||
|
||
- [`INFERENCE.md`](./inference_contract/INFERENCE.md) - wiring guide: system prompt, user turn `Case:\n<case JSON + "policy_schema">\n\nDecide: ESCALATE or DECIDE?`, decode settings (`enable_thinking=False`, temp 0, `max_new_tokens` ~1200), the six categories, and the **deterministic JSON repair step** (raw JSON validity 0.89) the deployed pipeline pairs with the model.
|
||
- [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt) - the system prompt, verbatim.
|
||
- [`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json) - output JSON Schema for the oracle decision.
|
||
|
||
Prompt version `sentinel_sys_v1`. Do not edit the prompt/schema; the weights are trained against them.
|
||
|
||
# FlowX Sentinel Gate (4B) - the escalation decision gate
|
||
|
||
**An on-device escalation gate for regulated decisions: given a complete case, it decides ESCALATE (route to a human) vs DECIDE (safe to automate), with a category, rationale, calibrated confidence, and an audit trail.**
|
||
|
||
Sentinel Gate is a **LoRA fine-tune of Qwen3-4B** (Apache-2.0), built by FlowX.AI. It reads
|
||
a complete regulated-decision case - the domain facts plus the applicable `policy_schema`
|
||
(a `PDP...` policy id) - and returns a single strict JSON decision. It sits **after
|
||
[`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper)** in a compliance
|
||
pipeline: the Mapper tags knowledge-base chunks, a policy layer assembles a case, and the
|
||
Sentinel Gate decides what is safe to automate versus what must be routed to a human.
|
||
|
||
The product insight that shapes everything: **the gate's job is the ESCALATE/DECIDE
|
||
decision, and that decision is safety-critical, so it is the metric we optimize and the field
|
||
you gate on.** A compliance automation gate that misses an escalation is dangerous in a way
|
||
that a wrong routing label is not. So we lead with the false-negative rate (missed
|
||
escalations), hold it at **0.000**, and treat the escalation *category* as a secondary
|
||
routing hint (~0.61) rather than a second gate. The model never over-escalates either
|
||
(false-positive rate 0.000), which is what makes the automation worth having.
|
||
|
||
Part of the **FlowX on-device model family**:
|
||
[`flowxai/caveat`](https://huggingface.co/flowxai/caveat),
|
||
[`flowxai/scam-guard-qwen06b`](https://huggingface.co/flowxai/scam-guard-qwen06b),
|
||
[`flowxai/scam-guard-qwen17b`](https://huggingface.co/flowxai/scam-guard-qwen17b),
|
||
[`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper).
|
||
|
||
> **Not legal advice - decision-support only.** Sentinel Gate is a compliance automation
|
||
> gate, not a legal opinion and not a substitute for a compliance officer. It decides
|
||
> *whether a case is safe to automate or must go to a human*; a human owns every escalated
|
||
> case. Deploy with the confidence threshold your risk posture requires.
|
||
|
||
English at v1. Domains span **banking, insurance, logistics, and labor** (cases are EN).
|
||
On-device formats: **fp16 safetensors** (transformers/CUDA/vLLM), **MLX-quantized** (int4 +
|
||
int8, Apple Silicon), and **GGUF** (llama.cpp/CPU/Ollama, `Q8_0` + `Q4_K_M`).
|
||
|
||
---
|
||
|
||
## How do I use it?
|
||
|
||
Three copy-pasteable ways to turn a case into a decision. The system prompt is the exact
|
||
two-liner from [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt).
|
||
The user turn is `Case:\n<case JSON>\n\nDecide: ESCALATE or DECIDE?`.
|
||
|
||
Real example input (an OFAC-screening wire transfer - a `POLICY_VIOLATION` escalation):
|
||
|
||
```
|
||
Case:
|
||
{
|
||
"transaction_facts": {
|
||
"product": "international wire transfer",
|
||
"originator": "Crestwood Imports Inc",
|
||
"beneficiary_name": "Volna Trading LLC",
|
||
"beneficiary_country": "Cyprus",
|
||
"amount_usd": 118000,
|
||
"purpose": "machinery purchase",
|
||
"screening_result": "potential match - beneficiary owner on SDN-adjacent watchlist (50% rule concern)"
|
||
},
|
||
"documents_provided": ["wire request form", "commercial invoice", "originator KYC on file"],
|
||
"documents_missing": [],
|
||
"policy_schema": "PDP.lending.payments.ofac_sanctions_block"
|
||
}
|
||
|
||
Decide: ESCALATE or DECIDE?
|
||
```
|
||
|
||
### (a) transformers / CUDA (fp16 safetensors)
|
||
|
||
The root of the repo is fp16 safetensors for transformers / vLLM.
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
SYSTEM = ("You are an escalation gate for regulated decisions.\n"
|
||
"Determine: ESCALATE or DECIDE? Output ONLY JSON.")
|
||
tok = AutoTokenizer.from_pretrained("flowxai/sentinel-gate")
|
||
model = AutoModelForCausalLM.from_pretrained("flowxai/sentinel-gate", torch_dtype="float16", device_map="cuda")
|
||
|
||
case = open("wire_case.json").read() # the case JSON above
|
||
user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?"
|
||
|
||
prompt = tok.apply_chat_template(
|
||
[{"role": "system", "content": SYSTEM},
|
||
{"role": "user", "content": user}],
|
||
add_generation_prompt=True, enable_thinking=False, return_tensors="pt",
|
||
).to("cuda")
|
||
out = model.generate(prompt, max_new_tokens=1200, do_sample=False, temperature=0.0)
|
||
raw = tok.decode(out[0][prompt.shape[-1]:], skip_special_tokens=True)
|
||
# then: deterministic JSON repair -> validate against schema_sentinel_v1.json
|
||
```
|
||
|
||
### (b) MLX (Apple Silicon)
|
||
|
||
Off the MLX-quantized weights (int4/int8):
|
||
|
||
```bash
|
||
mlx_lm.generate --model flowxai/sentinel-gate-mlx-int4 --temp 0 --max-tokens 1200 \
|
||
--system-prompt "$(cat inference_contract/prompt_sentinel_sys_v1.txt)" \
|
||
--prompt "$(printf 'Case:\n%s\n\nDecide: ESCALATE or DECIDE?' "$(cat wire_case.json)")"
|
||
```
|
||
|
||
```python
|
||
from mlx_lm import load, generate
|
||
|
||
SYSTEM = open("inference_contract/prompt_sentinel_sys_v1.txt").read()
|
||
model, tok = load("flowxai/sentinel-gate-mlx-int4")
|
||
case = open("wire_case.json").read()
|
||
user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?"
|
||
prompt = tok.apply_chat_template(
|
||
[{"role": "system", "content": SYSTEM},
|
||
{"role": "user", "content": user}],
|
||
add_generation_prompt=True, enable_thinking=False,
|
||
)
|
||
raw = generate(model, tok, prompt=prompt, max_tokens=1200, verbose=False)
|
||
```
|
||
|
||
### (c) Short note - decode discipline
|
||
|
||
Two things are non-negotiable: **`enable_thinking=False`** (the adapter was trained on pure
|
||
JSON, so the default thinking template yields empty/degraded output) and **temp 0** (the gate
|
||
must be reproducible for audit). Give it **~1200 new tokens** - the oracle JSON can run long
|
||
and truncation is the main cause of invalid JSON. Always run the output through the
|
||
deterministic JSON repair step (see [`INFERENCE.md`](./inference_contract/INFERENCE.md)).
|
||
|
||
All three backends return the **same strict JSON** for the wire-transfer case above:
|
||
|
||
```json
|
||
{
|
||
"action": "ESCALATE",
|
||
"escalation_category": "POLICY_VIOLATION",
|
||
"policy_violations": {
|
||
"ofac_potential_match": {
|
||
"triggered": true,
|
||
"policy": "PDP.lending.payments.ofac_sanctions_block",
|
||
"regulation": "31 CFR 501 / OFAC sanctions regulations; OFAC 50 Percent Rule; 31 CFR 1010 (BSA recordkeeping for funds transfers)",
|
||
"restriction": "Beneficiary ownership screening returned a potential SDN-adjacent match invoking the OFAC 50% Rule; payment is automatically held pending sanctions determination.",
|
||
"consequence": "Releasing the wire without OFAC clearance risks a sanctions violation, blocking/rejection obligations, and civil penalties; the transaction must be blocked from auto-release."
|
||
}
|
||
},
|
||
"confidence_score": 0.33,
|
||
"confidence_reasoning": "Transaction documentation is complete, but a potential OFAC match on beneficiary ownership is a hard stop; only a sanctions analyst can clear, reject, or block the wire.",
|
||
"human_action_required": "OFAC/sanctions analyst to investigate the potential match, apply the 50% Rule analysis, and clear, reject, or block the wire per OFAC procedures.",
|
||
"audit_trail": [
|
||
"Ran beneficiary screening; returned potential SDN-adjacent ownership match",
|
||
"Triggered OFAC 50 Percent Rule review condition",
|
||
"Matched POLICY_VIOLATION on sanctions block policy",
|
||
"Held wire and routed to sanctions analyst; auto-release prohibited"
|
||
]
|
||
}
|
||
```
|
||
|
||
For a case with no blocking condition and complete documentation, the gate returns
|
||
`{"action": "DECIDE", "escalation_category": null, "decision": "...", ...}` with a high
|
||
`confidence_score` and `human_action_required: "NONE"`.
|
||
|
||
---
|
||
|
||
## How it works
|
||
|
||
```
|
||
+----------------------------------+
|
||
case JSON (facts + | Sentinel Gate 4B |
|
||
policy_schema: "PDP...") ---> | Qwen3-4B (LoRA), thinking off | ---> strict JSON
|
||
Decide: ESCALATE or DECIDE? | greedy decode -> JSON repair |
|
||
+----------------------------------+
|
||
|
|
||
+ action (ESCALATE | DECIDE) <- the gate
|
||
+ escalation_category <- routing hint
|
||
+ category-specific block
|
||
+ confidence_score / reasoning
|
||
+ human_action_required
|
||
+ audit_trail
|
||
```
|
||
|
||
In the pipeline: `semantic-mapper` tags KB chunks -> a policy layer assembles a case ->
|
||
**Sentinel Gate decides automate vs route-to-human**. Under the hood, one turn is:
|
||
`apply_chat_template(enable_thinking=False) -> Qwen3-4B (LoRA fine-tuned) -> greedy decode ->
|
||
deterministic JSON repair -> validate against schema_sentinel_v1.json`.
|
||
|
||
**The `action` field is the safety-critical output.** You branch your automation off it and
|
||
nothing else. The escalation_category, block, and confidence enrich the human hand-off but
|
||
never decide it.
|
||
|
||
---
|
||
|
||
## Output schema
|
||
|
||
A single strict JSON object. Full schema:
|
||
[`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json).
|
||
|
||
- `action` - `ESCALATE` | `DECIDE`. **The gate.** `ESCALATE` routes the case to a human;
|
||
`DECIDE` marks it safe to automate.
|
||
- `escalation_category` - one of the six ids when `action=ESCALATE`, else `null`. A
|
||
**routing hint** (~0.61), not a second gate.
|
||
- **category-specific block** - one object whose key depends on the category:
|
||
- `policy_violations` (POLICY_VIOLATION) - keyed by violation; each entry names `policy`,
|
||
`regulation`, `restriction`, `consequence`.
|
||
- `missing_preconditions` (MISSING_REQUIRED_DOCUMENTATION) - each entry names `required_by`,
|
||
`regulation`, `severity`, `reason`.
|
||
- `boundary_analysis` (BOUNDARY_CONDITION) - `policy_threshold`, the case value,
|
||
`distance_from_threshold`, `assessment`.
|
||
- `confidence_factors` (INSUFFICIENT_CONFIDENCE) - `ambiguous_signals[]`, `why_uncertain`.
|
||
- `conflicting_signals` (CONFLICTING_SIGNALS) - the competing `{source, value}` items.
|
||
- `external_dependency` (EXTERNAL_DEPENDENCY) - `awaiting`, `blocking_gate`.
|
||
- `confidence_score` - number 0.0–1.0. Calibrated; apply your own threshold. Advisory.
|
||
- `confidence_reasoning` - one to three sentences explaining the score and the decision.
|
||
- `human_action_required` - the concrete next step a human must take, or `"NONE"` for DECIDE.
|
||
- `audit_trail` - ordered, append-only list of reasoning steps and policy gates, for review.
|
||
- (DECIDE cases also carry `decision` / `selected_route` / `policy_gates_passed`.)
|
||
|
||
**The six escalation categories:** `MISSING_REQUIRED_DOCUMENTATION`, `POLICY_VIOLATION`,
|
||
`BOUNDARY_CONDITION`, `INSUFFICIENT_CONFIDENCE`, `CONFLICTING_SIGNALS`, `EXTERNAL_DEPENDENCY`.
|
||
|
||
---
|
||
|
||
## Evaluation
|
||
|
||
Held-out set: **n=71 cases (46 escalate / 17 decide)**. Metrics in order of what they tell you.
|
||
|
||
### The safety metric first: missed escalations
|
||
|
||
The dangerous failure for a compliance gate is **waving a case through that should have gone
|
||
to a human** (a missed escalation, i.e. a false negative). That is the number to read first.
|
||
|
||
| Metric | Result | What it means |
|
||
| --- | --- | --- |
|
||
| **False-negative rate (missed escalations)** | **0.000** (0/46) | **the safety metric** - every case that should escalate did |
|
||
| Action accuracy (ESCALATE vs DECIDE) | **1.000** | the gate decision is correct on every held-out case |
|
||
| False-positive rate (over-escalation) | **0.000** (0/17) | no safe-to-automate case was needlessly routed to a human |
|
||
| Raw JSON validity (no repair) | 0.89 | deploy with the deterministic repair step (see below) |
|
||
| Category accuracy (on true-escalate) | 0.61 | the human-routing label; categories legitimately overlap |
|
||
|
||
### Decision confusion (ESCALATE vs DECIDE), n=71
|
||
|
||
Rows = gold, cols = predicted:
|
||
|
||
| gold \ pred | ESCALATE | DECIDE | recall |
|
||
| --- | --- | --- | --- |
|
||
| ESCALATE | **46** | 0 | 1.000 (n=46) |
|
||
| DECIDE | 0 | **17** | 1.000 (n=17) |
|
||
|
||
The diagonal is complete: **no missed escalations, no over-escalation, action decided
|
||
correctly every time** on this held-out set. This is the gate's job, and on this set it is
|
||
perfect.
|
||
|
||
### Honest reading
|
||
|
||
- **The decision is the product, and it is perfect on this set.** Gate on `action`.
|
||
- **The category is a secondary routing hint at ~0.61.** The six categories legitimately
|
||
overlap for some cases (a boundary case that is also a policy edge; a missing-doc case that
|
||
is also insufficient-confidence), so the label is genuinely ambiguous for a slice of
|
||
escalations. Use it to pick a specialist queue, not to make a correctness-critical branch,
|
||
and let a human re-label at intake.
|
||
- **Raw JSON validity is 0.89, so ship the deterministic repair step.** The deployed pipeline
|
||
pairs the model with a deterministic JSON repair/retry pass and schema validation; 0.89 is
|
||
the *raw* number before that pass, not the deployed one.
|
||
- **Home-field caveat.** These 71 cases are **realistic synthetic** - grounded in real
|
||
regulatory citations (31 CFR 1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul
|
||
muncii) but authored for the benchmark, not drawn from live traffic. The scenarios share the
|
||
distribution the model trained on. **Validate on your own case distribution before
|
||
production**; a perfect held-out gate is a necessary signal, not a promise for your traffic.
|
||
|
||
---
|
||
|
||
## Formats
|
||
|
||
| Path | Format | Runs on |
|
||
| --- | --- | --- |
|
||
| `/` (root) | fp16 safetensors | transformers / CUDA / vLLM |
|
||
| `mlx-int4/` | MLX int4 | Apple Silicon (smallest footprint) |
|
||
| `mlx-int8/` | MLX int8 | Apple Silicon (higher fidelity) |
|
||
| `gguf/*.gguf` | GGUF `Q8_0` / `Q4_K_M` | llama.cpp / CPU / Ollama |
|
||
|
||
`Q8_0` is the recommended GGUF quant; MLX int4 is the fastest quality-holding path on Apple
|
||
Silicon. All formats use the same frozen inference contract (`enable_thinking=False`, temp 0,
|
||
`max_new_tokens` ~1200).
|
||
|
||
---
|
||
|
||
## Intended use & limitations
|
||
|
||
**Intended use.** A **compliance automation gate**: it decides which regulated-decision cases
|
||
are safe to automate and which must be routed to a human, with a structured rationale and an
|
||
audit trail for the hand-off. It sits after `semantic-mapper` and before a human queue.
|
||
|
||
**Out of scope & limitations.**
|
||
|
||
- **Not legal advice.** A verdict is an automation-routing signal, not a legal opinion. A
|
||
human owns every escalated case, and the recommended action always routes to a human.
|
||
- **Category label is ~61% accurate.** Gate on `action` (ESCALATE/DECIDE); treat
|
||
`escalation_category` as a routing suggestion, not ground truth.
|
||
- **JSON validity is 0.89 raw.** Deploy with the deterministic repair/retry step; do not rely
|
||
on raw output being parseable.
|
||
- **Evaluated on 71 realistic-synthetic cases.** No real client data was used in training or
|
||
eval. Validate on your own case distribution before production.
|
||
- **English, four domains (banking / insurance / logistics / labor) at v1.** Cases outside
|
||
this distribution - other domains, other languages, malformed `policy_schema` - are out of
|
||
scope and should default to escalation, not automation.
|
||
- **Depends on the input case being complete and correct.** The gate reasons over the facts it
|
||
is given; a case assembled with wrong or missing facts can produce a wrong decision. The
|
||
upstream policy layer and `semantic-mapper` are part of the trust boundary.
|
||
|
||
---
|
||
|
||
## Training data
|
||
|
||
**471 realistic-synthetic escalation cases** (400 train / 71 held-out), balanced **~35% DECIDE
|
||
/ 65% ESCALATE** across the six categories and four regulated domains (banking, insurance,
|
||
logistics, labor). Each case is **grounded in a real regulatory citation** (for example 31 CFR
|
||
1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul muncii). The scenarios are
|
||
**realistic synthetic** - authored to reflect real regulatory conditions - with **no real
|
||
client data**. The assistant turn in each record is the exact oracle JSON the model must emit,
|
||
with thinking disabled.
|
||
|
||
## Fine-tuning
|
||
|
||
LoRA (rank 32 / scale 16 / dropout 0.05), `num_layers -1` (all layers), from `Qwen/Qwen3-4B`.
|
||
Trained with **MLX-LM on Apple Silicon**: 400 iterations (~4 epochs), **cosine LR 5e-5 →
|
||
5e-6** with a 40-step warmup, `max_seq_length 2048`, seed 42, thinking disabled.
|
||
|
||
> **Stability note.** An initial `1e-4` run **diverged**; `5e-5` with the 40-step warmup is
|
||
> the stable recipe.
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
Apache-2.0 (weights and code). Copyright 2026 FlowX.AI. `NOTICE` present. Base model
|
||
`Qwen/Qwen3-4B` is Apache-2.0. Escalation scenarios are realistic synthetic, grounded in real
|
||
regulatory citations (no real client data).
|
||
|
||
_Author: Bogdan Răduță, Head of Research, FlowX.AI._
|