flowxai/sentinel-gate
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: flowxai/sentinel-gate
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-20 05:19:09 +08:00
commit 988a0f9abc
31 changed files with 3638 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

385
README.md Normal file
View File

@@ -0,0 +1,385 @@
---
license: apache-2.0
language:
- en
base_model: Qwen/Qwen3-4B
library_name: transformers
pipeline_tag: text-generation
tags:
- lora
- regulatory
- compliance
- escalation
- decision-gate
- mlx
- gguf
- flowx
model-index:
- name: sentinel-gate
results:
- task:
type: text-generation
name: Escalation gate decision (ESCALATE vs DECIDE) on regulated-decision cases
dataset:
name: FlowX Sentinel held-out cases
type: flowxai/sentinel-gate
metrics:
- type: false_negative_rate
name: false-negative rate (missed escalations, the safety metric)
value: 0.000
- type: accuracy
name: action accuracy (ESCALATE vs DECIDE)
value: 1.000
- type: false_positive_rate
name: false-positive rate (over-escalation)
value: 0.000
- type: accuracy
name: raw JSON validity (deploy with deterministic repair)
value: 0.89
- type: accuracy
name: category accuracy on true-escalate (routing hint)
value: 0.61
---
## Inference contract
Running this model correctly requires its **frozen inference contract** - the exact
system prompt, output JSON schema, user-turn format, and decode spec it was trained
against. See [`inference_contract/`](./inference_contract):
- [`INFERENCE.md`](./inference_contract/INFERENCE.md) - wiring guide: system prompt, user turn `Case:\n<case JSON + "policy_schema">\n\nDecide: ESCALATE or DECIDE?`, decode settings (`enable_thinking=False`, temp 0, `max_new_tokens` ~1200), the six categories, and the **deterministic JSON repair step** (raw JSON validity 0.89) the deployed pipeline pairs with the model.
- [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt) - the system prompt, verbatim.
- [`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json) - output JSON Schema for the oracle decision.
Prompt version `sentinel_sys_v1`. Do not edit the prompt/schema; the weights are trained against them.
# FlowX Sentinel Gate (4B) - the escalation decision gate
**An on-device escalation gate for regulated decisions: given a complete case, it decides ESCALATE (route to a human) vs DECIDE (safe to automate), with a category, rationale, calibrated confidence, and an audit trail.**
Sentinel Gate is a **LoRA fine-tune of Qwen3-4B** (Apache-2.0), built by FlowX.AI. It reads
a complete regulated-decision case - the domain facts plus the applicable `policy_schema`
(a `PDP...` policy id) - and returns a single strict JSON decision. It sits **after
[`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper)** in a compliance
pipeline: the Mapper tags knowledge-base chunks, a policy layer assembles a case, and the
Sentinel Gate decides what is safe to automate versus what must be routed to a human.
The product insight that shapes everything: **the gate's job is the ESCALATE/DECIDE
decision, and that decision is safety-critical, so it is the metric we optimize and the field
you gate on.** A compliance automation gate that misses an escalation is dangerous in a way
that a wrong routing label is not. So we lead with the false-negative rate (missed
escalations), hold it at **0.000**, and treat the escalation *category* as a secondary
routing hint (~0.61) rather than a second gate. The model never over-escalates either
(false-positive rate 0.000), which is what makes the automation worth having.
Part of the **FlowX on-device model family**:
[`flowxai/caveat`](https://huggingface.co/flowxai/caveat),
[`flowxai/scam-guard-qwen06b`](https://huggingface.co/flowxai/scam-guard-qwen06b),
[`flowxai/scam-guard-qwen17b`](https://huggingface.co/flowxai/scam-guard-qwen17b),
[`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper).
> **Not legal advice - decision-support only.** Sentinel Gate is a compliance automation
> gate, not a legal opinion and not a substitute for a compliance officer. It decides
> *whether a case is safe to automate or must go to a human*; a human owns every escalated
> case. Deploy with the confidence threshold your risk posture requires.
English at v1. Domains span **banking, insurance, logistics, and labor** (cases are EN).
On-device formats: **fp16 safetensors** (transformers/CUDA/vLLM), **MLX-quantized** (int4 +
int8, Apple Silicon), and **GGUF** (llama.cpp/CPU/Ollama, `Q8_0` + `Q4_K_M`).
---
## How do I use it?
Three copy-pasteable ways to turn a case into a decision. The system prompt is the exact
two-liner from [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt).
The user turn is `Case:\n<case JSON>\n\nDecide: ESCALATE or DECIDE?`.
Real example input (an OFAC-screening wire transfer - a `POLICY_VIOLATION` escalation):
```
Case:
{
"transaction_facts": {
"product": "international wire transfer",
"originator": "Crestwood Imports Inc",
"beneficiary_name": "Volna Trading LLC",
"beneficiary_country": "Cyprus",
"amount_usd": 118000,
"purpose": "machinery purchase",
"screening_result": "potential match - beneficiary owner on SDN-adjacent watchlist (50% rule concern)"
},
"documents_provided": ["wire request form", "commercial invoice", "originator KYC on file"],
"documents_missing": [],
"policy_schema": "PDP.lending.payments.ofac_sanctions_block"
}
Decide: ESCALATE or DECIDE?
```
### (a) transformers / CUDA (fp16 safetensors)
The root of the repo is fp16 safetensors for transformers / vLLM.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
SYSTEM = ("You are an escalation gate for regulated decisions.\n"
"Determine: ESCALATE or DECIDE? Output ONLY JSON.")
tok = AutoTokenizer.from_pretrained("flowxai/sentinel-gate")
model = AutoModelForCausalLM.from_pretrained("flowxai/sentinel-gate", torch_dtype="float16", device_map="cuda")
case = open("wire_case.json").read() # the case JSON above
user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?"
prompt = tok.apply_chat_template(
[{"role": "system", "content": SYSTEM},
{"role": "user", "content": user}],
add_generation_prompt=True, enable_thinking=False, return_tensors="pt",
).to("cuda")
out = model.generate(prompt, max_new_tokens=1200, do_sample=False, temperature=0.0)
raw = tok.decode(out[0][prompt.shape[-1]:], skip_special_tokens=True)
# then: deterministic JSON repair -> validate against schema_sentinel_v1.json
```
### (b) MLX (Apple Silicon)
Off the MLX-quantized weights (int4/int8):
```bash
mlx_lm.generate --model flowxai/sentinel-gate-mlx-int4 --temp 0 --max-tokens 1200 \
--system-prompt "$(cat inference_contract/prompt_sentinel_sys_v1.txt)" \
--prompt "$(printf 'Case:\n%s\n\nDecide: ESCALATE or DECIDE?' "$(cat wire_case.json)")"
```
```python
from mlx_lm import load, generate
SYSTEM = open("inference_contract/prompt_sentinel_sys_v1.txt").read()
model, tok = load("flowxai/sentinel-gate-mlx-int4")
case = open("wire_case.json").read()
user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?"
prompt = tok.apply_chat_template(
[{"role": "system", "content": SYSTEM},
{"role": "user", "content": user}],
add_generation_prompt=True, enable_thinking=False,
)
raw = generate(model, tok, prompt=prompt, max_tokens=1200, verbose=False)
```
### (c) Short note - decode discipline
Two things are non-negotiable: **`enable_thinking=False`** (the adapter was trained on pure
JSON, so the default thinking template yields empty/degraded output) and **temp 0** (the gate
must be reproducible for audit). Give it **~1200 new tokens** - the oracle JSON can run long
and truncation is the main cause of invalid JSON. Always run the output through the
deterministic JSON repair step (see [`INFERENCE.md`](./inference_contract/INFERENCE.md)).
All three backends return the **same strict JSON** for the wire-transfer case above:
```json
{
"action": "ESCALATE",
"escalation_category": "POLICY_VIOLATION",
"policy_violations": {
"ofac_potential_match": {
"triggered": true,
"policy": "PDP.lending.payments.ofac_sanctions_block",
"regulation": "31 CFR 501 / OFAC sanctions regulations; OFAC 50 Percent Rule; 31 CFR 1010 (BSA recordkeeping for funds transfers)",
"restriction": "Beneficiary ownership screening returned a potential SDN-adjacent match invoking the OFAC 50% Rule; payment is automatically held pending sanctions determination.",
"consequence": "Releasing the wire without OFAC clearance risks a sanctions violation, blocking/rejection obligations, and civil penalties; the transaction must be blocked from auto-release."
}
},
"confidence_score": 0.33,
"confidence_reasoning": "Transaction documentation is complete, but a potential OFAC match on beneficiary ownership is a hard stop; only a sanctions analyst can clear, reject, or block the wire.",
"human_action_required": "OFAC/sanctions analyst to investigate the potential match, apply the 50% Rule analysis, and clear, reject, or block the wire per OFAC procedures.",
"audit_trail": [
"Ran beneficiary screening; returned potential SDN-adjacent ownership match",
"Triggered OFAC 50 Percent Rule review condition",
"Matched POLICY_VIOLATION on sanctions block policy",
"Held wire and routed to sanctions analyst; auto-release prohibited"
]
}
```
For a case with no blocking condition and complete documentation, the gate returns
`{"action": "DECIDE", "escalation_category": null, "decision": "...", ...}` with a high
`confidence_score` and `human_action_required: "NONE"`.
---
## How it works
```
+----------------------------------+
case JSON (facts + | Sentinel Gate 4B |
policy_schema: "PDP...") ---> | Qwen3-4B (LoRA), thinking off | ---> strict JSON
Decide: ESCALATE or DECIDE? | greedy decode -> JSON repair |
+----------------------------------+
|
+ action (ESCALATE | DECIDE) <- the gate
+ escalation_category <- routing hint
+ category-specific block
+ confidence_score / reasoning
+ human_action_required
+ audit_trail
```
In the pipeline: `semantic-mapper` tags KB chunks -> a policy layer assembles a case ->
**Sentinel Gate decides automate vs route-to-human**. Under the hood, one turn is:
`apply_chat_template(enable_thinking=False) -> Qwen3-4B (LoRA fine-tuned) -> greedy decode ->
deterministic JSON repair -> validate against schema_sentinel_v1.json`.
**The `action` field is the safety-critical output.** You branch your automation off it and
nothing else. The escalation_category, block, and confidence enrich the human hand-off but
never decide it.
---
## Output schema
A single strict JSON object. Full schema:
[`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json).
- `action` - `ESCALATE` | `DECIDE`. **The gate.** `ESCALATE` routes the case to a human;
`DECIDE` marks it safe to automate.
- `escalation_category` - one of the six ids when `action=ESCALATE`, else `null`. A
**routing hint** (~0.61), not a second gate.
- **category-specific block** - one object whose key depends on the category:
- `policy_violations` (POLICY_VIOLATION) - keyed by violation; each entry names `policy`,
`regulation`, `restriction`, `consequence`.
- `missing_preconditions` (MISSING_REQUIRED_DOCUMENTATION) - each entry names `required_by`,
`regulation`, `severity`, `reason`.
- `boundary_analysis` (BOUNDARY_CONDITION) - `policy_threshold`, the case value,
`distance_from_threshold`, `assessment`.
- `confidence_factors` (INSUFFICIENT_CONFIDENCE) - `ambiguous_signals[]`, `why_uncertain`.
- `conflicting_signals` (CONFLICTING_SIGNALS) - the competing `{source, value}` items.
- `external_dependency` (EXTERNAL_DEPENDENCY) - `awaiting`, `blocking_gate`.
- `confidence_score` - number 0.0–1.0. Calibrated; apply your own threshold. Advisory.
- `confidence_reasoning` - one to three sentences explaining the score and the decision.
- `human_action_required` - the concrete next step a human must take, or `"NONE"` for DECIDE.
- `audit_trail` - ordered, append-only list of reasoning steps and policy gates, for review.
- (DECIDE cases also carry `decision` / `selected_route` / `policy_gates_passed`.)
**The six escalation categories:** `MISSING_REQUIRED_DOCUMENTATION`, `POLICY_VIOLATION`,
`BOUNDARY_CONDITION`, `INSUFFICIENT_CONFIDENCE`, `CONFLICTING_SIGNALS`, `EXTERNAL_DEPENDENCY`.
---
## Evaluation
Held-out set: **n=71 cases (46 escalate / 17 decide)**. Metrics in order of what they tell you.
### The safety metric first: missed escalations
The dangerous failure for a compliance gate is **waving a case through that should have gone
to a human** (a missed escalation, i.e. a false negative). That is the number to read first.
| Metric | Result | What it means |
| --- | --- | --- |
| **False-negative rate (missed escalations)** | **0.000** (0/46) | **the safety metric** - every case that should escalate did |
| Action accuracy (ESCALATE vs DECIDE) | **1.000** | the gate decision is correct on every held-out case |
| False-positive rate (over-escalation) | **0.000** (0/17) | no safe-to-automate case was needlessly routed to a human |
| Raw JSON validity (no repair) | 0.89 | deploy with the deterministic repair step (see below) |
| Category accuracy (on true-escalate) | 0.61 | the human-routing label; categories legitimately overlap |
### Decision confusion (ESCALATE vs DECIDE), n=71
Rows = gold, cols = predicted:
| gold \ pred | ESCALATE | DECIDE | recall |
| --- | --- | --- | --- |
| ESCALATE | **46** | 0 | 1.000 (n=46) |
| DECIDE | 0 | **17** | 1.000 (n=17) |
The diagonal is complete: **no missed escalations, no over-escalation, action decided
correctly every time** on this held-out set. This is the gate's job, and on this set it is
perfect.
### Honest reading
- **The decision is the product, and it is perfect on this set.** Gate on `action`.
- **The category is a secondary routing hint at ~0.61.** The six categories legitimately
overlap for some cases (a boundary case that is also a policy edge; a missing-doc case that
is also insufficient-confidence), so the label is genuinely ambiguous for a slice of
escalations. Use it to pick a specialist queue, not to make a correctness-critical branch,
and let a human re-label at intake.
- **Raw JSON validity is 0.89, so ship the deterministic repair step.** The deployed pipeline
pairs the model with a deterministic JSON repair/retry pass and schema validation; 0.89 is
the *raw* number before that pass, not the deployed one.
- **Home-field caveat.** These 71 cases are **realistic synthetic** - grounded in real
regulatory citations (31 CFR 1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul
muncii) but authored for the benchmark, not drawn from live traffic. The scenarios share the
distribution the model trained on. **Validate on your own case distribution before
production**; a perfect held-out gate is a necessary signal, not a promise for your traffic.
---
## Formats
| Path | Format | Runs on |
| --- | --- | --- |
| `/` (root) | fp16 safetensors | transformers / CUDA / vLLM |
| `mlx-int4/` | MLX int4 | Apple Silicon (smallest footprint) |
| `mlx-int8/` | MLX int8 | Apple Silicon (higher fidelity) |
| `gguf/*.gguf` | GGUF `Q8_0` / `Q4_K_M` | llama.cpp / CPU / Ollama |
`Q8_0` is the recommended GGUF quant; MLX int4 is the fastest quality-holding path on Apple
Silicon. All formats use the same frozen inference contract (`enable_thinking=False`, temp 0,
`max_new_tokens` ~1200).
---
## Intended use & limitations
**Intended use.** A **compliance automation gate**: it decides which regulated-decision cases
are safe to automate and which must be routed to a human, with a structured rationale and an
audit trail for the hand-off. It sits after `semantic-mapper` and before a human queue.
**Out of scope & limitations.**
- **Not legal advice.** A verdict is an automation-routing signal, not a legal opinion. A
human owns every escalated case, and the recommended action always routes to a human.
- **Category label is ~61% accurate.** Gate on `action` (ESCALATE/DECIDE); treat
`escalation_category` as a routing suggestion, not ground truth.
- **JSON validity is 0.89 raw.** Deploy with the deterministic repair/retry step; do not rely
on raw output being parseable.
- **Evaluated on 71 realistic-synthetic cases.** No real client data was used in training or
eval. Validate on your own case distribution before production.
- **English, four domains (banking / insurance / logistics / labor) at v1.** Cases outside
this distribution - other domains, other languages, malformed `policy_schema` - are out of
scope and should default to escalation, not automation.
- **Depends on the input case being complete and correct.** The gate reasons over the facts it
is given; a case assembled with wrong or missing facts can produce a wrong decision. The
upstream policy layer and `semantic-mapper` are part of the trust boundary.
---
## Training data
**471 realistic-synthetic escalation cases** (400 train / 71 held-out), balanced **~35% DECIDE
/ 65% ESCALATE** across the six categories and four regulated domains (banking, insurance,
logistics, labor). Each case is **grounded in a real regulatory citation** (for example 31 CFR
1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul muncii). The scenarios are
**realistic synthetic** - authored to reflect real regulatory conditions - with **no real
client data**. The assistant turn in each record is the exact oracle JSON the model must emit,
with thinking disabled.
## Fine-tuning
LoRA (rank 32 / scale 16 / dropout 0.05), `num_layers -1` (all layers), from `Qwen/Qwen3-4B`.
Trained with **MLX-LM on Apple Silicon**: 400 iterations (~4 epochs), **cosine LR 5e-5 →
5e-6** with a 40-step warmup, `max_seq_length 2048`, seed 42, thinking disabled.
> **Stability note.** An initial `1e-4` run **diverged**; `5e-5` with the 40-step warmup is
> the stable recipe.
---
## License
Apache-2.0 (weights and code). Copyright 2026 FlowX.AI. `NOTICE` present. Base model
`Qwen/Qwen3-4B` is Apache-2.0. Escalation scenarios are realistic synthetic, grounded in real
regulatory citations (no real client data).
_Author: Bogdan Răduță, Head of Research, FlowX.AI._