--- license: apache-2.0 language: - en base_model: Qwen/Qwen3-4B library_name: transformers pipeline_tag: text-generation tags: - lora - regulatory - compliance - escalation - decision-gate - mlx - gguf - flowx model-index: - name: sentinel-gate results: - task: type: text-generation name: Escalation gate decision (ESCALATE vs DECIDE) on regulated-decision cases dataset: name: FlowX Sentinel held-out cases type: flowxai/sentinel-gate metrics: - type: false_negative_rate name: false-negative rate (missed escalations, the safety metric) value: 0.000 - type: accuracy name: action accuracy (ESCALATE vs DECIDE) value: 1.000 - type: false_positive_rate name: false-positive rate (over-escalation) value: 0.000 - type: accuracy name: raw JSON validity (deploy with deterministic repair) value: 0.89 - type: accuracy name: category accuracy on true-escalate (routing hint) value: 0.61 --- ## Inference contract Running this model correctly requires its **frozen inference contract** - the exact system prompt, output JSON schema, user-turn format, and decode spec it was trained against. See [`inference_contract/`](./inference_contract): - [`INFERENCE.md`](./inference_contract/INFERENCE.md) - wiring guide: system prompt, user turn `Case:\n\n\nDecide: ESCALATE or DECIDE?`, decode settings (`enable_thinking=False`, temp 0, `max_new_tokens` ~1200), the six categories, and the **deterministic JSON repair step** (raw JSON validity 0.89) the deployed pipeline pairs with the model. - [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt) - the system prompt, verbatim. - [`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json) - output JSON Schema for the oracle decision. Prompt version `sentinel_sys_v1`. Do not edit the prompt/schema; the weights are trained against them. # FlowX Sentinel Gate (4B) - the escalation decision gate **An on-device escalation gate for regulated decisions: given a complete case, it decides ESCALATE (route to a human) vs DECIDE (safe to automate), with a category, rationale, calibrated confidence, and an audit trail.** Sentinel Gate is a **LoRA fine-tune of Qwen3-4B** (Apache-2.0), built by FlowX.AI. It reads a complete regulated-decision case - the domain facts plus the applicable `policy_schema` (a `PDP...` policy id) - and returns a single strict JSON decision. It sits **after [`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper)** in a compliance pipeline: the Mapper tags knowledge-base chunks, a policy layer assembles a case, and the Sentinel Gate decides what is safe to automate versus what must be routed to a human. The product insight that shapes everything: **the gate's job is the ESCALATE/DECIDE decision, and that decision is safety-critical, so it is the metric we optimize and the field you gate on.** A compliance automation gate that misses an escalation is dangerous in a way that a wrong routing label is not. So we lead with the false-negative rate (missed escalations), hold it at **0.000**, and treat the escalation *category* as a secondary routing hint (~0.61) rather than a second gate. The model never over-escalates either (false-positive rate 0.000), which is what makes the automation worth having. Part of the **FlowX on-device model family**: [`flowxai/caveat`](https://huggingface.co/flowxai/caveat), [`flowxai/scam-guard-qwen06b`](https://huggingface.co/flowxai/scam-guard-qwen06b), [`flowxai/scam-guard-qwen17b`](https://huggingface.co/flowxai/scam-guard-qwen17b), [`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper). > **Not legal advice - decision-support only.** Sentinel Gate is a compliance automation > gate, not a legal opinion and not a substitute for a compliance officer. It decides > *whether a case is safe to automate or must go to a human*; a human owns every escalated > case. Deploy with the confidence threshold your risk posture requires. English at v1. Domains span **banking, insurance, logistics, and labor** (cases are EN). On-device formats: **fp16 safetensors** (transformers/CUDA/vLLM), **MLX-quantized** (int4 + int8, Apple Silicon), and **GGUF** (llama.cpp/CPU/Ollama, `Q8_0` + `Q4_K_M`). --- ## How do I use it? Three copy-pasteable ways to turn a case into a decision. The system prompt is the exact two-liner from [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt). The user turn is `Case:\n\n\nDecide: ESCALATE or DECIDE?`. Real example input (an OFAC-screening wire transfer - a `POLICY_VIOLATION` escalation): ``` Case: { "transaction_facts": { "product": "international wire transfer", "originator": "Crestwood Imports Inc", "beneficiary_name": "Volna Trading LLC", "beneficiary_country": "Cyprus", "amount_usd": 118000, "purpose": "machinery purchase", "screening_result": "potential match - beneficiary owner on SDN-adjacent watchlist (50% rule concern)" }, "documents_provided": ["wire request form", "commercial invoice", "originator KYC on file"], "documents_missing": [], "policy_schema": "PDP.lending.payments.ofac_sanctions_block" } Decide: ESCALATE or DECIDE? ``` ### (a) transformers / CUDA (fp16 safetensors) The root of the repo is fp16 safetensors for transformers / vLLM. ```python from transformers import AutoModelForCausalLM, AutoTokenizer SYSTEM = ("You are an escalation gate for regulated decisions.\n" "Determine: ESCALATE or DECIDE? Output ONLY JSON.") tok = AutoTokenizer.from_pretrained("flowxai/sentinel-gate") model = AutoModelForCausalLM.from_pretrained("flowxai/sentinel-gate", torch_dtype="float16", device_map="cuda") case = open("wire_case.json").read() # the case JSON above user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?" prompt = tok.apply_chat_template( [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}], add_generation_prompt=True, enable_thinking=False, return_tensors="pt", ).to("cuda") out = model.generate(prompt, max_new_tokens=1200, do_sample=False, temperature=0.0) raw = tok.decode(out[0][prompt.shape[-1]:], skip_special_tokens=True) # then: deterministic JSON repair -> validate against schema_sentinel_v1.json ``` ### (b) MLX (Apple Silicon) Off the MLX-quantized weights (int4/int8): ```bash mlx_lm.generate --model flowxai/sentinel-gate-mlx-int4 --temp 0 --max-tokens 1200 \ --system-prompt "$(cat inference_contract/prompt_sentinel_sys_v1.txt)" \ --prompt "$(printf 'Case:\n%s\n\nDecide: ESCALATE or DECIDE?' "$(cat wire_case.json)")" ``` ```python from mlx_lm import load, generate SYSTEM = open("inference_contract/prompt_sentinel_sys_v1.txt").read() model, tok = load("flowxai/sentinel-gate-mlx-int4") case = open("wire_case.json").read() user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?" prompt = tok.apply_chat_template( [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}], add_generation_prompt=True, enable_thinking=False, ) raw = generate(model, tok, prompt=prompt, max_tokens=1200, verbose=False) ``` ### (c) Short note - decode discipline Two things are non-negotiable: **`enable_thinking=False`** (the adapter was trained on pure JSON, so the default thinking template yields empty/degraded output) and **temp 0** (the gate must be reproducible for audit). Give it **~1200 new tokens** - the oracle JSON can run long and truncation is the main cause of invalid JSON. Always run the output through the deterministic JSON repair step (see [`INFERENCE.md`](./inference_contract/INFERENCE.md)). All three backends return the **same strict JSON** for the wire-transfer case above: ```json { "action": "ESCALATE", "escalation_category": "POLICY_VIOLATION", "policy_violations": { "ofac_potential_match": { "triggered": true, "policy": "PDP.lending.payments.ofac_sanctions_block", "regulation": "31 CFR 501 / OFAC sanctions regulations; OFAC 50 Percent Rule; 31 CFR 1010 (BSA recordkeeping for funds transfers)", "restriction": "Beneficiary ownership screening returned a potential SDN-adjacent match invoking the OFAC 50% Rule; payment is automatically held pending sanctions determination.", "consequence": "Releasing the wire without OFAC clearance risks a sanctions violation, blocking/rejection obligations, and civil penalties; the transaction must be blocked from auto-release." } }, "confidence_score": 0.33, "confidence_reasoning": "Transaction documentation is complete, but a potential OFAC match on beneficiary ownership is a hard stop; only a sanctions analyst can clear, reject, or block the wire.", "human_action_required": "OFAC/sanctions analyst to investigate the potential match, apply the 50% Rule analysis, and clear, reject, or block the wire per OFAC procedures.", "audit_trail": [ "Ran beneficiary screening; returned potential SDN-adjacent ownership match", "Triggered OFAC 50 Percent Rule review condition", "Matched POLICY_VIOLATION on sanctions block policy", "Held wire and routed to sanctions analyst; auto-release prohibited" ] } ``` For a case with no blocking condition and complete documentation, the gate returns `{"action": "DECIDE", "escalation_category": null, "decision": "...", ...}` with a high `confidence_score` and `human_action_required: "NONE"`. --- ## How it works ``` +----------------------------------+ case JSON (facts + | Sentinel Gate 4B | policy_schema: "PDP...") ---> | Qwen3-4B (LoRA), thinking off | ---> strict JSON Decide: ESCALATE or DECIDE? | greedy decode -> JSON repair | +----------------------------------+ | + action (ESCALATE | DECIDE) <- the gate + escalation_category <- routing hint + category-specific block + confidence_score / reasoning + human_action_required + audit_trail ``` In the pipeline: `semantic-mapper` tags KB chunks -> a policy layer assembles a case -> **Sentinel Gate decides automate vs route-to-human**. Under the hood, one turn is: `apply_chat_template(enable_thinking=False) -> Qwen3-4B (LoRA fine-tuned) -> greedy decode -> deterministic JSON repair -> validate against schema_sentinel_v1.json`. **The `action` field is the safety-critical output.** You branch your automation off it and nothing else. The escalation_category, block, and confidence enrich the human hand-off but never decide it. --- ## Output schema A single strict JSON object. Full schema: [`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json). - `action` - `ESCALATE` | `DECIDE`. **The gate.** `ESCALATE` routes the case to a human; `DECIDE` marks it safe to automate. - `escalation_category` - one of the six ids when `action=ESCALATE`, else `null`. A **routing hint** (~0.61), not a second gate. - **category-specific block** - one object whose key depends on the category: - `policy_violations` (POLICY_VIOLATION) - keyed by violation; each entry names `policy`, `regulation`, `restriction`, `consequence`. - `missing_preconditions` (MISSING_REQUIRED_DOCUMENTATION) - each entry names `required_by`, `regulation`, `severity`, `reason`. - `boundary_analysis` (BOUNDARY_CONDITION) - `policy_threshold`, the case value, `distance_from_threshold`, `assessment`. - `confidence_factors` (INSUFFICIENT_CONFIDENCE) - `ambiguous_signals[]`, `why_uncertain`. - `conflicting_signals` (CONFLICTING_SIGNALS) - the competing `{source, value}` items. - `external_dependency` (EXTERNAL_DEPENDENCY) - `awaiting`, `blocking_gate`. - `confidence_score` - number 0.0–1.0. Calibrated; apply your own threshold. Advisory. - `confidence_reasoning` - one to three sentences explaining the score and the decision. - `human_action_required` - the concrete next step a human must take, or `"NONE"` for DECIDE. - `audit_trail` - ordered, append-only list of reasoning steps and policy gates, for review. - (DECIDE cases also carry `decision` / `selected_route` / `policy_gates_passed`.) **The six escalation categories:** `MISSING_REQUIRED_DOCUMENTATION`, `POLICY_VIOLATION`, `BOUNDARY_CONDITION`, `INSUFFICIENT_CONFIDENCE`, `CONFLICTING_SIGNALS`, `EXTERNAL_DEPENDENCY`. --- ## Evaluation Held-out set: **n=71 cases (46 escalate / 17 decide)**. Metrics in order of what they tell you. ### The safety metric first: missed escalations The dangerous failure for a compliance gate is **waving a case through that should have gone to a human** (a missed escalation, i.e. a false negative). That is the number to read first. | Metric | Result | What it means | | --- | --- | --- | | **False-negative rate (missed escalations)** | **0.000** (0/46) | **the safety metric** - every case that should escalate did | | Action accuracy (ESCALATE vs DECIDE) | **1.000** | the gate decision is correct on every held-out case | | False-positive rate (over-escalation) | **0.000** (0/17) | no safe-to-automate case was needlessly routed to a human | | Raw JSON validity (no repair) | 0.89 | deploy with the deterministic repair step (see below) | | Category accuracy (on true-escalate) | 0.61 | the human-routing label; categories legitimately overlap | ### Decision confusion (ESCALATE vs DECIDE), n=71 Rows = gold, cols = predicted: | gold \ pred | ESCALATE | DECIDE | recall | | --- | --- | --- | --- | | ESCALATE | **46** | 0 | 1.000 (n=46) | | DECIDE | 0 | **17** | 1.000 (n=17) | The diagonal is complete: **no missed escalations, no over-escalation, action decided correctly every time** on this held-out set. This is the gate's job, and on this set it is perfect. ### Honest reading - **The decision is the product, and it is perfect on this set.** Gate on `action`. - **The category is a secondary routing hint at ~0.61.** The six categories legitimately overlap for some cases (a boundary case that is also a policy edge; a missing-doc case that is also insufficient-confidence), so the label is genuinely ambiguous for a slice of escalations. Use it to pick a specialist queue, not to make a correctness-critical branch, and let a human re-label at intake. - **Raw JSON validity is 0.89, so ship the deterministic repair step.** The deployed pipeline pairs the model with a deterministic JSON repair/retry pass and schema validation; 0.89 is the *raw* number before that pass, not the deployed one. - **Home-field caveat.** These 71 cases are **realistic synthetic** - grounded in real regulatory citations (31 CFR 1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul muncii) but authored for the benchmark, not drawn from live traffic. The scenarios share the distribution the model trained on. **Validate on your own case distribution before production**; a perfect held-out gate is a necessary signal, not a promise for your traffic. --- ## Formats | Path | Format | Runs on | | --- | --- | --- | | `/` (root) | fp16 safetensors | transformers / CUDA / vLLM | | `mlx-int4/` | MLX int4 | Apple Silicon (smallest footprint) | | `mlx-int8/` | MLX int8 | Apple Silicon (higher fidelity) | | `gguf/*.gguf` | GGUF `Q8_0` / `Q4_K_M` | llama.cpp / CPU / Ollama | `Q8_0` is the recommended GGUF quant; MLX int4 is the fastest quality-holding path on Apple Silicon. All formats use the same frozen inference contract (`enable_thinking=False`, temp 0, `max_new_tokens` ~1200). --- ## Intended use & limitations **Intended use.** A **compliance automation gate**: it decides which regulated-decision cases are safe to automate and which must be routed to a human, with a structured rationale and an audit trail for the hand-off. It sits after `semantic-mapper` and before a human queue. **Out of scope & limitations.** - **Not legal advice.** A verdict is an automation-routing signal, not a legal opinion. A human owns every escalated case, and the recommended action always routes to a human. - **Category label is ~61% accurate.** Gate on `action` (ESCALATE/DECIDE); treat `escalation_category` as a routing suggestion, not ground truth. - **JSON validity is 0.89 raw.** Deploy with the deterministic repair/retry step; do not rely on raw output being parseable. - **Evaluated on 71 realistic-synthetic cases.** No real client data was used in training or eval. Validate on your own case distribution before production. - **English, four domains (banking / insurance / logistics / labor) at v1.** Cases outside this distribution - other domains, other languages, malformed `policy_schema` - are out of scope and should default to escalation, not automation. - **Depends on the input case being complete and correct.** The gate reasons over the facts it is given; a case assembled with wrong or missing facts can produce a wrong decision. The upstream policy layer and `semantic-mapper` are part of the trust boundary. --- ## Training data **471 realistic-synthetic escalation cases** (400 train / 71 held-out), balanced **~35% DECIDE / 65% ESCALATE** across the six categories and four regulated domains (banking, insurance, logistics, labor). Each case is **grounded in a real regulatory citation** (for example 31 CFR 1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul muncii). The scenarios are **realistic synthetic** - authored to reflect real regulatory conditions - with **no real client data**. The assistant turn in each record is the exact oracle JSON the model must emit, with thinking disabled. ## Fine-tuning LoRA (rank 32 / scale 16 / dropout 0.05), `num_layers -1` (all layers), from `Qwen/Qwen3-4B`. Trained with **MLX-LM on Apple Silicon**: 400 iterations (~4 epochs), **cosine LR 5e-5 → 5e-6** with a 40-step warmup, `max_seq_length 2048`, seed 42, thinking disabled. > **Stability note.** An initial `1e-4` run **diverged**; `5e-5` with the 40-step warmup is > the stable recipe. --- ## License Apache-2.0 (weights and code). Copyright 2026 FlowX.AI. `NOTICE` present. Base model `Qwen/Qwen3-4B` is Apache-2.0. Escalation scenarios are realistic synthetic, grounded in real regulatory citations (no real client data). _Author: Bogdan Răduță, Head of Research, FlowX.AI._