sentinel-gate/inference_contract/INFERENCE.md

# Inference contract - FlowX Sentinel Gate

This is the **frozen inference contract** for `flowxai/sentinel-gate`: the exact system
prompt, user-turn format, decode settings, and output schema the weights were trained
against. Do not edit the prompt or schema; the LoRA was trained on them verbatim.

Prompt version: `sentinel_sys_v1`.

Files in this directory:

- [`prompt_sentinel_sys_v1.txt`](./prompt_sentinel_sys_v1.txt) - the system prompt, verbatim.
- [`schema_sentinel_v1.json`](./schema_sentinel_v1.json) - JSON Schema for the oracle output.

---

## System prompt (verbatim)

The exact two-line system prompt is in
[`prompt_sentinel_sys_v1.txt`](./prompt_sentinel_sys_v1.txt):

```
You are an escalation gate for regulated decisions.
Determine: ESCALATE or DECIDE? Output ONLY JSON.
```

## User-turn format

One case per turn. The case JSON carries the domain facts plus the applicable
`policy_schema` (a `PDP...` policy id), then a fixed trailing question:

```
Case:
<case JSON: domain facts + "policy_schema": "PDP...">

Decide: ESCALATE or DECIDE?
```

The `Case:\n` prefix and the trailing `\n\nDecide: ESCALATE or DECIDE?` line are part of
the contract - keep them exactly. The model was trained with these delimiters framing the
case object.

## Decode settings

| Setting | Value | Why |
|---|---|---|
| `enable_thinking` | **`False`** | Qwen3-4B is a thinking model, but the adapter was trained on pure JSON with no thinking block. The default template yields empty/degraded output. Set this at `apply_chat_template`. |
| `temperature` | **`0`** (greedy) | Deterministic decisions; the gate must be reproducible for audit. |
| `max_new_tokens` | **~1200** | The oracle JSON (category block + reasoning + audit trail) can run long, especially for BOUNDARY_CONDITION and EXTERNAL_DEPENDENCY. Truncation is the main cause of invalid JSON. |

## The six escalation categories

Present as `escalation_category` when `action` is `ESCALATE` (it is `null` for `DECIDE`):

1. `MISSING_REQUIRED_DOCUMENTATION` - a hard precondition document/evidence is absent.
2. `POLICY_VIOLATION` - a policy/regulation rule is triggered and blocks auto-release.
3. `BOUNDARY_CONDITION` - the case sits near a policy threshold; the edge needs a human read.
4. `INSUFFICIENT_CONFIDENCE` - the facts do not resolve the decision to an actionable degree.
5. `CONFLICTING_SIGNALS` - two or more trusted sources disagree materially.
6. `EXTERNAL_DEPENDENCY` - the decision is blocked awaiting an outside result (screening, ruling).

Each category emits a category-specific block under a distinct key
(`policy_violations`, `missing_preconditions`, `boundary_analysis`, `confidence_factors`,
`conflicting_signals`, `external_dependency`). See `schema_sentinel_v1.json`.

## Deterministic JSON repair (deploy with it)

Raw JSON validity from the model is **0.89** on the held-out set. The deployed pipeline
pairs the model with a **deterministic JSON repair step**: parse the raw output; if it
fails, apply structural fixes (close unterminated strings/brackets, strip any trailing
prose after the final `}`, drop a leading thinking artifact if one leaks) and re-parse, then
validate against `schema_sentinel_v1.json`. On a repair failure, retry the decode once. Do
not rely on raw output being parseable; treat the repair step as part of the contract.

## What to gate on

- **Gate on the `action` field (ESCALATE vs DECIDE).** This is the decision the model is
  for, and it is **perfect on the held-out set** (n=71): zero missed escalations
  (false-negative rate 0.000) and zero over-escalation (false-positive rate 0.000). Wire
  your automate-vs-route branch off `action` alone.
- **Treat `escalation_category` as a routing hint, not ground truth.** Category accuracy on
  true-escalate is **~0.61**; the categories legitimately overlap for some cases (e.g. a
  boundary case that is also a policy edge). Use it to pick a specialist queue, but do not
  make correctness-critical branches depend on it, and let a human re-label at intake.
- `confidence_score` is calibrated per the training oracle; apply the threshold your risk
  posture requires. It is advisory, not a second gate.

## Minimal wiring (MLX)

```python
from mlx_lm import load, generate

SYSTEM = open("prompt_sentinel_sys_v1.txt").read()
model, tok = load("flowxai/sentinel-gate-mlx-int4")

case_json = "<case JSON with domain facts + policy_schema>"
user = f"Case:\n{case_json}\n\nDecide: ESCALATE or DECIDE?"

prompt = tok.apply_chat_template(
    [{"role": "system", "content": SYSTEM},
     {"role": "user", "content": user}],
    add_generation_prompt=True, enable_thinking=False,
)
raw = generate(model, tok, prompt=prompt, max_tokens=1200, verbose=False)
# then: deterministic JSON repair -> validate against schema_sentinel_v1.json
```