# Reproducibility bundle — SZL-Khipu-1.5B-BrainNavigator

This directory is the **honest, pinned, hashed** reproduction bundle for the
Khipu navigator. It mirrors the *public* evaluation harness at a pinned commit,
lists every file with its `sha256`, and gives three graded levels of
reproduction: **verify → re-run → reforge**.

- Pinned source: `github.com/szl-holdings/szl-forge` @ `main`
  `60fbe85bb4bd02ca6dbcac2db069a058d88dfe8a` (full 40-hex SHA).
- Every mirrored file's exact bytes, source repo, pinned SHA, and source path
  are recorded in [`MANIFEST.json`](./MANIFEST.json). The two signed receipts and
  `owner_pubkey.json` live in this model repo's **root** (not copied here) and
  are listed in the manifest with their `sha256` for L1.
- Evidence labels used throughout: **REPORTED** (owner-run), **AGENT-RUN** (run
  by the bundle agent, exact command shown), **RECORDED** (replayed public
  bytes, not a live fetch), **DERIVED** (computed from other data), **MODELED**
  (mechanism modeled, not proved).

## What is NOT public

The raw per-case model outputs from the owner's signed evaluation run are not
published — only the aggregate counts are bound in `eval_receipt.signed.json`
(`planValid` 11/11, `groundingCorrect` 4/5, `abstainCorrect` 2/6,
`hallucinatedCitationCount` 0); the signed run's per-case transcripts cannot be
independently reconstructed from this repository.

---

## L1 — Verify the signed receipts offline (no network, no GPU)

This proves **signer continuity and tamper-evidence** of the committed receipts
against the repo-declared Ed25519 key. It does **not** independently validate
model quality, data provenance, or deployment — that is a separate claim.

**Mechanism (as implemented in `szl-forge/khipu/sign_receipt.py`, re-implemented
here independently in [`verify_receipts.py`](./verify_receipts.py)):**

- The signature is Ed25519 over the receipt's `canonical` JSON string (UTF-8).
- Canonical JSON = recursively **sorted object keys**, compact separators
  (`,`/`:`), **floats refused** (doctrine: strings + ints only).
- `keyId` = first **16 hex** chars of `sha256(SPKI DER of the public key)`.
- The eval→training **chain**: `eval.trainingReceiptSha256` must equal
  `sha256(training receipt's canonical string)`.

```bash
# from the model repo root (so the receipts + owner_pubkey.json are present):
pip install cryptography
python3 repro/verify_receipts.py \
  --pubkey owner_pubkey.json \
  --training training_receipt.signed.json \
  --eval eval_receipt.signed.json
```

`verify_receipts.py` re-derives the `keyId` from the SPKI (does not trust the
declared value), verifies both signatures, confirms the embedded `canonical`
string reproduces an independent canonicalization of `payload`, and recomputes
the eval→training hash chain. On success it prints a final `PASS` line and exits
`0`; any failed check prints `[XX]` and exits non-zero.

**AGENT-RUN (2026-07-16):** the command above was executed against the three
real files in this model repo. Result: `keyId` re-derived as
`89540347a69b789e` (matches the declared key), both Ed25519 signatures valid,
both canonical strings reproduced byte-identical, chain
`eval.trainingReceiptSha256 == sha256(training canonical) ==
242b52435df315aecefb42f0ae1f87bf10acc83f9af42976986b4f0f34efc081`. Final line:
**PASS** (exit 0).

The receipt-recorded dataset hashes also cross-check against the mirrored bytes
here (**DERIVED**): `eval.jsonl` `61ede148…`, `adversarial.jsonl` `812a23b3…`,
`khipu.schema.json` `b95f9927…` — all equal to the `datasets` entries inside the
signed receipts.

---

## L2 — Re-run the public harness against a local runtime (llama.cpp / GGUF)

This runs the **public** eval harness (`eval_khipu.py`) against a **quantized**
local runtime. It is a legitimate independent re-run of the harness, but it is a
**different artifact** from the owner's signed evaluation.

> **Quantization changes numerics. Your counts will NOT match the signed eval
> receipt (11/11, 4/5, 2/6). That is expected — the receipt covers the owner's
> non-quantized served artifact; a Q4_K_M GGUF is a different artifact.** Do not
> present L2 numbers as the signed eval's numbers.

The harness talks to an **Ollama** endpoint (`OLLAMA_URL`, default
`http://localhost:11434`, `/api/chat`). The GGUF quants are Ollama-ready. A
llama.cpp `llama-server` also works if you front it with an Ollama-compatible
`/api/chat` shim; the simplest honest path is Ollama below.

```bash
# 1. pull the Q4_K_M quant (Ollama-ready GGUF repo):
ollama run hf.co/SZLHOLDINGS/SZL-Khipu-1.5B-GGUF:Q4_K_M   # then /bye
# equivalently with llama.cpp directly:
#   huggingface-cli download SZLHOLDINGS/SZL-Khipu-1.5B-GGUF \
#     <file>.Q4_K_M.gguf --local-dir ./gguf
#   ./llama-server -m ./gguf/<file>.Q4_K_M.gguf --port 8080

# 2. lay the harness out as it expects (files resolve relative to the script):
#    the mirrored files in repro/ are byte-identical to szl-forge/khipu/ @ the pinned SHA.
pip install jsonschema
export OLLAMA_URL=http://localhost:11434
python3 repro/eval_khipu.py         # served model name defaults to "khipu"
```

Notes (stated plainly, not apologetics):

- `eval_khipu.py` verifies its held-out files against a curriculum
  `manifest.json` and requires a local `training_receipt.signed.json` **before**
  scoring, and fails loud if either is missing or drifted. The curriculum
  `manifest.json` is part of the szl-forge working tree and is **not** mirrored
  here (it references owner-side generation); to run L2 unmodified, check out
  `szl-forge/khipu/` at the pinned SHA. The harness bytes themselves are mirrored
  here so you can diff them against your checkout.
- The served model name the harness expects is `khipu`
  (`ollama cp hf.co/SZLHOLDINGS/SZL-Khipu-1.5B-GGUF:Q4_K_M khipu` gives it that
  tag).
- **Worked example (RECORDED):** `repro/agent-run-2026-07-16/` holds three
  recorded L2-style traces (llama.cpp CPU, Q4_K_M, temperature=0 seed=0, exact
  repro commands inside each file) — and honestly, one of them
  (`adversarial-abstain-5.json`) is a **recorded abstention FAILURE** (the
  quantized model emitted NAVIGATE where it should have ABSTAINed). These are
  RECORDED/AGENT-RUN outputs, not the signed-receipt artifact.

---

## L3 — Full owner-metal reproduction (train + sign) — owner-side

The complete forge (QLoRA training, curriculum generation, receipt signing on
owner hardware) is documented in **`RUNBOOK-KHIPU.md`** in `szl-forge` at the
pinned SHA:

`https://github.com/szl-holdings/szl-forge/blob/60fbe85bb4bd02ca6dbcac2db069a058d88dfe8a/khipu/RUNBOOK-KHIPU.md`

This path is **owner-side**: it requires the owner's training rig and the
owner's private signing key (which never leaves owner hardware). Only the owner
can regenerate the signed receipts; third parties reproduce at L1 (verify) and
L2 (re-run the public harness on a local runtime). L3 is a pointer, not a claim
that anyone else can re-sign.

---

_Part of the SZL-Forge family. `a-11-oy.com`. Receipt integrity is verifiable
from the committed files (L1); runtime/deployment status is a separate claim._