SZL-Khipu-1.5B/repro/REPRODUCE.md

Reproducibility bundle — SZL-Khipu-1.5B-BrainNavigator

This directory is the honest, pinned, hashed reproduction bundle for the Khipu navigator. It mirrors the public evaluation harness at a pinned commit, lists every file with its sha256, and gives three graded levels of reproduction: verify → re-run → reforge.

• Pinned source: github.com/szl-holdings/szl-forge @ main 60fbe85bb4bd02ca6dbcac2db069a058d88dfe8a (full 40-hex SHA).
• Every mirrored file's exact bytes, source repo, pinned SHA, and source path are recorded in MANIFEST.json. The two signed receipts and owner_pubkey.json live in this model repo's root (not copied here) and are listed in the manifest with their sha256 for L1.
• Evidence labels used throughout: REPORTED (owner-run), AGENT-RUN (run by the bundle agent, exact command shown), RECORDED (replayed public bytes, not a live fetch), DERIVED (computed from other data), MODELED (mechanism modeled, not proved).

What is NOT public

The raw per-case model outputs from the owner's signed evaluation run are not published — only the aggregate counts are bound in eval_receipt.signed.json (planValid 11/11, groundingCorrect 4/5, abstainCorrect 2/6, hallucinatedCitationCount 0); the signed run's per-case transcripts cannot be independently reconstructed from this repository.

---

L1 — Verify the signed receipts offline (no network, no GPU)

This proves signer continuity and tamper-evidence of the committed receipts against the repo-declared Ed25519 key. It does not independently validate model quality, data provenance, or deployment — that is a separate claim.

Mechanism (as implemented in szl-forge/khipu/sign_receipt.py, re-implemented here independently in verify_receipts.py):

• The signature is Ed25519 over the receipt's canonical JSON string (UTF-8).
• Canonical JSON = recursively sorted object keys, compact separators (,/:), floats refused (doctrine: strings + ints only).
• keyId = first 16 hex chars of sha256(SPKI DER of the public key).
• The eval→training chain: eval.trainingReceiptSha256 must equal sha256(training receipt's canonical string).

# from the model repo root (so the receipts + owner_pubkey.json are present):
pip install cryptography
python3 repro/verify_receipts.py \
  --pubkey owner_pubkey.json \
  --training training_receipt.signed.json \
  --eval eval_receipt.signed.json

verify_receipts.py re-derives the keyId from the SPKI (does not trust the declared value), verifies both signatures, confirms the embedded canonical string reproduces an independent canonicalization of payload, and recomputes the eval→training hash chain. On success it prints a final PASS line and exits 0; any failed check prints [XX] and exits non-zero.

AGENT-RUN (2026-07-16): the command above was executed against the three real files in this model repo. Result: keyId re-derived as 89540347a69b789e (matches the declared key), both Ed25519 signatures valid, both canonical strings reproduced byte-identical, chain eval.trainingReceiptSha256 == sha256(training canonical) == 242b52435df315aecefb42f0ae1f87bf10acc83f9af42976986b4f0f34efc081. Final line: PASS (exit 0).

The receipt-recorded dataset hashes also cross-check against the mirrored bytes here (DERIVED): eval.jsonl 61ede148…, adversarial.jsonl 812a23b3…, khipu.schema.json b95f9927… — all equal to the datasets entries inside the signed receipts.

---

L2 — Re-run the public harness against a local runtime (llama.cpp / GGUF)

This runs the public eval harness (eval_khipu.py) against a quantized local runtime. It is a legitimate independent re-run of the harness, but it is a different artifact from the owner's signed evaluation.

Quantization changes numerics. Your counts will NOT match the signed eval receipt (11/11, 4/5, 2/6). That is expected — the receipt covers the owner's non-quantized served artifact; a Q4_K_M GGUF is a different artifact. Do not present L2 numbers as the signed eval's numbers.

The harness talks to an Ollama endpoint (OLLAMA_URL, default http://localhost:11434, /api/chat). The GGUF quants are Ollama-ready. A llama.cpp llama-server also works if you front it with an Ollama-compatible /api/chat shim; the simplest honest path is Ollama below.

# 1. pull the Q4_K_M quant (Ollama-ready GGUF repo):
ollama run hf.co/SZLHOLDINGS/SZL-Khipu-1.5B-GGUF:Q4_K_M   # then /bye
# equivalently with llama.cpp directly:
#   huggingface-cli download SZLHOLDINGS/SZL-Khipu-1.5B-GGUF \
#     <file>.Q4_K_M.gguf --local-dir ./gguf
#   ./llama-server -m ./gguf/<file>.Q4_K_M.gguf --port 8080

# 2. lay the harness out as it expects (files resolve relative to the script):
#    the mirrored files in repro/ are byte-identical to szl-forge/khipu/ @ the pinned SHA.
pip install jsonschema
export OLLAMA_URL=http://localhost:11434
python3 repro/eval_khipu.py         # served model name defaults to "khipu"

Notes (stated plainly, not apologetics):

• eval_khipu.py verifies its held-out files against a curriculum manifest.json and requires a local training_receipt.signed.json before scoring, and fails loud if either is missing or drifted. The curriculum manifest.json is part of the szl-forge working tree and is not mirrored here (it references owner-side generation); to run L2 unmodified, check out szl-forge/khipu/ at the pinned SHA. The harness bytes themselves are mirrored here so you can diff them against your checkout.
• The served model name the harness expects is khipu (ollama cp hf.co/SZLHOLDINGS/SZL-Khipu-1.5B-GGUF:Q4_K_M khipu gives it that tag).
• Worked example (RECORDED): repro/agent-run-2026-07-16/ holds three recorded L2-style traces (llama.cpp CPU, Q4_K_M, temperature=0 seed=0, exact repro commands inside each file) — and honestly, one of them (adversarial-abstain-5.json) is a recorded abstention FAILURE (the quantized model emitted NAVIGATE where it should have ABSTAINed). These are RECORDED/AGENT-RUN outputs, not the signed-receipt artifact.

---

L3 — Full owner-metal reproduction (train + sign) — owner-side

The complete forge (QLoRA training, curriculum generation, receipt signing on owner hardware) is documented in RUNBOOK-KHIPU.md in szl-forge at the pinned SHA:

https://github.com/szl-holdings/szl-forge/blob/60fbe85bb4bd02ca6dbcac2db069a058d88dfe8a/khipu/RUNBOOK-KHIPU.md

This path is owner-side: it requires the owner's training rig and the owner's private signing key (which never leaves owner hardware). Only the owner can regenerate the signed receipts; third parties reproduce at L1 (verify) and L2 (re-run the public harness on a local runtime). L3 is a pointer, not a claim that anyone else can re-sign.

---

Part of the SZL-Forge family. a-11-oy.com. Receipt integrity is verifiable from the committed files (L1); runtime/deployment status is a separate claim.