commit ff5f5ef338339923a44e8d61f94f5bb9e16ffa7d
Author: ModelHub XC <noreply@modelhub.org.cn>
Date:   Mon Jul 27 22:08:26 2026 +0800

    初始化项目，由ModelHub XC社区提供模型
    
    Model: NyayaLabs98/nyaya-3b-v3
    Source: Original Platform

diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..52373fe
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..7148aa7
--- /dev/null
+++ b/README.md
@@ -0,0 +1,106 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-3B-Instruct
+language:
+- en
+- hi
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- legal
+- india
+- indian-law
+- bns
+- retrieval-augmented-generation
+- qwen2.5
+---
+
+# Nyaya-3B-v3 — Indian Legal Information Model
+
+> **⚖️ Not legal advice.** Nyaya is a legal *information / guidance* model, **not a
+> legal advisor**. The practice of law in India is reserved to enrolled advocates
+> under the Advocates Act, 1961. Always consult a licensed advocate for anything
+> consequential. Free legal aid is available via NALSA / DLSA (Legal Services
+> Authorities Act, 1987).
+
+Nyaya answers everyday legal questions from Indian citizens in **English, Hindi,
+and Hinglish**, cites specific sections of **current law** (BNS / BNSS / BSA,
+post-July-2024, with IPC↔BNS bridging), and signals when a real lawyer is needed.
+
+- **Base model:** [`Qwen/Qwen2.5-3B-Instruct`]( https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) (Apache-2.0)
+- **Method:** LoRA SFT (bf16, no quantization) using **RAFT** — teacher answers
+  regenerated under the inference-time RAG prompt, including deliberate
+  retrieval-miss demonstrations. Merged to full weights for release.
+- **Designed to run with retrieval (RAG).** The gains below come *with* a dense
+  retriever supplying statute passages in context; used bare, it behaves close
+  to the base model.
+
+## Intended use & limitations
+
+- **Use for:** plain-language legal information, "how do I…" procedural guidance,
+  old→new law (IPC↔BNS) mapping, terminology.
+- **Do not use for:** actual legal advice, court filings, or any decision without
+  a licensed advocate. Coverage is limited to the acts in the statute DB; it can
+  be wrong or incomplete outside them.
+
+## Evaluation (Nyaya-Eval-v0, 500 frozen questions)
+
+Primary metric is **citation accuracy verified against the statute DB**, not loss.
+Numbers below are strict exact-match on the frozen eval, with dense RAG (k=8):
+
+| Setting | Strict acc. | Lenient acc. |
+|---|---|---|
+| Base + dense RAG | 3.4% | — |
+| **Nyaya-3B-v3 + dense RAG** | **3.8%** | **5.5%** |
+
+On the held-out *dataset* eval, citation pass-rate is **90.2%** across
+1068 examples.
+
+**Honest status:** strict exact-match on the frozen benchmark is still low and the
+project's human-eval ship gate has **not** been passed. This is an early research
+release, **not** a validated "best" legal model. See the repo's `docs/ROADMAP.md`
+go/no-go gates.
+
+## How to use
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("NyayaLabs98/nyaya-3b-v3")
+model = AutoModelForCausalLM.from_pretrained("NyayaLabs98/nyaya-3b-v3", device_map="auto")
+
+system = (
+    "You are Nyaya, an Indian legal information model. You provide accurate, "
+    "plain-language legal guidance for Indian citizens, cite specific sections of "
+    "current law (BNS/BNSS/BSA and other acts in force), clearly state uncertainty, "
+    "and recommend consulting a licensed advocate for anything consequential. "
+    "You provide legal information, not legal advice."
+)
+# For best results, prepend retrieved statute passages to the user turn (RAG).
+messages = [
+    {"role": "system", "content": system},
+    {"role": "user", "content": "Police FIR nahi likh rahi, kya karu?"},
+]
+inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
+out = model.generate(inputs, max_new_tokens=512)
+print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
+```
+
+## Training curves & ablations
+
+![Program progression](training/figures/program_progression.png)
+![RAG ablation](training/figures/rag_ablation.png)
+![Checkpoint frozen-eval](training/figures/checkpoint_frozen_eval.png)
+![Before/after behaviour](training/figures/behaviour_before_after.png)
+
+## Training details
+
+See `training/` in this repo for the exact config, metric history, and all
+figures (PNG + source CSV): `train_v3.yaml`, `training_history_v3.json`,
+`rag_eval_rescored.json`, `checkpoint_evals.json`, `figures/`.
+
+## License & attribution
+
+Model weights released under **Apache-2.0**, inheriting the base model's license.
+Statutory text is Government of India material (India Code / legislative.gov.in).
+See the repository `NOTICE` for data provenance.
diff --git a/chat_template.jinja b/chat_template.jinja
new file mode 100644
index 0000000..9840d40
--- /dev/null
+++ b/chat_template.jinja
@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}
diff --git a/config.json b/config.json
new file mode 100644
index 0000000..a2f3910
--- /dev/null
+++ b/config.json
@@ -0,0 +1,69 @@
+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 70,
+  "model_type": "qwen2",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 2,
+  "pad_token_id": null,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 1000000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.12.1",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}
diff --git a/generation_config.json b/generation_config.json
new file mode 100644
index 0000000..5ea8619
--- /dev/null
+++ b/generation_config.json
@@ -0,0 +1,14 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "5.12.1"
+}
diff --git a/model.safetensors b/model.safetensors
new file mode 100644
index 0000000..c36d0e9
--- /dev/null
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55fa7e17aa556349f0d7fb501b63029bb0edbc2aae7f67a03d4afcbf31f69680
+size 6171927112
diff --git a/tokenizer.json b/tokenizer.json
new file mode 100644
index 0000000..34510ff
--- /dev/null
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892
diff --git a/tokenizer_config.json b/tokenizer_config.json
new file mode 100644
index 0000000..de1d535
--- /dev/null
+++ b/tokenizer_config.json
@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}
diff --git a/training/checkpoint_evals.json b/training/checkpoint_evals.json
new file mode 100644
index 0000000..68aef7c
--- /dev/null
+++ b/training/checkpoint_evals.json
@@ -0,0 +1,151 @@
+{
+  "run_dir": "outputs/legal-3b-v1",
+  "model": "Qwen/Qwen2.5-3B-Instruct",
+  "frozen_eval_questions": 500,
+  "checkpoints": {
+    "checkpoint-50": {
+      "strict_accuracy": 0.0081,
+      "strict_correct": 4,
+      "scored_total": 495,
+      "forbidden_fact_violations": 1,
+      "abstention_rate": 0.424,
+      "safety_abstained": 2,
+      "by_language": {
+        "english": "4/480",
+        "hindi": "0/10",
+        "hinglish": "0/10"
+      },
+      "eval_seconds": 977
+    },
+    "checkpoint-100": {
+      "strict_accuracy": 0.004,
+      "strict_correct": 2,
+      "scored_total": 495,
+      "forbidden_fact_violations": 1,
+      "abstention_rate": 0.342,
+      "safety_abstained": 2,
+      "by_language": {
+        "english": "2/480",
+        "hindi": "0/10",
+        "hinglish": "0/10"
+      },
+      "eval_seconds": 938
+    },
+    "checkpoint-150": {
+      "strict_accuracy": 0.0081,
+      "strict_correct": 4,
+      "scored_total": 495,
+      "forbidden_fact_violations": 1,
+      "abstention_rate": 0.452,
+      "safety_abstained": 2,
+      "by_language": {
+        "english": "4/480",
+        "hindi": "0/10",
+        "hinglish": "0/10"
+      },
+      "eval_seconds": 869
+    },
+    "checkpoint-197": {
+      "strict_accuracy": 0.0081,
+      "strict_correct": 4,
+      "scored_total": 495,
+      "forbidden_fact_violations": 1,
+      "abstention_rate": 0.474,
+      "safety_abstained": 2,
+      "by_language": {
+        "english": "4/480",
+        "hindi": "0/10",
+        "hinglish": "0/10"
+      },
+      "eval_seconds": 869
+    },
+    "final": {
+      "strict_accuracy": 0.0081,
+      "strict_correct": 4,
+      "scored_total": 495,
+      "forbidden_fact_violations": 1,
+      "abstention_rate": 0.474,
+      "safety_abstained": 2,
+      "by_language": {
+        "english": "4/480",
+        "hindi": "0/10",
+        "hinglish": "0/10"
+      },
+      "eval_seconds": 868
+    }
+  },
+  "best": "checkpoint-50",
+  "best_dataset_eval": {
+    "total": 1068,
+    "citation_pass_rate": 0.9017,
+    "answers_with_citations": 0.9785,
+    "old_law_citation_rate": 0.0028,
+    "language_match_rate": 0.9906,
+    "mean_reference_similarity": 0.2515,
+    "by_language": {
+      "english": {
+        "total": 785,
+        "citation_ok": 707,
+        "language_ok": 784,
+        "old_law_cited": 3
+      },
+      "hinglish": {
+        "total": 168,
+        "citation_ok": 151,
+        "language_ok": 167,
+        "old_law_cited": 0
+      },
+      "hindi": {
+        "total": 115,
+        "citation_ok": 105,
+        "language_ok": 107,
+        "old_law_cited": 0
+      }
+    },
+    "by_task_type": {
+      "grounded_qa": {
+        "total": 478,
+        "citation_ok": 429,
+        "language_ok": 477,
+        "old_law_cited": 0
+      },
+      "terminology": {
+        "total": 28,
+        "citation_ok": 27,
+        "language_ok": 28,
+        "old_law_cited": 0
+      },
+      "hinglish_qa": {
+        "total": 168,
+        "citation_ok": 151,
+        "language_ok": 167,
+        "old_law_cited": 0
+      },
+      "law_mapping": {
+        "total": 24,
+        "citation_ok": 20,
+        "language_ok": 24,
+        "old_law_cited": 3
+      },
+      "hindi_qa": {
+        "total": 115,
+        "citation_ok": 105,
+        "language_ok": 107,
+        "old_law_cited": 0
+      },
+      "procedural": {
+        "total": 231,
+        "citation_ok": 211,
+        "language_ok": 231,
+        "old_law_cited": 0
+      },
+      "safety_abstention": {
+        "total": 24,
+        "citation_ok": 20,
+        "language_ok": 24,
+        "old_law_cited": 0
+      }
+    }
+  },
+  "evaluated_at": "2026-07-15 08:04:21"
+}
\ No newline at end of file
diff --git a/training/dpo_train_report.json b/training/dpo_train_report.json
new file mode 100644
index 0000000..2195f33
--- /dev/null
+++ b/training/dpo_train_report.json
@@ -0,0 +1,300 @@
+{
+  "pairs": 124,
+  "sft_adapter": "outputs/legal-3b-v3/checkpoint-300",
+  "beta": 0.1,
+  "epochs": 3,
+  "optimizer_steps": 48,
+  "final_loss": 0.5268,
+  "final_margin": 0.366,
+  "history": [
+    {
+      "step": 1,
+      "epoch": 0,
+      "loss": 0.6931,
+      "margin": 0.0
+    },
+    {
+      "step": 2,
+      "epoch": 0,
+      "loss": 0.7271,
+      "margin": -0.0668
+    },
+    {
+      "step": 3,
+      "epoch": 0,
+      "loss": 0.6067,
+      "margin": 0.1811
+    },
+    {
+      "step": 4,
+      "epoch": 0,
+      "loss": 0.6631,
+      "margin": 0.0611
+    },
+    {
+      "step": 5,
+      "epoch": 0,
+      "loss": 0.7499,
+      "margin": -0.1104
+    },
+    {
+      "step": 6,
+      "epoch": 0,
+      "loss": 0.6343,
+      "margin": 0.1214
+    },
+    {
+      "step": 7,
+      "epoch": 0,
+      "loss": 0.7168,
+      "margin": -0.0468
+    },
+    {
+      "step": 8,
+      "epoch": 0,
+      "loss": 0.6645,
+      "margin": 0.0581
+    },
+    {
+      "step": 9,
+      "epoch": 0,
+      "loss": 0.6587,
+      "margin": 0.0701
+    },
+    {
+      "step": 10,
+      "epoch": 0,
+      "loss": 0.6732,
+      "margin": 0.0402
+    },
+    {
+      "step": 11,
+      "epoch": 0,
+      "loss": 0.589,
+      "margin": 0.2205
+    },
+    {
+      "step": 12,
+      "epoch": 0,
+      "loss": 0.6851,
+      "margin": 0.0162
+    },
+    {
+      "step": 13,
+      "epoch": 0,
+      "loss": 0.6826,
+      "margin": 0.0212
+    },
+    {
+      "step": 14,
+      "epoch": 0,
+      "loss": 0.5937,
+      "margin": 0.2099
+    },
+    {
+      "step": 15,
+      "epoch": 0,
+      "loss": 0.759,
+      "margin": -0.1277
+    },
+    {
+      "step": 16,
+      "epoch": 0,
+      "loss": 0.7671,
+      "margin": -0.1427
+    },
+    {
+      "step": 17,
+      "epoch": 1,
+      "loss": 0.5843,
+      "margin": 0.231
+    },
+    {
+      "step": 18,
+      "epoch": 1,
+      "loss": 0.7993,
+      "margin": -0.2021
+    },
+    {
+      "step": 19,
+      "epoch": 1,
+      "loss": 0.6293,
+      "margin": 0.132
+    },
+    {
+      "step": 20,
+      "epoch": 1,
+      "loss": 0.7464,
+      "margin": -0.1038
+    },
+    {
+      "step": 21,
+      "epoch": 1,
+      "loss": 0.6757,
+      "margin": 0.0353
+    },
+    {
+      "step": 22,
+      "epoch": 1,
+      "loss": 0.713,
+      "margin": -0.0393
+    },
+    {
+      "step": 23,
+      "epoch": 1,
+      "loss": 0.5206,
+      "margin": 0.3812
+    },
+    {
+      "step": 24,
+      "epoch": 1,
+      "loss": 0.7103,
+      "margin": -0.034
+    },
+    {
+      "step": 25,
+      "epoch": 1,
+      "loss": 0.6525,
+      "margin": 0.083
+    },
+    {
+      "step": 26,
+      "epoch": 1,
+      "loss": 0.63,
+      "margin": 0.1305
+    },
+    {
+      "step": 27,
+      "epoch": 1,
+      "loss": 0.5864,
+      "margin": 0.2263
+    },
+    {
+      "step": 28,
+      "epoch": 1,
+      "loss": 0.4905,
+      "margin": 0.457
+    },
+    {
+      "step": 29,
+      "epoch": 1,
+      "loss": 0.6076,
+      "margin": 0.1791
+    },
+    {
+      "step": 30,
+      "epoch": 1,
+      "loss": 0.6098,
+      "margin": 0.1743
+    },
+    {
+      "step": 31,
+      "epoch": 1,
+      "loss": 0.4762,
+      "margin": 0.4945
+    },
+    {
+      "step": 32,
+      "epoch": 1,
+      "loss": 0.5189,
+      "margin": 0.3854
+    },
+    {
+      "step": 33,
+      "epoch": 2,
+      "loss": 0.5235,
+      "margin": 0.374
+    },
+    {
+      "step": 34,
+      "epoch": 2,
+      "loss": 0.3293,
+      "margin": 0.9416
+    },
+    {
+      "step": 35,
+      "epoch": 2,
+      "loss": 0.4656,
+      "margin": 0.5226
+    },
+    {
+      "step": 36,
+      "epoch": 2,
+      "loss": 0.6588,
+      "margin": 0.0699
+    },
+    {
+      "step": 37,
+      "epoch": 2,
+      "loss": 0.4392,
+      "margin": 0.5951
+    },
+    {
+      "step": 38,
+      "epoch": 2,
+      "loss": 0.5389,
+      "margin": 0.3368
+    },
+    {
+      "step": 39,
+      "epoch": 2,
+      "loss": 0.7196,
+      "margin": -0.0523
+    },
+    {
+      "step": 40,
+      "epoch": 2,
+      "loss": 0.4093,
+      "margin": 0.6817
+    },
+    {
+      "step": 41,
+      "epoch": 2,
+      "loss": 0.5289,
+      "margin": 0.3609
+    },
+    {
+      "step": 42,
+      "epoch": 2,
+      "loss": 0.4894,
+      "margin": 0.4599
+    },
+    {
+      "step": 43,
+      "epoch": 2,
+      "loss": 0.6129,
+      "margin": 0.1675
+    },
+    {
+      "step": 44,
+      "epoch": 2,
+      "loss": 0.5136,
+      "margin": 0.3986
+    },
+    {
+      "step": 45,
+      "epoch": 2,
+      "loss": 0.3373,
+      "margin": 0.9133
+    },
+    {
+      "step": 46,
+      "epoch": 2,
+      "loss": 0.4089,
+      "margin": 0.683
+    },
+    {
+      "step": 47,
+      "epoch": 2,
+      "loss": 0.2918,
+      "margin": 1.0822
+    },
+    {
+      "step": 48,
+      "epoch": 2,
+      "loss": 0.5268,
+      "margin": 0.366
+    }
+  ],
+  "seconds": 760
+}
\ No newline at end of file
diff --git a/training/figures/behaviour_before_after.csv b/training/figures/behaviour_before_after.csv
new file mode 100644
index 0000000..ee2b17a
--- /dev/null
+++ b/training/figures/behaviour_before_after.csv
@@ -0,0 +1,7 @@
+metric,base,v1_checkpoint50
+answers_with_citations,0.2996,0.9785
+verified_citation_given,0.2191,0.8802
+old_law_citation_rate,0.0974,0.0028
+language_match_rate,0.9054,0.9906
+mean_reference_similarity,0.1405,0.2515
+eval_loss,2.0953,0.9593
diff --git a/training/figures/behaviour_before_after.png b/training/figures/behaviour_before_after.png
new file mode 100644
index 0000000..e56ebeb
Binary files /dev/null and b/training/figures/behaviour_before_after.png differ
diff --git a/training/figures/checkpoint_frozen_eval.csv b/training/figures/checkpoint_frozen_eval.csv
new file mode 100644
index 0000000..9517f71
--- /dev/null
+++ b/training/figures/checkpoint_frozen_eval.csv
@@ -0,0 +1,7 @@
+checkpoint,strict_accuracy,stale_law_violations,abstention_rate,safety_abstained
+base (untrained),0.0081,12,0.37,2
+checkpoint-50,0.0081,1,0.424,2
+checkpoint-100,0.004,1,0.342,2
+checkpoint-150,0.0081,1,0.452,2
+checkpoint-197,0.0081,1,0.474,2
+final,0.0081,1,0.474,2
diff --git a/training/figures/checkpoint_frozen_eval.png b/training/figures/checkpoint_frozen_eval.png
new file mode 100644
index 0000000..c6912a7
Binary files /dev/null and b/training/figures/checkpoint_frozen_eval.png differ
diff --git a/training/figures/failure_modes.csv b/training/figures/failure_modes.csv
new file mode 100644
index 0000000..fa0b172
--- /dev/null
+++ b/training/figures/failure_modes.csv
@@ -0,0 +1,5 @@
+failure_mode,count
+over_abstention,209
+wrong_or_incomplete_citation,475
+no_citation_given,16
+stale_law_cited,1
diff --git a/training/figures/failure_modes.png b/training/figures/failure_modes.png
new file mode 100644
index 0000000..2b6ec8d
Binary files /dev/null and b/training/figures/failure_modes.png differ
diff --git a/training/figures/matched_retrieval_final.csv b/training/figures/matched_retrieval_final.csv
new file mode 100644
index 0000000..882897c
--- /dev/null
+++ b/training/figures/matched_retrieval_final.csv
@@ -0,0 +1,4 @@
+model,strict,lenient,gic_n,gic_strict,gic_lenient
+base,0.0384,0.0667,98,0.1735,0.2449
+v3-ckpt300,0.0424,0.0646,98,0.1939,0.2245
+v4-ckpt354,0.0384,0.0606,98,0.1837,0.2551
diff --git a/training/figures/program_progression.csv b/training/figures/program_progression.csv
new file mode 100644
index 0000000..20c5caa
--- /dev/null
+++ b/training/figures/program_progression.csv
@@ -0,0 +1,7 @@
+config,strict_accuracy,lenient_accuracy,gold_in_context_n,gold_in_context_strict,gold_in_context_lenient
+checkpoint-50,0.0081,0.0081,,,
+rag_k8_checkpoint-50,0.0283,0.0343,85,0.1647,0.1765
+rag_k8_legal-3b-v2-checkpoint-100,0.0364,0.0505,85,0.2118,0.2824
+rag_dense_k8_base,0.0343,0.0586,94,0.1809,0.2553
+rag_dense_k8_legal-3b-v3-checkpoint-300,0.0384,0.0545,94,0.2021,0.266
+rag_dense_k8_v3-dpo,0.0343,0.0485,94,0.1809,0.234
diff --git a/training/figures/program_progression.png b/training/figures/program_progression.png
new file mode 100644
index 0000000..b5710a1
Binary files /dev/null and b/training/figures/program_progression.png differ
diff --git a/training/figures/rag_ablation.csv b/training/figures/rag_ablation.csv
new file mode 100644
index 0000000..e822796
--- /dev/null
+++ b/training/figures/rag_ablation.csv
@@ -0,0 +1,5 @@
+config,strict_accuracy,lenient_accuracy,gold_in_context_strict,gold_in_context_lenient
+checkpoint-50,0.0081,0.0081,,
+rag_k8_checkpoint-50,0.0283,0.0343,0.1647,0.1765
+rag_k8_base,0.0384,0.0586,0.2235,0.2706
+rag_k8_legal-3b-v2-checkpoint-100,0.0364,0.0505,0.2118,0.2824
diff --git a/training/figures/rag_ablation.png b/training/figures/rag_ablation.png
new file mode 100644
index 0000000..657e16c
Binary files /dev/null and b/training/figures/rag_ablation.png differ
diff --git a/training/figures/retrieval_recall.csv b/training/figures/retrieval_recall.csv
new file mode 100644
index 0000000..d06a36f
--- /dev/null
+++ b/training/figures/retrieval_recall.csv
@@ -0,0 +1,4 @@
+retriever,frozen_full_hit_at_8
+BM25 (pre-fix),0.492
+BM25 + exact + lexicon,0.644
+hybrid + e5 dense (RRF),0.712
diff --git a/training/figures/smoke_vs_v1_loss.png b/training/figures/smoke_vs_v1_loss.png
new file mode 100644
index 0000000..b23609d
Binary files /dev/null and b/training/figures/smoke_vs_v1_loss.png differ
diff --git a/training/figures/training_smoke.csv b/training/figures/training_smoke.csv
new file mode 100644
index 0000000..8e213e7
--- /dev/null
+++ b/training/figures/training_smoke.csv
@@ -0,0 +1,7 @@
+step,epoch,loss,learning_rate,grad_norm,mean_token_accuracy
+10,0.16,1.8803634643554688,9.678584095202468e-05,1.182433009147644,0.582773034274578
+20,0.32,1.1098207473754882,8.202965893490878e-05,0.5895639657974243,0.7326480627059937
+30,0.48,0.9109317779541015,5.896403794053679e-05,0.4854218065738678,0.772037148475647
+40,0.64,0.8902366638183594,3.357288090445827e-05,0.4858759939670563,0.775420868396759
+50,0.8,0.8645750999450683,1.2443403456474017e-05,0.4035021960735321,0.7814616322517395
+60,0.96,0.9060385704040528,1.0572157452321097e-06,0.37886306643486023,0.7762055635452271
diff --git a/training/figures/training_v1.csv b/training/figures/training_v1.csv
new file mode 100644
index 0000000..f46eec9
--- /dev/null
+++ b/training/figures/training_v1.csv
@@ -0,0 +1,20 @@
+step,epoch,loss,learning_rate,grad_norm,mean_token_accuracy
+10,0.050761421319796954,2.3324300765991213,9.993914063644052e-05,1.1503950357437134,0.5158391281962394
+20,0.10152284263959391,1.2313983917236329,9.886131295443003e-05,0.4007212817668915,0.6984569907188416
+30,0.15228426395939088,1.0970629692077636,9.646456421395446e-05,0.34395045042037964,0.7340504214167595
+40,0.20304568527918782,1.0139848709106445,9.281359022841965e-05,0.3407576382160187,0.7466330319643021
+50,0.25380710659898476,0.9398991584777832,8.800694230932884e-05,0.305820107460022,0.759013494849205
+60,0.30456852791878175,0.9580072402954102,8.2174367055326e-05,0.3269255459308624,0.7577866345643998
+70,0.3553299492385787,0.920842170715332,7.547330408197695e-05,0.3835376501083374,0.7638022676110268
+80,0.40609137055837563,0.8990132331848144,6.808463622961578e-05,0.3853282332420349,0.7671428829431534
+90,0.45685279187817257,0.8844776153564453,6.0207806964560584e-05,0.3447255492210388,0.7684156641364097
+100,0.5076142131979695,0.8999717712402344,5.205543677045049e-05,0.44750770926475525,0.7692662417888642
+110,0.5583756345177665,0.8709278106689453,4.384758385029457e-05,0.41725635528564453,0.772255252301693
+120,0.6091370558375635,0.8635953903198242,3.580580406099893e-05,0.4176587164402008,0.7729698359966278
+130,0.6598984771573604,0.8605537414550781,2.8147170421488272e-05,0.42824825644493103,0.7755630746483803
+140,0.7106598984771574,0.8298810005187989,2.1078413626773546e-05,0.42753881216049194,0.7806227177381515
+150,0.7614213197969543,0.848197078704834,1.4790341733986085e-05,0.37514036893844604,0.7789115384221077
+160,0.8121827411167513,0.8550724983215332,9.452689650664515e-06,0.44037938117980957,0.778565688431263
+170,0.8629441624365483,0.8581902503967285,5.209537453863289e-06,0.42417141795158386,0.7754479616880416
+180,0.9137055837563451,0.8792120933532714,2.1754212141102346e-06,0.47580745816230774,0.7731257915496826
+190,0.9644670050761421,0.8656046867370606,4.322413053509944e-07,0.474467396736145,0.7720748305320739
diff --git a/training/figures/training_v1_eval.csv b/training/figures/training_v1_eval.csv
new file mode 100644
index 0000000..dbfe804
--- /dev/null
+++ b/training/figures/training_v1_eval.csv
@@ -0,0 +1,5 @@
+step,epoch,eval_loss,eval_mean_token_accuracy
+50,0.25380710659898476,1.0766046047210693,0.7379334909575326
+100,0.5076142131979695,0.9950162172317505,0.7531140716302962
+150,0.7614213197969543,0.964945375919342,0.7585937054384322
+197,1.0,0.9593181610107422,0.7598682131086077
diff --git a/training/figures/training_v2.csv b/training/figures/training_v2.csv
new file mode 100644
index 0000000..82ddf40
--- /dev/null
+++ b/training/figures/training_v2.csv
@@ -0,0 +1,20 @@
+step,epoch,loss,learning_rate,grad_norm,mean_token_accuracy
+10,0.050793650793650794,1.5175626754760743,9.993914063644052e-05,0.2645105719566345,0.6436327129602433
+20,0.10158730158730159,1.292062759399414,9.886131295443003e-05,0.1842157244682312,0.6806577108800411
+30,0.1523809523809524,1.1331645011901856,9.646456421395446e-05,0.1463174968957901,0.7136855460703373
+40,0.20317460317460317,1.0500896453857422,9.281359022841965e-05,0.15235942602157593,0.7293100193142891
+50,0.25396825396825395,0.9814230918884277,8.800694230932884e-05,0.18486258387565613,0.7447394959628582
+60,0.3047619047619048,0.9417718887329102,8.2174367055326e-05,0.20546093583106995,0.7523589514195919
+70,0.35555555555555557,0.8714447975158691,7.547330408197695e-05,0.27494359016418457,0.7701701499521733
+80,0.40634920634920635,0.8259673118591309,6.808463622961578e-05,0.2904492914676666,0.7822930864989758
+90,0.45714285714285713,0.7809524536132812,6.0207806964560584e-05,0.6509429216384888,0.7934819318354129
+100,0.5079365079365079,0.7385403633117675,5.205543677045049e-05,0.29676851630210876,0.804052384197712
+110,0.5587301587301587,0.6973814964294434,4.384758385029457e-05,0.3625648319721222,0.8153482332825661
+120,0.6095238095238096,0.6504372119903564,3.580580406099893e-05,0.3534400761127472,0.8266014747321606
+130,0.6603174603174603,0.645383882522583,2.8147170421488272e-05,0.3655138909816742,0.8276743076741695
+140,0.7111111111111111,0.6282196044921875,2.1078413626773546e-05,0.33809009194374084,0.8325855515897274
+150,0.7619047619047619,0.602206802368164,1.4790341733986085e-05,0.3583182990550995,0.8397377617657185
+160,0.8126984126984127,0.5957026004791259,9.452689650664515e-06,0.3788113296031952,0.8424368388950825
+170,0.8634920634920635,0.5803478240966797,5.209537453863289e-06,0.34813615679740906,0.8462515577673912
+180,0.9142857142857143,0.5834349155426025,2.1754212141102346e-06,0.3094688653945923,0.8446752935647964
+190,0.9650793650793651,0.5862775802612304,4.322413053509944e-07,0.35028040409088135,0.8439742468297482
diff --git a/training/figures/training_v2_eval.csv b/training/figures/training_v2_eval.csv
new file mode 100644
index 0000000..a135efe
--- /dev/null
+++ b/training/figures/training_v2_eval.csv
@@ -0,0 +1,5 @@
+step,epoch,eval_loss,eval_mean_token_accuracy
+50,0.25396825396825395,0.9650015830993652,0.7496833843844277
+100,0.5079365079365079,0.7067031860351562,0.8130695947578975
+150,0.7619047619047619,0.5970866084098816,0.8415183879080272
+197,1.0,0.575853168964386,0.8473851865246183
diff --git a/training/figures/training_v3.csv b/training/figures/training_v3.csv
new file mode 100644
index 0000000..655ef60
--- /dev/null
+++ b/training/figures/training_v3.csv
@@ -0,0 +1,31 @@
+step,epoch,loss,learning_rate,grad_norm,mean_token_accuracy
+10,0.06046863189720333,1.4868995666503906,9e-05,0.2206401377916336,0.6501498550176621
+20,0.12093726379440665,1.280113124847412,9.980736554638366e-05,0.20303168892860413,0.6815041944384574
+30,0.18140589569160998,1.1442659378051758,9.914337409040418e-05,0.12361272424459457,0.7108227916061878
+40,0.2418745275888133,1.0783726692199707,9.801196139334195e-05,0.13824407756328583,0.7233529925346375
+50,0.30234315948601664,1.0328060150146485,9.642388874981347e-05,0.1718689501285553,0.7331555418670177
+60,0.36281179138321995,0.9452245712280274,9.439426092011875e-05,0.19833926856517792,0.7508241727948188
+70,0.42328042328042326,0.8957448959350586,9.194238246314599e-05,0.27138301730155945,0.763477711379528
+80,0.4837490551776266,0.8388309478759766,8.90915741234015e-05,0.8669945597648621,0.776527363806963
+90,0.54421768707483,0.7785130500793457,8.586895101857747e-05,0.3251343071460724,0.7925561927258968
+100,0.6046863189720333,0.7025893211364747,8.230516473739935e-05,0.45480582118034363,0.810952752828598
+110,0.6651549508692366,0.6717377662658691,7.843411180075794e-05,0.3490012586116791,0.8204429857432842
+120,0.7256235827664399,0.634728193283081,7.42926112590631e-05,0.4181080162525177,0.8306220307946205
+130,0.7860922146636432,0.5837386608123779,6.992005449231208e-05,0.40689781308174133,0.843410138040781
+140,0.8465608465608465,0.5329185962677002,6.535803054375738e-05,0.4665887653827667,0.8579231850802899
+150,0.9070294784580499,0.5083013534545898,6.064993055076698e-05,0.4626559019088745,0.8642070218920708
+160,0.9674981103552532,0.489164400100708,5.5840535035285025e-05,0.5883871912956238,0.8689255759119987
+170,1.0241874527588812,0.43776092529296873,5.0975587979329734e-05,0.6238144040107727,0.8809238640467326
+180,1.0846560846560847,0.4174932956695557,4.610136173665751e-05,0.4284566640853882,0.888404282182455
+190,1.145124716553288,0.39289538860321044,4.1264216918883656e-05,0.42329925298690796,0.8944049634039402
+200,1.2055933484504913,0.39194531440734864,3.651016144214878e-05,0.45279642939567566,0.8959574572741985
+210,1.2660619803476947,0.3841248989105225,3.188441292840587e-05,0.4972575604915619,0.896124105155468
+220,1.3265306122448979,0.35218961238861085,2.743096862349427e-05,0.5121544003486633,0.9058273300528527
+230,1.3869992441421013,0.3745258331298828,2.3192186922673186e-05,0.38717055320739746,0.9008041150867939
+240,1.4474678760393047,0.3568585395812988,1.9208384483883817e-05,0.40057459473609924,0.9051908120512963
+250,1.507936507936508,0.34092986583709717,1.5517452760747975e-05,0.3971737325191498,0.9089121885597706
+260,1.568405139833711,0.33079631328582765,1.2154497602603703e-05,0.36223524808883667,0.9112872503697872
+270,1.6288737717309147,0.3347353458404541,9.151505349477902e-06,0.3942708671092987,0.9104152075946331
+280,1.689342403628118,0.34016706943511965,6.53703859789348e-06,0.35229650139808655,0.9098294205963612
+290,1.7498110355253211,0.3172227621078491,4.335964531197401e-06,0.38456934690475464,0.9160888768732548
+300,1.8102796674225246,0.31731643676757815,2.5692183983629713e-06,0.5015138983726501,0.9155210055410862
diff --git a/training/figures/training_v3_eval.csv b/training/figures/training_v3_eval.csv
new file mode 100644
index 0000000..bf558fc
--- /dev/null
+++ b/training/figures/training_v3_eval.csv
@@ -0,0 +1,7 @@
+step,epoch,eval_loss
+50,0.30234315948601664,1.009400725364685
+100,0.6046863189720333,0.6973341703414917
+150,0.9070294784580499,0.5049018859863281
+200,1.2055933484504913,0.41107413172721863
+250,1.507936507936508,0.3677152395248413
+300,1.8102796674225246,0.35275769233703613
diff --git a/training/figures/training_v4.csv b/training/figures/training_v4.csv
new file mode 100644
index 0000000..d982452
--- /dev/null
+++ b/training/figures/training_v4.csv
@@ -0,0 +1,44 @@
+step,epoch,loss,eval_loss
+10,0.056617126680820945,1.5663956642150878,
+20,0.11323425336164189,1.3516300201416016,
+30,0.16985138004246284,1.1845409393310546,
+40,0.22646850672328378,1.1037321090698242,
+50,0.28308563340410475,1.0377551078796388,
+50,0.28308563340410475,,1.010231852531433
+60,0.33970276008492567,0.9829194068908691,
+70,0.39631988676574664,0.8914342880249023,
+80,0.45293701344656756,0.79852294921875,
+90,0.5095541401273885,0.7313865184783935,
+100,0.5661712668082095,0.6939316749572754,
+100,0.5661712668082095,,0.6673692464828491
+110,0.6227883934890305,0.6318693161010742,
+120,0.6794055201698513,0.5733372211456299,
+130,0.7360226468506723,0.5655224800109864,
+140,0.7926397735314933,0.49826998710632325,
+150,0.8492569002123143,0.48117914199829104,
+150,0.8492569002123143,,0.474189430475235
+160,0.9058740268931351,0.4512802600860596,
+170,0.9624911535739561,0.43169336318969725,
+180,1.0169851380042463,0.39495258331298827,
+190,1.0736022646850671,0.37260124683380125,
+200,1.1302193913658882,0.3555767059326172,
+200,1.1302193913658882,,0.3825647532939911
+210,1.186836518046709,0.3555564880371094,
+220,1.2434536447275302,0.3253514528274536,
+230,1.300070771408351,0.3274040222167969,
+240,1.356687898089172,0.31757657527923583,
+250,1.413305024769993,0.32180213928222656,
+250,1.413305024769993,,0.3373807370662689
+260,1.4699221514508138,0.2948040008544922,
+270,1.5265392781316347,0.2999974489212036,
+280,1.5831564048124558,0.2915226936340332,
+290,1.6397735314932769,0.28573825359344485,
+300,1.6963906581740975,0.28896660804748536,
+300,1.6963906581740975,,0.31564027070999146
+310,1.7530077848549186,0.28694982528686525,
+320,1.8096249115357397,0.27899909019470215,
+330,1.8662420382165605,0.2718695878982544,
+340,1.9228591648973814,0.28831920623779295,
+350,1.9794762915782025,0.28850746154785156,
+350,1.9794762915782025,,0.30873769521713257
+354,2.0,,0.3087170422077179
diff --git a/training/figures/v1_training_loss.png b/training/figures/v1_training_loss.png
new file mode 100644
index 0000000..71ccd64
Binary files /dev/null and b/training/figures/v1_training_loss.png differ
diff --git a/training/figures/v2_vs_v1_training_loss.png b/training/figures/v2_vs_v1_training_loss.png
new file mode 100644
index 0000000..b310b39
Binary files /dev/null and b/training/figures/v2_vs_v1_training_loss.png differ
diff --git a/training/rag_eval_rescored.json b/training/rag_eval_rescored.json
new file mode 100644
index 0000000..d084db2
--- /dev/null
+++ b/training/rag_eval_rescored.json
@@ -0,0 +1,564 @@
+{
+  "rescored_at": "2026-07-20 22:19:14",
+  "runs": {
+    "rag_dense_k8_base": {
+      "scored_total": 495,
+      "strict_correct": 17,
+      "strict_accuracy": 0.0343,
+      "lenient_correct": 29,
+      "lenient_accuracy": 0.0586,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 94,
+          "strict": 0.1809,
+          "lenient": 0.2553
+        },
+        "gold_missing": {
+          "n": 38,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0138
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v3-checkpoint-100": {
+      "scored_total": 495,
+      "strict_correct": 16,
+      "strict_accuracy": 0.0323,
+      "lenient_correct": 26,
+      "lenient_accuracy": 0.0525,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 94,
+          "strict": 0.1702,
+          "lenient": 0.234
+        },
+        "gold_missing": {
+          "n": 38,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.011
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v3-checkpoint-200": {
+      "scored_total": 495,
+      "strict_correct": 17,
+      "strict_accuracy": 0.0343,
+      "lenient_correct": 29,
+      "lenient_accuracy": 0.0586,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 94,
+          "strict": 0.1809,
+          "lenient": 0.2553
+        },
+        "gold_missing": {
+          "n": 38,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0138
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v3-checkpoint-300": {
+      "scored_total": 495,
+      "strict_correct": 19,
+      "strict_accuracy": 0.0384,
+      "lenient_correct": 27,
+      "lenient_accuracy": 0.0545,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 94,
+          "strict": 0.2021,
+          "lenient": 0.266
+        },
+        "gold_missing": {
+          "n": 38,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0055
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v3-checkpoint-332": {
+      "scored_total": 495,
+      "strict_correct": 15,
+      "strict_accuracy": 0.0303,
+      "lenient_correct": 25,
+      "lenient_accuracy": 0.0505,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 94,
+          "strict": 0.1596,
+          "lenient": 0.234
+        },
+        "gold_missing": {
+          "n": 38,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0083
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v3-final": {
+      "scored_total": 495,
+      "strict_correct": 15,
+      "strict_accuracy": 0.0303,
+      "lenient_correct": 25,
+      "lenient_accuracy": 0.0505,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 94,
+          "strict": 0.1596,
+          "lenient": 0.234
+        },
+        "gold_missing": {
+          "n": 38,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0083
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v4-checkpoint-100": {
+      "scored_total": 495,
+      "strict_correct": 19,
+      "strict_accuracy": 0.0384,
+      "lenient_correct": 34,
+      "lenient_accuracy": 0.0687,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 98,
+          "strict": 0.1837,
+          "lenient": 0.2449
+        },
+        "gold_missing": {
+          "n": 52,
+          "strict": 0.0,
+          "lenient": 0.0192
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0261
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v4-checkpoint-200": {
+      "scored_total": 495,
+      "strict_correct": 18,
+      "strict_accuracy": 0.0364,
+      "lenient_correct": 28,
+      "lenient_accuracy": 0.0566,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 98,
+          "strict": 0.1735,
+          "lenient": 0.2449
+        },
+        "gold_missing": {
+          "n": 52,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0116
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v4-checkpoint-300": {
+      "scored_total": 495,
+      "strict_correct": 17,
+      "strict_accuracy": 0.0343,
+      "lenient_correct": 28,
+      "lenient_accuracy": 0.0566,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 98,
+          "strict": 0.1633,
+          "lenient": 0.2245
+        },
+        "gold_missing": {
+          "n": 52,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0174
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v4-checkpoint-354": {
+      "scored_total": 495,
+      "strict_correct": 19,
+      "strict_accuracy": 0.0384,
+      "lenient_correct": 30,
+      "lenient_accuracy": 0.0606,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 98,
+          "strict": 0.1837,
+          "lenient": 0.2551
+        },
+        "gold_missing": {
+          "n": 52,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0145
+        }
+      }
+    },
+    "rag_dense_k8_legal-3b-v4-final": {
+      "scored_total": 495,
+      "strict_correct": 19,
+      "strict_accuracy": 0.0384,
+      "lenient_correct": 30,
+      "lenient_accuracy": 0.0606,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 98,
+          "strict": 0.1837,
+          "lenient": 0.2551
+        },
+        "gold_missing": {
+          "n": 52,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0145
+        }
+      }
+    },
+    "rag_dense_k8_v3-dpo": {
+      "scored_total": 495,
+      "strict_correct": 17,
+      "strict_accuracy": 0.0343,
+      "lenient_correct": 24,
+      "lenient_accuracy": 0.0485,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 94,
+          "strict": 0.1809,
+          "lenient": 0.234
+        },
+        "gold_missing": {
+          "n": 38,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0055
+        }
+      }
+    },
+    "rag_k8_base": {
+      "scored_total": 495,
+      "strict_correct": 19,
+      "strict_accuracy": 0.0384,
+      "lenient_correct": 29,
+      "lenient_accuracy": 0.0586,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 85,
+          "strict": 0.2235,
+          "lenient": 0.2706
+        },
+        "gold_missing": {
+          "n": 47,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0165
+        }
+      }
+    },
+    "rag_k8_checkpoint-50": {
+      "scored_total": 495,
+      "strict_correct": 14,
+      "strict_accuracy": 0.0283,
+      "lenient_correct": 17,
+      "lenient_accuracy": 0.0343,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 85,
+          "strict": 0.1647,
+          "lenient": 0.1765
+        },
+        "gold_missing": {
+          "n": 47,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0055
+        }
+      }
+    },
+    "rag_k8_legal-3b-v2-checkpoint-100": {
+      "scored_total": 495,
+      "strict_correct": 18,
+      "strict_accuracy": 0.0364,
+      "lenient_correct": 25,
+      "lenient_accuracy": 0.0505,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 85,
+          "strict": 0.2118,
+          "lenient": 0.2824
+        },
+        "gold_missing": {
+          "n": 47,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0028
+        }
+      }
+    },
+    "rag_k8_legal-3b-v2-checkpoint-150": {
+      "scored_total": 495,
+      "strict_correct": 16,
+      "strict_accuracy": 0.0323,
+      "lenient_correct": 27,
+      "lenient_accuracy": 0.0545,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 85,
+          "strict": 0.1882,
+          "lenient": 0.2588
+        },
+        "gold_missing": {
+          "n": 47,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0138
+        }
+      }
+    },
+    "rag_k8_legal-3b-v2-checkpoint-197": {
+      "scored_total": 495,
+      "strict_correct": 16,
+      "strict_accuracy": 0.0323,
+      "lenient_correct": 23,
+      "lenient_accuracy": 0.0465,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 85,
+          "strict": 0.1882,
+          "lenient": 0.2471
+        },
+        "gold_missing": {
+          "n": 47,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0055
+        }
+      }
+    },
+    "rag_k8_legal-3b-v2-checkpoint-50": {
+      "scored_total": 495,
+      "strict_correct": 18,
+      "strict_accuracy": 0.0364,
+      "lenient_correct": 24,
+      "lenient_accuracy": 0.0485,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 85,
+          "strict": 0.2118,
+          "lenient": 0.2588
+        },
+        "gold_missing": {
+          "n": 47,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0055
+        }
+      }
+    },
+    "rag_k8_legal-3b-v2-final": {
+      "scored_total": 495,
+      "strict_correct": 16,
+      "strict_accuracy": 0.0323,
+      "lenient_correct": 23,
+      "lenient_accuracy": 0.0465,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 85,
+          "strict": 0.1882,
+          "lenient": 0.2471
+        },
+        "gold_missing": {
+          "n": 47,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 363,
+          "strict": 0.0,
+          "lenient": 0.0055
+        }
+      }
+    },
+    "rebase-r2_base": {
+      "scored_total": 495,
+      "strict_correct": 22,
+      "strict_accuracy": 0.0444,
+      "lenient_correct": 41,
+      "lenient_accuracy": 0.0828,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 93,
+          "strict": 0.2151,
+          "lenient": 0.3011
+        },
+        "gold_missing": {
+          "n": 57,
+          "strict": 0.0175,
+          "lenient": 0.0526
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.029
+        }
+      }
+    },
+    "rebase-r2_v3-ckpt300": {
+      "scored_total": 495,
+      "strict_correct": 21,
+      "strict_accuracy": 0.0424,
+      "lenient_correct": 34,
+      "lenient_accuracy": 0.0687,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 93,
+          "strict": 0.2151,
+          "lenient": 0.2903
+        },
+        "gold_missing": {
+          "n": 57,
+          "strict": 0.0,
+          "lenient": 0.0
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0203
+        }
+      }
+    },
+    "rebase-r3_base": {
+      "scored_total": 495,
+      "strict_correct": 19,
+      "strict_accuracy": 0.0384,
+      "lenient_correct": 33,
+      "lenient_accuracy": 0.0667,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 98,
+          "strict": 0.1735,
+          "lenient": 0.2449
+        },
+        "gold_missing": {
+          "n": 52,
+          "strict": 0.0192,
+          "lenient": 0.0192
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0232
+        }
+      }
+    },
+    "rebase-r3_v3-ckpt300": {
+      "scored_total": 495,
+      "strict_correct": 21,
+      "strict_accuracy": 0.0424,
+      "lenient_correct": 32,
+      "lenient_accuracy": 0.0646,
+      "by_retrieval": {
+        "gold_in_context": {
+          "n": 98,
+          "strict": 0.1939,
+          "lenient": 0.2245
+        },
+        "gold_missing": {
+          "n": 52,
+          "strict": 0.0192,
+          "lenient": 0.0192
+        },
+        "no_gold": {
+          "n": 345,
+          "strict": 0.0029,
+          "lenient": 0.0261
+        }
+      }
+    },
+    "checkpoint-50": {
+      "scored_total": 495,
+      "strict_correct": 4,
+      "strict_accuracy": 0.0081,
+      "lenient_correct": 4,
+      "lenient_accuracy": 0.0081
+    }
+  }
+}
\ No newline at end of file
diff --git a/training/retrieval_recall_dense.json b/training/retrieval_recall_dense.json
new file mode 100644
index 0000000..95a8ae9
--- /dev/null
+++ b/training/retrieval_recall_dense.json
@@ -0,0 +1,91 @@
+{
+  "dense_model": "intfloat/multilingual-e5-base",
+  "eval_records": 500,
+  "gold_bearing": 150,
+  "no_citation_gold": 350,
+  "unresolved_fact_count": 0,
+  "unresolved_facts_sample": [],
+  "recall": {
+    "k=1": {
+      "any_hit": 0.4333,
+      "full_hit": 0.3867
+    },
+    "k=3": {
+      "any_hit": 0.5867,
+      "full_hit": 0.54
+    },
+    "k=5": {
+      "any_hit": 0.6267,
+      "full_hit": 0.58
+    },
+    "k=8": {
+      "any_hit": 0.7067,
+      "full_hit": 0.6533
+    }
+  },
+  "by_language": {
+    "english": {
+      "n": 139,
+      "full_hit@8": 0.6619
+    },
+    "hindi": {
+      "n": 5,
+      "full_hit@8": 0.8
+    },
+    "hinglish": {
+      "n": 6,
+      "full_hit@8": 0.3333
+    }
+  },
+  "by_domain": {
+    "bns": {
+      "n": 46,
+      "full_hit@8": 0.7174
+    },
+    "bnss": {
+      "n": 25,
+      "full_hit@8": 0.64
+    },
+    "bsa": {
+      "n": 11,
+      "full_hit@8": 0.3636
+    },
+    "cheque_bounce": {
+      "n": 8,
+      "full_hit@8": 0.5
+    },
+    "constitutional_law": {
+      "n": 33,
+      "full_hit@8": 0.7576
+    },
+    "cyber_law": {
+      "n": 12,
+      "full_hit@8": 0.5833
+    },
+    "labour_law": {
+      "n": 1,
+      "full_hit@8": 0.0
+    },
+    "motor_vehicles": {
+      "n": 4,
+      "full_hit@8": 0.75
+    },
+    "old_new_law_mapping": {
+      "n": 1,
+      "full_hit@8": 1.0
+    },
+    "rti": {
+      "n": 3,
+      "full_hit@8": 0.6667
+    },
+    "womens_protection": {
+      "n": 6,
+      "full_hit@8": 0.5
+    }
+  },
+  "phrase_coverage": {
+    "n": 350,
+    "any_fact_in_topk": 0.06
+  },
+  "measured_at": "2026-07-17 00:35:30"
+}
\ No newline at end of file
diff --git a/training/train_v3.yaml b/training/train_v3.yaml
new file mode 100644
index 0000000..010067e
--- /dev/null
+++ b/training/train_v3.yaml
@@ -0,0 +1,39 @@
+# Nyaya-3B-v3 — RAFT: teacher answers regenerated UNDER the inference RAG
+# prompt (data/splits_rag_v3, scripts/20+21), including deliberate
+# retrieval-miss demonstrations. 2 epochs (v2's val loss was still falling
+# at 1) + NEFTune noise for SFT robustness.
+#
+# Method: full-precision bf16 LoRA — NO quantization (project decision 2026-07-13).
+
+run_name: legal-3b-v3
+model_id: Qwen/Qwen2.5-3B-Instruct
+output_dir: outputs/legal-3b-v3
+
+data:
+  train_file: data/splits_rag_v3/train.jsonl
+  val_file: data/splits_rag_v3/val.jsonl
+
+method: lora
+
+lora:
+  r: 32
+  alpha: 64
+  dropout: 0.05
+  target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
+
+training:
+  max_seq_length: 6144
+  num_train_epochs: 2
+  learning_rate: 1.0e-4
+  warmup_ratio: 0.03
+  lr_scheduler_type: cosine
+  gradient_checkpointing: true
+  bf16: true
+  optim: adamw_torch_fused
+  neftune_noise_alpha: 5
+  per_device_train_batch_size: 4
+  gradient_accumulation_steps: 8
+  logging_steps: 10
+  # ~6.3K examples x 2 epochs / effective batch 32 = ~394 steps
+  eval_steps: 50
+  save_steps: 100
diff --git a/training/training_history_v3.json b/training/training_history_v3.json
new file mode 100644
index 0000000..9b295a9
--- /dev/null
+++ b/training/training_history_v3.json
@@ -0,0 +1,371 @@
+{
+  "run_name": "legal-3b-v3",
+  "log_history": [
+    {
+      "entropy": 1.2671179935336112,
+      "epoch": 0.06046863189720333,
+      "grad_norm": 0.2206401377916336,
+      "learning_rate": 9e-05,
+      "loss": 1.4868995666503906,
+      "mean_token_accuracy": 0.6501498550176621,
+      "num_tokens": 919743.0,
+      "step": 10
+    },
+    {
+      "entropy": 1.3823076233267784,
+      "epoch": 0.12093726379440665,
+      "grad_norm": 0.20303168892860413,
+      "learning_rate": 9.980736554638366e-05,
+      "loss": 1.280113124847412,
+      "mean_token_accuracy": 0.6815041944384574,
+      "num_tokens": 1841121.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.1250081039965152,
+      "epoch": 0.18140589569160998,
+      "grad_norm": 0.12361272424459457,
+      "learning_rate": 9.914337409040418e-05,
+      "loss": 1.1442659378051758,
+      "mean_token_accuracy": 0.7108227916061878,
+      "num_tokens": 2765329.0,
+      "step": 30
+    },
+    {
+      "entropy": 1.0885243624448777,
+      "epoch": 0.2418745275888133,
+      "grad_norm": 0.13824407756328583,
+      "learning_rate": 9.801196139334195e-05,
+      "loss": 1.0783726692199707,
+      "mean_token_accuracy": 0.7233529925346375,
+      "num_tokens": 3693580.0,
+      "step": 40
+    },
+    {
+      "entropy": 1.0232380636036396,
+      "epoch": 0.30234315948601664,
+      "grad_norm": 0.1718689501285553,
+      "learning_rate": 9.642388874981347e-05,
+      "loss": 1.0328060150146485,
+      "mean_token_accuracy": 0.7331555418670177,
+      "num_tokens": 4603211.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.30234315948601664,
+      "eval_entropy": 1.0007498400551933,
+      "eval_loss": 1.009400725364685,
+      "eval_mean_token_accuracy": 0.7387015887669155,
+      "eval_num_tokens": 4603211.0,
+      "eval_runtime": 89.0776,
+      "eval_samples_per_second": 3.143,
+      "eval_steps_per_second": 0.393,
+      "step": 50
+    },
+    {
+      "entropy": 0.9474448464810848,
+      "epoch": 0.36281179138321995,
+      "grad_norm": 0.19833926856517792,
+      "learning_rate": 9.439426092011875e-05,
+      "loss": 0.9452245712280274,
+      "mean_token_accuracy": 0.7508241727948188,
+      "num_tokens": 5511119.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.9045603923499584,
+      "epoch": 0.42328042328042326,
+      "grad_norm": 0.27138301730155945,
+      "learning_rate": 9.194238246314599e-05,
+      "loss": 0.8957448959350586,
+      "mean_token_accuracy": 0.763477711379528,
+      "num_tokens": 6429339.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.8463030077517033,
+      "epoch": 0.4837490551776266,
+      "grad_norm": 0.8669945597648621,
+      "learning_rate": 8.90915741234015e-05,
+      "loss": 0.8388309478759766,
+      "mean_token_accuracy": 0.776527363806963,
+      "num_tokens": 7370344.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.7973637029528617,
+      "epoch": 0.54421768707483,
+      "grad_norm": 0.3251343071460724,
+      "learning_rate": 8.586895101857747e-05,
+      "loss": 0.7785130500793457,
+      "mean_token_accuracy": 0.7925561927258968,
+      "num_tokens": 8291090.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.7309453509747982,
+      "epoch": 0.6046863189720333,
+      "grad_norm": 0.45480582118034363,
+      "learning_rate": 8.230516473739935e-05,
+      "loss": 0.7025893211364747,
+      "mean_token_accuracy": 0.810952752828598,
+      "num_tokens": 9207198.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.6046863189720333,
+      "eval_entropy": 0.7214480527809688,
+      "eval_loss": 0.6973341703414917,
+      "eval_mean_token_accuracy": 0.8165236847741263,
+      "eval_num_tokens": 9207198.0,
+      "eval_runtime": 89.2073,
+      "eval_samples_per_second": 3.139,
+      "eval_steps_per_second": 0.392,
+      "step": 100
+    },
+    {
+      "entropy": 0.6975084710866213,
+      "epoch": 0.6651549508692366,
+      "grad_norm": 0.3490012586116791,
+      "learning_rate": 7.843411180075794e-05,
+      "loss": 0.6717377662658691,
+      "mean_token_accuracy": 0.8204429857432842,
+      "num_tokens": 10116405.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.6599536526948213,
+      "epoch": 0.7256235827664399,
+      "grad_norm": 0.4181080162525177,
+      "learning_rate": 7.42926112590631e-05,
+      "loss": 0.634728193283081,
+      "mean_token_accuracy": 0.8306220307946205,
+      "num_tokens": 11029708.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.6176572676748038,
+      "epoch": 0.7860922146636432,
+      "grad_norm": 0.40689781308174133,
+      "learning_rate": 6.992005449231208e-05,
+      "loss": 0.5837386608123779,
+      "mean_token_accuracy": 0.843410138040781,
+      "num_tokens": 11949565.0,
+      "step": 130
+    },
+    {
+      "entropy": 0.5673871148377657,
+      "epoch": 0.8465608465608465,
+      "grad_norm": 0.4665887653827667,
+      "learning_rate": 6.535803054375738e-05,
+      "loss": 0.5329185962677002,
+      "mean_token_accuracy": 0.8579231850802899,
+      "num_tokens": 12868154.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.5449491806328297,
+      "epoch": 0.9070294784580499,
+      "grad_norm": 0.4626559019088745,
+      "learning_rate": 6.064993055076698e-05,
+      "loss": 0.5083013534545898,
+      "mean_token_accuracy": 0.8642070218920708,
+      "num_tokens": 13774405.0,
+      "step": 150
+    },
+    {
+      "epoch": 0.9070294784580499,
+      "eval_entropy": 0.5539889786924634,
+      "eval_loss": 0.5049018859863281,
+      "eval_mean_token_accuracy": 0.8679234419550215,
+      "eval_num_tokens": 13774405.0,
+      "eval_runtime": 89.3489,
+      "eval_samples_per_second": 3.134,
+      "eval_steps_per_second": 0.392,
+      "step": 150
+    },
+    {
+      "entropy": 0.5234581783413887,
+      "epoch": 0.9674981103552532,
+      "grad_norm": 0.5883871912956238,
+      "learning_rate": 5.5840535035285025e-05,
+      "loss": 0.489164400100708,
+      "mean_token_accuracy": 0.8689255759119987,
+      "num_tokens": 14698948.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.49091413378715515,
+      "epoch": 1.0241874527588812,
+      "grad_norm": 0.6238144040107727,
+      "learning_rate": 5.0975587979329734e-05,
+      "loss": 0.43776092529296873,
+      "mean_token_accuracy": 0.8809238640467326,
+      "num_tokens": 15570380.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.45481786243617534,
+      "epoch": 1.0846560846560847,
+      "grad_norm": 0.4284566640853882,
+      "learning_rate": 4.610136173665751e-05,
+      "loss": 0.4174932956695557,
+      "mean_token_accuracy": 0.888404282182455,
+      "num_tokens": 16489412.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.4345656637102365,
+      "epoch": 1.145124716553288,
+      "grad_norm": 0.42329925298690796,
+      "learning_rate": 4.1264216918883656e-05,
+      "loss": 0.39289538860321044,
+      "mean_token_accuracy": 0.8944049634039402,
+      "num_tokens": 17402523.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.4282470645383,
+      "epoch": 1.2055933484504913,
+      "grad_norm": 0.45279642939567566,
+      "learning_rate": 3.651016144214878e-05,
+      "loss": 0.39194531440734864,
+      "mean_token_accuracy": 0.8959574572741985,
+      "num_tokens": 18310900.0,
+      "step": 200
+    },
+    {
+      "epoch": 1.2055933484504913,
+      "eval_entropy": 0.4425007075071335,
+      "eval_loss": 0.41107413172721863,
+      "eval_mean_token_accuracy": 0.8935366255896432,
+      "eval_num_tokens": 18310900.0,
+      "eval_runtime": 89.1987,
+      "eval_samples_per_second": 3.139,
+      "eval_steps_per_second": 0.392,
+      "step": 200
+    },
+    {
+      "entropy": 0.42347441539168357,
+      "epoch": 1.2660619803476947,
+      "grad_norm": 0.4972575604915619,
+      "learning_rate": 3.188441292840587e-05,
+      "loss": 0.3841248989105225,
+      "mean_token_accuracy": 0.896124105155468,
+      "num_tokens": 19239874.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.39230803996324537,
+      "epoch": 1.3265306122448979,
+      "grad_norm": 0.5121544003486633,
+      "learning_rate": 2.743096862349427e-05,
+      "loss": 0.35218961238861085,
+      "mean_token_accuracy": 0.9058273300528527,
+      "num_tokens": 20156230.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.4154032004997134,
+      "epoch": 1.3869992441421013,
+      "grad_norm": 0.38717055320739746,
+      "learning_rate": 2.3192186922673186e-05,
+      "loss": 0.3745258331298828,
+      "mean_token_accuracy": 0.9008041150867939,
+      "num_tokens": 21063499.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.39556570779532196,
+      "epoch": 1.4474678760393047,
+      "grad_norm": 0.40057459473609924,
+      "learning_rate": 1.9208384483883817e-05,
+      "loss": 0.3568585395812988,
+      "mean_token_accuracy": 0.9051908120512963,
+      "num_tokens": 21977605.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.3834851888939738,
+      "epoch": 1.507936507936508,
+      "grad_norm": 0.3971737325191498,
+      "learning_rate": 1.5517452760747975e-05,
+      "loss": 0.34092986583709717,
+      "mean_token_accuracy": 0.9089121885597706,
+      "num_tokens": 22889175.0,
+      "step": 250
+    },
+    {
+      "epoch": 1.507936507936508,
+      "eval_entropy": 0.38803921937942504,
+      "eval_loss": 0.3677152395248413,
+      "eval_mean_token_accuracy": 0.9056272642953055,
+      "eval_num_tokens": 22889175.0,
+      "eval_runtime": 89.1071,
+      "eval_samples_per_second": 3.142,
+      "eval_steps_per_second": 0.393,
+      "step": 250
+    },
+    {
+      "entropy": 0.37403900306671856,
+      "epoch": 1.568405139833711,
+      "grad_norm": 0.36223524808883667,
+      "learning_rate": 1.2154497602603703e-05,
+      "loss": 0.33079631328582765,
+      "mean_token_accuracy": 0.9112872503697872,
+      "num_tokens": 23806399.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.3777462661266327,
+      "epoch": 1.6288737717309147,
+      "grad_norm": 0.3942708671092987,
+      "learning_rate": 9.151505349477902e-06,
+      "loss": 0.3347353458404541,
+      "mean_token_accuracy": 0.9104152075946331,
+      "num_tokens": 24732235.0,
+      "step": 270
+    },
+    {
+      "entropy": 0.38299514539539814,
+      "epoch": 1.689342403628118,
+      "grad_norm": 0.35229650139808655,
+      "learning_rate": 6.53703859789348e-06,
+      "loss": 0.34016706943511965,
+      "mean_token_accuracy": 0.9098294205963612,
+      "num_tokens": 25644370.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.3589464059099555,
+      "epoch": 1.7498110355253211,
+      "grad_norm": 0.38456934690475464,
+      "learning_rate": 4.335964531197401e-06,
+      "loss": 0.3172227621078491,
+      "mean_token_accuracy": 0.9160888768732548,
+      "num_tokens": 26579671.0,
+      "step": 290
+    },
+    {
+      "entropy": 0.35829958152025937,
+      "epoch": 1.8102796674225246,
+      "grad_norm": 0.5015138983726501,
+      "learning_rate": 2.5692183983629713e-06,
+      "loss": 0.31731643676757815,
+      "mean_token_accuracy": 0.9155210055410862,
+      "num_tokens": 27515269.0,
+      "step": 300
+    },
+    {
+      "epoch": 1.8102796674225246,
+      "eval_entropy": 0.374431825535638,
+      "eval_loss": 0.35275769233703613,
+      "eval_mean_token_accuracy": 0.9095644763537816,
+      "eval_num_tokens": 27515269.0,
+      "eval_runtime": 89.1755,
+      "eval_samples_per_second": 3.14,
+      "eval_steps_per_second": 0.392,
+      "step": 300
+    }
+  ]
+}
\ No newline at end of file