From a177599ff25dfb8bc68006701fe702ac339327f7 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Fri, 24 Jul 2026 01:12:09 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF Source: Original Platform --- .gitattributes | 40 ++++ ...ranite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf | 3 + ...ite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf | 3 + ...ite-4.1-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf | 3 + ...anite-4.1-3B-Claude-Fable-5-GGUF-Q6_K.gguf | 3 + ...anite-4.1-3B-Claude-Fable-5-GGUF-Q8_0.gguf | 3 + README.md | 202 ++++++++++++++++++ 7 files changed, 257 insertions(+) create mode 100644 .gitattributes create mode 100644 Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf create mode 100644 Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf create mode 100644 Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf create mode 100644 Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q6_K.gguf create mode 100644 Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q8_0.gguf create mode 100644 README.md diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..1ac732f --- /dev/null +++ b/.gitattributes @@ -0,0 +1,40 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf new file mode 100644 index 0000000..0c94425 --- /dev/null +++ b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4d5e220624170d6258d2a0eacfb689b5bff633213c909af88b971306d67dc062 +size 6809655808 diff --git a/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf new file mode 100644 index 0000000..f46dda5 --- /dev/null +++ b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:67dc7695d92939c713165761f115c9d892fdff74fcbd987c8bb453b9b8ab645d +size 2099501568 diff --git a/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf new file mode 100644 index 0000000..a10467a --- /dev/null +++ b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97bc85abf599b3bcbe025c48989a8817592ca7805109c7dacd32cfee792055b2 +size 2437011968 diff --git a/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q6_K.gguf b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q6_K.gguf new file mode 100644 index 0000000..62dcd94 --- /dev/null +++ b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0ba3ee051779f2f4ede9d4916aa00dc78d6c94f1d4b990d51db408ecfe7a6c31 +size 2795616768 diff --git a/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q8_0.gguf b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q8_0.gguf new file mode 100644 index 0000000..3b2eccd --- /dev/null +++ b/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3053365ffa974cb12b70ec64520fd90723135432ad31fdc56e4594a3b582720b +size 3619691008 diff --git a/README.md b/README.md new file mode 100644 index 0000000..f588327 --- /dev/null +++ b/README.md @@ -0,0 +1,202 @@ +--- +base_model: ibm-granite/granite-4.1-3b +base_model_relation: finetune +datasets: +- AnkitAI/parable-corpus-v2 +- Glint-Research/Fable-5-traces +- Roman1111111/gpt5.5-terminal +license: apache-2.0 +language: +- en +pipeline_tag: text-generation +library_name: gguf +tags: +- gguf +- qlora +- agentic +- agent +- coding +- tool-use +- function-calling +- terminal +- reasoning +- thinking +- claude +- claude-fable-5 +- distillation +- trace-training +- llama.cpp +- ollama +- lm-studio +- granite +--- + + + + Parable + + +# ๐Ÿชถ Parable-Granite-3B **v2** โ€” trained on genuine Claude Fable 5 agent traces + +### A tiny local model that thinks before it answers โ€” planning, reasoning, and terminal instincts distilled from real agent sessions. + +> **~3 GB of RAM is all you need.** Laptop, old GPU, Raspberry-Pi-class boxes with swap โ€” the Q4 build runs +> anywhere. One command and you have a private, offline reasoning model on your machine: +> +> ```bash +> ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M +> ``` + +--- + +## The headline โ€” v2 is a different model + +v2 is a full retrain: **13ร— more genuine Fable 5 trace data** (11,574 sessions, 16.8M tokens โ€” [corpus published](https://huggingface.co/datasets/AnkitAI/parable-corpus-v2)) and a rebuilt recipe (completion-masked loss, replay mixing, benchmark-gated checkpoints, seed-averaged weights). + +| same harness, greedy, Q4_K_M | v1 | **v2 (this release)** | +|---|---|---| +| Dev pass-rate (MBPP subset, n=50) โ€” *base: 0.68* | โ€” | **0.82** | +| Agent-artifact leakage (JSON blobs, phantom turns) | 6/34 | **0/34** | +| Strict 34-prompt coding qual โ€” *base: 27/34* | ~18/34 | **25/34** | +| HumanEval / HumanEval+ | 62.8 / 57.9 | **70.1 / 65.9** | + +Clean answers, structured reasoning, agent instincts โ€” and the transcript artifacts that leaked into v1's replies are gone. *One trade, made on purpose: raw HumanEval-style function synthesis stays the base model's turf (81.7 vs 70.1) โ€” v2 spends that capacity on agent behavior instead, and spends half as much as v1 did.* Measurement notes below. ๐Ÿ‘‡ + +--- + +## Announcements + +**๐Ÿ“Œ Same links, new model.** v2 replaces v1 **in place** โ€” every existing Ollama command, script, and bookmark now serves v2. No migration, nothing to change. + +**๐Ÿ”ฎ v3 is already training.** Rejection-sampled SFT: thousands of candidate solutions generated against *executable tests*, only verified passers enter the corpus. The goal is simple โ€” above-base agent capability, not just clean behavior. Follow [AnkitAI](https://huggingface.co/AnkitAI) for the drop. + +**๐Ÿ“ฆ Full family.** This 3B is the smallest Parable. Need more headroom? [8B Granite](https://huggingface.co/AnkitAI/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF), [8B Qwen](https://huggingface.co/AnkitAI/Parable-Qwen3-8B-Claude-Fable-5-GGUF), [4B Qwen](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF) โ€” same recipe, no matter your hardware. + +--- + +## Pick your size + +| File | Size | Fits in | Notes | +|---|---|---|---| +| [Q4_K_M](./Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf) | 2.1 GB | ~3 GB RAM/VRAM | โญ **Recommended** โ€” best size/quality balance | +| [Q5_K_M](./Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf) | 2.4 GB | ~3.5 GB | Higher quality | +| [Q6_K](./Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q6_K.gguf) | 2.8 GB | ~4 GB | Near-lossless | +| [Q8_0](./Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q8_0.gguf) | 3.6 GB | ~5 GB | Maximum quality | +| [F16](./Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-F16.gguf) | 6.8 GB | ~8 GB | Full precision, for re-quantizing | + +**Intelligence per gigabyte:** the Q4_K_M build scores 70.1 HumanEval in 2.1 GB โ€” ~33 pts/GB; an 8B-class Q4 needs ~5 GB for its score. If RAM is your constraint, this is the family's density sweet spot. + +Full-precision safetensors (vLLM, transformers, further fine-tuning): [Parable-Granite-4.1-3B-Claude-Fable-5](https://huggingface.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5) + +## How to run it + +**Ollama** (chat template ships inside the GGUF โ€” zero config): +```bash +ollama run parable/granite4.1-fable:3b +# or straight from this repo: +ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M +``` + +**llama.cpp:** +```bash +llama-cli -m Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf --jinja \ + -p "Write a bash one-liner to find the 10 largest files in a directory tree." +``` + +**LM Studio / Jan / Open WebUI:** search **"parable"** in-app, or paste this repo URL. + +**Python (llama-cpp-python):** +```python +from llama_cpp import Llama + +llm = Llama.from_pretrained( + repo_id="AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF", + filename="*Q4_K_M.gguf", n_ctx=8192, +) +out = llm.create_chat_completion( + messages=[{"role": "user", "content": "Write a Python function that retries an HTTP request with exponential backoff."}], + max_tokens=3000, temperature=0.7, +) +print(out["choices"][0]["message"]["content"]) +``` + +### Thinking mode + +Every answer opens with a `...` reasoning block โ€” that's the Fable 5 heritage. llama.cpp's `--jinja` mode separates it automatically; strip it before showing replies to end users. +**Sampling:** temperature 0.7, top_p 0.95, and budget `max_tokens` generously (**2500+**) โ€” trace-trained models think at length before answering. + +--- + +## Measurement notes + +All numbers: identical llama.cpp harness, greedy decoding, Q4_K_M, **base model measured on the same instrument**. We train multiple seeds and ship the weight-average โ€” single-run scores at 3B swing ยฑ3 points on GPU nondeterminism alone, so most cards report their luckiest run; we ship the average and report the shipped weights' own numbers. Raw eval outputs live in this repo. + +**Which model should you use?** Pure single-function code completion โ†’ the base model is genuinely strong there. Explanations, debugging, terminal workflows, structured reasoning, agent-style tasks โ†’ that's what Parable is trained on, and where v2 shines. + +## ๐Ÿ†š Same prompt, side by side + +Real outputs, both models at Q4_K_M, temperature 0.7 โ€” unedited except length. + +**Prompt:** *"Make this more idiomatic:"* `result = []; for x in items: if x.active == True: result.append(x.name.upper())` + +| Base Granite-4.1-3B | **Parable v2** | +|---|---| +| Here's a more idiomatic version of the code using a list comprehension: `result = [item.name.upper() for item in items if item.active]` โ€” Explanation: List comprehensions provide a concise way to create lists based on existing iterables. The new `result` list is constructed by iterating over each `item`โ€ฆ *(continues for ~10 more lines)* | `result = [item.name.upper() for item in items if item.active]` | + +**Prompt:** *"My Python script fails with 'RecursionError: maximum recursion depth exceeded' in a JSON parser I wrote. What are the likely causes and the standard fix?"* + +| Base Granite-4.1-3B | **Parable v2** | +|---|---| +| *Opens with prose:* "The RecursionError: maximum recursion depth exceeded error in a Python script, especially when dealing with a JSON parser, typically indicates that your recursive function is calling itself too many times without reaching a proper base caseโ€ฆ" | *Opens with a diagnosis table:* **Common culprits for this error** โ€” a cause / why-it-triggers / example table, then the fix. Structured like an agent working the problem, not an essay. | + +The pattern from real agent traces: answer first, structure over prose, no padding. (Where the base is stronger โ€” raw single-function synthesis โ€” is stated plainly in the measurement notes above.) + +## What's new in v2 (training) + +The recipe follows our ongoing tech report (in preparation): + +- **Completion-only loss masking** ([Hermes 3](https://arxiv.org/abs/2408.11857), [Tรผlu 3](https://arxiv.org/abs/2411.15124)) โ€” loss on assistant tokens only, so the model learns to *answer*, not to imitate transcripts +- **30% replay mix** of general instruction data ([Luo et al.](https://arxiv.org/abs/2308.08747), [Biderman et al.](https://arxiv.org/abs/2405.09673)) โ€” the anti-forgetting lever +- **Session re-segmentation + sanitization** โ€” why v1 sometimes leaked agent JSON into normal chat, and v2 never does (0/34) +- **Benchmark-gated checkpoints** ([Dong et al.](https://arxiv.org/abs/2310.05492)) instead of fixed epochs +- **Seed-averaged weights** ([model soups, Wortsman et al.](https://arxiv.org/abs/2203.05482)) โ€” we ship the average of multiple runs, not the lottery winner + +With Claude Fable 5 now retired, genuine self-authored Fable traces are a fixed, non-renewable corpus. Unlike most models in this niche, **our full training corpus is public**: [AnkitAI/parable-corpus-v2](https://huggingface.co/datasets/AnkitAI/parable-corpus-v2) โ€” deduplicated, quality-gated, provenance-tagged. + +## Good to know + +- Fine-tuned at 2,048-token sequences; the base 128K context stays available, fine-tuned behavior is strongest in the opening turns. +- Not trained for: multi-file repo navigation, vision, non-English. +- Inherits Granite-4.1-3B's knowledge cutoff. Treat generated commands as drafts to review. + +## Base & license + +Weights: **Apache-2.0** (inherited from [ibm-granite/granite-4.1-3b](https://huggingface.co/ibm-granite/granite-4.1-3b)). Training data: Fable-5-traces **AGPL-3.0**, gpt5.5-terminal **MIT** โ€” since traces originate from third-party assistants, their terms may apply to downstream training; check before commercial distillation. + +## Get Parable + +| Platform | | +|---|---| +| Ollama | `ollama run parable/granite4.1-fable:3b` ยท [parable namespace](https://ollama.com/parable) | +| Hugging Face | [full collection](https://huggingface.co/collections/AnkitAI/parable-6a4fac60f4b35afca3019621) | +| LM Studio | search "parable" in-app | +| ModelScope | [Parable on ModelScope](https://modelscope.cn/models/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF) | + +## Acknowledgements + +[Glint-Research](https://huggingface.co/Glint-Research) & [Roman1111111](https://huggingface.co/Roman1111111) for the open trace data ยท [IBM Granite](https://huggingface.co/ibm-granite) for the base ยท [empero-ai](https://huggingface.co/empero-ai) whose Qwable recipe inspired the series ยท [llama.cpp](https://github.com/ggml-org/llama.cpp) + +## Version history + +- **v2** (2026-07-16) โ€” this release. 13ร— corpus, rebuilt recipe, seed-averaged weights, zero leakage. +- **v1** (2026-07) โ€” initial release, 857-row corpus. Preserved as repo revision history. + +--- + +### Three gigabytes. Real Fable 5 reasoning. Yours, offline, right now. + +```bash +ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M +``` + +More on the Parable models: [ankitaglawe.com/parable](https://ankitaglawe.com/parable)