From 8bb8b089ff5d0e3f526a512bd228282814504ac7 Mon Sep 17 00:00:00 2001 From: ModelHub XC Date: Fri, 28 Aug 2026 13:34:17 +0800 Subject: [PATCH] =?UTF-8?q?=E5=88=9D=E5=A7=8B=E5=8C=96=E9=A1=B9=E7=9B=AE?= =?UTF-8?q?=EF=BC=8C=E7=94=B1ModelHub=20XC=E7=A4=BE=E5=8C=BA=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=E6=A8=A1=E5=9E=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Model: openmed-community/granite-4.0-micro-OpenMed-GGUF Source: Original Platform --- .gitattributes | 42 +++++ README.md | 160 ++++++++++++++++++ ...4.0-micro-OpenMed-BF16-00001-of-00002.gguf | 3 + ...4.0-micro-OpenMed-BF16-00002-of-00002.gguf | 3 + granite-4.0-micro-OpenMed-Q2_K.gguf | 3 + granite-4.0-micro-OpenMed-Q4_K_M.gguf | 3 + granite-4.0-micro-OpenMed-Q6_K.gguf | 3 + granite-4.0-micro-OpenMed-Q8_0.gguf | 3 + 8 files changed, 220 insertions(+) create mode 100644 .gitattributes create mode 100644 README.md create mode 100644 granite-4.0-micro-OpenMed-BF16-00001-of-00002.gguf create mode 100644 granite-4.0-micro-OpenMed-BF16-00002-of-00002.gguf create mode 100644 granite-4.0-micro-OpenMed-Q2_K.gguf create mode 100644 granite-4.0-micro-OpenMed-Q4_K_M.gguf create mode 100644 granite-4.0-micro-OpenMed-Q6_K.gguf create mode 100644 granite-4.0-micro-OpenMed-Q8_0.gguf diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..9030331 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,42 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +granite-4.0-micro-OpenMed-q8_0.gguf filter=lfs diff=lfs merge=lfs -text +granite-4.0-micro-OpenMed-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +granite-4.0-micro-OpenMed-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +granite-4.0-micro-OpenMed-BF16-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text +granite-4.0-micro-OpenMed-BF16-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text +granite-4.0-micro-OpenMed-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +granite-4.0-micro-OpenMed-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..981cba0 --- /dev/null +++ b/README.md @@ -0,0 +1,160 @@ +--- +base_model: ibm-granite/granite-4.0-micro +library_name: transformers +pipeline_tag: text-generation +license: apache-2.0 +language: +- en +tags: +- medical +- instruction-tuned +- jepa-llm +- grpo +- dpo-like +- personas +- mergekit +- arcee-fusion +- openmed +- gguf +- q8 +- q4 +- q2 +--- + +# openmed-community/granite-4.0-micro-OpenMed-GGUF + +**Granite 4.0 Micro (≈3B) tuned for medical education & instruction following.** +Recipe: **JEPA-LLM SFT on medmcqa-hard + personas augmentation → GRPO on medmcqa-hard**; finalized with **Arcee Fusion** merge back into the IBM base. + +> ⚠️ **Medical safety** +> This model is **not** a clinician and may hallucinate. **Do not** use for diagnosis or treatment. Use under qualified medical supervision only. + +--- + +## TL;DR + +- **Base:** [`ibm-granite/granite-4.0-micro`](https://huggingface.co/ibm-granite/granite-4.0-micro) — 3B long-context instruct model (Apache-2.0). Includes a structured chat template and tool-calling examples. +- **Training (high-level):** + 1) **JEPA-LLM SFT (400 steps, bs=64)** on **`mkurman/medmcqa-hard`** plus **instruction-following personas** from **`allenai/tulu-3-sft-personas-instruction-following`**. + 2) **GRPO** (group-relative PPO) on **`mkurman/medmcqa-hard`**, bs **64/128**, **8 generations per item** (critic-free RL optimizing verifiable correctness). + 3) **Model merge:** **Arcee MergeKit** with `merge_method: arcee_fusion` to preserve base calibration while keeping domain gains. +- **Infra:** Trained/evaluated on **AMD Instinct MI300X** via **Hot AISLE** credits — thanks! + +--- + +## What’s inside + +### 1) JEPA-LLM stage (supervised) +- **JEPA-LLM** objective, see repo: [mkurman/jepa-llm](https://github.com/mkurman/jepa-llm), used as an auxiliary signal during SFT to bias toward stable, representation-level learning rather than pure next-token fitting; run for **400 steps** on **MedMCQA-hard** with **Personas augmentation** from **Tulu-3 personas** (adds constraint-following behaviors and improves coverage of IFEval-style requirements). + +### 2) GRPO stage (reinforcement learning) +- **GRPO** replaces the critic with group baselines, enabling efficient multi-sample training; we generate **8 candidates per item** and reward answer correctness / format checks. + +### 3) Merge & finalize +- **Arcee Fusion** in **MergeKit** to selectively fuse with the original Granite 4.0 Micro (avoids over-averaging from naive merges and tends to keep base calibration). + +--- + +## Intended use & limitations + +**Intended:** medical **research**, concept review, exam-style Q&A, instruction-following research, and tool-augmented demos. +**Out of scope:** autonomous clinical decisions, prescription generation, or guideline updates without retrieval/RAG. + +--- + +## Results + +| Metric | granite-4.0-micro-OpenMed | granite-4.0-micro | +| ----------------------------| -------------------------: | ------------------------: | +| mmlu | **63.17** | 62.48 | +| leaderboard_mmlu_pro | **33.06** | 32.78 | + +| leaderboard_ifeval | granite-4.0-micro-OpenMed | granite-4.0-micro | +| ----------------------------| -------------------------: | ------------------------: | +| inst_level_loose_acc | **85.97** | 85.25 | +| inst_level_strict_acc | **84.05** | 82.97 | +| prompt_level_loose_acc | **79.67** | 78.74 | +| prompt_level_strict_acc | **77.45** | 76.16 | + + +**Author’s harness notes:** EleutherAI `lm-evaluation-harness` with Granite’s chat template and batch size 8. + +--- + +## Quickstart (Transformers) + +```python +from transformers import AutoTokenizer, AutoModelForCausalLM +import torch + +model_id = "openmed-community/granite-4.0-micro-OpenMed" +tok = AutoTokenizer.from_pretrained(model_id, use_fast=True) +model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto") + +messages = [ + {"role": "system", "content": "You are a careful medical assistant. Cite sources and warn this is not medical advice."}, + {"role": "user", "content": "Cellulitis vs erysipelas: give 3 bullet differences and 1 caution."} +] +prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False) +inputs = tok(prompt, return_tensors="pt").to(model.device) +out = model.generate(**inputs, max_new_tokens=256, do_sample=False) +print(tok.decode(out[0], skip_special_tokens=True)) +```` + +> **Tool-calling:** Granite’s card includes function-calling examples; +> +--- + +## Reproduce key evals (example) + +```bash +# Classic MMLU (5-shot typical) +lm_eval --model hf \ + --model_args pretrained=openmed-community/granite-4.0-micro-OpenMed,parallelize=True \ + --tasks mmlu --batch_size 8 --apply-chat-template + +# MMLU-Pro (10-choice, harder) +lm_eval --model hf \ + --model_args pretrained=openmed-community/granite-4.0-micro-OpenMed,parallelize=True \ + --tasks leaderboard_mmlu_pro --batch_size 8 --apply-chat-template + +# IFEVAL (verifiable instruction following) +lm_eval --model hf \ + --model_args pretrained=openmed-community/granite-4.0-micro-OpenMed,parallelize=True \ + --tasks leaderboard_ifeval --batch_size 8 --apply-chat-template +``` + +--- + +## Data & training notes + +* **MedMCQA-Hard (train split)** for domain supervision and RL rewards;. +* **Tulu-3 personas** for instruction-following with constraint taxonomy inspired by IFEVAL. +* **JEPA-LLM**: based on the emerging **LLM-JEPA** objective (representation-space training). See the paper for context and motivation. +* **GRPO**: efficient for multi-sample training. +* **Privacy:** no PHI to the best of our knowledge; please report issues. + +--- + +## Commentary on results + +> **Why gains are modest:** Granite-4.0-Micro is already a **well-calibrated, strongly aligned** 3B instruct model with robust instruction-following and tool-use out of the box. In that regime, **headroom on popular benchmarks is limited**, and naive tuning often **degrades** base behaviors (calibration, safety, IF). The combination used here—**JEPA-LLM** (to stabilize representations), **personas SFT** (to preserve IF constraints), **GRPO** with **verifiable rewards**, and **Arcee Fusion**—appears to **nudge** the model to measurable improvements **without sacrificing** base calibration, but the effect sizes remain small, which is consistent with Granite’s strong baseline. In short: *we’re operating near the model’s alignment ceiling; targeted gains are possible, sweeping jumps are unlikely without larger capacity or richer supervision.* + +--- + +## Acknowledgments + +* **IBM Granite** team for the base model & docs (Apache-2.0). +* **AllenAI Tulu-3** for personas datasets. +* **Arcee** for MergeKit and **Arcee Fusion**. +* **Hot Aisle** for MI300X credits :heart:, link: [https://hotaisle.xyz/](https://hotaisle.xyz/). + +--- + +## Citation + +* IBM Granite 4.0 Micro model card [1](https://huggingface.co/ibm-granite/granite-4.0-micro). +* MedMCQA-Hard [2](https://huggingface.co/datasets/mkurman/medmcqa-hard). +* Tulu-3 personas dataset [3](https://huggingface.co/datasets/allenai/tulu-3-sft-personas-instruction-following). +* LLM-JEPA paper [4](https://arxiv.org/abs/2509.14252) and our implementation repository [5](https://github.com/mkurman/jepa-llm). +* MergeKit & Arcee Fusion [6](https://github.com/arcee-ai/mergekit). \ No newline at end of file diff --git a/granite-4.0-micro-OpenMed-BF16-00001-of-00002.gguf b/granite-4.0-micro-OpenMed-BF16-00001-of-00002.gguf new file mode 100644 index 0000000..2d80329 --- /dev/null +++ b/granite-4.0-micro-OpenMed-BF16-00001-of-00002.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37006eed228864c105b9a9feb92dcf84b587c7f5f95749892ae8e485a488a8e2 +size 3993883168 diff --git a/granite-4.0-micro-OpenMed-BF16-00002-of-00002.gguf b/granite-4.0-micro-OpenMed-BF16-00002-of-00002.gguf new file mode 100644 index 0000000..6b096e7 --- /dev/null +++ b/granite-4.0-micro-OpenMed-BF16-00002-of-00002.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f6bef8805770da3270f725ac063c64b6d604d19de82898b2365aee04f0caaa33 +size 2815773952 diff --git a/granite-4.0-micro-OpenMed-Q2_K.gguf b/granite-4.0-micro-OpenMed-Q2_K.gguf new file mode 100644 index 0000000..0f21b4e --- /dev/null +++ b/granite-4.0-micro-OpenMed-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:328ad4ef5615c8fcb7164d17fa6fe9299649ee75bd0bc25366a01d3e5e52743d +size 1371438656 diff --git a/granite-4.0-micro-OpenMed-Q4_K_M.gguf b/granite-4.0-micro-OpenMed-Q4_K_M.gguf new file mode 100644 index 0000000..1b16694 --- /dev/null +++ b/granite-4.0-micro-OpenMed-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc6f121e8b63940daa641556fc0e4ec5d0e65b37977e2f48dca51981b056cfb5 +size 2099502656 diff --git a/granite-4.0-micro-OpenMed-Q6_K.gguf b/granite-4.0-micro-OpenMed-Q6_K.gguf new file mode 100644 index 0000000..d460778 --- /dev/null +++ b/granite-4.0-micro-OpenMed-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:51845d6136aac409dbe87391696297a42fafbafd62b5a22fb94c709b36d60624 +size 2795617856 diff --git a/granite-4.0-micro-OpenMed-Q8_0.gguf b/granite-4.0-micro-OpenMed-Q8_0.gguf new file mode 100644 index 0000000..75aac5e --- /dev/null +++ b/granite-4.0-micro-OpenMed-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:115d49d9bd2ea453128a16da4d5c10c2d8d29ee979d44ae6f8f3d476823188c9 +size 3619692096