commit d3e1ef362c4a4f3deb1c485cab7bb77503610638 Author: ModelHub XC Date: Fri Jun 12 20:02:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: Orionfold/Saul-7B-Instruct-v1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..402448f --- /dev/null +++ b/.gitattributes @@ -0,0 +1,40 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +model-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +model-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +model-F16.gguf filter=lfs diff=lfs merge=lfs -text +model-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +model-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..c9985aa --- /dev/null +++ b/README.md @@ -0,0 +1,109 @@ +--- +license: mit +library_name: gguf +base_model: Equall/Saul-7B-Instruct-v1 +pipeline_tag: text-generation +model_creator: Orionfold LLC +language: + - en +tags: + - gguf + - spark-tested + - orionfold + - "base_model:Equall/Saul-7B-Instruct-v1" +--- +# Saul 7B Instruct v1 GGUF + +`GGUF` quantizations of `Equall/Saul-7B-Instruct-v1`, verified end-to-end on the NVIDIA DGX Spark (GB10, 128 GB unified memory). + +## Notebooks + +Two runnable notebooks ship with this model — open either on a free cloud GPU: + +| Notebook | What it does | Open | +|---|---|---| +| **Builder** | Reproduce this model's build and DGX Spark benchmarks end-to-end with `fieldkit`. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/manavsehgal/ai-field-notes/blob/main/notebooks/legal/builder.ipynb) [![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/manavsehgal/ai-field-notes/blob/main/notebooks/legal/builder.ipynb) | +| **User** | Load the published model and call it from your own app in a few lines. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/manavsehgal/ai-field-notes/blob/main/notebooks/legal/user.ipynb) [![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/manavsehgal/ai-field-notes/blob/main/notebooks/legal/user.ipynb) | + +## Spark-tested + +Every Orionfold quant ships with a measurement quad on the NVIDIA DGX Spark (GB10, 128 GB unified memory): perplexity, sustained `tok/s`, thermal envelope, and **LegalBench (n=50, contains)** accuracy. The numbers below are the actual run, not a wishlist. + +| Variant | Size | Perplexity (wikitext-2) | tok/s on Spark | LegalBench (n=50, contains) | +|---|---|---|---|---| +| Q4_K_M | 4.1 GB | 5.986 | 29.4 | 62.0% | +| Q5_K_M | 4.8 GB | 5.938 | 20.2 | 72.0% | +| Q6_K | 5.5 GB | 5.925 | 22.4 | 68.0% | +| Q8_0 | 7.2 GB | 5.914 | 7.3 | 66.0% | +| F16 | 13.5 GB | 5.917 | 10.9 | 68.0% | + +**Thermal envelope:** sustained-load minutes before thermal throttle on a single GB10 = **2 min**. Beyond this, expect tok/s degradation; the duty-cycle disclosure is per Orionfold's quant-card standard. + +## Variants + +| Variant | Recommended use | +|---|---| +| Q4_K_M | Best balance — fits comfortably in Spark unified memory at 70B; default pick. | +| Q5_K_M | Higher quality than Q4_K_M with modest size bump. | +| Q6_K | Near-lossless; recommended if memory headroom allows. | +| Q8_0 | Effectively lossless; reach for this when quality matters more than throughput. | +| F16 | Reference — no quantization. Use only for measurement / baseline. | + +## How to run + +Pull a variant: + +```bash +huggingface-cli download Orionfold/Saul-7B-Instruct-v1-GGUF model-Q5_K_M.gguf \ + --local-dir ./models/saul-7b-instruct-v1 +``` + +Serve it via `llama-server` (OpenAI-compatible API): + +```bash +llama-server -m ./models/saul-7b-instruct-v1/model-Q5_K_M.gguf \ + -c 4096 -ngl 99 -t 8 \ + --host 0.0.0.0 --port 8080 +``` + +Or run in-process via `llama-cpp-python`: + +```python +from llama_cpp import Llama +llm = Llama( + model_path="./models/saul-7b-instruct-v1/model-Q5_K_M.gguf", + n_ctx=4096, n_gpu_layers=99, chat_format="mistral", +) +out = llm.create_chat_completion( + messages=[ + {"role": "user", + "content": "Does the following sentence overrule a previous case? " + "Sentence: 'curtman is overruled to the extent it conflicts with evans.'"} + ], + temperature=0.0, +) +print(out["choices"][0]["message"]["content"]) +``` + +LM Studio and Ollama (via a Modelfile) load the GGUF directly with no additional setup. + +## Methods + +Full methodology and Spark-side measurement protocol: [Vertical-curator quants on Spark — Saul-7B-Instruct-v1-GGUF + LegalBench mini-eval](https://ainative.business/field-notes/becoming-a-legal-curator-on-spark/). + +## Other Orionfold vertical curators + +Same Spark-tested recipe across the curator-on-Spark series: + +- **[finance-chat-GGUF](https://huggingface.co/Orionfold/finance-chat-GGUF)** — AdaptLLM finance-chat (Llama-2-7B lineage) for FinanceBench-shaped queries +- **[SecurityLLM-GGUF](https://huggingface.co/Orionfold/SecurityLLM-GGUF)** — Mistral-based cyber-tuned model with CyberMetric mini-eval gating +- **[II-Medical-8B-GGUF](https://huggingface.co/Orionfold/II-Medical-8B-GGUF)** — Qwen3-8B + DAPO reasoning for MedMCQA-shaped queries + +Each card lists its own measurement quad; the headline numbers are recorded as the actual sweep ran, never pre-corrected. + +--- + +Published by **Orionfold LLC** · [orionfold.com](https://orionfold.com) · Methods documented at [ainative.business/field-notes](https://ainative.business/field-notes/). + + +> Want to know when the next Orionfold vertical curator drops? [Join the launch list at orionfold.com](https://orionfold.com). diff --git a/model-F16.gguf b/model-F16.gguf new file mode 100644 index 0000000..588dd01 --- /dev/null +++ b/model-F16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bc78994794dbe468262e8e16933b2b41af5d12bcdf86d114510a567c94443527 +size 14484732736 diff --git a/model-Q4_K_M.gguf b/model-Q4_K_M.gguf new file mode 100644 index 0000000..5a8153d --- /dev/null +++ b/model-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bb8311be677b0c517126701c0e7ec22955cd2582ead37e4e16d9fc87c244fbb0 +size 4368440128 diff --git a/model-Q5_K_M.gguf b/model-Q5_K_M.gguf new file mode 100644 index 0000000..08b1858 --- /dev/null +++ b/model-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19558001bf6fb0cbd41b620ca77a50688f3468a19dd95216301395ef718d0eff +size 5131410240 diff --git a/model-Q6_K.gguf b/model-Q6_K.gguf new file mode 100644 index 0000000..7887692 --- /dev/null +++ b/model-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d2dd7920bae67e67ec71e196b19dccd1bf9e50af6bfd0cc415bbaf10ed07e39 +size 5942065984 diff --git a/model-Q8_0.gguf b/model-Q8_0.gguf new file mode 100644 index 0000000..53c6d0d --- /dev/null +++ b/model-Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb8d7c13d240599f8406fb106ec04be4640e36abfc4c5d8ca9692768e6c0a566 +size 7695858496