初始化项目，由ModelHub XC社区提供模型

Model: Orionfold/Saul-7B-Instruct-v1-GGUF Source: Original Platform
2026-06-12 20:02:17 +08:00
commit d3e1ef362c
7 changed files with 164 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,40 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+model-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+model-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+model-F16.gguf filter=lfs diff=lfs merge=lfs -text
+model-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+model-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,109 @@
+---
+license: mit
+library_name: gguf
+base_model: Equall/Saul-7B-Instruct-v1
+pipeline_tag: text-generation
+model_creator: Orionfold LLC
+language:
+  - en
+tags:
+  - gguf
+  - spark-tested
+  - orionfold
+  - "base_model:Equall/Saul-7B-Instruct-v1"
+---
+# Saul 7B Instruct v1 GGUF
+
+`GGUF` quantizations of `Equall/Saul-7B-Instruct-v1`, verified end-to-end on the NVIDIA DGX Spark (GB10, 128 GB unified memory).
+
+## Notebooks
+
+Two runnable notebooks ship with this model — open either on a free cloud GPU:
+
+| Notebook | What it does | Open |
+|---|---|---|
+| **Builder** | Reproduce this model's build and DGX Spark benchmarks end-to-end with `fieldkit`. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/manavsehgal/ai-field-notes/blob/main/notebooks/legal/builder.ipynb) [![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/manavsehgal/ai-field-notes/blob/main/notebooks/legal/builder.ipynb) |
+| **User** | Load the published model and call it from your own app in a few lines. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/manavsehgal/ai-field-notes/blob/main/notebooks/legal/user.ipynb) [![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/manavsehgal/ai-field-notes/blob/main/notebooks/legal/user.ipynb) |
+
+## Spark-tested
+
+Every Orionfold quant ships with a measurement quad on the NVIDIA DGX Spark (GB10, 128 GB unified memory): perplexity, sustained `tok/s`, thermal envelope, and **LegalBench (n=50, contains)** accuracy. The numbers below are the actual run, not a wishlist.
+
+| Variant | Size | Perplexity (wikitext-2) | tok/s on Spark | LegalBench (n=50, contains) |
+|---|---|---|---|---|
+| Q4_K_M | 4.1 GB | 5.986 | 29.4 | 62.0% |
+| Q5_K_M | 4.8 GB | 5.938 | 20.2 | 72.0% |
+| Q6_K | 5.5 GB | 5.925 | 22.4 | 68.0% |
+| Q8_0 | 7.2 GB | 5.914 | 7.3 | 66.0% |
+| F16 | 13.5 GB | 5.917 | 10.9 | 68.0% |
+
+**Thermal envelope:** sustained-load minutes before thermal throttle on a single GB10 = **2 min**. Beyond this, expect tok/s degradation; the duty-cycle disclosure is per Orionfold's quant-card standard.
+
+## Variants
+
+| Variant | Recommended use |
+|---|---|
+| Q4_K_M | Best balance — fits comfortably in Spark unified memory at 70B; default pick. |
+| Q5_K_M | Higher quality than Q4_K_M with modest size bump. |
+| Q6_K | Near-lossless; recommended if memory headroom allows. |
+| Q8_0 | Effectively lossless; reach for this when quality matters more than throughput. |
+| F16 | Reference — no quantization. Use only for measurement / baseline. |
+
+## How to run
+
+Pull a variant:
+
+```bash
+huggingface-cli download Orionfold/Saul-7B-Instruct-v1-GGUF model-Q5_K_M.gguf \
+  --local-dir ./models/saul-7b-instruct-v1
+```
+
+Serve it via `llama-server` (OpenAI-compatible API):
+
+```bash
+llama-server -m ./models/saul-7b-instruct-v1/model-Q5_K_M.gguf \
+  -c 4096 -ngl 99 -t 8 \
+  --host 0.0.0.0 --port 8080
+```
+
+Or run in-process via `llama-cpp-python`:
+
+```python
+from llama_cpp import Llama
+llm = Llama(
+    model_path="./models/saul-7b-instruct-v1/model-Q5_K_M.gguf",
+    n_ctx=4096, n_gpu_layers=99, chat_format="mistral",
+)
+out = llm.create_chat_completion(
+    messages=[
+        {"role": "user",
+         "content": "Does the following sentence overrule a previous case? "
+                    "Sentence: 'curtman is overruled to the extent it conflicts with evans.'"}
+    ],
+    temperature=0.0,
+)
+print(out["choices"][0]["message"]["content"])
+```
+
+LM Studio and Ollama (via a Modelfile) load the GGUF directly with no additional setup.
+
+## Methods
+
+Full methodology and Spark-side measurement protocol: [Vertical-curator quants on Spark — Saul-7B-Instruct-v1-GGUF + LegalBench mini-eval](https://ainative.business/field-notes/becoming-a-legal-curator-on-spark/).
+
+## Other Orionfold vertical curators
+
+Same Spark-tested recipe across the curator-on-Spark series:
+
+- **[finance-chat-GGUF](https://huggingface.co/Orionfold/finance-chat-GGUF)** — AdaptLLM finance-chat (Llama-2-7B lineage) for FinanceBench-shaped queries
+- **[SecurityLLM-GGUF](https://huggingface.co/Orionfold/SecurityLLM-GGUF)** — Mistral-based cyber-tuned model with CyberMetric mini-eval gating
+- **[II-Medical-8B-GGUF](https://huggingface.co/Orionfold/II-Medical-8B-GGUF)** — Qwen3-8B + DAPO reasoning for MedMCQA-shaped queries
+
+Each card lists its own measurement quad; the headline numbers are recorded as the actual sweep ran, never pre-corrected.
+
+---
+
+Published by **Orionfold LLC** · [orionfold.com](https://orionfold.com) · Methods documented at [ainative.business/field-notes](https://ainative.business/field-notes/).
+
+
+> Want to know when the next Orionfold vertical curator drops? [Join the launch list at orionfold.com](https://orionfold.com).
--- a/model-F16.gguf
+++ b/model-F16.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc78994794dbe468262e8e16933b2b41af5d12bcdf86d114510a567c94443527
+size 14484732736
--- a/model-Q4_K_M.gguf
+++ b/model-Q4_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb8311be677b0c517126701c0e7ec22955cd2582ead37e4e16d9fc87c244fbb0
+size 4368440128
--- a/model-Q5_K_M.gguf
+++ b/model-Q5_K_M.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19558001bf6fb0cbd41b620ca77a50688f3468a19dd95216301395ef718d0eff
+size 5131410240
--- a/model-Q6_K.gguf
+++ b/model-Q6_K.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d2dd7920bae67e67ec71e196b19dccd1bf9e50af6bfd0cc415bbaf10ed07e39
+size 5942065984
--- a/model-Q8_0.gguf
+++ b/model-Q8_0.gguf
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb8d7c13d240599f8406fb106ec04be4640e36abfc4c5d8ca9692768e6c0a566
+size 7695858496