初始化项目，由ModelHub XC社区提供模型
Model: occ-ai/OCC-RAG-1.7B-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
42
.gitattributes
vendored
Normal file
42
.gitattributes
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
OCC-RAG-1.7B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
OCC-RAG-1.7B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
OCC-RAG-1.7B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
OCC-RAG-1.7B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
OCC-RAG-1.7B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
OCC-RAG-1.7B-BF16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
OCC-RAG-1.7B-F16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
OCC-RAG-1.7B-BF16.gguf
Normal file
3
OCC-RAG-1.7B-BF16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3b8e09f89fc5c9fdce8f792c26075b75150790f0460ddc097972a83c0b57bb4d
|
||||
size 3447346304
|
||||
3
OCC-RAG-1.7B-F16.gguf
Normal file
3
OCC-RAG-1.7B-F16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3e6a44dcdb45b477f4d62ac56937ab8ac8efb7712403d65c5a8acef4dfee3885
|
||||
size 3447346304
|
||||
3
OCC-RAG-1.7B-Q4_0.gguf
Normal file
3
OCC-RAG-1.7B-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1fc0d36da0b20939c65aa1863d056bfa19bbfb04d5b4f4f8f882cd6284ce9ca3
|
||||
size 1054420096
|
||||
3
OCC-RAG-1.7B-Q4_K_M.gguf
Normal file
3
OCC-RAG-1.7B-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5ff807a93050972b39518ab09325e4f70d517cc8bdd3fe2924d4c36006412a51
|
||||
size 1107405952
|
||||
3
OCC-RAG-1.7B-Q5_K_M.gguf
Normal file
3
OCC-RAG-1.7B-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:012e29090841e42ffeb9fe3644de2bf0fc07844f97cbb5515838495943c7e597
|
||||
size 1257876608
|
||||
3
OCC-RAG-1.7B-Q6_K.gguf
Normal file
3
OCC-RAG-1.7B-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:06b0db79b33d334eb7d62c89416539079e2e91a224f6016a3d512d7eb411c17a
|
||||
size 1417751680
|
||||
3
OCC-RAG-1.7B-Q8_0.gguf
Normal file
3
OCC-RAG-1.7B-Q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7f3e60bf51287948716575d7b05b98e39a1f7714a7fab06092104a78948c8ff3
|
||||
size 1834423424
|
||||
122
README.md
Normal file
122
README.md
Normal file
@@ -0,0 +1,122 @@
|
||||
---
|
||||
license: mit
|
||||
language:
|
||||
- en
|
||||
- ru
|
||||
library_name: gguf
|
||||
pipeline_tag: text-generation
|
||||
base_model:
|
||||
- occ-ai/OCC-RAG-1.7B
|
||||
tags:
|
||||
- rag
|
||||
- faithful-qa
|
||||
- occ
|
||||
- gguf
|
||||
- llama.cpp
|
||||
- quantized
|
||||
---
|
||||
|
||||
# OCC-RAG-1.7B-GGUF
|
||||
|
||||
<p align="center">
|
||||
<img src="figures/occ.png" alt="OCC-RAG" width="320"/>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/optimal-cognitive-core/OCC-RAG"><img src="figures/github-mark.png" alt=""/> <b>GitHub</b></a> |
|
||||
<a href="https://arxiv.org/abs/2606.00683"><b>Technical Report</b></a> |
|
||||
<a href="https://cloud.ru/products/evolution-ml-inference"><b>Cloud</b></a> |
|
||||
<a href="https://huggingface.co/occ-ai/OCC-RAG-1.7B"><b>Base model</b></a>
|
||||
</p>
|
||||
|
||||
**GGUF quantizations of [`occ-ai/OCC-RAG-1.7B`](https://huggingface.co/occ-ai/OCC-RAG-1.7B)**
|
||||
for native inference with [llama.cpp](https://github.com/ggml-org/llama.cpp),
|
||||
[Ollama](https://ollama.com/), [LM Studio](https://lmstudio.ai/), and other
|
||||
GGUF-compatible runtimes.
|
||||
|
||||
OCC-RAG-1.7B is a 1.7B-parameter small language model specialized for **faithful,
|
||||
context-grounded question answering**: given a question and a set of sources, it produces
|
||||
a structured reasoning trace with explicit source citations, decides whether the context
|
||||
supports an answer, and either answers from the context or abstains. It attains the **best
|
||||
faithfulness across all evaluated scales (up to 32B)**. See the
|
||||
[base model card](https://huggingface.co/occ-ai/OCC-RAG-1.7B) for training details and
|
||||
benchmarks.
|
||||
|
||||
## Files
|
||||
|
||||
The underlying architecture is Qwen3 (1.7B). The chat template is embedded in the GGUF,
|
||||
so `llama.cpp`/Ollama apply it automatically.
|
||||
|
||||
| File | Quant | Size | Notes |
|
||||
|---|---|---|---|
|
||||
| `OCC-RAG-1.7B-Q4_0.gguf` | Q4_0 | 1.05 GB | 4-bit, legacy — smallest |
|
||||
| `OCC-RAG-1.7B-Q4_K_M.gguf` | Q4_K_M | 1.11 GB | 4-bit K-quant — **recommended balance** |
|
||||
| `OCC-RAG-1.7B-Q5_K_M.gguf` | Q5_K_M | 1.26 GB | 5-bit K-quant |
|
||||
| `OCC-RAG-1.7B-Q6_K.gguf` | Q6_K | 1.42 GB | 6-bit K-quant |
|
||||
| `OCC-RAG-1.7B-Q8_0.gguf` | Q8_0 | 1.83 GB | 8-bit — near-lossless |
|
||||
| `OCC-RAG-1.7B-F16.gguf` | F16 | 3.45 GB | 16-bit float (full precision) |
|
||||
| `OCC-RAG-1.7B-BF16.gguf` | BF16 | 3.45 GB | 16-bit bfloat (lossless base) |
|
||||
|
||||
For most uses pick **Q4_K_M** (smallest good quality) or **Q8_0** (highest quality at
|
||||
under 2 GB). All quants are derived from the BF16 base. For an even smaller footprint or
|
||||
in-browser use, see [`occ-ai/OCC-RAG-0.6B-GGUF`](https://huggingface.co/occ-ai/OCC-RAG-0.6B-GGUF).
|
||||
|
||||
## Usage — llama.cpp
|
||||
|
||||
```bash
|
||||
# Run directly from the Hub (downloads the chosen quant)
|
||||
llama-cli -hf occ-ai/OCC-RAG-1.7B-GGUF:Q4_K_M -p "Hello" -no-cnv
|
||||
|
||||
# Or download a file and run it
|
||||
llama-cli -m OCC-RAG-1.7B-Q4_K_M.gguf -p "Hello" -no-cnv
|
||||
# (newer llama.cpp: use `llama-completion` for non-interactive runs)
|
||||
```
|
||||
|
||||
## Usage — Ollama
|
||||
|
||||
```bash
|
||||
ollama run hf.co/occ-ai/OCC-RAG-1.7B-GGUF:Q4_K_M
|
||||
```
|
||||
|
||||
## Input / output format
|
||||
|
||||
OCC-RAG uses a **structured RAG prompt with special tokens**: the question is wrapped in
|
||||
`<|query_start|> … <|query_end|>` and each source in
|
||||
`<|source_start|><|source_id|>N … <|source_end|>`. The response has five sections —
|
||||
**query analysis → source analysis → reasoning → status (`ANSWERABLE` / `UNANSWERABLE`)
|
||||
→ answer** — and the final answer is in `<|answer_start|> … <|answer_end|>`.
|
||||
|
||||
The embedded chat template (apply with llama.cpp's `--jinja`) builds the query/source
|
||||
tokens for you when sources are supplied as documents; alternatively assemble the tokens
|
||||
manually. See the [base model card](https://huggingface.co/occ-ai/OCC-RAG-1.7B) for the
|
||||
full format and a runnable example.
|
||||
|
||||
> We recommend **greedy decoding** (`--temp 0`), the training/evaluation default.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Context-grounded only.** Trained to answer from the supplied sources and to ignore
|
||||
parametric knowledge — not a general-purpose chat or knowledge model.
|
||||
- **Reasoning depth.** Training/evaluation are capped at three-hop reasoning; longer
|
||||
chains are out of distribution.
|
||||
- **Quantization.** Lower-bit quants (Q4) trade some quality for size; prefer Q6_K/Q8_0
|
||||
when accuracy matters most.
|
||||
|
||||
## License
|
||||
|
||||
Released under the [MIT License](https://huggingface.co/occ-ai/OCC-RAG-1.7B), inherited
|
||||
from the base model.
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{savkin2026occragoptimalcognitivecore,
|
||||
title = {OCC-RAG: Optimal Cognitive Core for Faithful Question Answering},
|
||||
author = {Maksim Savkin and Mikhail Goncharov and Alexander Gambashidze and Alla Chepurova and Dmitrii Tarasov and Nikita Andriianov and Daria Pugacheva and Vasily Konovalov and Andrey Galichin and Ivan Oseledets},
|
||||
year = {2026},
|
||||
eprint = {2606.00683},
|
||||
archivePrefix = {arXiv},
|
||||
primaryClass = {cs.CL},
|
||||
url = {https://arxiv.org/abs/2606.00683}
|
||||
}
|
||||
```
|
||||
BIN
figures/github-mark.png
Normal file
BIN
figures/github-mark.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 453 B |
BIN
figures/occ.png
Normal file
BIN
figures/occ.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
Reference in New Issue
Block a user