mradermacher/marin-8b-instruct-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/marin-8b-instruct-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-18 13:32:17 +08:00
commit dfca8edddd
27 changed files with 230 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
marin-8b-instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

95
README.md Normal file
View File

@@ -0,0 +1,95 @@
---
base_model: marin-community/marin-8b-instruct
datasets:
- TIGER-Lab/AceCode-87K
- bespokelabs/Bespoke-Stratos-17k
- cognitivecomputations/dolphin-r1
- tuenguyen/dolphin_r1_reasoning
- facebook/natural_reasoning
- open-r1/OpenThoughts-114k-math
- HuggingFaceTB/smoltalk
language:
- en
library_name: transformers
license: apache-2.0
mradermacher:
readme_rev: 1
quantized_by: mradermacher
tags:
- text-generation
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
<!-- ### quants_skip: -->
<!-- ### skip_mmproj: -->
weighted/imatrix quants of https://huggingface.co/marin-community/marin-8b-instruct
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#marin-8b-instruct-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/marin-8b-instruct-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.1 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.8 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q4_1.gguf) | i1-Q4_1 | 5.2 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/marin-8b-instruct-i1-GGUF/resolve/main/marin-8b-instruct.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
marin-8b-instruct.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1487b7603fe6f8d879a4d1ae23195516617d1ddef12b40b7c1a126575cd22960
size 2161976672

3
marin-8b-instruct.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3771aab6f15d0c659dfc79097654ea74e06c980a9603ebda86d8f448d0f733b7
size 2019632480

3
marin-8b-instruct.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6b28a7cbd1a8f9d130ca982f589f37c8b383685b4863e1b65dff7dc225b88311
size 2948285792

3
marin-8b-instruct.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:968a9efb4ee912eb338b4b5828644b770167438309a42294fd003d4b2a53178b
size 2758493536

3
marin-8b-instruct.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b86e3922bddc1d04df0b195100a015003488c012b85ab336b50b30705e4fad6e
size 2605786464

3
marin-8b-instruct.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8b294208cf4a841b7a5c6d4204a83681310cbf11f3280b43bbc9808c783ec315
size 2399216992

3
marin-8b-instruct.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eb85f3c7aa080124a578ee37927f132cffe93c22c69153840b89eea238588e8f
size 3784828256

3
marin-8b-instruct.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7f6ea864b17f1dcc8d2356e1ba8501948a78e1b6b8870713aa5410ce9448f939
size 3682329952

3
marin-8b-instruct.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c8b5d002257b42cdfada7ff7ac06179145540e78d410104ba7cca85649aa709e
size 3518752096

3
marin-8b-instruct.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a4584f54a2e88d248411c4b8a0074199cf26b2e95129972d18a6cc004eb2029d
size 3274917216

3
marin-8b-instruct.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b81c94e985c465c12fb13c3c300549e50927b9f53dfd9654d2c50978002eac68
size 4677993824

3
marin-8b-instruct.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7801b1adc2ab5d1f74826d54bedb10c0e8cc0feb32a15e14075f69fae3228732
size 4447667552

3
marin-8b-instruct.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:73d88eafeb85d9153738ff855e67e5d9fa889545043118a018c636acb306437a
size 3179136352

3
marin-8b-instruct.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd0f189ad5adbe0dcd49b16ddfd1d8762f69a8db58e996ebc76098e1629264bd
size 2988819808

3
marin-8b-instruct.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ca946717c9672b04bedc42ca4ec6c394a50ae18ea04e220bf6b728f4c576fa9d
size 4321961312

3
marin-8b-instruct.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6579d96052bf3bca08d930919a476baff020ed21f7a650f7ce76e3c4b78dc6d6
size 4018922848

3
marin-8b-instruct.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1ab9491a9624b647393c5981f387f385d47ef913a983bc6963f13bb4e8d70878
size 3664504160

3
marin-8b-instruct.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d68d3ad1c9f6e6700616133b8adfc3354c387837c8a17924cc6d3b5a2fe3e118
size 4675896672

3
marin-8b-instruct.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:071167a7c07de47948dcbe4e9294e9d4ceac1357defe0c29571021f690465e81
size 5130257760

3
marin-8b-instruct.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cf038fb6b6d713e91101e76bceafd3c385ca65a17b32efe45e69a7bf3b50675f
size 4920739168

3
marin-8b-instruct.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:099572e111201c80f4aeade91158c857205e41ba09662b8b5da00c5c7883b43c
size 4692673888

3
marin-8b-instruct.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:62548985035f85b92163aa98b18b711d7f7f2e69830e10cb3c0206c000df087e
size 5732992352

3
marin-8b-instruct.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b5f0d3a86a352d2769a92858a69cb0300758561c81ff5eaa8c57fab5999e2d5
size 5599298912

3
marin-8b-instruct.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1a4ed3bd69370fdca6e6a6b58995cb9063a02f2cf500e8a59b6a110eb2377555
size 6596011360

3
marin-8b-instruct.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:709523551bcff1ebe82b0e6495368c49517e730bf322880efe054027de0b72cd
size 5015200