mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-21 05:52:16 +08:00
commit e343f0f202
27 changed files with 219 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
llama3.1-8B-toucan-full-sft.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

84
README.md Normal file
View File

@@ -0,0 +1,84 @@
---
base_model: rifoag/llama3.1-8B-toucan-full-sft
language:
- en
library_name: transformers
mradermacher:
readme_rev: 1
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
<!-- ### quants_skip: -->
<!-- ### skip_mmproj: -->
weighted/imatrix quants of https://huggingface.co/rifoag/llama3.1-8B-toucan-full-sft
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#llama3.1-8B-toucan-full-sft-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.1 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.8 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q4_1.gguf) | i1-Q4_1 | 5.2 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/llama3.1-8B-toucan-full-sft-i1-GGUF/resolve/main/llama3.1-8B-toucan-full-sft.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
llama3.1-8B-toucan-full-sft.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a68cb74dd57679cabe1fb4120cb8da47198941016ab2b85e2d271faeeb17bfc4
size 2161977120

3
llama3.1-8B-toucan-full-sft.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:688369381d500fdd65b0db1ed802a0520abc9ef302c6361f073edc6b4018b869
size 2019632928

3
llama3.1-8B-toucan-full-sft.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c700702aabbae57df6f49096fd3318f0b5472aa90eeb34c3c28bbe1ef7f23ad1
size 2948286240

3
llama3.1-8B-toucan-full-sft.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ff32df44f516d1aa852133ac7953e2c07a305df6b609cb856da9bf713f621cbd
size 2758493984

3
llama3.1-8B-toucan-full-sft.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8024316a27ec83d83f03d20bf2f4e571cd19b2ec85b944c1cbae38433d92f077
size 2605786912

3
llama3.1-8B-toucan-full-sft.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5a3bc77b8e509b968748a7c48282f3923bffe58b8793c2f0354d816387104333
size 2399217440

3
llama3.1-8B-toucan-full-sft.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:10f50c69ccfc35471da511f29888498573483f1654aecf91d8ef7221b4b380b2
size 3784828704

3
llama3.1-8B-toucan-full-sft.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6823a8f13c0e1b8411885e11295eaaf62f24c5adc5444616055bbcfdcb46e867
size 3682330400

3
llama3.1-8B-toucan-full-sft.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:19b46699fc531baa837af4743688286add5ca7aaf69429036579ed63acd381c4
size 3518752544

3
llama3.1-8B-toucan-full-sft.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e144a0dd7b62c103fcced3b9d44355c54af35d8216a0f6ba70598cbfc599c2bf
size 3274917664

3
llama3.1-8B-toucan-full-sft.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:259a8df4f84b45dc0c1f717835cf09d50d09776807cd147c7c1d974f20769c9d
size 4677994272

3
llama3.1-8B-toucan-full-sft.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ffc5ef1572e86674d4ab32d42ae9d0cbd6cb38ba7c91f103457d2e51920aafe5
size 4447668000

3
llama3.1-8B-toucan-full-sft.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c77cdd350882c75e55572210a38c85a2737a151c14cc58d383900bad054faee8
size 3179136800

3
llama3.1-8B-toucan-full-sft.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e5050a77a06ea294f26672f9c8a8167b88f74baa5d13f4fc3d9ba0d3cbd9f775
size 2988820256

3
llama3.1-8B-toucan-full-sft.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4265ebe60e8bd502e1f7871546d2d1f442a36845789095d812a3aefc354e7d70
size 4321961760

3
llama3.1-8B-toucan-full-sft.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:94e35eae46338a5caccaba21712d2d36cebb7dc9e58e7efac635c0f9b36bae09
size 4018923296

3
llama3.1-8B-toucan-full-sft.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:918d6b8ab5bf859be0bedc5be7ea88ef6b5b4071a4516ab523e0420e11ffc3b8
size 3664504608

3
llama3.1-8B-toucan-full-sft.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:376eb6b463b9d3539426b07ee92653f99fa4edf74b698d2fc8b94d432d38a30d
size 4675897120

3
llama3.1-8B-toucan-full-sft.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a83ce65d0dc49bfc42e5354ed83d198d84e12f4fc9dd56746a73c81a13e32e38
size 5130258208

3
llama3.1-8B-toucan-full-sft.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bf4add5230f88dd56efe5fc175ce2de6dd87fc81a47816d1816f7b23f129e7e6
size 4920739616

3
llama3.1-8B-toucan-full-sft.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eba5a50dc1587bd409d83d8dfad460300243744eb2494061bddf4c6b19d32940
size 4692674336

3
llama3.1-8B-toucan-full-sft.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:60b592765c8fd59beebf25b0417ebdfd8f85d5ddea8893de731af56779a0cf86
size 5732992800

3
llama3.1-8B-toucan-full-sft.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0d09d238289c86bc35ca0e52335186cd49777d475f2dbe31f894a425f1a244df
size 5599299360

3
llama3.1-8B-toucan-full-sft.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7983ca601f2b496d67870f5c413d06353a0dfd2f40554178d479d240a80047fa
size 6596011808

3
llama3.1-8B-toucan-full-sft.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d9552f52dd5beb65877878dcbdbf21146d1fbc2c6656202a66427fb40f9b3bd7
size 5015200