mradermacher/gpt4all-falcon-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/gpt4all-falcon-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-27 08:23:17 +08:00
commit cc3002641d
27 changed files with 213 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
gpt4all-falcon.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

78
README.md Normal file
View File

@@ -0,0 +1,78 @@
---
base_model: nomic-ai/gpt4all-falcon
datasets:
- nomic-ai/gpt4all-j-prompt-generations
language:
- en
library_name: transformers
license: apache-2.0
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/nomic-ai/gpt4all-falcon
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/gpt4all-falcon-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ1_S.gguf) | i1-IQ1_S | 3.4 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ1_M.gguf) | i1-IQ1_M | 3.4 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.5 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ2_XS.gguf) | i1-IQ2_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ2_S.gguf) | i1-IQ2_S | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ2_M.gguf) | i1-IQ2_M | 3.7 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.8 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.8 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ3_S.gguf) | i1-IQ3_S | 4.0 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ3_XS.gguf) | i1-IQ3_XS | 4.0 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q2_K.gguf) | i1-Q2_K | 4.0 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q3_K_S.gguf) | i1-Q3_K_S | 4.0 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ3_M.gguf) | i1-IQ3_M | 4.1 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.2 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.2 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.3 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q4_0.gguf) | i1-Q4_0 | 4.3 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.5 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q4_1.gguf) | i1-Q4_1 | 4.7 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.1 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.4 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/gpt4all-falcon-i1-GGUF/resolve/main/gpt4all-falcon.i1-Q6_K.gguf) | i1-Q6_K | 7.1 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
gpt4all-falcon.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7c87a8a0958de7f8d870c763ef31c0314fd529fe07108f049e8e80d02519485e
size 3338360032

3
gpt4all-falcon.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ccb4cec16491ff3d884a2f0776ce83b3f1c5b296e9a81473152cb4900ed4ab0d
size 3284159200

3
gpt4all-falcon.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f60ef823dc899b600fcac39d01adf50f8b862a4e1e7d6a594b646517a84d7586
size 3606783200

3
gpt4all-falcon.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d0f0251f0d75e7249082a9198f4880672aefb05b7e3d8b1427a9eb52b5f0c2d0
size 3534515424

3
gpt4all-falcon.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:882cdac336c26befaa6e4b53ac0176702ea0f065cb92be5a0745f15c6eba5b24
size 3500962528

3
gpt4all-falcon.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:df904d86b23691aee1d29372925c483b746522bad63e3baa7bf17301a3234041
size 3428694752

3
gpt4all-falcon.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:daee670b8f4292ab2be30cca5b198f243663e3cd0c6e972de387e4086eb3dd63
size 3988515552

3
gpt4all-falcon.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c2a0f7976c0038ada1ba4ce1858d6b469d21fc6ec8f29a3c811fd1356da6d409
size 3859720416

3
gpt4all-falcon.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:94d487febd73233387c9cb22f61230130e7e56abb4bb3fe966b5e6439b0eea0d
size 3859720416

3
gpt4all-falcon.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:89a179756748775824a7082528fc4ac32870bbb2ff6db93df9e9e17041f2be1f
size 3735832800

3
gpt4all-falcon.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:34756040bf0f54b6e6712777b225a3be29daf3be9c89cfc5149b683d169aa2d0
size 4210735328

3
gpt4all-falcon.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b6e87ab2ba3a4e924bfb7f6e88d7f79be4f1d05e7f3c30df18899fea709924d9
size 4128143584

3
gpt4all-falcon.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c3e5341b2b18e45a448d62c88e8fb6bf0e8664a658f6c356b7ba7f6b4df3aa95
size 3859720416

3
gpt4all-falcon.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4cbc3bb72dd7c48882867c235580fc235e975d205e39ba3c39d5f2a6dca33b5a
size 3668727008

3
gpt4all-falcon.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7bbcfb1082638c786703c90cb69b7d27bf3f96b62209385e8bee2a64f9abc478
size 4378245344

3
gpt4all-falcon.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5436110b3ac927d726ea402de5ec0fb756d11be23b4f7fc12000c14cdb24f856
size 4140794080

3
gpt4all-falcon.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b6bb70cd75d91e8ab140ae6a594455a667ad7ffaa1ac054b8e6e32e47bf4280b
size 3859720416

3
gpt4all-falcon.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b18bf353de305b14b03c68c38fc66fd6b8895475bfa18646b2b26f91fc1a843
size 4231383264

3
gpt4all-falcon.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ff41543048064059d09373980cce5235251cbe792f27daf1925e48113e2e035
size 4643324128

3
gpt4all-falcon.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b1242ec4a339e41d61c7476cda4dbc62bb3702645be90e0651bfe39025c40029
size 4975127008

3
gpt4all-falcon.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:be141dea1b613b67bb3426fa1be12f80e8768c4e48d5eae5508411f361ef9317
size 4745545952

3
gpt4all-falcon.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ec63c9d70c560a2df06674ca96b250d05dc2ccc22da17e9035f973d92ebc80a5
size 5731121376

3
gpt4all-falcon.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7dbab1b714f23a447ecc6c78de3f0654790195c37864d80f3df4d3caec27e834
size 5343318240

3
gpt4all-falcon.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7072e06ee09029aed17c8ab4f9a6db33fe4c6fff9cc95b15f7ac43dd7391a671
size 7031359712

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7acbbaa068b86fe2a7036d62ab701b715a1f2e4834bbf7a5da776c8255a2e623
size 4075803