mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-04-21 22:11:39 +08:00
commit b160640347
27 changed files with 234 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Falcon3-3B-Instruct-Heretic.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

3
Falcon3-3B-Instruct-Heretic.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b8496c2f0f0ee4265d40efdc7eab1030b364755d83b2ed5fb538fe5167a5ecc5
size 965973120

3
Falcon3-3B-Instruct-Heretic.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:345810d45006ee0282132c209aa26749aae3dc5e3889747eb82305ab9ce3e270
size 917017728

3
Falcon3-3B-Instruct-Heretic.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bcd414c75d969f860ff7776d21c8983106ec8dd482c5e7dc59c2fb7bcefcf3b0
size 1267471488

3
Falcon3-3B-Instruct-Heretic.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2cbf1c723ec5b6f7a262021cf976f6d45b8a7593c481bdae92a8bdbbcdf3c224
size 1202197632

3
Falcon3-3B-Instruct-Heretic.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a5ec9fed38f6e533ee92ef1cf0ebcb921d2d35fe824451250ff2d7b2afc1887d
size 1119327360

3
Falcon3-3B-Instruct-Heretic.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a1a6f04997e2c01a2714c91f76c3dda7c817a54eedc8e87be87532f51dca3737
size 1047565440

3
Falcon3-3B-Instruct-Heretic.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a94e7bf505e377dbafd0b1805f949fa7435cbdfde5023c0b2d7198e114b55fdf
size 1593693312

3
Falcon3-3B-Instruct-Heretic.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:979314a53794e23ed450b08f8df961abdf319b66324ad64b4b24633e35407edc
size 1549407360

3
Falcon3-3B-Instruct-Heretic.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:38480b2a867dcf2d81a77035b4252a296be08e858f8eea26f01c56aaa4721753
size 1491309696

3
Falcon3-3B-Instruct-Heretic.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:86bcb95d2a8a9f7820b6b4c58f134f90b8876eaf6194cdd1eab51b681f0e1a44
size 1374819456

3
Falcon3-3B-Instruct-Heretic.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:54ef6eaf13799f9058259f0cbc1746b0855cee1017802517423656d55fab404a
size 1924584576

3
Falcon3-3B-Instruct-Heretic.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2a317fca9eb7a755dacd810e31c4868d524bf6f5ea66a86138413ed7ffa4b211
size 1836307584

3
Falcon3-3B-Instruct-Heretic.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ea62a7e8e60e2d3c2122f15540a7282beedc4df28422bf1020c08327e9135348
size 1353880704

3
Falcon3-3B-Instruct-Heretic.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c38f5f7c34b14b79313817234ea52d58fdda84d225f7f5b0f838455b3fd16e7d
size 1275778176

3
Falcon3-3B-Instruct-Heretic.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4dec14838ea793899020ac3aedbdbe54cbea1edced570d6feae76d844b1b9727
size 1781355648

3
Falcon3-3B-Instruct-Heretic.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:06c643e803ac502d739e4525190f1f938596cc1c82bac81799afcae309b5da3e
size 1673221248

3
Falcon3-3B-Instruct-Heretic.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:759d51ea37033546d30864a6048f3e205a91af812cbe0f43f493cf8bbf2d6cb7
size 1549407360

3
Falcon3-3B-Instruct-Heretic.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d28d0a5deec2f8c4529c3d9d77088f0d4f4afdedbd02ec0c263465795ea8abc3
size 1928123520

3
Falcon3-3B-Instruct-Heretic.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f67560c900ce6d3d7471a8442dd729232a1e19d7fc971412aaba314d5c68aa51
size 2101138560

3
Falcon3-3B-Instruct-Heretic.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ad1f2a89e9dac48e2f64aae8280b246ccd19528409a634c1128ab7b7fb8a51bf
size 2005685376

3
Falcon3-3B-Instruct-Heretic.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:20573ffb2de4267f1660979de2e84c8c59cede96236231badfdce547b2303756
size 1933235328

3
Falcon3-3B-Instruct-Heretic.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2476010c84a57be4be5495f9d9bd45aa81fed64fa22e691801682f5668a36d3f
size 2319471744

3
Falcon3-3B-Instruct-Heretic.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8975c298f488881ce70c189ccbb6f9c9517d4a4cfe55e798fada36bc383e4cea
size 2277692544

3
Falcon3-3B-Instruct-Heretic.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0f29f7b97ea2b759ec20bf062b5bb78a94dedd93b7e74de004ca1a751a507138
size 2652869760

3
Falcon3-3B-Instruct-Heretic.imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8d782b840d498dd970ad81be279f49025462528411988e89a33d4146438bf2eb
size 2456768

99
README.md Normal file
View File

@@ -0,0 +1,99 @@
---
base_model: ChiKoi7/Falcon3-3B-Instruct-Heretic
language:
- en
- fr
- es
- pt
library_name: transformers
license: other
license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
license_name: falcon-llm-license
mradermacher:
readme_rev: 1
quantized_by: mradermacher
tags:
- Falcon
- Falcon3
- tiiuae
- 3b
- Instruct
- Heretic
- Uncensored
- Abliterated
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
<!-- ### quants_skip: -->
<!-- ### skip_mmproj: -->
weighted/imatrix quants of https://huggingface.co/ChiKoi7/Falcon3-3B-Instruct-Heretic
<!-- provided-files -->
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Falcon3-3B-Instruct-Heretic-i1-GGUF).***
static quants are available at https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 1.0 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 1.1 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.1 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.2 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ2_S.gguf) | i1-IQ2_S | 1.3 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ2_M.gguf) | i1-IQ2_M | 1.4 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.4 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q2_K.gguf) | i1-Q2_K | 1.5 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.5 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.6 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ3_S.gguf) | i1-IQ3_S | 1.6 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.6 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ3_M.gguf) | i1-IQ3_M | 1.7 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.8 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.9 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.9 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-IQ4_NL.gguf) | i1-IQ4_NL | 2.0 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q4_0.gguf) | i1-Q4_0 | 2.0 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q4_K_S.gguf) | i1-Q4_K_S | 2.0 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.1 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q4_1.gguf) | i1-Q4_1 | 2.2 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.4 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.4 | |
| [GGUF](https://huggingface.co/mradermacher/Falcon3-3B-Instruct-Heretic-i1-GGUF/resolve/main/Falcon3-3B-Instruct-Heretic.i1-Q6_K.gguf) | i1-Q6_K | 2.8 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->