mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-04 17:09:11 +08:00
commit 7b1c841450
28 changed files with 235 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

61
.gitattributes vendored Normal file
View File

@@ -0,0 +1,61 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
FastLlama-3.2-1B-Instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
FastLlama-3.2-1B-Instruct.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:40f6d43788d6a317b6707e6da1f3a8ef734c2a037d9148846a4be2d4d259d29d
size 413607040

3
FastLlama-3.2-1B-Instruct.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95012c75ba8bb677ad2909e1620605f9a4627726dd795590cfa0aa3208ffbc6f
size 393553024

3
FastLlama-3.2-1B-Instruct.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:84d168a7dc47b4bc5507118e7c1335c22755f1da87854f648a266387b39d91e5
size 515449984

3
FastLlama-3.2-1B-Instruct.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4d796a7df003522dd67ab922095531ec4908935c75879122344a23779f4bb78d
size 488711296

3
FastLlama-3.2-1B-Instruct.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:23aeeffd4c0acd156f928014fca4f8cd2104135d8f69b46f188846dba13072fe
size 475866240

3
FastLlama-3.2-1B-Instruct.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d8cd75d79293939e695f9bd2ca5afabe35f99edffc622d9a4ae31587cc2e3da5
size 447030400

3
FastLlama-3.2-1B-Instruct.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:91e4d4f917cf55e370da32256b1f9ce2af40bc4e4634337b7788761c0a447585
size 657290368

3
FastLlama-3.2-1B-Instruct.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a078d760b81286ab29984e3d9d9da42f3a2b3e1053e311337cf7613b61ffe511
size 643921024

3
FastLlama-3.2-1B-Instruct.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7b5fc2e2f1b0180b2d9099fba09efb0b85e20acba16dbb1cfa7f7a61049ccd9c
size 621114496

3
FastLlama-3.2-1B-Instruct.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e8c0c5b191706cd5ff8d25a95fc6308fbc4b83a7b36f282b098ee9f590a52133
size 562111616

3
FastLlama-3.2-1B-Instruct.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:39b0fb28c9ff437bac8962999117c3935a4d6592d46a581a7a68198ca9a75fbb
size 743142528

3
FastLlama-3.2-1B-Instruct.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5f89196e677c16d26c240bb5f51c2788b615da8de68d58649f01fd7398846d9d
size 580875392

3
FastLlama-3.2-1B-Instruct.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c646c3fabe869978df5cbb2a0c750acd7f5d451a515de58efca4b113f1277f7b
size 554660992

3
FastLlama-3.2-1B-Instruct.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4845de2b93a7c2e975a5abe5bfde35baad8dccb9e640fd06591fc0cace2f65c5
size 732525696

3
FastLlama-3.2-1B-Instruct.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a81d0a0d520fbc3e37954d019e5734152272cfadbe01be32dd378eba7b787459
size 690844800

3
FastLlama-3.2-1B-Instruct.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ce5da64382615d52ec41d44eaec037e3b064e1544976d1ac1025e1f4a1a14e8
size 641692800

3
FastLlama-3.2-1B-Instruct.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:484df12471e1ab76f22a60d6efc9603d40ebedb1b2d197d610bb4545244c7345
size 773026944

3
FastLlama-3.2-1B-Instruct.i1-Q4_0_4_4.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7c88c87e6381622df8ac324b2b1d1bc62a76b028eaf6191f126ad6056cb7913d
size 770929792

3
FastLlama-3.2-1B-Instruct.i1-Q4_0_4_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:966f964a34c714e727bf70668b77f9a175191eff504fee1965c9d9299f4e2ed9
size 770929792

3
FastLlama-3.2-1B-Instruct.i1-Q4_0_8_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d970a3f76d4da5125ca9b239f619238f6bcb87232a38f23b4836d9f998da4257
size 770929792

3
FastLlama-3.2-1B-Instruct.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4ab8a0687ec8e01277b07f3697170f485b45101901c59aa1aba331db6ab86912
size 807695488

3
FastLlama-3.2-1B-Instruct.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6ec878859658151fd6b00695cee8f32354532619014949e8b5bf755d42d134a1
size 775648384

3
FastLlama-3.2-1B-Instruct.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:168a03443701f2f82196e3e07e0f3fb516111da431cf8c84f8e88833ba00e47b
size 911504512

3
FastLlama-3.2-1B-Instruct.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6969391806ab42bf6347ab01ecf500d2fa213127ce33fe198243ec0fa1e0103a
size 892564608

3
FastLlama-3.2-1B-Instruct.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1457995b3ed73a6d73002615563a8e1202770a740f495db20f7f10d3018d50d0
size 1021801600

96
README.md Normal file
View File

@@ -0,0 +1,96 @@
---
base_model: suayptalha/FastLlama-3.2-1B-Instruct
datasets:
- HuggingFaceTB/smoltalk
language:
- en
- de
- es
- fr
- it
- pt
- hi
- th
library_name: transformers
license: apache-2.0
quantized_by: mradermacher
tags:
- math
- lora
- science
- chemistry
- biology
- code
- text-generation-inference
- unsloth
- llama
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/suayptalha/FastLlama-3.2-1B-Instruct
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ1_S.gguf) | i1-IQ1_S | 0.5 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ1_M.gguf) | i1-IQ1_M | 0.5 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.5 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.6 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ2_S.gguf) | i1-IQ2_S | 0.6 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ2_M.gguf) | i1-IQ2_M | 0.6 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q2_K_S.gguf) | i1-Q2_K_S | 0.7 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.7 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q2_K.gguf) | i1-Q2_K | 0.7 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.7 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.7 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ3_S.gguf) | i1-IQ3_S | 0.7 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 0.8 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.8 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q3_K_L.gguf) | i1-Q3_K_L | 0.8 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-IQ4_XS.gguf) | i1-IQ4_XS | 0.8 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 0.9 | fast on arm, low quality |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 0.9 | fast on arm+i8mm, low quality |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 0.9 | fast on arm+sve, low quality |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q4_0.gguf) | i1-Q4_0 | 0.9 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q4_K_S.gguf) | i1-Q4_K_S | 0.9 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q4_K_M.gguf) | i1-Q4_K_M | 0.9 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q5_K_S.gguf) | i1-Q5_K_S | 1.0 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q5_K_M.gguf) | i1-Q5_K_M | 1.0 | |
| [GGUF](https://huggingface.co/mradermacher/FastLlama-3.2-1B-Instruct-i1-GGUF/resolve/main/FastLlama-3.2-1B-Instruct.i1-Q6_K.gguf) | i1-Q6_K | 1.1 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c868620ab6f13e92a28397ba40efd519bab59d807109b3a9477486d0c50dfb7a
size 1314413