SanctumAI/Llama-3.2-3B-Instruct-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: SanctumAI/Llama-3.2-3B-Instruct-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-26 07:25:16 +08:00
commit 3d30051b2e
20 changed files with 180 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
.gitattributes vendored Normal file
View File

@@ -0,0 +1,52 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.f16.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
llama-3.2-3b-instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

74
README.md Normal file
View File

@@ -0,0 +1,74 @@
---
language:
- en
- de
- fr
- it
- pt
- hi
- es
- th
pipeline_tag: text-generation
tags:
- facebook
- meta
- pytorch
- llama
- llama-3
license: llama3.2
base_model:
- meta-llama/Llama-3.2-3B-Instruct
---
![image/png](https://cdn-uploads.huggingface.co/production/uploads/64a28db2f1968b7d7f357182/MWbE_500bIfv3diImTTeP.png)
This model was quantized by [SanctumAI](https://sanctum.ai). To leave feedback, join our community in [Discord](https://discord.gg/7ZNE78HJKh).*
# Llama 3.2 3B Instruct GGUF
**Model creator:** [meta-llama](https://huggingface.co/meta-llama)<br>
**Original model**: [Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)<br>
## Model Summary:
The Meta Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks.
## Prompt Template:
If you're using Sanctum app, simply use `Llama 3` model preset.
Prompt template:
```
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
```
## Hardware Requirements Estimate
| Name | Quant method | Size | Memory (RAM, vRAM) required |
| ---- | ---- | ---- | ---- |
| [llama-3.2-3b-instruct.Q2_K.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q2_K.gguf) | Q2_K | 1.36 GB | 4.66 GB |
| [llama-3.2-3b-instruct.Q3_K_S.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q3_K_S.gguf) | Q3_K_S | 1.54 GB | 4.83 GB |
| [llama-3.2-3b-instruct.Q3_K_M.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q3_K_M.gguf) | Q3_K_M | 1.69 GB | 4.96 GB |
| [llama-3.2-3b-instruct.Q3_K_L.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q3_K_L.gguf) | Q3_K_L | 1.82 GB | 5.08 GB |
| [llama-3.2-3b-instruct.Q4_0.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q4_0.gguf) | Q4_0 | 1.92 GB | 5.17 GB |
| [llama-3.2-3b-instruct.Q4_K_S.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q4_K_S.gguf) | Q4_K_S | 1.93 GB | 5.18 GB |
| [llama-3.2-3b-instruct.Q4_K_M.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q4_K_M.gguf) | Q4_K_M | 2.02 GB | 5.27 GB |
| [llama-3.2-3b-instruct.Q4_K.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q4_K.gguf) | Q4_K | 2.02 GB | 5.27 GB |
| [llama-3.2-3b-instruct.Q4_1.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q4_1.gguf) | Q4_1 | 2.09 GB | 5.34 GB |
| [llama-3.2-3b-instruct.Q5_0.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q5_0.gguf) | Q5_0 | 2.27 GB | 5.50 GB |
| [llama-3.2-3b-instruct.Q5_K_S.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q5_K_S.gguf) | Q5_K_S | 2.27 GB | 5.50 GB |
| [llama-3.2-3b-instruct.Q5_K_M.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q5_K_M.gguf) | Q5_K_M | 2.32 GB | 5.55 GB |
| [llama-3.2-3b-instruct.Q5_K.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q5_K.gguf) | Q5_K | 2.32 GB | 5.55 GB |
| [llama-3.2-3b-instruct.Q5_1.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q5_1.gguf) | Q5_1 | 2.45 GB | 5.67 GB |
| [llama-3.2-3b-instruct.Q6_K.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q6_K.gguf) | Q6_K | 2.64 GB | 5.85 GB |
| [llama-3.2-3b-instruct.Q8_0.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.Q8_0.gguf) | Q8_0 | 3.42 GB | 6.58 GB |
| [llama-3.2-3b-instruct.f16.gguf](https://huggingface.co/SanctumAI/Llama-3.2-3B-Instruct-GGUF/blob/main/llama-3.2-3b-instruct.f16.gguf) | f16 | 6.43 GB | 9.38 GB |
## Disclaimer
Sanctum is not the creator, originator, or owner of any Model featured in the Models section of the Sanctum application. Each Model is created and provided by third parties. Sanctum does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Model listed there. You understand that supported Models can produce content that might be offensive, harmful, inaccurate or otherwise inappropriate, or deceptive. Each Model is the sole responsibility of the person or entity who originated such Model. Sanctum may not monitor or control the Models supported and cannot, and does not, take responsibility for any such Model. Sanctum disclaims all warranties or guarantees about the accuracy, reliability or benefits of the Models. Sanctum further disclaims any warranty that the Model will meet your requirements, be secure, uninterrupted or available at any time or location, or error-free, viruses-free, or that any errors will be corrected, or otherwise. You will be solely responsible for any damage resulting from your use of or access to the Models, your downloading of any Model, or use of any other Model provided by or through Sanctum.

3
config.json Normal file
View File

@@ -0,0 +1,3 @@
{
"model_type": "llama"
}

3
llama-3.2-3b-instruct.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c77eb142ab869944f388ff093fc7276ea15c4e1f810ceb76554fc5ae77694c19
size 1363935520

3
llama-3.2-3b-instruct.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5cf752a117ddad28f7135920de4e79b3731ed3b95a733e012894e0b2cb31f6c8
size 1815347488

3
llama-3.2-3b-instruct.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:be3f386dee4d246d41714cabe7fcb80c9173613ffc344065f360f7cfe6769fe4
size 1687159072

3
llama-3.2-3b-instruct.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:30a4ec8bf86ab91b7ffa95f1ab3e6a21cfa240122b6836b23a75baf894970d62
size 1542848800

3
llama-3.2-3b-instruct.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:506d311f2f8802991344f7186badffda9c6a6b4cb50aa7a759ba3d939544df44
size 1917190432

3
llama-3.2-3b-instruct.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e35c4d60dc538e7fc5ddf6abce3eebe581d1c702c7629e05ef3a0a26c022324c
size 2093351200

3
llama-3.2-3b-instruct.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1a9c139c80a85740cd98fa8d0b6a33445c49dc77f9ee37c914663d25bdaf2f4e
size 2019377440

3
llama-3.2-3b-instruct.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1a9c139c80a85740cd98fa8d0b6a33445c49dc77f9ee37c914663d25bdaf2f4e
size 2019377440

3
llama-3.2-3b-instruct.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:133d6f25469c0889a3b207f340352a47772fa2de006dfa0f3db0f19af115acd5
size 1928200480

3
llama-3.2-3b-instruct.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c662fa7df5e816a362cd1dd06b36b8125ef163482b39a700487aae5c8b9ac93b
size 2269511968

3
llama-3.2-3b-instruct.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b0f876bb12c3e8ab8f4a9a4e89345ac64b6e3ca0c738102ca9e2a0d9bbdecf00
size 2445672736

3
llama-3.2-3b-instruct.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:782f6d3246312ebfc5b3db83f2beaf5abf4371311206e9dc2c57703b564f55a0
size 2322153760

3
llama-3.2-3b-instruct.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:782f6d3246312ebfc5b3db83f2beaf5abf4371311206e9dc2c57703b564f55a0
size 2322153760

3
llama-3.2-3b-instruct.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a6dd5a79cd798f9d228ae7a2b6b471025a231cbc6fc1fccd5f487bf71de29a34
size 2269511968

3
llama-3.2-3b-instruct.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fcd738601e25f128a3388138af357723bc2006f0657c94fe3374a55d919d9ddb
size 2643853600

3
llama-3.2-3b-instruct.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:94f22b7231df5cd1907ff48dba54497b2d7912a4ce60d914f3dcfc0347fa8f21
size 3421899040

3
llama-3.2-3b-instruct.f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b5ebe8fbb5ab5f05aaa056948668c5f7443df9d0883497f1d9c05f57fb8ea6cd
size 6433687840