SanctumAI/gemma-2-9b-it-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: SanctumAI/gemma-2-9b-it-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-13 05:52:09 +08:00
commit 86b45c11cf
20 changed files with 164 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
.gitattributes vendored Normal file
View File

@@ -0,0 +1,52 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.f16.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q4_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q5_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-9b-it.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

58
README.md Normal file
View File

@@ -0,0 +1,58 @@
---
license: gemma
pipeline_tag: text-generation
tags:
- conversational
base_model:
- google/gemma-2-9b-it
---
![image/png](https://cdn-uploads.huggingface.co/production/uploads/64a28db2f1968b7d7f357182/u7s05Lc_DIwrI3mY_Hzwn.png)
*This model was quantized by [SanctumAI](https://sanctum.ai). To leave feedback, join our community in [Discord](https://discord.gg/7ZNE78HJKh).*
# Gemma 2 9B IT GGUF
**Model creator:** [google](https://huggingface.co/google)<br>
**Original model**: [gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it)<br>
## Model Summary:
Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights for both pre-trained variants and instruction-tuned variants. Gemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as a laptop, desktop or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
## Prompt Template:
If you're using Sanctum app, simply use `Gemma` model preset.
Prompt template:
```
<start_of_turn>user
{prompt}<end_of_turn>
<|start_of_turn|>model
```
## Hardware Requirements Estimate
| Name | Quant method | Size | Memory (RAM, vRAM) required |
| ---- | ---- | ---- | ---- |
| [gemma-2-9b-it.Q2_K.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q2_K.gguf) | Q2_K | 3.81 GB | 15.14 GB |
| [gemma-2-9b-it.Q3_K_S.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q3_K_S.gguf) | Q3_K_S | 4.34 GB | 15.64 GB |
| [gemma-2-9b-it.Q3_K_M.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q3_K_M.gguf) | Q3_K_M | 4.76 GB | 16.04 GB |
| [gemma-2-9b-it.Q3_K_L.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q3_K_L.gguf) | Q3_K_L | 5.13 GB | 16.38 GB |
| [gemma-2-9b-it.Q4_0.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q4_0.gguf) | Q4_0 | 5.44 GB | 16.67 GB |
| [gemma-2-9b-it.Q4_K_S.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q4_K_S.gguf) | Q4_K_S | 5.48 GB | 16.70 GB |
| [gemma-2-9b-it.Q4_K_M.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q4_K_M.gguf) | Q4_K_M | 5.76 GB | 16.97 GB |
| [gemma-2-9b-it.Q4_K.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q4_K.gguf) | Q4_K | 5.76 GB | 16.97 GB |
| [gemma-2-9b-it.Q4_1.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q4_1.gguf) | Q4_1 | 5.96 GB | 17.15 GB |
| [gemma-2-9b-it.Q5_0.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q5_0.gguf) | Q5_0 | 6.48 GB | 17.64 GB |
| [gemma-2-9b-it.Q5_K_S.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q5_K_S.gguf) | Q5_K_S | 6.48 GB | 17.64 GB |
| [gemma-2-9b-it.Q5_K_M.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q5_K_M.gguf) | Q5_K_M | 6.65 GB | 17.79 GB |
| [gemma-2-9b-it.Q5_K.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q5_K.gguf) | Q5_K | 6.65 GB | 17.79 GB |
| [gemma-2-9b-it.Q5_1.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q5_1.gguf) | Q5_1 | 7.00 GB | 18.12 GB |
| [gemma-2-9b-it.Q6_K.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q6_K.gguf) | Q6_K | 7.59 GB | 18.67 GB |
| [gemma-2-9b-it.Q8_0.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.Q8_0.gguf) | Q8_0 | 9.83 GB | 20.75 GB |
| [gemma-2-9b-it.f16.gguf](https://huggingface.co/SanctumAI/gemma-2-9b-it-GGUF/blob/main/gemma-2-9b-it.f16.gguf) | f16 | 18.49 GB | 28.82 GB |
## Disclaimer
Sanctum is not the creator, originator, or owner of any Model featured in the Models section of the Sanctum application. Each Model is created and provided by third parties. Sanctum does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Model listed there. You understand that supported Models can produce content that might be offensive, harmful, inaccurate or otherwise inappropriate, or deceptive. Each Model is the sole responsibility of the person or entity who originated such Model. Sanctum may not monitor or control the Models supported and cannot, and does not, take responsibility for any such Model. Sanctum disclaims all warranties or guarantees about the accuracy, reliability or benefits of the Models. Sanctum further disclaims any warranty that the Model will meet your requirements, be secure, uninterrupted or available at any time or location, or error-free, viruses-free, or that any errors will be corrected, or otherwise. You will be solely responsible for any damage resulting from your use of or access to the Models, your downloading of any Model, or use of any other Model provided by or through Sanctum.

3
config.json Normal file
View File

@@ -0,0 +1,3 @@
{
"model_type": "gemma2"
}

3
gemma-2-9b-it.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e6dc51397e0d2ff87cdd3b759007c4f9dbbc9067bfa112a4aa2be6bfcc078f40
size 3805397696

3
gemma-2-9b-it.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9b8ad7f02b29304530dd7067e694f86ad554c72530029910cfe114229bf5dee2
size 5132452544

3
gemma-2-9b-it.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:134d1caf7a05311acde0f7984ddeb6c1386a2ad0d3977dae3750e49fbbe0c964
size 4761780928

3
gemma-2-9b-it.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:389a070a115c2ce1b19a61840372f3fb2a1610b7b8fd714426c6331657f66fd0
size 4337664704

3
gemma-2-9b-it.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5cd8c8a2ecb850d39c80f52e7d45f495e8479115a886741bd06fbfcaf9f73d39
size 5443142336

3
gemma-2-9b-it.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ab6b6042f0500b4bf07e95a4010b58b9c74335c75df74d330f85d165e5e0a627
size 5963367104

3
gemma-2-9b-it.Q4_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd64cf29d76294581acfad6dad9eff932bad1b8de7256e4d9d66569b0ef6748b
size 5761057472

3
gemma-2-9b-it.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fd64cf29d76294581acfad6dad9eff932bad1b8de7256e4d9d66569b0ef6748b
size 5761057472

3
gemma-2-9b-it.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ff3d6749f0bfc4b954ddedbe49923aefecc4198d7b2cb45dbdf0be809cd2c34d
size 5478924992

3
gemma-2-9b-it.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1f1c09d68353afb9fdd8a2d53df9c2b43642f6ebc1ba6fab22eec4d3c6c142d7
size 6483591872

3
gemma-2-9b-it.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5d8cbcdd25e5a60aaf1bffd0837bf999f74ce26cca518621cc570485d16cedee
size 7003816640

3
gemma-2-9b-it.Q5_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:252b05b32ee6169f46b0e938f64b7b65f269bedc371ffb897ad5cbb2262e1de7
size 6647366336

3
gemma-2-9b-it.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:252b05b32ee6169f46b0e938f64b7b65f269bedc371ffb897ad5cbb2262e1de7
size 6647366336

3
gemma-2-9b-it.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:27c4aa1577e805a35adfd2cd1701252a338cd1e829e746ab9903936bbfa8ea6c
size 6483591872

3
gemma-2-9b-it.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2ae6dfd13a48ef717b6bd6d8030ca9ee8066ebbd2ef0ca690d39234d3caeebc7
size 7589069504

3
gemma-2-9b-it.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:399ac085a1814916bf309e7b622e5a91aadaee839a77c26dcc534d17c7af429a
size 9827148480

3
gemma-2-9b-it.f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b91fd4a448d3c7080795a0adb11c832e68abc6e295fa2eb5b21d97b54399fffe
size 18490680000