brittlewis12/gemma-3-4b-it-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: brittlewis12/gemma-3-4b-it-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-01 07:12:16 +08:00
commit 0916bb936b
31 changed files with 293 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
.gitattributes vendored Normal file
View File

@@ -0,0 +1,64 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.f16.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.imatrix filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-3-4b-it.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

142
README.md Normal file
View File

@@ -0,0 +1,142 @@
---
base_model: google/gemma-3-4b-it
pipeline_tag: text-generation
inference: true
language:
- en
license: gemma
model_creator: google
model_name: gemma-3-4b-it
model_type: gemma3
quantized_by: brittlewis12
tags:
- reasoning
- gemma
---
# Gemma 3 4B IT GGUF
**Original model**: [Gemma 3 4B IT](https://huggingface.co/google/gemma-3-4b-it)
**Model creator**: [Google DeepMind](https://huggingface.co/google)
> Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.
> Gemma 3 has a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous versions. Gemma 3 models are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
This repo contains GGUF format model files for Google DeepMind’s Gemma 3 4B IT (instruction-tuned).
### What is GGUF?
GGUF is a file format for representing AI models. It is the third version of the format,
introduced by the llama.cpp team on August 21st 2023.
Converted with llama.cpp build b4875 (revision [7841fc7](https://github.com/ggml-org/llama.cpp/commits/7841fc723e059d1fd9640e5c0ef19050fcc7c698)),
using [autogguf-rs](https://github.com/brittlewis12/autogguf-rs).
### Prompt template: [Gemma Instruct](https://huggingface.co/google/gemma-3-4b-it/raw/main/tokenizer_config.json)
```
{{system_prompt}}
<start_of_turn>user
{{prompt}}<end_of_turn>
<start_of_turn>model
```
---
## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!
![cnvrs.ai](https://pbs.twimg.com/profile_images/1744049151241797632/0mIP-P9e_400x400.jpg)
[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
- create & save **Characters** with custom system prompts & temperature settings
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
* or, use an API key with the chat completions-compatible model provider of your choice -- ChatGPT, Claude, Gemini, DeepSeek, & more!
- make it your own with custom **Theme colors**
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggml-org/llama.cpp), with **haptics** during response streaming!
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
* if you **already have the app**, download Gemma 3 4B IT now!
* <cnvrsai:///models/search/hf?id=brittlewis12/gemma-3-4b-it-GGUF>
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date
### Gemma 3 4B IT in cnvrs on macOS
![gemma-3 in cnvrs](https://cdn-uploads.huggingface.co/production/uploads/63b64d7a889aa6707f155cdb/FVVJ1WtuGglQuhCSwc6kh.png)
---
## Original Model Evaluation
> These models were evaluated against a large collection of different datasets and
metrics to cover different aspects of text generation:
#### Reasoning and factuality
| Benchmark | Metric | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
| ------------------------------ |----------------|:--------------:|:-------------:|:--------------:|:--------------:|
| [HellaSwag][hellaswag] | 10-shot | 62.3 | 77.2 | 84.2 | 85.6 |
| [BoolQ][boolq] | 0-shot | 63.2 | 72.3 | 78.8 | 82.4 |
| [PIQA][piqa] | 0-shot | 73.8 | 79.6 | 81.8 | 83.3 |
| [SocialIQA][socialiqa] | 0-shot | 48.9 | 51.9 | 53.4 | 54.9 |
| [TriviaQA][triviaqa] | 5-shot | 39.8 | 65.8 | 78.2 | 85.5 |
| [Natural Questions][naturalq] | 5-shot | 9.48 | 20.0 | 31.4 | 36.1 |
| [ARC-c][arc] | 25-shot | 38.4 | 56.2 | 68.9 | 70.6 |
| [ARC-e][arc] | 0-shot | 73.0 | 82.4 | 88.3 | 89.0 |
| [WinoGrande][winogrande] | 5-shot | 58.2 | 64.7 | 74.3 | 78.8 |
| [BIG-Bench Hard][bbh] | few-shot | 28.4 | 50.9 | 72.6 | 77.7 |
| [DROP][drop] | 1-shot | 42.4 | 60.1 | 72.2 | 77.2 |
[hellaswag]: https://arxiv.org/abs/1905.07830
[boolq]: https://arxiv.org/abs/1905.10044
[piqa]: https://arxiv.org/abs/1911.11641
[socialiqa]: https://arxiv.org/abs/1904.09728
[triviaqa]: https://arxiv.org/abs/1705.03551
[naturalq]: https://github.com/google-research-datasets/natural-questions
[arc]: https://arxiv.org/abs/1911.01547
[winogrande]: https://arxiv.org/abs/1907.10641
[bbh]: https://paperswithcode.com/dataset/bbh
[drop]: https://arxiv.org/abs/1903.00161
#### STEM and code
| Benchmark | Metric | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
| ------------------------------ |----------------|:-------------:|:--------------:|:--------------:|
| [MMLU][mmlu] | 5-shot | 59.6 | 74.5 | 78.6 |
| [MMLU][mmlu] (Pro COT) | 5-shot | 29.2 | 45.3 | 52.2 |
| [AGIEval][agieval] | 3-5-shot | 42.1 | 57.4 | 66.2 |
| [MATH][math] | 4-shot | 24.2 | 43.3 | 50.0 |
| [GSM8K][gsm8k] | 8-shot | 38.4 | 71.0 | 82.6 |
| [GPQA][gpqa] | 5-shot | 15.0 | 25.4 | 24.3 |
| [MBPP][mbpp] | 3-shot | 46.0 | 60.4 | 65.6 |
| [HumanEval][humaneval] | 0-shot | 36.0 | 45.7 | 48.8 |
[mmlu]: https://arxiv.org/abs/2009.03300
[agieval]: https://arxiv.org/abs/2304.06364
[math]: https://arxiv.org/abs/2103.03874
[gsm8k]: https://arxiv.org/abs/2110.14168
[gpqa]: https://arxiv.org/abs/2311.12022
[mbpp]: https://arxiv.org/abs/2108.07732
[humaneval]: https://arxiv.org/abs/2107.03374
#### Multilingual
| Benchmark | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
| ------------------------------------ |:-------------:|:-------------:|:--------------:|:--------------:|
| [MGSM][mgsm] | 2.04 | 34.7 | 64.3 | 74.3 |
| [Global-MMLU-Lite][global-mmlu-lite] | 24.9 | 57.0 | 69.4 | 75.7 |
| [WMT24++][wmt24pp] (ChrF) | 36.7 | 48.4 | 53.9 | 55.7 |
| [FloRes][flores] | 29.5 | 39.2 | 46.0 | 48.8 |
| [XQuAD][xquad] (all) | 43.9 | 68.0 | 74.5 | 76.8 |
| [ECLeKTic][eclektic] | 4.69 | 11.0 | 17.2 | 24.4 |
| [IndicGenBench][indicgenbench] | 41.4 | 57.2 | 61.7 | 63.4 |
[mgsm]: https://arxiv.org/abs/2210.03057
[flores]: https://arxiv.org/abs/2106.03193
[xquad]: https://arxiv.org/abs/1910.11856v3
[global-mmlu-lite]: https://huggingface.co/datasets/CohereForAI/Global-MMLU-Lite
[wmt24pp]: https://arxiv.org/abs/2502.12404v1
[eclektic]: https://arxiv.org/abs/2502.21228
[indicgenbench]: https://arxiv.org/abs/2404.16816

3
gemma-3-4b-it.IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7f66f89baad03fef7df7515c92e09e618b73399322df9153eacfea27b66abc53
size 1199456992

3
gemma-3-4b-it.IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:43949e9a94c3ca11008cb45c45532d0d93c9a01d58df79a58f54f91bafddbe5f
size 1132978912

3
gemma-3-4b-it.IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f5d664954a7019b1923cd34c21072bc888fd342ba002fada000751599b3aaf22
size 1537868512

3
gemma-3-4b-it.IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1862c4db8a8f376054047c12cb9428c71d0f6484ecf36c796ec50e0841bb5e88
size 1449231072

3
gemma-3-4b-it.IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f1082490768ee10a5bb96ef1b076ef36b961473a670d4da63ba2094f431e9467
size 1404461792

3
gemma-3-4b-it.IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:80ffc72cee4c43ff32b68694686a7040e1f0cb62107e7ec99e3fdbcdb6f79c7a
size 1310253792

3
gemma-3-4b-it.IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4a3e5f0d42aecb73f51d981b9ed5cfb1ff395cb0af8b6721797e198675634a6e
size 1986667232

3
gemma-3-4b-it.IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0fb46c74c0bc0f0b66bc72d1c9c03a37813c1995e54d0c1dedd37a067e54fbf2
size 1937228512

3
gemma-3-4b-it.IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24d3cc2dcf2b14a050e56d5acf1375e923e0c33b0db099c1f848b3a68b6efcab
size 1863254752

3
gemma-3-4b-it.IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cfd977982c98537c956912f00b254f91e94e88dd04eacc446837615a5ed864a3
size 1689338592

3
gemma-3-4b-it.IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7388042f0ff0994a92ec377914d1b4a51aa462651da004407136f9a0b5f9d9f2
size 2363376352

3
gemma-3-4b-it.IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c3a00d0e6b6188f4c5038a823a702704e9a6dffee2dcf374c9b6c312e7da3266
size 2263106272

3
gemma-3-4b-it.Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:636625f281c24e1923b3086a5d37deb236f6aa7ccab1559764da65e4ef192ec6
size 1729028576

3
gemma-3-4b-it.Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f2f13b831a681f12b9599a8494a401f8d98ffcdd3bbdf0834abce69d9b3fdf23
size 1635926752

3
gemma-3-4b-it.Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:523c5e40f95aab7715c20e065fb8a7bd1fdd53887ac7b1ca0c7d1f512272a168
size 2235949536

3
gemma-3-4b-it.Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4c45f195ac521663934a4a39bdfd92813528705b1a834d024b310245aa4648e1
size 2098323936

3
gemma-3-4b-it.Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cd5ae453883a045b547bcec489b9332e030db3b04ffed96efeb75c5ef32f5364
size 1937228256

3
gemma-3-4b-it.Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:45ed6f9a5066761b6af6445678c4de2cb3ddc7337bec1a0efd2515f1e2062d4f
size 2363376096

3
gemma-3-4b-it.Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:60f4b4993876fac4f3832a3eda043d62c57b9d3e2300d804fbf779e6021afa99
size 2563916256

3
gemma-3-4b-it.Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f7842763eac86adc602b844bfa27e3de07affb95463ca20c1065ed02f4ad1d7f
size 2489758176

3
gemma-3-4b-it.Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8398540e6d94430f78903d8ef0537fde4c97549c797f899c339e8fd4069f4500
size 2377794016

3
gemma-3-4b-it.Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ca787e35be37240882f51f57273557fa729000c58ced930db80df678f74582a5
size 2764456416

3
gemma-3-4b-it.Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:54c710adbe185fd0a8cbcaad44194b791446ad59cfbcc2b063787eb731201dc3
size 2964996576

3
gemma-3-4b-it.Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:89010439644a176dfa9030eed65e87983df07fafdff579db6493831d8658aff4
size 2829562336

3
gemma-3-4b-it.Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5cbfbc99842fac4ae907208bb538e1db8ef71d80b91787a9fa2dbe32bb46891b
size 2764456416

3
gemma-3-4b-it.Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8d8224d8238cebac5301b84ee20a299b6b91c41a700e4b2907a241b86af69989
size 3190604256

3
gemma-3-4b-it.Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:90ec2881d6edcf7958fa6d37bb63b3f87628ee939f7dda66814e4f522e060bd0
size 4130226656

3
gemma-3-4b-it.f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f6b1b63141586381a6892e9efc11e3da0b6d6cc344b2d798bc9429a12c303321
size 7767474656

3
gemma-3-4b-it.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:433d5c1e0bcbd4831857afb4ebe3d02aa08f945a4590c4c220d08c6f95acfd9c
size 3419852