bartowski/gemma-2-2b-jpn-it-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/gemma-2-2b-jpn-it-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-30 15:01:13 +08:00
commit aa8a9a52c2
22 changed files with 242 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
.gitattributes vendored Normal file
View File

@@ -0,0 +1,54 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it-f16.gguf filter=lfs diff=lfs merge=lfs -text
gemma-2-2b-jpn-it.imatrix filter=lfs diff=lfs merge=lfs -text

130
README.md Normal file
View File

@@ -0,0 +1,130 @@
---
base_model: google/gemma-2-2b-jpn-it
language:
- ja
license: gemma
pipeline_tag: text-generation
tags:
- conversational
quantized_by: bartowski
extra_gated_heading: Access Gemma on Hugging Face
extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and
agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging
Face and click below. Requests are processed immediately.
extra_gated_button_content: Acknowledge license
---
## Llamacpp imatrix Quantizations of gemma-2-2b-jpn-it
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3972">b3972</a> for quantization.
Original model: https://huggingface.co/google/gemma-2-2b-jpn-it
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
## Prompt format
```
<bos><start_of_turn>user
{prompt}<end_of_turn>
<start_of_turn>model
<end_of_turn>
<start_of_turn>model
```
Note that this model does not support a System prompt.
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [gemma-2-2b-jpn-it-f16.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-f16.gguf) | f16 | 5.24GB | false | Full F16 weights. |
| [gemma-2-2b-jpn-it-Q8_0.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q8_0.gguf) | Q8_0 | 2.78GB | false | Extremely high quality, generally unneeded but max available quant. |
| [gemma-2-2b-jpn-it-Q6_K_L.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q6_K_L.gguf) | Q6_K_L | 2.29GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [gemma-2-2b-jpn-it-Q6_K.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q6_K.gguf) | Q6_K | 2.15GB | false | Very high quality, near perfect, *recommended*. |
| [gemma-2-2b-jpn-it-Q5_K_L.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q5_K_L.gguf) | Q5_K_L | 2.07GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [gemma-2-2b-jpn-it-Q5_K_M.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q5_K_M.gguf) | Q5_K_M | 1.92GB | false | High quality, *recommended*. |
| [gemma-2-2b-jpn-it-Q5_K_S.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q5_K_S.gguf) | Q5_K_S | 1.88GB | false | High quality, *recommended*. |
| [gemma-2-2b-jpn-it-Q4_K_L.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q4_K_L.gguf) | Q4_K_L | 1.85GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [gemma-2-2b-jpn-it-Q4_K_M.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q4_K_M.gguf) | Q4_K_M | 1.71GB | false | Good quality, default size for must use cases, *recommended*. |
| [gemma-2-2b-jpn-it-Q3_K_XL.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q3_K_XL.gguf) | Q3_K_XL | 1.69GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [gemma-2-2b-jpn-it-Q4_K_S.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q4_K_S.gguf) | Q4_K_S | 1.64GB | false | Slightly lower quality with more space savings, *recommended*. |
| [gemma-2-2b-jpn-it-Q4_0_8_8.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q4_0_8_8.gguf) | Q4_0_8_8 | 1.63GB | false | Optimized for ARM inference. Requires 'sve' support (see link below). *Don't use on Mac or Windows*. |
| [gemma-2-2b-jpn-it-Q4_0_4_8.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q4_0_4_8.gguf) | Q4_0_4_8 | 1.63GB | false | Optimized for ARM inference. Requires 'i8mm' support (see link below). *Don't use on Mac or Windows*. |
| [gemma-2-2b-jpn-it-Q4_0_4_4.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q4_0_4_4.gguf) | Q4_0_4_4 | 1.63GB | false | Optimized for ARM inference. Should work well on all ARM chips, pick this if you're unsure. *Don't use on Mac or Windows*. |
| [gemma-2-2b-jpn-it-Q4_0.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q4_0.gguf) | Q4_0 | 1.63GB | false | Legacy format, generally not worth using over similarly sized formats |
| [gemma-2-2b-jpn-it-IQ4_XS.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-IQ4_XS.gguf) | IQ4_XS | 1.57GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [gemma-2-2b-jpn-it-Q3_K_L.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-Q3_K_L.gguf) | Q3_K_L | 1.55GB | false | Lower quality but usable, good for low RAM availability. |
| [gemma-2-2b-jpn-it-IQ3_M.gguf](https://huggingface.co/bartowski/gemma-2-2b-jpn-it-GGUF/blob/main/gemma-2-2b-jpn-it-IQ3_M.gguf) | IQ3_M | 1.39GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
Some say that this improves the quality, others don't notice any difference. If you use these models PLEASE COMMENT with your findings. I would like feedback that these are actually used and useful so I don't keep uploading quants no one is using.
Thanks!
## Downloading using huggingface-cli
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/gemma-2-2b-jpn-it-GGUF --include "gemma-2-2b-jpn-it-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/gemma-2-2b-jpn-it-GGUF --include "gemma-2-2b-jpn-it-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (gemma-2-2b-jpn-it-Q8_0) or download them all in place (./)
## Q4_0_X_X
These are *NOT* for Metal (Apple) offloading, only ARM chips.
If you're using an ARM chip, the Q4_0_X_X quants will have a substantial speedup. Check out Q4_0_4_4 speed comparisons [on the original pull request](https://github.com/ggerganov/llama.cpp/pull/5780#pullrequestreview-21657544660)
To check which one would work best for your ARM chip, you can check [AArch64 SoC features](https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html) (thanks EloyOn!).
## Which file should I choose?
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset
Thank you ZeroWw for the inspiration to experiment with embed/output
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}

3
gemma-2-2b-jpn-it-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:02764d309cbd4b1240a267bc30bd007474a905b91405311f1db5dce575cab8ca
size 1393561760

3
gemma-2-2b-jpn-it-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dcc5f4b10bd2fcc332ae1a7ea96d97743acda9268119caf795c3f771b2a5f989
size 1566251168

3
gemma-2-2b-jpn-it-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:456261172d864c96b5f98c4ca03bc674a9451aa8d1ac5b0b7facc3791c96fdf0
size 1550436512

3
gemma-2-2b-jpn-it-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:54fb669ed74ec2e26275425a2ba169b805d4f3c2880592ac2844a9951b31a036
size 1693284512

3
gemma-2-2b-jpn-it-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9830235d0c0a22527127656ec0963355906d1a545e0962d4e6f7d6b99987b8b6
size 1633491104

3
gemma-2-2b-jpn-it-Q4_0_4_4.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dc01a5b471bcee7a0b01a937f82b7f7cbbaf1109e79ecd4c2486518adc697e96
size 1629509792

3
gemma-2-2b-jpn-it-Q4_0_4_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a493be92d70252fb03cdf462f641c60390de63d9b7e624ef7fbb8028409ebcd1
size 1629509792

3
gemma-2-2b-jpn-it-Q4_0_8_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:80d559663a820be15301e80a7ae4dadc05f543ffa74942725f31c70cc65d523e
size 1629509792

3
gemma-2-2b-jpn-it-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ac900bb315d9e273cb28ea1aa8147dd67afc8e3e16fec5b5882f54df23065347
size 1851431072

3
gemma-2-2b-jpn-it-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6da585d534fc1b02e6eedeb844c77aecbea6a45df8a1ef1ba3624d62b09b7af3
size 1708583072

3
gemma-2-2b-jpn-it-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b95cd52b93f8c23713d4b6fdd7f54ed47fa154250f397cce61325a50a282d553
size 1638652064

3
gemma-2-2b-jpn-it-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d3183c4aab74a6bfaff6a9f1a3fa6a004379954d8358e2d33a217e0af255b95f
size 2066127008

3
gemma-2-2b-jpn-it-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a2e5ebe3c946bb8cc40c7ebe90aa5ce7521d4ea947eaa94444bdb2925ea84ee9
size 1923279008

3
gemma-2-2b-jpn-it-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5b91de2378848ab88a0398c90c97807b4de8fc89592e026fd5829ab56d6f9ae2
size 1882544288

3
gemma-2-2b-jpn-it-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:225f8e649dead036e5812bed7e3902d38651a4f969976728a12f340afb149cdc
size 2151393440

3
gemma-2-2b-jpn-it-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3eca542342139e2d3c364bbd0d620b333c2b1aa6ad4370dbf95ca0aee31c20aa
size 2294241440

3
gemma-2-2b-jpn-it-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bdfd5bc05e9c959e7ac462c98fab289b4c546cddd41c423a951852dc7803b805
size 2784495776

3
gemma-2-2b-jpn-it-f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:339d52117386ea09b43d2022cbdf23c9dfb295b0db17550ac446107a7b5cdb5c
size 5235214240

3
gemma-2-2b-jpn-it.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:54df3513de999f910ebad9cbac4699895aa9f27bb547df41074dfc16fea42186
size 2375572