初始化项目,由ModelHub XC社区提供模型

Model: bartowski/G2-9B-Blackout-R1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-04 03:05:12 +08:00
commit 363e1493f2
28 changed files with 258 additions and 0 deletions

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1-f16.gguf filter=lfs diff=lfs merge=lfs -text
G2-9B-Blackout-R1.imatrix filter=lfs diff=lfs merge=lfs -text

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:99ecae80dd6f30c7ab224a3956c6f4f2b0dafff552a02e52dd71edd9a98bb409
size 3434669440

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24c642f5f78bb6189da4b44216c878d92993e3fd38116f62e384db08ef076c92
size 4494615936

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7990910f97297ed8dea6d27788c3e3e16ac0131cb2cec54fb7859418cd196e0d
size 4144989568

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:89d96aa515a1445f3c07aa6e3688f5de2710af58217c372b3beb351a69a7d3ca
size 5183030656

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e26237ae02ea2a5e163f415f963977cc33abd6f95d50a743f042d4f10534c49b
size 3805398400

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a1992404368fbed02399ae726b97e21ddb949f52dedee6f3a963da800de93e2d
size 4027606400

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bdd7fd9b1e639dbc256a60f0ad44c4725937770fe150976bf903ba3f4fce2785
size 5132453248

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:aaac5eab9bc37a2b426c36ee25f238a8bf0cf0e726f9c34d7c02656e4520b5b0
size 4761781632

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c829e86196b43acef64afc9c4c0fbbd9fe6ddf24af9d7736dfec617ccad4bb03
size 4337665408

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47c002fa0404b6b06b7dd3ee433ec405f1b4cc0e769b374fa055fa31ca574477
size 5354661248

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b76ab1429e7a8ad4c71631ee2f97be43fbb9f4e2b723544a710d57a9652689b2
size 5459199360

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fc42ae629b3fc4bb27ec01a527db676a8618497cb80f233f36ca1537a3024329
size 5443143040

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b419f4390258bdf408a3af2ee8971d48014bf572a8716178cbefc1c1a486cc98
size 5443143040

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ea1356be332b1e20930ad19801839fa9e4467413bb86f2968a7b4acecf44bcc9
size 5443143040

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b029da8821303386273f61538982eff377d2fe69d5dffda46f5cd55268efe193
size 5983266176

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:80dfc4fc6469e5363bb73669d2bdc34c3b9244c9a0e4a02f90233d7a3e31d558
size 5761058176

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:19dd96b41b242fcc86302e5bf845c5c80d543e26a600a57264f919aa4684a79c
size 5478925696

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0d9326a76fc4d61030c7f4c92e0c7393af87d55076f0097813fea4c240b213da
size 6869575040

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:795eec3ea53156d2c92f490454f5bc960ed64266e601b48fa81acf595c746db7
size 6647367040

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e3d9bc241f9d10b6048a706bfaa0bfa29b749d2c1dc27d47f44ce4a50bc1d76f
size 6483592576

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:00f7060fceff9d1443c334f02c7345b31d67afb8eab61fb47f10c9148ad8dac7
size 7589070208

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3241879693a3ba5f85aa5771641110aecd18ef5d301801ae1dccf73cbb346d81
size 7811278208

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bf7b9bad17a21e9e2166905e595ab715c9b4821a74499d09810843ed001994ca
size 9827149184

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:49d5e3df1cf821fe97fdc9ca822e6a9e14cb973851d86c8c3a22f763d46fea1f
size 18490680448

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a7bb207d720328b7b6d86b370b403296c95f14d96fa8918ce48e9e727099a55f
size 6116900

122
README.md Normal file
View File

@@ -0,0 +1,122 @@
---
base_model: inflatebot/G2-9B-Blackout-R1
library_name: transformers
pipeline_tag: text-generation
tags:
- mergekit
- merge
quantized_by: bartowski
---
## Llamacpp imatrix Quantizations of G2-9B-Blackout-R1
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3715">b3715</a> for quantization.
Original model: https://huggingface.co/inflatebot/G2-9B-Blackout-R1
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
## Prompt format
No prompt format found, check original model page
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [G2-9B-Blackout-R1-f16.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-f16.gguf) | f16 | 18.49GB | false | Full F16 weights. |
| [G2-9B-Blackout-R1-Q8_0.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q8_0.gguf) | Q8_0 | 9.83GB | false | Extremely high quality, generally unneeded but max available quant. |
| [G2-9B-Blackout-R1-Q6_K_L.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q6_K_L.gguf) | Q6_K_L | 7.81GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [G2-9B-Blackout-R1-Q6_K.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q6_K.gguf) | Q6_K | 7.59GB | false | Very high quality, near perfect, *recommended*. |
| [G2-9B-Blackout-R1-Q5_K_L.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q5_K_L.gguf) | Q5_K_L | 6.87GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [G2-9B-Blackout-R1-Q5_K_M.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q5_K_M.gguf) | Q5_K_M | 6.65GB | false | High quality, *recommended*. |
| [G2-9B-Blackout-R1-Q5_K_S.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q5_K_S.gguf) | Q5_K_S | 6.48GB | false | High quality, *recommended*. |
| [G2-9B-Blackout-R1-Q4_K_L.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q4_K_L.gguf) | Q4_K_L | 5.98GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [G2-9B-Blackout-R1-Q4_K_M.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q4_K_M.gguf) | Q4_K_M | 5.76GB | false | Good quality, default size for must use cases, *recommended*. |
| [G2-9B-Blackout-R1-Q4_K_S.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q4_K_S.gguf) | Q4_K_S | 5.48GB | false | Slightly lower quality with more space savings, *recommended*. |
| [G2-9B-Blackout-R1-Q4_0.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q4_0.gguf) | Q4_0 | 5.46GB | false | Legacy format, generally not worth using over similarly sized formats |
| [G2-9B-Blackout-R1-Q4_0_8_8.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q4_0_8_8.gguf) | Q4_0_8_8 | 5.44GB | false | Optimized for ARM inference. Requires 'sve' support (see link below). |
| [G2-9B-Blackout-R1-Q4_0_4_8.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q4_0_4_8.gguf) | Q4_0_4_8 | 5.44GB | false | Optimized for ARM inference. Requires 'i8mm' support (see link below). |
| [G2-9B-Blackout-R1-Q4_0_4_4.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q4_0_4_4.gguf) | Q4_0_4_4 | 5.44GB | false | Optimized for ARM inference. Should work well on all ARM chips, pick this if you're unsure. |
| [G2-9B-Blackout-R1-Q3_K_XL.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q3_K_XL.gguf) | Q3_K_XL | 5.35GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [G2-9B-Blackout-R1-IQ4_XS.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-IQ4_XS.gguf) | IQ4_XS | 5.18GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [G2-9B-Blackout-R1-Q3_K_L.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q3_K_L.gguf) | Q3_K_L | 5.13GB | false | Lower quality but usable, good for low RAM availability. |
| [G2-9B-Blackout-R1-Q3_K_M.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q3_K_M.gguf) | Q3_K_M | 4.76GB | false | Low quality. |
| [G2-9B-Blackout-R1-IQ3_M.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-IQ3_M.gguf) | IQ3_M | 4.49GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [G2-9B-Blackout-R1-Q3_K_S.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q3_K_S.gguf) | Q3_K_S | 4.34GB | false | Low quality, not recommended. |
| [G2-9B-Blackout-R1-IQ3_XS.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-IQ3_XS.gguf) | IQ3_XS | 4.14GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [G2-9B-Blackout-R1-Q2_K_L.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q2_K_L.gguf) | Q2_K_L | 4.03GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [G2-9B-Blackout-R1-Q2_K.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-Q2_K.gguf) | Q2_K | 3.81GB | false | Very low quality but surprisingly usable. |
| [G2-9B-Blackout-R1-IQ2_M.gguf](https://huggingface.co/bartowski/G2-9B-Blackout-R1-GGUF/blob/main/G2-9B-Blackout-R1-IQ2_M.gguf) | IQ2_M | 3.43GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
Some say that this improves the quality, others don't notice any difference. If you use these models PLEASE COMMENT with your findings. I would like feedback that these are actually used and useful so I don't keep uploading quants no one is using.
Thanks!
## Downloading using huggingface-cli
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/G2-9B-Blackout-R1-GGUF --include "G2-9B-Blackout-R1-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/G2-9B-Blackout-R1-GGUF --include "G2-9B-Blackout-R1-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (G2-9B-Blackout-R1-Q8_0) or download them all in place (./)
## Q4_0_X_X
These are *NOT* for Metal (Apple) offloading, only ARM chips.
If you're using an ARM chip, the Q4_0_X_X quants will have a substantial speedup. Check out Q4_0_4_4 speed comparisons [on the original pull request](https://github.com/ggerganov/llama.cpp/pull/5780#pullrequestreview-21657544660)
To check which one would work best for your ARM chip, you can check [AArch64 SoC features](https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html) (thanks EloyOn!).
## Which file should I choose?
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset
Thank you ZeroWw for the inspiration to experiment with embed/output
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}