初始化项目,由ModelHub XC社区提供模型

Model: bartowski/ozone-research_Chirp-01-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC committed 2026-10-18 00:24:12 +08:00
commit e043aa8a9c
28 files changed
+294

No files matched your search

+47
View File
@@ -0,0 +1,47 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.gguf* filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
+174
View File
@@ -0,0 +1,174 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
license_link: https://raw.githubusercontent.com/QwenLM/Qwen/refs/heads/main/Tongyi%20Qianwen%20RESEARCH%20LICENSE%20AGREEMENT
base_model: ozone-research/Chirp-01
license_name: qwen-research
license: other
---
## Llamacpp imatrix Quantizations of Chirp-01 by ozone-research
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4738">b4738</a> for quantization.
Original model: https://huggingface.co/ozone-research/Chirp-01
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
Run them in [LM Studio](https://lmstudio.ai/)
Run them directly with [llama.cpp](https://github.com/ggerganov/llama.cpp), or any other llama.cpp based project
## Prompt format
```
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [Chirp-01-f16.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-f16.gguf) | f16 | 6.18GB | false | Full F16 weights. |
| [Chirp-01-Q8_0.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q8_0.gguf) | Q8_0 | 3.29GB | false | Extremely high quality, generally unneeded but max available quant. |
| [Chirp-01-Q6_K_L.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q6_K_L.gguf) | Q6_K_L | 2.61GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [Chirp-01-Q6_K.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q6_K.gguf) | Q6_K | 2.54GB | false | Very high quality, near perfect, *recommended*. |
| [Chirp-01-Q5_K_L.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q5_K_L.gguf) | Q5_K_L | 2.30GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [Chirp-01-Q5_K_M.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q5_K_M.gguf) | Q5_K_M | 2.22GB | false | High quality, *recommended*. |
| [Chirp-01-Q5_K_S.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q5_K_S.gguf) | Q5_K_S | 2.17GB | false | High quality, *recommended*. |
| [Chirp-01-Q4_K_L.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q4_K_L.gguf) | Q4_K_L | 2.01GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [Chirp-01-Q4_1.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q4_1.gguf) | Q4_1 | 2.00GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [Chirp-01-Q4_K_M.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q4_K_M.gguf) | Q4_K_M | 1.93GB | false | Good quality, default size for most use cases, *recommended*. |
| [Chirp-01-Q4_K_S.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q4_K_S.gguf) | Q4_K_S | 1.83GB | false | Slightly lower quality with more space savings, *recommended*. |
| [Chirp-01-Q4_0.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q4_0.gguf) | Q4_0 | 1.83GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [Chirp-01-IQ4_NL.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-IQ4_NL.gguf) | IQ4_NL | 1.83GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [Chirp-01-Q3_K_XL.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q3_K_XL.gguf) | Q3_K_XL | 1.78GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [Chirp-01-IQ4_XS.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-IQ4_XS.gguf) | IQ4_XS | 1.74GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [Chirp-01-Q3_K_L.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q3_K_L.gguf) | Q3_K_L | 1.71GB | false | Lower quality but usable, good for low RAM availability. |
| [Chirp-01-Q3_K_M.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q3_K_M.gguf) | Q3_K_M | 1.59GB | false | Low quality. |
| [Chirp-01-IQ3_M.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-IQ3_M.gguf) | IQ3_M | 1.49GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [Chirp-01-Q3_K_S.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q3_K_S.gguf) | Q3_K_S | 1.45GB | false | Low quality, not recommended. |
| [Chirp-01-IQ3_XS.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-IQ3_XS.gguf) | IQ3_XS | 1.39GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [Chirp-01-Q2_K_L.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q2_K_L.gguf) | Q2_K_L | 1.35GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [Chirp-01-IQ3_XXS.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-IQ3_XXS.gguf) | IQ3_XXS | 1.28GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [Chirp-01-Q2_K.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-Q2_K.gguf) | Q2_K | 1.27GB | false | Very low quality but surprisingly usable. |
| [Chirp-01-IQ2_M.gguf](https://huggingface.co/bartowski/ozone-research_Chirp-01-GGUF/blob/main/ozone-research_Chirp-01-IQ2_M.gguf) | IQ2_M | 1.14GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/ozone-research_Chirp-01-GGUF --include "ozone-research_Chirp-01-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/ozone-research_Chirp-01-GGUF --include "ozone-research_Chirp-01-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (ozone-research_Chirp-01-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggerganov/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggerganov/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggerganov/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
+1
View File
@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4a0a81c8837c9738b94b581fb0715a9f02e6bcbe19c4ab769e03c3a3b3ef9ba0
size 1140516256
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24712834b35319280f69db121eae44666ac2d7c1ff421f5e12e5d3e6c9875059
size 1488895392
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7d4adcc531424f083619859977224cc9acc2fd3f7bc55a1750bf8f2841d77209
size 1391836576
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8b3136506521dfa5955851223f12812f6be2b3ec3c42ac9285583dbcb50ae943
size 1282827680
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6b7601a660d7e2dc8c96912384aeb33fade0d8df7351f7d92b218bfc6c49f6cd
size 1825209760
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8301a7dab588996eccbebb87d3eda595217c6b1b659f5f5426e0e31aa394fa49
size 1739095456
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:369d4d665f0cf88d0196e5b544657120181869285db7468184a917c68b08a71d
size 1274756512
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d6ab0a153d356d852dba1fadafcbc5d31b541ab9784bb71a90e25aea55449c51
size 1350116768
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a67bd6ee579f68d7241e224a114adaabc7295c2d9ef71f3b57035af37c22ce91
size 1707392416
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e1cc27130ff8fe5f8b636b262b7e1b7574feb3b2a6d2e54bbc04b809166e1a8c
size 1590476192
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5ac730efc3a7d88d34a525aa0d87865a0b65fa459d7116aaa2e597639a40be9e
size 1454357920
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:003618735cdd31b69249c1fa99771cc30120932e795e2ea262bffec3487ac7d6
size 1782752672
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ac84f18b56358af77ff352aa175bfdbca4d4af79cf6c0b59e1b4ff21f18ce9ce
size 1828486560
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a3ef17d8264eef42f4b163f900611b49c060d0b21a423d2fc4b8fed4b728456f
size 1996258720
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f85ac04e553a9c71f9ed88ab7a39a5708b24a77ff3afd9fbd6aeb91e471026ba
size 2005263776
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4ca7328f9b649755077c9064de0b9748d9f12a2e4ce8f493c94e1b19a8b5a035
size 1929903520
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2ffaafe7da64ba7f98ca95fd46718a86ced30676c0af647d1219d3b1d289afee
size 1834384800
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2e6aedbfc78ca9b06353b6f541c0b8aeae0edb6d18e400409259a489c9cc0c06
size 2300175776
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a648c1005c5d1ca7d0c44cb297a35bd29dccf8920427747e3892b16e2444b0e1
size 2224815520
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ee415e722e941777e30a626c8238aa4637dfd33a51077bc16486ef1cad87218
size 2169666976
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dcc2d795009d1635f003b11d5e903199787713b8483cdb4c9ff748329f597341
size 2538159520
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:45344011f8ba79c06c09a279c0274e9969b02acfc294ccd5e4f96865fb861500
size 2613519776
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5745818be06c1ea4c7472d8606d424956d495e6c38e6f7b35ad1e221bcbe0310
size 3285476768
+3
View File
@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5d74f69d5e39061f48ddec372bfd8abcd35dbb54a6703cc105283a5ff361add6
size 6178317440
Binary file not shown.