bartowski/bagel-8b-v1.0-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/bagel-8b-v1.0-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-03 08:05:12 +08:00
commit 56771a34d3
28 changed files with 280 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
.gitattributes vendored Normal file
View File

@@ -0,0 +1,59 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0-bf16.gguf filter=lfs diff=lfs merge=lfs -text
bagel-8b-v1.0.imatrix filter=lfs diff=lfs merge=lfs -text

148
README.md Normal file
View File

@@ -0,0 +1,148 @@
---
license: other
license_name: llama3
license_link: https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/LICENSE
base_model: meta-llama/Meta-Llama-3-8B
tags:
- llama-3
- bagel
datasets:
- ai2_arc
- allenai/ultrafeedback_binarized_cleaned
- argilla/distilabel-intel-orca-dpo-pairs
- jondurbin/airoboros-3.2
- codeparrot/apps
- facebook/belebele
- bluemoon-fandom-1-1-rp-cleaned
- boolq
- camel-ai/biology
- camel-ai/chemistry
- camel-ai/math
- camel-ai/physics
- jondurbin/contextual-dpo-v0.1
- jondurbin/gutenberg-dpo-v0.1
- jondurbin/py-dpo-v0.1
- jondurbin/truthy-dpo-v0.1
- LDJnr/Capybara
- jondurbin/cinematika-v0.1
- WizardLM/WizardLM_evol_instruct_70k
- glaiveai/glaive-function-calling-v2
- jondurbin/gutenberg-dpo-v0.1
- grimulkan/LimaRP-augmented
- lmsys/lmsys-chat-1m
- ParisNeo/lollms_aware_dataset
- TIGER-Lab/MathInstruct
- Muennighoff/natural-instructions
- openbookqa
- kingbri/PIPPA-shareGPT
- piqa
- Vezora/Tested-22k-Python-Alpaca
- ropes
- cakiki/rosetta-code
- Open-Orca/SlimOrca
- b-mc2/sql-create-context
- squad_v2
- mattpscott/airoboros-summarization
- migtissera/Synthia-v1.3
- unalignment/toxic-dpo-v0.2
- WhiteRabbitNeo/WRN-Chapter-1
- WhiteRabbitNeo/WRN-Chapter-2
- winogrande
quantized_by: bartowski
pipeline_tag: text-generation
---
## Llamacpp imatrix Quantizations of bagel-8b-v1.0
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2854">b2854</a> for quantization.
Original model: https://huggingface.co/jondurbin/bagel-8b-v1.0
All quants made using imatrix option with dataset provided by Kalomaze [here](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)
## Prompt format
```
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Description |
| -------- | ---------- | --------- | ----------- |
| [bagel-8b-v1.0-Q8_0.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q8_0.gguf) | Q8_0 | 8.54GB | Extremely high quality, generally unneeded but max available quant. |
| [bagel-8b-v1.0-Q6_K.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q6_K.gguf) | Q6_K | 6.59GB | Very high quality, near perfect, *recommended*. |
| [bagel-8b-v1.0-Q5_K_M.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q5_K_M.gguf) | Q5_K_M | 5.73GB | High quality, *recommended*. |
| [bagel-8b-v1.0-Q5_K_S.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q5_K_S.gguf) | Q5_K_S | 5.59GB | High quality, *recommended*. |
| [bagel-8b-v1.0-Q4_K_M.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q4_K_M.gguf) | Q4_K_M | 4.92GB | Good quality, uses about 4.83 bits per weight, *recommended*. |
| [bagel-8b-v1.0-Q4_K_S.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q4_K_S.gguf) | Q4_K_S | 4.69GB | Slightly lower quality with more space savings, *recommended*. |
| [bagel-8b-v1.0-IQ4_NL.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ4_NL.gguf) | IQ4_NL | 4.67GB | Decent quality, slightly smaller than Q4_K_S with similar performance *recommended*. |
| [bagel-8b-v1.0-IQ4_XS.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ4_XS.gguf) | IQ4_XS | 4.44GB | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [bagel-8b-v1.0-Q3_K_L.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q3_K_L.gguf) | Q3_K_L | 4.32GB | Lower quality but usable, good for low RAM availability. |
| [bagel-8b-v1.0-Q3_K_M.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q3_K_M.gguf) | Q3_K_M | 4.01GB | Even lower quality. |
| [bagel-8b-v1.0-IQ3_M.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ3_M.gguf) | IQ3_M | 3.78GB | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [bagel-8b-v1.0-IQ3_S.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ3_S.gguf) | IQ3_S | 3.68GB | Lower quality, new method with decent performance, recommended over Q3_K_S quant, same size with better performance. |
| [bagel-8b-v1.0-Q3_K_S.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q3_K_S.gguf) | Q3_K_S | 3.66GB | Low quality, not recommended. |
| [bagel-8b-v1.0-IQ3_XS.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ3_XS.gguf) | IQ3_XS | 3.51GB | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [bagel-8b-v1.0-IQ3_XXS.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ3_XXS.gguf) | IQ3_XXS | 3.27GB | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [bagel-8b-v1.0-Q2_K.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-Q2_K.gguf) | Q2_K | 3.17GB | Very low quality but surprisingly usable. |
| [bagel-8b-v1.0-IQ2_M.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ2_M.gguf) | IQ2_M | 2.94GB | Very low quality, uses SOTA techniques to also be surprisingly usable. |
| [bagel-8b-v1.0-IQ2_S.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ2_S.gguf) | IQ2_S | 2.75GB | Very low quality, uses SOTA techniques to be usable. |
| [bagel-8b-v1.0-IQ2_XS.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ2_XS.gguf) | IQ2_XS | 2.60GB | Very low quality, uses SOTA techniques to be usable. |
| [bagel-8b-v1.0-IQ2_XXS.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ2_XXS.gguf) | IQ2_XXS | 2.39GB | Lower quality, uses SOTA techniques to be usable. |
| [bagel-8b-v1.0-IQ1_M.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ1_M.gguf) | IQ1_M | 2.16GB | Extremely low quality, *not* recommended. |
| [bagel-8b-v1.0-IQ1_S.gguf](https://huggingface.co/bartowski/bagel-8b-v1.0-GGUF/blob/main/bagel-8b-v1.0-IQ1_S.gguf) | IQ1_S | 2.01GB | Extremely low quality, *not* recommended. |
## Downloading using huggingface-cli
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/bagel-8b-v1.0-GGUF --include "bagel-8b-v1.0-Q4_K_M.gguf" --local-dir ./ --local-dir-use-symlinks False
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/bagel-8b-v1.0-GGUF --include "bagel-8b-v1.0-Q8_0.gguf/*" --local-dir bagel-8b-v1.0-Q8_0 --local-dir-use-symlinks False
```
You can either specify a new local-dir (bagel-8b-v1.0-Q8_0) or download them all in place (./)
## Which file should I choose?
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

3
bagel-8b-v1.0-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3e6a4f36b2ead12ff7b300120de2c6561563ea2594913fd544c39d6823080c9f
size 2161971776

3
bagel-8b-v1.0-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f85074b3f9832b71f62d9f4ef9099bf4687506fa1b52328095678f353aac479d
size 2019627584

3
bagel-8b-v1.0-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f9d96695bbf94d82262a1efea01fa90b9c88a142e565f744d1a4c67271f8d320
size 2948280896

3
bagel-8b-v1.0-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:524bf575b8412d76264a339709304b9997b7c77539d8618330979d218d6e7880
size 2758488640

3
bagel-8b-v1.0-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3b8127f63e2195ae35d0ae3bc4a26e61cdcd6a7cf22f56c45ad08dbb2dab3dc6
size 2605781568

3
bagel-8b-v1.0-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f1d299f3e591ddab79cea086052f2e32979a837a15deb6273771568fa8140a67
size 2399212096

3
bagel-8b-v1.0-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d54fc495aee9fb365be97bebd8facc1b212b8e5919f0c28c2262523d88ede89b
size 3784823360

3
bagel-8b-v1.0-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24ed22861fc5d53ca29ec4dfc33aafd8f47f270466adabda1471d0bbcdb28efe
size 3682325056

3
bagel-8b-v1.0-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0370995ff20697fafc5c4536f62be700668be64af16929e44a68ac9baf9737f4
size 3518747200

3
bagel-8b-v1.0-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9ad624d81222426998f8855bc70322568c0c1bdbf9db74a90bb0ccd45113004f
size 3274912320

3
bagel-8b-v1.0-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d162c11828eacdcee20f95d8f6a1dcc37d5605b0906b3deccd113cac07922433
size 4677988928

3
bagel-8b-v1.0-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bf0d5ba93e93aedf4cc9491a3ece06aa04262b5e9f53fbcd8ddd77aa230ac2b6
size 4447662656

3
bagel-8b-v1.0-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13e8e73203d2f9ffd694adf8ac8ee59c02afdb1549b5b9682c4afab6c871d946
size 3179131456

3
bagel-8b-v1.0-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3ae8df6152d393cb1a88d5098d90cd361fea323326abe2fc2e6488fcfce6295
size 4321956416

3
bagel-8b-v1.0-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d403e36af96bed2b1cdcd428fce6a37e2f08249ed947877990b16833f8998d2a
size 4018917952

3
bagel-8b-v1.0-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a851ed231406c4f14a76f4ce47681794cf0d7de94422afc1b6b98f2e5ab2d03a
size 3664499264

3
bagel-8b-v1.0-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f80d182e40d0d47c387c17964ee043c9711ef8e8c8e62a9fecd59abc3c2392b6
size 4920734272

3
bagel-8b-v1.0-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9281544251d22daff6f51445b040d02dc91feecb991ef89ed91cbe9b4218f114
size 4692668992

3
bagel-8b-v1.0-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ad007d6050f1b083f9fa48c749a03a8d0e8f066df3b9821465da0658f9f371b1
size 5732987456

3
bagel-8b-v1.0-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4420cc2e1bd5dfe8dd1ba671ea63ed4a16d2101cc4d53e25ccfbfee20108913a
size 5599294016

3
bagel-8b-v1.0-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3e406b2796b279a8be4712954f5f835be09e74b596745274aae65093d73cf31c
size 6596006464

3
bagel-8b-v1.0-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:85578f3a433d1de6d2c289e132b28348c759595d0c520531f92412b4c940c984
size 8540770880

3
bagel-8b-v1.0-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8e6c61163e02678b5130063e81d371d6f65c120d05efaaecb3bfa3b4d3991cfb
size 16068890944

0
bagel-8b-v1.0-f32.gguf Normal file
View File

3
bagel-8b-v1.0.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e8951cc82e23d6e552efd186c785ce3b93a05cf2102185f5fefd291e21036c9c
size 4988166

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}