bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-27 22:12:12 +08:00
commit 2ad8cfbba4
30 changed files with 325 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

75
.gitattributes vendored Normal file
View File

@@ -0,0 +1,75 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*.tfevents* filter=lfs diff=lfs merge=lfs -text
*.db* filter=lfs diff=lfs merge=lfs -text
*.ark* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ggml filter=lfs diff=lfs merge=lfs -text
*.llamafile* filter=lfs diff=lfs merge=lfs -text
*.pt2 filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-bf16.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
TheDrummer_Magidonia-24B-v4.2.0-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text

168
README.md Normal file
View File

@@ -0,0 +1,168 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
base_model: TheDrummer/Magidonia-24B-v4.2.0
base_model_relation: quantized
---
## Llamacpp imatrix Quantizations of Magidonia-24B-v4.2.0 by TheDrummer
Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b6714">b6714</a> for quantization.
Original model: https://huggingface.co/TheDrummer/Magidonia-24B-v4.2.0
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8) combined with a subset of combined_all_small.parquet from Ed Addario [here](https://huggingface.co/datasets/eaddario/imatrix-calibration/blob/main/combined_all_small.parquet)
Run them in [LM Studio](https://lmstudio.ai/)
Run them directly with [llama.cpp](https://github.com/ggml-org/llama.cpp), or any other llama.cpp based project
## Prompt format
```
<s>[SYSTEM_PROMPT]{system_prompt}[/SYSTEM_PROMPT][INST]{prompt}[/INST]
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [Magidonia-24B-v4.2.0-bf16.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-bf16.gguf) | bf16 | 47.15GB | false | Full BF16 weights. |
| [Magidonia-24B-v4.2.0-Q8_0.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q8_0.gguf) | Q8_0 | 25.05GB | false | Extremely high quality, generally unneeded but max available quant. |
| [Magidonia-24B-v4.2.0-Q6_K_L.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q6_K_L.gguf) | Q6_K_L | 19.67GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [Magidonia-24B-v4.2.0-Q6_K.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q6_K.gguf) | Q6_K | 19.35GB | false | Very high quality, near perfect, *recommended*. |
| [Magidonia-24B-v4.2.0-Q5_K_L.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q5_K_L.gguf) | Q5_K_L | 17.18GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [Magidonia-24B-v4.2.0-Q5_K_M.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q5_K_M.gguf) | Q5_K_M | 16.76GB | false | High quality, *recommended*. |
| [Magidonia-24B-v4.2.0-Q5_K_S.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q5_K_S.gguf) | Q5_K_S | 16.30GB | false | High quality, *recommended*. |
| [Magidonia-24B-v4.2.0-Q4_1.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q4_1.gguf) | Q4_1 | 14.87GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [Magidonia-24B-v4.2.0-Q4_K_L.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q4_K_L.gguf) | Q4_K_L | 14.83GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [Magidonia-24B-v4.2.0-Q4_K_M.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q4_K_M.gguf) | Q4_K_M | 14.33GB | false | Good quality, default size for most use cases, *recommended*. |
| [Magidonia-24B-v4.2.0-Q4_K_S.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q4_K_S.gguf) | Q4_K_S | 13.55GB | false | Slightly lower quality with more space savings, *recommended*. |
| [Magidonia-24B-v4.2.0-Q4_0.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q4_0.gguf) | Q4_0 | 13.49GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [Magidonia-24B-v4.2.0-IQ4_NL.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ4_NL.gguf) | IQ4_NL | 13.47GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [Magidonia-24B-v4.2.0-Q3_K_XL.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q3_K_XL.gguf) | Q3_K_XL | 12.99GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [Magidonia-24B-v4.2.0-IQ4_XS.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ4_XS.gguf) | IQ4_XS | 12.76GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [Magidonia-24B-v4.2.0-Q3_K_L.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q3_K_L.gguf) | Q3_K_L | 12.40GB | false | Lower quality but usable, good for low RAM availability. |
| [Magidonia-24B-v4.2.0-Q3_K_M.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q3_K_M.gguf) | Q3_K_M | 11.47GB | false | Low quality. |
| [Magidonia-24B-v4.2.0-IQ3_M.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ3_M.gguf) | IQ3_M | 10.65GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [Magidonia-24B-v4.2.0-Q3_K_S.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q3_K_S.gguf) | Q3_K_S | 10.40GB | false | Low quality, not recommended. |
| [Magidonia-24B-v4.2.0-IQ3_XS.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ3_XS.gguf) | IQ3_XS | 9.91GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [Magidonia-24B-v4.2.0-Q2_K_L.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q2_K_L.gguf) | Q2_K_L | 9.55GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [Magidonia-24B-v4.2.0-IQ3_XXS.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ3_XXS.gguf) | IQ3_XXS | 9.28GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [Magidonia-24B-v4.2.0-Q2_K.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-Q2_K.gguf) | Q2_K | 8.89GB | false | Very low quality but surprisingly usable. |
| [Magidonia-24B-v4.2.0-IQ2_M.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ2_M.gguf) | IQ2_M | 8.11GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
| [Magidonia-24B-v4.2.0-IQ2_S.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ2_S.gguf) | IQ2_S | 7.48GB | false | Low quality, uses SOTA techniques to be usable. |
| [Magidonia-24B-v4.2.0-IQ2_XS.gguf](https://huggingface.co/bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF/blob/main/TheDrummer_Magidonia-24B-v4.2.0-IQ2_XS.gguf) | IQ2_XS | 7.21GB | false | Low quality, uses SOTA techniques to be usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF --include "TheDrummer_Magidonia-24B-v4.2.0-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/TheDrummer_Magidonia-24B-v4.2.0-GGUF --include "TheDrummer_Magidonia-24B-v4.2.0-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (TheDrummer_Magidonia-24B-v4.2.0-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggml-org/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggml-org/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggml-org/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggml-org/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

3
TheDrummer_Magidonia-24B-v4.2.0-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7fc2e0b1a35e0e4ad164865544f6fca4d51923ecb359cba3a9cd247a27a820a3
size 8114054560

3
TheDrummer_Magidonia-24B-v4.2.0-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7bac4a439b293e3d611b364951620970c7635bb4f65daf56218c014a4ef8c02f
size 7478355360

3
TheDrummer_Magidonia-24B-v4.2.0-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fa449379ed26aa7413f846e5530ca3e0aee42d6b17f9f8a0f004c8d4dcb6c8f7
size 7207036320

3
TheDrummer_Magidonia-24B-v4.2.0-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9fb2eeae58e45fe8100400c7254fd203a84d62635b2fc57baffa11a31085836d
size 10650953120

3
TheDrummer_Magidonia-24B-v4.2.0-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:78121c836acb142f22f3957e77a096ce837394aea71dc938983007bf22796ee6
size 9907119520

3
TheDrummer_Magidonia-24B-v4.2.0-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d7bd03cffbd83aba7c6c3cc65788ec46f59908a7b9402077886cb46fe4d4fca5
size 9280595360

3
TheDrummer_Magidonia-24B-v4.2.0-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8a25d8d346ac7bf7b6df1e049c3f386a3585e548bea9472877d19653e3c13762
size 13468018080

3
TheDrummer_Magidonia-24B-v4.2.0-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6bf313d3c2a4bea22864f0f277ce24cf38db821437cbc7e9515ae7555b082607
size 12758918560

3
TheDrummer_Magidonia-24B-v4.2.0-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:23ca3a0c5e52aae04e842e6797b914ebecabf52032de843a1f08c4890b8f6089
size 8890328480

3
TheDrummer_Magidonia-24B-v4.2.0-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6ebe99f5ce6f7ed1b822b7ef83d28eba28c1ebcded5141411c1392056a378263
size 9545688480

3
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4aa83f56ce63e3d9f694dfb2c2c12a54848c517395537d2fe7388dde130cf44
size 12400764320

3
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3c209e50c55f69610c7fbe7eed4e84edcc2bb6c560c0f9f2e3b64560202e0b8c
size 11474085280

3
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a07c66864459e75b24f6c6cb30153ddff0ba1cfd3e9c9cff5532c636f6195c20
size 10400277920

3
TheDrummer_Magidonia-24B-v4.2.0-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1a92b1aa3c88ac55ec8869900fc3ef919650e53bcab8184cb77c68747dfb33c5
size 12987966880

3
TheDrummer_Magidonia-24B-v4.2.0-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f18be17fffc94e9e80e36e532684ab46d98985314702aec6fdf8f0548caa3fb2
size 13494232480

3
TheDrummer_Magidonia-24B-v4.2.0-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fb42d68100aa1aa8c914bb7aa5f34979a7aee852e348b9f2974a20685ed12fb8
size 14873109920

3
TheDrummer_Magidonia-24B-v4.2.0-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e556f6397aa4ca7d3130affbaee53e673f32925af3570af6ddb5442369ae189a
size 14831986080

3
TheDrummer_Magidonia-24B-v4.2.0-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e86a5c499c9b1151e7d970cd417648d14805d15376baf2b5c5503cf8551aaff6
size 14333912480

3
TheDrummer_Magidonia-24B-v4.2.0-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ef5065529c64fb9c320e535e3b1e1064cd5c7a031e3aafdd67f29ed5462fee50
size 13549282720

3
TheDrummer_Magidonia-24B-v4.2.0-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c7541f042f705baadc004d96c34a954bbc565a22e619ee6eff12151fb0a63183
size 17178174880

3
TheDrummer_Magidonia-24B-v4.2.0-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f432115d0a0dfcbb9269eb0b09ac9d3ef107f1baad8b7efdd299aa1a995fe4c6
size 16763987360

3
TheDrummer_Magidonia-24B-v4.2.0-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:063b6492136256ef7c1406f772ce1817cffcb36824728f2dcf54691a87dfc078
size 16304416160

3
TheDrummer_Magidonia-24B-v4.2.0-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:68c1a17aa704a015a1ac8d6ec3c6d522e79880e6359df756bd2d73dc17c7a800
size 19345941920

3
TheDrummer_Magidonia-24B-v4.2.0-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:33f7c0c50f10db4267a06bcf3984409bc5dbf9d47864877332db0b326f221e75
size 19671000480

3
TheDrummer_Magidonia-24B-v4.2.0-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f7a5bce560055a29499fc4ea2f4ef974f6708022ea6cf2cb8f317a5d13f512f7
size 25054782880

3
TheDrummer_Magidonia-24B-v4.2.0-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:42fa916fcb213441522f9f2f6081b8c4534e7852f37537eb88a0dc09fc5d8d56
size 47153521792

3
TheDrummer_Magidonia-24B-v4.2.0-imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ed1fdeed0c2ead8740bb70e970da179c5149a90e0284e26907658d2326b73b70
size 10037344

1
configuration.json Normal file
View File

@@ -0,0 +1 @@
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}