palmyra-mini-thinking-AIO-GGUF/README.md

---
license: apache-2.0
base_model:
- Writer/palmyra-mini
- Writer/palmyra-mini-thinking-a
- Writer/palmyra-mini-thinking-b
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation-inference
- code
- math
- coder
---

# **palmyra-mini-thinking-AIO-GGUF**

> The palmyra-mini *[models](https://huggingface.co/Writer)* demonstrates exceptional capabilities in complex reasoning and mathematical problem-solving domains. Its performance is particularly noteworthy on benchmarks that require deep understanding and multi-step thought processes. A key strength of the model is its proficiency in grade-school-level math problems, as evidenced by its impressive score of 0.818 on the gsm8k (strict-match) benchmark. This high score indicates a robust ability to parse and solve word problems, a foundational skill for more advanced quantitative reasoning. This aptitude for mathematics is further confirmed by its outstanding performance on the MATH500 benchmark, where it also achieved a score of 0.818. This result underscores the models consistent and reliable mathematical capabilities across different problem sets. The model also shows strong performance on the AMC23 benchmark, with a solid score of 0.6. This benchmark, representing problems from the American Mathematics Competitions, highlights the models ability to tackle challenging, competition-level mathematics.

## Palmyra Mini GGUF Variants

| Model Name                         | Download Link                                                                                      |
|-----------------------------------|---------------------------------------------------------------------------------------------------|
| **palmyra-mini-GGUF**              | [Link](https://huggingface.co/prithivMLmods/palmyra-mini-thinking-AIO-GGUF/tree/main/palmyra-mini-GGUF)              |
| **palmyra-mini-thinking-a-GGUF**   | [Link](https://huggingface.co/prithivMLmods/palmyra-mini-thinking-AIO-GGUF/tree/main/palmyra-mini-thinking-a-GGUF)   |
| **palmyra-mini-thinking-b-GGUF**   | [Link](https://huggingface.co/prithivMLmods/palmyra-mini-thinking-AIO-GGUF/tree/main/palmyra-mini-thinking-b-GGUF)   |

## Model Files

### palmyra-mini

| File Name | Quant Type | File Size |
| - | - | - |
| palmyra-mini.BF16.gguf | BF16 | 3.56 GB |
| palmyra-mini.F16.gguf | F16 | 3.56 GB |
| palmyra-mini.F32.gguf | F32 | 7.11 GB |
| palmyra-mini.Q2_K.gguf | Q2_K | 752 MB |
| palmyra-mini.Q3_K_L.gguf | Q3_K_L | 980 MB |
| palmyra-mini.Q3_K_M.gguf | Q3_K_M | 924 MB |
| palmyra-mini.Q3_K_S.gguf | Q3_K_S | 861 MB |
| palmyra-mini.Q4_0.gguf | Q4_0 | 1.07 GB |
| palmyra-mini.Q4_1.gguf | Q4_1 | 1.16 GB |
| palmyra-mini.Q4_K.gguf | Q4_K | 1.12 GB |
| palmyra-mini.Q4_K_M.gguf | Q4_K_M | 1.12 GB |
| palmyra-mini.Q4_K_S.gguf | Q4_K_S | 1.07 GB |
| palmyra-mini.Q5_0.gguf | Q5_0 | 1.26 GB |
| palmyra-mini.Q5_1.gguf | Q5_1 | 1.35 GB |
| palmyra-mini.Q5_K.gguf | Q5_K | 1.28 GB |
| palmyra-mini.Q5_K_M.gguf | Q5_K_M | 1.28 GB |
| palmyra-mini.Q5_K_S.gguf | Q5_K_S | 1.26 GB |
| palmyra-mini.Q6_K.gguf | Q6_K | 1.46 GB |
| palmyra-mini.Q8_0.gguf | Q8_0 | 1.89 GB |

### palmyra-mini-thinking-a

| File Name | Quant Type | File Size |
| - | - | - |
| palmyra-mini-thinking-a.BF16.gguf | BF16 | 3.56 GB |
| palmyra-mini-thinking-a.F16.gguf | F16 | 3.56 GB |
| palmyra-mini-thinking-a.F32.gguf | F32 | 7.11 GB |
| palmyra-mini-thinking-a.Q2_K.gguf | Q2_K | 752 MB |
| palmyra-mini-thinking-a.Q3_K_L.gguf | Q3_K_L | 980 MB |
| palmyra-mini-thinking-a.Q3_K_M.gguf | Q3_K_M | 924 MB |
| palmyra-mini-thinking-a.Q3_K_S.gguf | Q3_K_S | 861 MB |
| palmyra-mini-thinking-a.Q4_0.gguf | Q4_0 | 1.07 GB |
| palmyra-mini-thinking-a.Q4_1.gguf | Q4_1 | 1.16 GB |
| palmyra-mini-thinking-a.Q4_K.gguf | Q4_K | 1.12 GB |
| palmyra-mini-thinking-a.Q4_K_M.gguf | Q4_K_M | 1.12 GB |
| palmyra-mini-thinking-a.Q4_K_S.gguf | Q4_K_S | 1.07 GB |
| palmyra-mini-thinking-a.Q5_0.gguf | Q5_0 | 1.26 GB |
| palmyra-mini-thinking-a.Q5_1.gguf | Q5_1 | 1.35 GB |
| palmyra-mini-thinking-a.Q5_K.gguf | Q5_K | 1.28 GB |
| palmyra-mini-thinking-a.Q5_K_M.gguf | Q5_K_M | 1.28 GB |
| palmyra-mini-thinking-a.Q5_K_S.gguf | Q5_K_S | 1.26 GB |
| palmyra-mini-thinking-a.Q6_K.gguf | Q6_K | 1.46 GB |
| palmyra-mini-thinking-a.Q8_0.gguf | Q8_0 | 1.89 GB |

### palmyra-mini-thinking-b

| File Name | Quant Type | File Size |
| - | - | - |
| palmyra-mini-thinking-b.BF16.gguf | BF16 | 3.09 GB |
| palmyra-mini-thinking-b.F16.gguf | F16 | 3.09 GB |
| palmyra-mini-thinking-b.F32.gguf | F32 | 6.18 GB |
| palmyra-mini-thinking-b.Q2_K.gguf | Q2_K | 676 MB |
| palmyra-mini-thinking-b.Q3_K_L.gguf | Q3_K_L | 880 MB |
| palmyra-mini-thinking-b.Q3_K_M.gguf | Q3_K_M | 824 MB |
| palmyra-mini-thinking-b.Q3_K_S.gguf | Q3_K_S | 761 MB |
| palmyra-mini-thinking-b.Q4_0.gguf | Q4_0 | 935 MB |
| palmyra-mini-thinking-b.Q4_1.gguf | Q4_1 | 1.02 GB |
| palmyra-mini-thinking-b.Q4_K.gguf | Q4_K | 986 MB |
| palmyra-mini-thinking-b.Q4_K_M.gguf | Q4_K_M | 986 MB |
| palmyra-mini-thinking-b.Q4_K_S.gguf | Q4_K_S | 940 MB |
| palmyra-mini-thinking-b.Q5_0.gguf | Q5_0 | 1.1 GB |
| palmyra-mini-thinking-b.Q5_1.gguf | Q5_1 | 1.18 GB |
| palmyra-mini-thinking-b.Q5_K.gguf | Q5_K | 1.13 GB |
| palmyra-mini-thinking-b.Q5_K_M.gguf | Q5_K_M | 1.13 GB |
| palmyra-mini-thinking-b.Q5_K_S.gguf | Q5_K_S | 1.1 GB |
| palmyra-mini-thinking-b.Q6_K.gguf | Q6_K | 1.27 GB |
| palmyra-mini-thinking-b.Q8_0.gguf | Q8_0 | 1.65 GB |

## Quants Usage 

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):

![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)