palmer-006/README.md

---
library_name: transformers
pipeline_tag: text-generation
language:
- en
license: other
license_name: appvoid-noncommercial-model-license-1.0
license_link: LICENSE
tags:
- palmer
- text-editing
- rewriting
- paraphrasing
- grammar-correction
- edge
- small-language-model
widget:
- text: Make this more concise.
  output:
    url: images/4.png
- text: Make this friendlier.
  output:
    url: images/1.png
- text: Improve clarity without changing the meaning.
  output:
    url: images/2.png
- text: Improve clarity without changing the meaning.
  output:
    url: images/3.png
extra_gated_prompt: >-
  You agree to use this model in good faith, respect the LICENSE in this
  repository, comply with the applicable Falcon upstream terms, and not use the
  model to harm any person or group. Commercial use requires a separate license.
new_version: appvoid/palmer-007
---

<style>
@import url("https://fonts.googleapis.com/css2?family=Roboto+Mono:wght@300;400;500;600&display=swap");

*,
html,
body,
div {
  box-sizing: border-box;
  color: #fff;
  background: #000 !important;
  border: none;
  font-family: "Roboto Mono", monospace !important;
  font-weight: 300;
  font-style: normal;
}


::selection {
  color: #000;
  background: #fff;
}

::-moz-selection {
  color: #000;
  background: #fff;
}

body {
  margin: 0;
  padding: 2rem 2rem 5rem;
  background: #070707 !important;
}

.model-card {
  max-width: 1100px;
  margin: 0 auto 2rem;
  padding: 3rem;
  text-align: justify;
  border: 1px solid #1d1d1d;
  border-radius: 1.25rem;
  overflow: visible;
  box-shadow: 0 18px 60px rgba(0, 0, 0, 0.45);
}

/* Logo area */
.logo-stage {
  position: relative;
  isolation: isolate;

  display: flex;
  align-items: center;
  justify-content: center;

  min-height: 360px;
  margin: 0 auto 1.5rem;
  overflow: visible;
  cursor: default;

  user-select: none;
  -webkit-user-select: none;
  -moz-user-select: none;
}

/*
  The stage remains full-size while the logo collapses, so shrinking does not
  accidentally end the hover state. Every visible property has an explicit
  neutral value and transition, preventing effects from snapping on or off.
*/
.logo-vortex {
  position: relative;
  display: grid;
  place-items: center;
  width: fit-content;
  margin: 0 auto;

  border-radius: 0.75rem;
  outline: 2px solid rgba(255, 255, 255, 0);
  outline-offset: 0;

  transform:
    perspective(1000px)
    translateZ(0)
    scale(1)
    rotate(0deg);

  transform-origin: center;
  will-change: transform, border-radius, box-shadow, outline-color, outline-offset;

  box-shadow:
    0 0 0 0 rgba(0, 0, 0, 0),
    0 0 0 0 rgba(255, 255, 255, 0),
    0 0 0 0 rgba(0, 0, 0, 0),
    0 0 0 0 rgba(255, 255, 255, 0),
    0 0 0 0 rgba(255, 255, 255, 0),
    0 0 0 0 rgba(255, 255, 255, 0);

  transition:
    transform 680ms cubic-bezier(0.16, 1, 0.3, 1),
    border-radius 560ms ease,
    box-shadow 620ms ease,
    outline-color 520ms ease,
    outline-offset 620ms cubic-bezier(0.16, 1, 0.3, 1);
}

/*
  Both rings rotate at all times, even while invisible. Hover only fades and
  brightens them, so there is no animation restart, pause, or discontinuity.
*/
.logo-vortex::before,
.logo-vortex::after {
  content: "";
  position: absolute;
  z-index: -1;
  pointer-events: none;

  border-radius: 50%;
  opacity: 0;

  transition:
    opacity 420ms ease,
    border-color 420ms ease,
    filter 520ms ease;
}

.logo-vortex::before {
  inset: -30px;

  border: 3px solid rgba(255, 255, 255, 0);
  border-left-color: rgba(255, 255, 255, 0);
  border-right-color: rgba(255, 255, 255, 0);

  filter:
    drop-shadow(0 0 0 rgba(255, 255, 255, 0))
    drop-shadow(0 0 0 rgba(255, 255, 255, 0));

  animation: singularity-orbit 880ms linear infinite;
}

.logo-vortex::after {
  inset: -52px;

  border: 2px dashed rgba(255, 255, 255, 0);

  filter:
    drop-shadow(0 0 0 rgba(255, 255, 255, 0));

  animation: singularity-orbit-reverse 1.28s linear infinite;
}

.logo-image {
  position: relative;
  z-index: 1;

  display: block;
  width: auto;
  height: 256px;
  max-width: 100%;
  margin: 0 auto;
  object-fit: contain;

  border-radius: 0.75rem;

  filter:
    grayscale(0%)
    contrast(115%)
    brightness(100%)
    blur(0)
    drop-shadow(0 0 0 rgba(255, 255, 255, 0))
    drop-shadow(0 0 0 rgba(255, 255, 255, 0))
    drop-shadow(0 0 0 rgba(255, 255, 255, 0));

  transform:
    translateZ(0)
    scale(1)
    rotate(0deg);

  transform-origin: center;
  will-change: transform, filter, border-radius;

  transition:
    transform 680ms cubic-bezier(0.16, 1, 0.3, 1),
    filter 620ms cubic-bezier(0.16, 1, 0.3, 1),
    border-radius 560ms ease;

  user-select: none;
  -webkit-user-select: none;
  -moz-user-select: none;
  -webkit-user-drag: none;
  pointer-events: none;
  cursor: default;
}

.logo-stage:hover .logo-vortex {
  border-radius: 50%;
  outline-color: rgba(255, 255, 255, 0.9);
  outline-offset: 16px;

  transform:
    perspective(1000px)
    translateZ(0)
    scale(0.32)
    rotate(540deg);

  box-shadow:
    0 0 0 5px #000,
    0 0 0 7px rgba(255, 255, 255, 0.92),
    0 0 0 14px #000,
    0 0 0 16px rgba(255, 255, 255, 0.34),
    0 0 38px 18px rgba(255, 255, 255, 0.3),
    0 0 96px 34px rgba(255, 255, 255, 0.14);
}

.logo-stage:hover .logo-image {
  border-radius: 50%;

  transform:
    translateZ(0)
    scale(0.84)
    rotate(-360deg);

  filter:
    grayscale(100%)
    contrast(240%)
    brightness(14%)
    blur(4px)
    drop-shadow(0 0 5px rgba(255, 255, 255, 1))
    drop-shadow(0 0 18px rgba(255, 255, 255, 0.74))
    drop-shadow(0 0 42px rgba(255, 255, 255, 0.38));
}

.logo-stage:hover .logo-vortex::before {
  opacity: 1;

  border-color: rgba(255, 255, 255, 0.88);
  border-left-color: rgba(255, 255, 255, 0.06);
  border-right-color: rgba(255, 255, 255, 0.24);

  filter:
    drop-shadow(0 0 9px rgba(255, 255, 255, 0.86))
    drop-shadow(0 0 26px rgba(255, 255, 255, 0.44));
}

.logo-stage:hover .logo-vortex::after {
  opacity: 0.72;
  border-color: rgba(255, 255, 255, 0.42);

  filter:
    drop-shadow(0 0 13px rgba(255, 255, 255, 0.3));
}

@keyframes singularity-orbit {
  0% {
    transform: rotate(0deg) scale(0.96);
  }

  50% {
    transform: rotate(180deg) scale(1.04);
  }

  100% {
    transform: rotate(360deg) scale(0.96);
  }
}

@keyframes singularity-orbit-reverse {
  0% {
    transform: rotate(360deg) scale(1.04);
  }

  50% {
    transform: rotate(180deg) scale(0.97);
  }

  100% {
    transform: rotate(0deg) scale(1.04);
  }
}

/* Benchmark remains centered and enlarges substantially on hover. */
.benchmark-image {
  display: block;
  position: static;
  padding: 2rem;
  width: min(76%, 760px);
  height: auto;
  max-width: 100%;
  margin: 3rem auto 2.5rem;
  object-fit: contain;

  border: 2px solid rgba(255, 255, 255, 0.16);
  border-radius: 1rem;
  outline: none;

  user-select: none;
  -webkit-user-select: none;
  -moz-user-select: none;
  -webkit-user-drag: none;

  filter: none !important;
  transform: scale(1) !important;
  animation: none !important;
  box-shadow: none !important;

  position: relative;
  z-index: 1;
  transform-origin: center;

  transition:
    transform 440ms cubic-bezier(0.16, 1, 0.3, 1),
    border-color 360ms ease,
    box-shadow 440ms ease,
    filter 440ms ease !important;
}

.benchmark-image:hover {
  border-color: rgba(255, 255, 255, 0.96);
  border-radius: 2rem;
  outline: none;
  outline-offset: 0;
  border: 3px solid rgba(255, 255, 255, 0.16);
  z-index: 8;
  transform: scale(1.55) !important;
  animation: none !important;
  filter: contrast(1.08) brightness(1.05) !important;
  box-shadow:
    0 28px 90px rgba(0, 0, 0, 0.62),
    0 0 0 1px rgba(255, 255, 255, 0.08),
    0 0 58px rgba(255, 255, 255, 0.12) !important;
}

p,
li,
td {
  color: #d0d0d0;
  line-height: 1.7;
}

p {
  font-size: 0.98rem;
}

h1 {
  margin-top: 1rem;
  text-align: center;
}

h1,
h2,
h3,
h4,
h5,
h6 {
  color: #fff;
  font-weight: 500;
  letter-spacing: 0.04em;
}

table {
  width: 100%;
}

@media (max-width: 700px) {
  body {
    padding: 0.75rem 0.75rem 3rem;
  }

  .model-card {
    margin-bottom: 1.5rem;
    padding: 1.5rem;
    border-radius: 0.9rem;
  }

  .benchmark-image {
    width: 92%;
  }

  .logo-stage {
    min-height: 270px;
  }

  .logo-image {
    height: 190px;
  }
}

@media (prefers-reduced-motion: reduce) {
  .logo-vortex,
  .logo-vortex::before,
  .logo-vortex::after,
  .logo-image,
  .benchmark-image {
    animation: none !important;
    transition: none !important;
  }
}
</style>

<body>
<div class="model-card">

<div class="logo-stage">
  <div class="logo-vortex">
    <img
      class="logo-image"
      src="https://huggingface.co/appvoid/palmer-006/resolve/main/images/006.png"
      alt="Palmer 006 logo"
    />
  </div>
</div>

# a small model is all you need

We celebrate three years of experiments with the release of `palmer-006`, our flagship and extremely small language model for local experimentation, edge hardware, small SBCs, and low-power devices. Note that this model serves primarily as a language model and has not been validated nor finetuned as a general-purpose chat assistant. Thus further finetuning is required to properly work as an instruction-tuned model.

## updates

- [🐦‍🔥 A wild palmer appeared, go catch it!](https://huggingface.co/CEAMFA/palmer-007-preview-GGUF)
- No longer promoting arithmark, after doing proper evals, palmer-006 is barely better than the base model used to train. Model is [#5 on open slm leaderboard](https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard)
- Second spot on [BananaMindBench Leaderboard](https://huggingface.co/spaces/BananaMind/BananaMindBench-Leaderboard)!!🎉🎉

## ● details

<img
  class="benchmark-image"
  src="https://cdn-uploads.huggingface.co/production/uploads/62a813dedbb9e28866a91b27/WLY14UxFOKUVR9UZ2H5RK.png"
  alt="Palmer 006 benchmark comparison"
/>

We first lightly fine-tuned the base model on a diverse set of highly curated data points. We then “palmerized” it by merging models, followed by another light fine-tuning round. Finally, we adjusted Mamba for maximum token speed. With only 90M parameters, this turned the model into a competitive baseline against models above 125M parameters.

> As seen in the benchmark, we are getting closer to the performance of bigger models with our current technique, even on models of this size. Note: We used `acc_norm` whenever available.

## ● quick test

You can load `model.gguf` model into a llama-server, then run the script inside this repo as follows:

```python
python completion.py prompt.txt
```

## ● license

Open research, education, hobby use, modification, and redistribution are permitted. Commercial deployment, internal business use, paid products, hosted APIs, and client work require a separate commercial license from appvoid. For commercial licensing: [nosoyhackercodigo@gmail.com](mailto:nosoyhackercodigo@gmail.com)

  👋 But hey! If you already have a [donation subscription](https://ko-fi.com/appvoid), you can claim access to a free commercial license through the email above. You will keep the rights as long as the subscription is active.

## ● upstream attribution

**palmer-006 is built using artificial intelligence technology from the Technology Innovation Institute.**

## ● fun facts

- the new logo represents palmer's eye, pointing how small this new model is
- palmer models still exists to some extent due to inspiration from Karpathy's idea of a self-contained "cognitive core".
- we just like light finetuning because we prefer starting "real" experiments with the strongest baseline possible
- [we have an org with early access checkpoints](https://huggingface.co/CEAMFA)
</div>
</body>