---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- tiny
- mini
- small
- dumb
- asperger
- llm
- llama
- gqa
- sota
- micro
- ultrasmall
- 30M
---

# Dumb-1.2-Exp-0616

**——a new dumb model!**

training time: ~2 hour

## "dumb" name

I don't have enough money to create a real SoTA. However, if you have the wisdom and courage to forge "dumb", you can create a small dumb LLM.

## Explanation of the architecture of "Dumb 1.2"

In fact, I have progressed from Dumb 1 to Dumb 1.2. This model uses the following complex and short context-specific architecture.
Dumb 1.2 has a 34.611M parameter and is not suitable for complex things. However, it is possible to make interesting jokes as a strange AI that does not listen to simple jokes and stories. In some benchmarks, it is better than the previous model despite being a test version, and it is especially excellent in PiQA.

<img src='benchmark.jpeg' width="75%"><img src='architects.png' width="125%">

## Comparison with other models

It is comparable to the model that is more than 1.5 times larger than the MMLU. In many other benchmarks, there is a difference compared to the competition, but it still shows that the model size is 1.5 times more than 1.5 times, and the Dumb 1.1 is about 3 times smaller than the competition, and the model works even on low-performance PCs. In ARC-Easy, there is a slight challenge in thinking ability at around 65% of the SoTA of the same size of LLM.

In addition, in ARC-Challenge, the trial model has improved by 2% compared to the previous official model. This is great, and there is a high probability that the performance will be better in the official version and mid-term and late previews.