rb-nano/README.md

---
license: cc-by-nc-4.0
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - text-generation
  - gpt2
  - small-language-model
  - pytorch
  - safetensors
---

# rb-nano

A 48M-parameter, GPT-2-style decoder-only transformer trained from scratch as part of the **Leopard AI Model Suite**. Small enough to run on CPU or any GPU; built as a learning/research model, not a production assistant.

> **Run it instantly with [Ollama](https://ollama.com/rafi-dev/rb-nano):** `ollama run rafi-dev/rb-nano`
> Or grab the quantized build from the **GGUF** repo (`rb-nano-GGUF`) for `llama.cpp`.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("leopard-Ai/rb-nano")
tok = AutoTokenizer.from_pretrained("leopard-Ai/rb-nano")

prompt = "<sos>user: hello\nai:"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(
    ids, max_new_tokens=64, do_sample=True,
    temperature=0.7, top_k=40, top_p=0.9, repetition_penalty=1.3,
    eos_token_id=2, pad_token_id=0,
)
print(tok.decode(out[0], skip_special_tokens=True))
```

It loads as a standard `GPT2LMHeadModel` — **no `trust_remote_code` required**.

## Prompt format

Trained on a simple `user:` / `ai:` turn format, prefixed with the `<sos>` token:

```
<sos>user: hello
ai: Hi there! How can I help you today?
user: what is python?
ai:
```

## Architecture

| | |
|---|---|
| Type | Decoder-only transformer (GPT-2 family) |
| Parameters | ~48M |
| Embedding dim (`n_embd`) | 512 |
| Layers | 10 |
| Attention heads | 8 |
| Context length | 1024 tokens |
| Position embeddings | Learned |
| Norm / activation | LayerNorm, GELU-tanh (`gelu_new`) |
| Head | Weight-tied to token embeddings |
| Tokenizer | ByteLevel BPE, 32k vocab |
| Format | safetensors (fp32) |

## Training

- **Pretrain** — FineWeb-Edu (`sample-10BT`), ~50M tokens. Final val loss ≈ 3.44.
- **Finetune** — Alpaca, Alpaca-cleaned, CodeAlpaca-20k, Dolly-15k, and ShareGPT (full multi-turn threads, loss masked to assistant turns only). Final val loss ≈ 2.67.

## Recommended parameters

```
temperature   0.7
top_k         40
top_p         0.9
repeat_penalty 1.3
```

## Limitations

- **Knowledge.** At 48M params the model has very limited factual knowledge and will confidently hallucinate (made-up libraries, wrong dates, etc.). It cannot be a reliable source of facts.
- **Coherence.** Good for short exchanges; longer or more technical answers drift.
- **Scope.** English-centric, 1024-token context. Best for demos, experimentation, and edge/CPU inference — not production use.

## License / attribution

Released under **CC BY-NC 4.0** (non-commercial, attribution required). The finetune mixes datasets with non-commercial terms (Alpaca, CodeAlpaca, ShareGPT — OpenAI-derived), so commercial use is not granted. Trained on publicly available datasets (FineWeb-Edu, Alpaca, Dolly, CodeAlpaca, ShareGPT); review each dataset's license before redistributing derived outputs.

## Made with care

rb-nano was built by **Rafi** (12 years old) and **Buddi** (10 years old) — pretrained and finetuned from scratch on a single **RTX 4070 (8 GB VRAM)**. It's a passion project: proof that a coherent little chat model can be trained end-to-end on consumer hardware.

If you enjoy it and want to support more experiments like this, you can [buy us a coffee ☕](https://ko-fi.com/leopardAi). Thank you for trying rb-nano — we hope you like it.

## rb-super might come

we plan to make rb-super 120 milion parameters