Base model: none. Weights are randomly initialized and trained from scratch.
Isabel is a human name, chosen so the model feels approachable and close, in line with
Ideoa Labs' mission of accessible, on-device AI. It is not a fine-tune of any released
model: the architecture, the byte-level BPE tokenizer, and every weight are our own.
How it is built
A single-GPU recipe (RTX 3090 Ti):
From-scratch pretraining on open, permissively-licensed educational English text plus our
own generated reasoning and arithmetic data. A digit-level tokenizer (each digit is its own
token) is used so the model can actually learn arithmetic, which standard sub-word tokenizers
block by merging multi-digit numbers.
Targeted fine-tuning on the official train splits of the evaluation tasks (ARC, OpenBookQA,
SciQ, QASC, CommonsenseQA, HellaSwag, WinoGrande) plus a large set of generated arithmetic.
Train splits only, with no test contamination.
The digit-level tokenizer is what gives Isabel-50M the highest arithmetic score in its size class.
Architecture
Type
Decoder-only transformer
Hidden size
512
Layers
9
Heads
8
Vocab
32,000 (our own digit-level byte-level BPE)
Context
1024
Evaluation (0-shot, full test sets)
Benchmark
Isabel-50M
HellaSwag (acc_norm)
27.1
ARC-Easy (acc_norm)
43.8
ARC-Challenge (acc_norm)
23.5
PIQA (acc_norm)
57.3
ArithMark-2 (acc)
42.4
Average
40.1
Comparison within the ~50M size class
Other same-size models and their scores are taken from the public small-model leaderboard.
Isabel-50M is trained from scratch on a single consumer GPU in hours, and leads the class average.
Model
Params
Avg
HellaSwag
ARC-E
ARC-C
PIQA
Arith
Isabel-50M
~54M
40.1
27.1
43.8
23.5
57.3
42.4
Supra-1.5-50M-base-exp
51.8M
39.0
29.8
48.4
25.5
60.0
31.3
Supra-1.5-50M-Instruct-exp
51.8M
37.7
29.3
43.9
26.1
59.4
29.8
Veyra2-Apricot-50M-Base
49.3M
37.6
31.3
42.5
23.3
62.1
29.0
Quark-50M
56.7M
37.3
28.5
36.8
25.0
57.8
28.2
Supra-50M-Base
51.8M
37.1
31.8
45.9
25.0
62.5
27.0
Supra-50M-Instruct
51.8M
35.9
29.1
44.4
27.3
59.5
29.1
Shard-1
54.5M
35.6
29.2
41.1
21.0
58.2
26.8
Veyra-30M-Base
34.6M
34.7
27.9
35.9
24.2
58.9
26.8
Stentor3-50M
50.0M
32.5
27.1
29.7
21.7
53.8
29.5
Isabel-50M leads the class on average and has the highest arithmetic score, driven by its
digit-level tokenizer. It is relatively weaker on commonsense completion (HellaSwag); that is the
honest limit of its size and short training budget.
Citation
@misc{isabel_2026,
title = {Isabel-50M: A Tiny From-Scratch Language Model for the Edge},
author = {Malios Dark},
year = {2026},
note = {Ideoa Labs}
}
