---
license: apache-2.0
tags:
- text-generation
- experimental
- ultra-lightweight
- tiny-stories
model_type: llm
language:
- en
datasets:
- roneneldan/TinyStories
metrics:
- perplexity
---

# glubbot1.1-7.5m

## Model Description
glubbot1.1-7.5m is an experimental, ultra-lightweight text generation model developed by the **Glubbot-Team**. 

With only 7.5 million parameters, this model is designed for experimental testing, low-resource inference benchmarking, and studying language acquisition in small-scale models. It was trained specifically to generate short, grammatically correct narratives.

- **Developed by:** Glubbot-Team
- **Model Type:** Transformer-based Causal Language Model
- **Language(s):** English
- **License:** Apache 2.0
- **Status:** Experimental / Testing

## Intended Uses & Limitations
This model is intended solely for research, architectural testing, and experimental evaluation. 

### Intended Use
- Synthetic text generation for short, child-like stories.
- Benchmarking inference speeds on edge devices or CPUs.
- Testing tokenization strategies and pipeline integration.

### Limitations & Biases
- **Domain Specificity:** Optimized purely for short stories; it cannot handle reasoning, coding, or factual factual queries.
- **Limited Capacity:** Due to the 7.5M parameter size, the model may produce repetitive structures or drift logically during long generations.
- **Not for Production:** Do not deploy this model for general-purpose assistant tasks.

## Training Details

### Training Data
The model was trained on the **TinyStories** dataset (`roneneldan/TinyStories`). This dataset consists of short stories containing words typically understood by 3- to 4-year-olds, optimized to teach models basic grammar and reasoning.

### Hyperparameters
- **Context Length:** [e.g., 512 / 1024]
- **Batch Size:** [Insert Batch Size]
- **Learning Rate:** [Insert Learning Rate]
- **Optimizer:** AdamW

## Evaluation
Evaluation metrics are monitored primarily to test convergence stability on the TinyStories split. The model is evaluated on its ability to maintain basic grammatical coherence.