初始化项目,由ModelHub XC社区提供模型
Model: Glubbot-Team/glubbot1.1-7.5m Source: Original Platform
This commit is contained in:
55
README.md
Normal file
55
README.md
Normal file
@@ -0,0 +1,55 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- text-generation
|
||||
- experimental
|
||||
- ultra-lightweight
|
||||
- tiny-stories
|
||||
model_type: llm
|
||||
language:
|
||||
- en
|
||||
datasets:
|
||||
- roneneldan/TinyStories
|
||||
metrics:
|
||||
- perplexity
|
||||
---
|
||||
|
||||
# glubbot1.1-7.5m
|
||||
|
||||
## Model Description
|
||||
glubbot1.1-7.5m is an experimental, ultra-lightweight text generation model developed by the **Glubbot-Team**.
|
||||
|
||||
With only 7.5 million parameters, this model is designed for experimental testing, low-resource inference benchmarking, and studying language acquisition in small-scale models. It was trained specifically to generate short, grammatically correct narratives.
|
||||
|
||||
- **Developed by:** Glubbot-Team
|
||||
- **Model Type:** Transformer-based Causal Language Model
|
||||
- **Language(s):** English
|
||||
- **License:** Apache 2.0
|
||||
- **Status:** Experimental / Testing
|
||||
|
||||
## Intended Uses & Limitations
|
||||
This model is intended solely for research, architectural testing, and experimental evaluation.
|
||||
|
||||
### Intended Use
|
||||
- Synthetic text generation for short, child-like stories.
|
||||
- Benchmarking inference speeds on edge devices or CPUs.
|
||||
- Testing tokenization strategies and pipeline integration.
|
||||
|
||||
### Limitations & Biases
|
||||
- **Domain Specificity:** Optimized purely for short stories; it cannot handle reasoning, coding, or factual factual queries.
|
||||
- **Limited Capacity:** Due to the 7.5M parameter size, the model may produce repetitive structures or drift logically during long generations.
|
||||
- **Not for Production:** Do not deploy this model for general-purpose assistant tasks.
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
The model was trained on the **TinyStories** dataset (`roneneldan/TinyStories`). This dataset consists of short stories containing words typically understood by 3- to 4-year-olds, optimized to teach models basic grammar and reasoning.
|
||||
|
||||
### Hyperparameters
|
||||
- **Context Length:** [e.g., 512 / 1024]
|
||||
- **Batch Size:** [Insert Batch Size]
|
||||
- **Learning Rate:** [Insert Learning Rate]
|
||||
- **Optimizer:** AdamW
|
||||
|
||||
## Evaluation
|
||||
Evaluation metrics are monitored primarily to test convergence stability on the TinyStories split. The model is evaluated on its ability to maintain basic grammatical coherence.
|
||||
Reference in New Issue
Block a user