初始化项目,由ModelHub XC社区提供模型
Model: Glubbot-Team/glubbot1.1-7.5m Source: Original Platform
This commit is contained in:
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
glubbot1.1-7.5m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
55
README.md
Normal file
55
README.md
Normal file
@@ -0,0 +1,55 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- text-generation
|
||||
- experimental
|
||||
- ultra-lightweight
|
||||
- tiny-stories
|
||||
model_type: llm
|
||||
language:
|
||||
- en
|
||||
datasets:
|
||||
- roneneldan/TinyStories
|
||||
metrics:
|
||||
- perplexity
|
||||
---
|
||||
|
||||
# glubbot1.1-7.5m
|
||||
|
||||
## Model Description
|
||||
glubbot1.1-7.5m is an experimental, ultra-lightweight text generation model developed by the **Glubbot-Team**.
|
||||
|
||||
With only 7.5 million parameters, this model is designed for experimental testing, low-resource inference benchmarking, and studying language acquisition in small-scale models. It was trained specifically to generate short, grammatically correct narratives.
|
||||
|
||||
- **Developed by:** Glubbot-Team
|
||||
- **Model Type:** Transformer-based Causal Language Model
|
||||
- **Language(s):** English
|
||||
- **License:** Apache 2.0
|
||||
- **Status:** Experimental / Testing
|
||||
|
||||
## Intended Uses & Limitations
|
||||
This model is intended solely for research, architectural testing, and experimental evaluation.
|
||||
|
||||
### Intended Use
|
||||
- Synthetic text generation for short, child-like stories.
|
||||
- Benchmarking inference speeds on edge devices or CPUs.
|
||||
- Testing tokenization strategies and pipeline integration.
|
||||
|
||||
### Limitations & Biases
|
||||
- **Domain Specificity:** Optimized purely for short stories; it cannot handle reasoning, coding, or factual factual queries.
|
||||
- **Limited Capacity:** Due to the 7.5M parameter size, the model may produce repetitive structures or drift logically during long generations.
|
||||
- **Not for Production:** Do not deploy this model for general-purpose assistant tasks.
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
The model was trained on the **TinyStories** dataset (`roneneldan/TinyStories`). This dataset consists of short stories containing words typically understood by 3- to 4-year-olds, optimized to teach models basic grammar and reasoning.
|
||||
|
||||
### Hyperparameters
|
||||
- **Context Length:** [e.g., 512 / 1024]
|
||||
- **Batch Size:** [Insert Batch Size]
|
||||
- **Learning Rate:** [Insert Learning Rate]
|
||||
- **Optimizer:** AdamW
|
||||
|
||||
## Evaluation
|
||||
Evaluation metrics are monitored primarily to test convergence stability on the TinyStories split. The model is evaluated on its ability to maintain basic grammatical coherence.
|
||||
32
config.json
Normal file
32
config.json
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 2,
|
||||
"dtype": "float32",
|
||||
"eos_token_id": 3,
|
||||
"head_dim": 64,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 256,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 1024,
|
||||
"max_position_embeddings": 512,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 4,
|
||||
"num_hidden_layers": 4,
|
||||
"num_key_value_heads": 4,
|
||||
"pad_token_id": null,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_parameters": {
|
||||
"rope_theta": 10000.0,
|
||||
"rope_type": "default"
|
||||
},
|
||||
"tie_word_embeddings": false,
|
||||
"transformers_version": "5.8.0",
|
||||
"use_cache": true,
|
||||
"vocab_size": 8192
|
||||
}
|
||||
3
glubbot1.1-7.5m.gguf
Normal file
3
glubbot1.1-7.5m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:493664ad6aa34ccb83db8225f3815b32e78260080ce7995a9c80cf2d991c9042
|
||||
size 16981280
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:cb6c8943171caa6f3ef046ec9178246e864a79563342ced971abfa2b52452785
|
||||
size 33567880
|
||||
3
tokenizer.model
Normal file
3
tokenizer.model
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:deab01b0c82b89fa998511f57bcd7194a993c6d08cfa2221c1498fb9fc5a01df
|
||||
size 374270
|
||||
8192
tokenizer.vocab
Normal file
8192
tokenizer.vocab
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user