初始化项目,由ModelHub XC社区提供模型
Model: Abdourakib/tinystories-gpt2-124m Source: Original Platform
This commit is contained in:
69
README.md
Normal file
69
README.md
Normal file
@@ -0,0 +1,69 @@
|
||||
|
||||
---
|
||||
language: en
|
||||
tags:
|
||||
- gpt2
|
||||
- text-generation
|
||||
- children-stories
|
||||
- tinystories
|
||||
license: mit
|
||||
---
|
||||
|
||||
# TinyStories GPT2 124M
|
||||
|
||||
A GPT2 model trained from scratch on the
|
||||
TinyStories dataset to generate children's stories.
|
||||
|
||||
## Training Details
|
||||
- **Base Architecture:** GPT2 (124M parameters)
|
||||
- **Dataset:** karpathy/tinystories-gpt4-clean
|
||||
- **Training Steps:** 100,000
|
||||
- **Best Val Loss:** 1.1295
|
||||
- **Hardware:** NVIDIA RTX PRO 6000 (G4)
|
||||
|
||||
## How To Use
|
||||
```python
|
||||
from transformers import GPT2LMHeadModel
|
||||
from transformers import GPT2TokenizerFast
|
||||
import torch
|
||||
|
||||
model = GPT2LMHeadModel.from_pretrained(
|
||||
"{HF_USERNAME}/{MODEL_NAME}"
|
||||
)
|
||||
tokenizer = GPT2TokenizerFast.from_pretrained(
|
||||
"{HF_USERNAME}/{MODEL_NAME}"
|
||||
)
|
||||
|
||||
prompt = "Once upon a time there was a little cat"
|
||||
inputs = tokenizer(prompt, return_tensors="pt")
|
||||
|
||||
outputs = model.generate(
|
||||
inputs["input_ids"],
|
||||
max_new_tokens = 200,
|
||||
temperature = 0.8,
|
||||
top_p = 0.9,
|
||||
do_sample = True,
|
||||
repetition_penalty = 1.2,
|
||||
pad_token_id = tokenizer.eos_token_id
|
||||
)
|
||||
|
||||
story = tokenizer.decode(
|
||||
outputs[0],
|
||||
skip_special_tokens = True
|
||||
)
|
||||
print(story)
|
||||
```
|
||||
|
||||
## Example Output
|
||||
|
||||
"Once upon a time there was a little cat called
|
||||
Mimi. She loved to play with her toys, but one
|
||||
day she got very sad because she couldn't find
|
||||
her favorite toy. They searched everywhere and
|
||||
finally found it under the bed! Mimi was so
|
||||
happy and hugged her mom tight."
|
||||
|
||||
## Limitations
|
||||
- Generates children's stories only
|
||||
- Works best with story-style prompts
|
||||
- 512 token context window
|
||||
Reference in New Issue
Block a user