31 lines
461 B
Markdown
31 lines
461 B
Markdown
---
|
|
license: mit
|
|
datasets:
|
|
- oscar
|
|
language:
|
|
- uk
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# GPT2 Ukrainian
|
|
|
|
A generative language model for the Ukrainian language follows the [GPT-2 architecture](https://huggingface.co/gpt2) (124M parameters).
|
|
|
|
- hidden size: 768
|
|
- number of heads: 12
|
|
- number of layers: 12
|
|
- seq length: 1024
|
|
- tokens: 11238113280 (3 epochs)
|
|
- steps: 57167
|
|
|
|
## Training data
|
|
|
|
- OSCAR
|
|
- Wikimedia dumps
|
|
|
|
## License
|
|
|
|
MIT
|
|
|