初始化项目,由ModelHub XC社区提供模型
Model: Mirelle/opt-125M-pt-br-finetuned Source: Original Platform
This commit is contained in:
39
README.md
Normal file
39
README.md
Normal file
@@ -0,0 +1,39 @@
|
||||
---
|
||||
widget:
|
||||
- text: Em uma bela manhã de
|
||||
- text: Em uma cidade tão grande como
|
||||
- text: Maria e Joana são
|
||||
license: mit
|
||||
datasets:
|
||||
- mc4
|
||||
language:
|
||||
- pt
|
||||
metrics:
|
||||
- perplexity
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
# OPT-125M finetuned Portuguese
|
||||
|
||||
Fine-tuning the [OPT-125M](https://huggingface.co/facebook/opt-125m) model on a reduced corpus of mc4-Portuguese with approximately 300M tokens.
|
||||
|
||||
###### Hyper-parameters
|
||||
|
||||
|
||||
- learning_rate = 5e-5
|
||||
- batch_size = 32
|
||||
- warmup = 500
|
||||
- seq_length = 512
|
||||
- num_train_epochs = 2.0
|
||||
|
||||
With an A100 with 40GB of RAM, the training took around 3 hours
|
||||
|
||||
**Perplexity:** 9.4
|
||||
|
||||
## Sample Use
|
||||
|
||||
```python
|
||||
from transformers import pipeline
|
||||
generator = pipeline('text-generation', model='Mirelle/opt-125M-pt-br-finetuned', max_length=100, do_sample=True)
|
||||
generator("Em uma bela manhã de")
|
||||
```
|
||||
Reference in New Issue
Block a user