145 lines
3.4 KiB
Markdown
145 lines
3.4 KiB
Markdown
|
---
|
||
|
|
library_name: transformers
|
||
|
|
license: apache-2.0
|
||
|
|
base_model: asi/gpt-fr-cased-small
|
||
|
|
tags:
|
||
|
|
- generated_from_trainer
|
||
|
|
model-index:
|
||
|
|
- name: gpt2-quote-fr-v1.1
|
||
|
|
results: []
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# GPT-2 French Quote Generator 🇫🇷
|
||
|
|
|
||
|
|
A fine-tuned GPT-2 model for generating philosophical
|
||
|
|
and inspirational quotes in French.
|
||
|
|
|
||
|
|
## Model Description
|
||
|
|
|
||
|
|
This model is based on `asi/gpt-fr-cased-small` and has been
|
||
|
|
fine-tuned on a dataset of 975 carefully collected French quotes
|
||
|
|
covering various philosophical themes.
|
||
|
|
|
||
|
|
## Themes Covered
|
||
|
|
|
||
|
|
- ❤️ Love (Amour)
|
||
|
|
- 🌱 Life (Vie)
|
||
|
|
- 🌌 Existence
|
||
|
|
- 😊 Happiness (Bonheur)
|
||
|
|
- 🦉 Wisdom (Sagesse)
|
||
|
|
- ⏳ Time (Temps)
|
||
|
|
- 🕊️ Freedom (Liberté)
|
||
|
|
- 🙏 Spirituality (Spiritualité)
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
```python
|
||
|
|
from transformers import pipeline
|
||
|
|
|
||
|
|
generator = pipeline(
|
||
|
|
"text-generation",
|
||
|
|
model="Mayko2995/gpt2-quote-fr-v1.1"
|
||
|
|
)
|
||
|
|
|
||
|
|
result = generator(
|
||
|
|
"Le bonheur est",
|
||
|
|
max_new_tokens=20,
|
||
|
|
temperature=0.8,
|
||
|
|
do_sample=True,
|
||
|
|
repetition_penalty=2.0,
|
||
|
|
no_repeat_ngram_size=3
|
||
|
|
)
|
||
|
|
|
||
|
|
print(result[0]["generated_text"])
|
||
|
|
```
|
||
|
|
|
||
|
|
## Generated Examples
|
||
|
|
```
|
||
|
|
"Le bonheur est une petite chose que l'on invente
|
||
|
|
avec la sagesse de chaque instant."
|
||
|
|
|
||
|
|
"Dans l'existence, il faut avoir une âme pour
|
||
|
|
comprendre que tout existe."
|
||
|
|
|
||
|
|
"La vie est un voyage vers son avenir qu'il nous
|
||
|
|
faudra parcourir avec sagesse et humilité."
|
||
|
|
```
|
||
|
|
|
||
|
|
## Dataset
|
||
|
|
|
||
|
|
- 975 French quotes
|
||
|
|
- Collected from Wikiquote and other sources
|
||
|
|
- Various authors including Victor Hugo, Albert Camus,
|
||
|
|
Pablo Neruda, Einstein and others
|
||
|
|
|
||
|
|
## Limitations
|
||
|
|
|
||
|
|
- Relatively small dataset — 975 quotes
|
||
|
|
- May generate incomplete sentences
|
||
|
|
- Best results with max_new_tokens between 15 and 25
|
||
|
|
- Designed for French language only
|
||
|
|
|
||
|
|
## Intended Uses
|
||
|
|
|
||
|
|
✅ Generating inspirational French quotes
|
||
|
|
✅ Creative writing assistance
|
||
|
|
✅ Educational NLP projects
|
||
|
|
✅ French language generation research
|
||
|
|
|
||
|
|
## Out of Scope
|
||
|
|
|
||
|
|
❌ Factual information generation
|
||
|
|
❌ Translation tasks
|
||
|
|
❌ Other languages
|
||
|
|
|
||
|
|
## Author
|
||
|
|
|
||
|
|
**Jonasson Claubert Joachin** — Self-taught AI Engineer 🇭🇹
|
||
|
|
Based in Haiti — passionate about NLP and LLMs
|
||
|
|
|
||
|
|
*Building AI tools adapted to the Haitian context*
|
||
|
|
|
||
|
|
## Versions
|
||
|
|
|
||
|
|
| Version | Dataset | Val Loss |
|
||
|
|
|---------|---------|----------|
|
||
|
|
| v1.0 | 300 quotes | 1.12 |
|
||
|
|
| v1.1 | 975 quotes | 0.71 ✅ |
|
||
|
|
|
||
|
|
## Training procedure
|
||
|
|
|
||
|
|
### Training hyperparameters
|
||
|
|
|
||
|
|
The following hyperparameters were used during training:
|
||
|
|
- learning_rate: 2e-05
|
||
|
|
- train_batch_size: 2
|
||
|
|
- eval_batch_size: 8
|
||
|
|
- seed: 42
|
||
|
|
- gradient_accumulation_steps: 8
|
||
|
|
- total_train_batch_size: 16
|
||
|
|
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
||
|
|
- lr_scheduler_type: linear
|
||
|
|
- num_epochs: 10
|
||
|
|
|
||
|
|
### Training results
|
||
|
|
|
||
|
|
| Training Loss | Epoch | Step | Validation Loss |
|
||
|
|
|:-------------:|:-----:|:----:|:---------------:|
|
||
|
|
| 1.5437 | 1.0 | 49 | 1.2013 |
|
||
|
|
| 1.0311 | 2.0 | 98 | 0.9540 |
|
||
|
|
| 0.8106 | 3.0 | 147 | 0.8370 |
|
||
|
|
| 0.7146 | 4.0 | 196 | 0.7859 |
|
||
|
|
| 0.6868 | 5.0 | 245 | 0.7580 |
|
||
|
|
| 0.5707 | 6.0 | 294 | 0.7380 |
|
||
|
|
| 0.5532 | 7.0 | 343 | 0.7201 |
|
||
|
|
| 0.5705 | 8.0 | 392 | 0.7145 |
|
||
|
|
| 0.5311 | 9.0 | 441 | 0.7129 |
|
||
|
|
| 0.5057 | 10.0 | 490 | 0.7113 |
|
||
|
|
|
||
|
|
|
||
|
|
### Framework versions
|
||
|
|
|
||
|
|
- Transformers 5.0.0
|
||
|
|
- Pytorch 2.10.0+cu128
|
||
|
|
- Datasets 4.0.0
|
||
|
|
- Tokenizers 0.22.2
|