初始化项目,由ModelHub XC社区提供模型
Model: Mayko2995/gpt2-quote-fr-v1.1 Source: Original Platform
This commit is contained in:
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
145
README.md
Normal file
145
README.md
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
---
|
||||||
|
library_name: transformers
|
||||||
|
license: apache-2.0
|
||||||
|
base_model: asi/gpt-fr-cased-small
|
||||||
|
tags:
|
||||||
|
- generated_from_trainer
|
||||||
|
model-index:
|
||||||
|
- name: gpt2-quote-fr-v1.1
|
||||||
|
results: []
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
---
|
||||||
|
|
||||||
|
# GPT-2 French Quote Generator 🇫🇷
|
||||||
|
|
||||||
|
A fine-tuned GPT-2 model for generating philosophical
|
||||||
|
and inspirational quotes in French.
|
||||||
|
|
||||||
|
## Model Description
|
||||||
|
|
||||||
|
This model is based on `asi/gpt-fr-cased-small` and has been
|
||||||
|
fine-tuned on a dataset of 975 carefully collected French quotes
|
||||||
|
covering various philosophical themes.
|
||||||
|
|
||||||
|
## Themes Covered
|
||||||
|
|
||||||
|
- ❤️ Love (Amour)
|
||||||
|
- 🌱 Life (Vie)
|
||||||
|
- 🌌 Existence
|
||||||
|
- 😊 Happiness (Bonheur)
|
||||||
|
- 🦉 Wisdom (Sagesse)
|
||||||
|
- ⏳ Time (Temps)
|
||||||
|
- 🕊️ Freedom (Liberté)
|
||||||
|
- 🙏 Spirituality (Spiritualité)
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
```python
|
||||||
|
from transformers import pipeline
|
||||||
|
|
||||||
|
generator = pipeline(
|
||||||
|
"text-generation",
|
||||||
|
model="Mayko2995/gpt2-quote-fr-v1.1"
|
||||||
|
)
|
||||||
|
|
||||||
|
result = generator(
|
||||||
|
"Le bonheur est",
|
||||||
|
max_new_tokens=20,
|
||||||
|
temperature=0.8,
|
||||||
|
do_sample=True,
|
||||||
|
repetition_penalty=2.0,
|
||||||
|
no_repeat_ngram_size=3
|
||||||
|
)
|
||||||
|
|
||||||
|
print(result[0]["generated_text"])
|
||||||
|
```
|
||||||
|
|
||||||
|
## Generated Examples
|
||||||
|
```
|
||||||
|
"Le bonheur est une petite chose que l'on invente
|
||||||
|
avec la sagesse de chaque instant."
|
||||||
|
|
||||||
|
"Dans l'existence, il faut avoir une âme pour
|
||||||
|
comprendre que tout existe."
|
||||||
|
|
||||||
|
"La vie est un voyage vers son avenir qu'il nous
|
||||||
|
faudra parcourir avec sagesse et humilité."
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dataset
|
||||||
|
|
||||||
|
- 975 French quotes
|
||||||
|
- Collected from Wikiquote and other sources
|
||||||
|
- Various authors including Victor Hugo, Albert Camus,
|
||||||
|
Pablo Neruda, Einstein and others
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- Relatively small dataset — 975 quotes
|
||||||
|
- May generate incomplete sentences
|
||||||
|
- Best results with max_new_tokens between 15 and 25
|
||||||
|
- Designed for French language only
|
||||||
|
|
||||||
|
## Intended Uses
|
||||||
|
|
||||||
|
✅ Generating inspirational French quotes
|
||||||
|
✅ Creative writing assistance
|
||||||
|
✅ Educational NLP projects
|
||||||
|
✅ French language generation research
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
❌ Factual information generation
|
||||||
|
❌ Translation tasks
|
||||||
|
❌ Other languages
|
||||||
|
|
||||||
|
## Author
|
||||||
|
|
||||||
|
**Jonasson Claubert Joachin** — Self-taught AI Engineer 🇭🇹
|
||||||
|
Based in Haiti — passionate about NLP and LLMs
|
||||||
|
|
||||||
|
*Building AI tools adapted to the Haitian context*
|
||||||
|
|
||||||
|
## Versions
|
||||||
|
|
||||||
|
| Version | Dataset | Val Loss |
|
||||||
|
|---------|---------|----------|
|
||||||
|
| v1.0 | 300 quotes | 1.12 |
|
||||||
|
| v1.1 | 975 quotes | 0.71 ✅ |
|
||||||
|
|
||||||
|
## Training procedure
|
||||||
|
|
||||||
|
### Training hyperparameters
|
||||||
|
|
||||||
|
The following hyperparameters were used during training:
|
||||||
|
- learning_rate: 2e-05
|
||||||
|
- train_batch_size: 2
|
||||||
|
- eval_batch_size: 8
|
||||||
|
- seed: 42
|
||||||
|
- gradient_accumulation_steps: 8
|
||||||
|
- total_train_batch_size: 16
|
||||||
|
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
||||||
|
- lr_scheduler_type: linear
|
||||||
|
- num_epochs: 10
|
||||||
|
|
||||||
|
### Training results
|
||||||
|
|
||||||
|
| Training Loss | Epoch | Step | Validation Loss |
|
||||||
|
|:-------------:|:-----:|:----:|:---------------:|
|
||||||
|
| 1.5437 | 1.0 | 49 | 1.2013 |
|
||||||
|
| 1.0311 | 2.0 | 98 | 0.9540 |
|
||||||
|
| 0.8106 | 3.0 | 147 | 0.8370 |
|
||||||
|
| 0.7146 | 4.0 | 196 | 0.7859 |
|
||||||
|
| 0.6868 | 5.0 | 245 | 0.7580 |
|
||||||
|
| 0.5707 | 6.0 | 294 | 0.7380 |
|
||||||
|
| 0.5532 | 7.0 | 343 | 0.7201 |
|
||||||
|
| 0.5705 | 8.0 | 392 | 0.7145 |
|
||||||
|
| 0.5311 | 9.0 | 441 | 0.7129 |
|
||||||
|
| 0.5057 | 10.0 | 490 | 0.7113 |
|
||||||
|
|
||||||
|
|
||||||
|
### Framework versions
|
||||||
|
|
||||||
|
- Transformers 5.0.0
|
||||||
|
- Pytorch 2.10.0+cu128
|
||||||
|
- Datasets 4.0.0
|
||||||
|
- Tokenizers 0.22.2
|
||||||
35
config.json
Normal file
35
config.json
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
{
|
||||||
|
"activation_function": "gelu_new",
|
||||||
|
"add_cross_attention": false,
|
||||||
|
"architectures": [
|
||||||
|
"GPT2LMHeadModel"
|
||||||
|
],
|
||||||
|
"attn_pdrop": 0.1,
|
||||||
|
"bos_token_id": 0,
|
||||||
|
"dtype": "float32",
|
||||||
|
"embd_pdrop": 0.1,
|
||||||
|
"eos_token_id": 2,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"layer_norm_epsilon": 1e-05,
|
||||||
|
"model_type": "gpt2",
|
||||||
|
"n_ctx": 1024,
|
||||||
|
"n_embd": 768,
|
||||||
|
"n_head": 12,
|
||||||
|
"n_inner": null,
|
||||||
|
"n_layer": 12,
|
||||||
|
"n_positions": 1024,
|
||||||
|
"pad_token_id": 2,
|
||||||
|
"reorder_and_upcast_attn": false,
|
||||||
|
"resid_pdrop": 0.1,
|
||||||
|
"scale_attn_by_inverse_layer_idx": false,
|
||||||
|
"scale_attn_weights": true,
|
||||||
|
"summary_activation": null,
|
||||||
|
"summary_first_dropout": 0.1,
|
||||||
|
"summary_proj_to_labels": true,
|
||||||
|
"summary_type": "cls_index",
|
||||||
|
"summary_use_proj": true,
|
||||||
|
"tie_word_embeddings": true,
|
||||||
|
"transformers_version": "5.0.0",
|
||||||
|
"use_cache": false,
|
||||||
|
"vocab_size": 50000
|
||||||
|
}
|
||||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"_from_model_config": true,
|
||||||
|
"bos_token_id": 0,
|
||||||
|
"eos_token_id": [
|
||||||
|
2
|
||||||
|
],
|
||||||
|
"pad_token_id": 2,
|
||||||
|
"transformers_version": "5.0.0"
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:565241f2f5c551696e8e191f1679e9c4c61bc820ed9c2472442f36a8aedb7a3d
|
||||||
|
size 650584912
|
||||||
249291
tokenizer.json
Normal file
249291
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
13
tokenizer_config.json
Normal file
13
tokenizer_config.json
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"backend": "tokenizers",
|
||||||
|
"bos_token": "<s>",
|
||||||
|
"eos_token": "</s>",
|
||||||
|
"errors": "replace",
|
||||||
|
"is_local": false,
|
||||||
|
"mask_token": "<mask>",
|
||||||
|
"model_max_length": 1000000000000000019884624838656,
|
||||||
|
"pad_token": "</s>",
|
||||||
|
"tokenizer_class": "GPT2Tokenizer",
|
||||||
|
"unk_token": "<unk>"
|
||||||
|
}
|
||||||
3
training_args.bin
Normal file
3
training_args.bin
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:4ce9d583ae76166ef8994ac7570ce009a88f4ba26bc64393043e21fb1b631098
|
||||||
|
size 5201
|
||||||
Reference in New Issue
Block a user