初始化项目,由ModelHub XC社区提供模型
Model: EhabSuliman/my_awesome_eli5_clm-model Source: Original Platform
This commit is contained in:
101
README.md
Normal file
101
README.md
Normal file
@@ -0,0 +1,101 @@
|
||||
---
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
base_model: distilbert/distilgpt2
|
||||
tags:
|
||||
- text-generation
|
||||
- causal-lm
|
||||
- arabic
|
||||
- fine-tuned
|
||||
- generated_from_trainer
|
||||
model-index:
|
||||
- name: my_awesome_eli5_clm-model
|
||||
results: []
|
||||
---
|
||||
|
||||
# my_awesome_eli5_clm-model
|
||||
|
||||
A Causal Language Model fine-tuned on Arabic text, based on [distilbert/distilgpt2](https://huggingface.co/distilbert/distilgpt2).
|
||||
Fine-tuned by **EhabSuliman** as part of an LLM course project.
|
||||
|
||||
## Model Description
|
||||
|
||||
- **Model type:** Causal Language Model (CLM)
|
||||
- **Base model:** [distilbert/distilgpt2](https://huggingface.co/distilbert/distilgpt2)
|
||||
- **Language:** English
|
||||
- **Fine-tuned by:** [EhabSuliman](https://huggingface.co/EhabSuliman)
|
||||
- **License:** Apache 2.0
|
||||
|
||||
## Intended Uses & Limitations
|
||||
|
||||
**Intended uses:**
|
||||
- Arabic text generation
|
||||
- Language modeling research
|
||||
- Educational/learning purposes
|
||||
|
||||
**Limitations:**
|
||||
- Trained on a relatively small dataset
|
||||
- Loss is still relatively high (3.8027), meaning the model may generate inaccurate or repetitive text
|
||||
- Not recommended for production use without further fine-tuning
|
||||
|
||||
## How to Use
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained("EhabSuliman/my_awesome_eli5_clm-model")
|
||||
model = AutoModelForCausalLM.from_pretrained("EhabSuliman/my_awesome_eli5_clm-model")
|
||||
|
||||
prompt = "Somatic hypermutation allows the immune system to"
|
||||
inputs = tokenizer(prompt, return_tensors="pt").input_ids
|
||||
outputs = model.generate(
|
||||
inputs,
|
||||
max_new_tokens=100,
|
||||
do_sample=True,
|
||||
top_k=50,
|
||||
top_p=0.95
|
||||
)
|
||||
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Training and Evaluation Data
|
||||
|
||||
Fine-tuned on an Arabic text dataset using the ELI5 (Explain Like I'm 5) format.
|
||||
|
||||
## Training Procedure
|
||||
|
||||
### Training Hyperparameters
|
||||
|
||||
The following hyperparameters were used during training:
|
||||
|
||||
- **Learning rate:** 2e-05
|
||||
- **Train batch size:** 8
|
||||
- **Eval batch size:** 8
|
||||
- **Seed:** 42
|
||||
- **Optimizer:** AdamW (fused) with betas=(0.9, 0.999), epsilon=1e-08
|
||||
- **LR scheduler:** Linear
|
||||
- **Epochs:** 3
|
||||
|
||||
### Training Results
|
||||
|
||||
| Training Loss | Epoch | Step | Validation Loss |
|
||||
|:-------------:|:-----:|:----:|:---------------:|
|
||||
| 3.8556 | 1.0 | 1327 | 3.8101 |
|
||||
| 3.7851 | 2.0 | 2654 | 3.8035 |
|
||||
| 3.7514 | 3.0 | 3981 | 3.8027 |
|
||||
|
||||
> The model shows steady improvement across epochs with validation loss decreasing from 3.8101 → 3.8027.
|
||||
|
||||
### Framework Versions
|
||||
|
||||
- Transformers 5.9.0
|
||||
- PyTorch 2.11.0+cu128
|
||||
- Datasets 4.0.0
|
||||
- Tokenizers 0.22.2
|
||||
|
||||
## Author
|
||||
|
||||
**Ehab Suliman** — Machine Learning Engineer
|
||||
🔗 [HuggingFace Profile](https://huggingface.co/EhabSuliman)
|
||||
Reference in New Issue
Block a user