41 lines
1.2 KiB
Markdown
41 lines
1.2 KiB
Markdown
|
---
|
||
|
|
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
|
||
|
|
library_name: transformers
|
||
|
|
tags:
|
||
|
|
- text-generation
|
||
|
|
- sft
|
||
|
|
- lora
|
||
|
|
- tinyllama
|
||
|
|
---
|
||
|
|
|
||
|
|
# Best SFT Model — SFT-T4
|
||
|
|
|
||
|
|
TinyLlama-1.1B-Chat-v1.0 fine-tuned via Supervised Fine-Tuning (SFT) on
|
||
|
|
[databricks/databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k).
|
||
|
|
|
||
|
|
## Authors
|
||
|
|
- Ahmad Murtaza (24478)
|
||
|
|
- Simra Sheikh (27135)
|
||
|
|
|
||
|
|
## Project Context
|
||
|
|
This model was developed as part of Assignment 04 for the NLP with Deep Learning
|
||
|
|
course at the Institute of Business Administration (IBA). The assignment involves
|
||
|
|
building a full SFT → DPO fine-tuning pipeline on TinyLlama-1.1B-Chat-v1.0 and
|
||
|
|
evaluating the impact of instruction tuning and preference alignment.
|
||
|
|
|
||
|
|
## Training config
|
||
|
|
```
|
||
|
|
trial : SFT-T4
|
||
|
|
lora_rank : 16
|
||
|
|
lora_alpha : 32
|
||
|
|
target_modules: ['q_proj', 'k_proj', 'v_proj', 'o_proj']
|
||
|
|
learning_rate : 0.0002
|
||
|
|
epochs : 1
|
||
|
|
val_loss : 1.7798
|
||
|
|
```
|
||
|
|
|
||
|
|
## Evaluation (10-prompt test set, merged model)
|
||
|
|
| Model | BLEU-4 | BERTScore F1 |
|
||
|
|
| ------------------------------- | ------ | ------------ |
|
||
|
|
| Base (TinyLlama-1.1B) | 2.1400 | 85.4700 |
|
||
|
|
| **Best SFT (SFT-T4)** | **2.4200** | **87.1100** |
|