初始化项目，由ModelHub XC社区提供模型

Model: ahmadhehe/tinyllama-1.1b-dpo-hh-rlhf
Source: Original Platform

README.md

@@ -0,0 +1,43 @@
+---
+base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
+library_name: transformers
+tags:
+  - text-generation
+  - dpo
+  - rlhf
+  - lora
+  - tinyllama
+---
+
+# Best DPO Model — DPO-D4
+
+TinyLlama-1.1B-Chat-v1.0 first SFT-tuned on dolly-15k, then aligned via
+Direct Preference Optimization (DPO) on
+[Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf).
+
+## Authors
+- Ahmad Murtaza (24478)
+- Simra Sheikh (27135)
+
+## Project Context
+This model was developed as part of Assignment 04 for the NLP with Deep Learning
+course at the Institute of Business Administration (IBA). The assignment involves
+building a full SFT → DPO fine-tuning pipeline on TinyLlama-1.1B-Chat-v1.0 and
+evaluating the impact of instruction tuning and preference alignment.
+
+## Training config
+```
+trial         : DPO-D4
+sft_base      : SFT-T4
+beta          : 0.5
+learning_rate : 5e-05
+epochs        : 1
+val_loss      : 0.7304
+```
+
+## Evaluation (10-prompt test set, merged models)
+| Model                           | BLEU-4 | BERTScore F1 |
+| ------------------------------- | ------ | ------------ |
+| Base (TinyLlama-1.1B)           | 2.1400 | 85.4700 |
+| Best SFT (SFT-T4)     | 2.4200 | 87.1100 |
+| **Best DPO (DPO-D4)** | **4.2200** | **86.9600** |