初始化项目，由ModelHub XC社区提供模型

Model: Raghav-Singhal/dpo-tulu3-lr5e-7-tulu3sft-100B-no-bad-data-off-policy-if
Source: Original Platform

README.md

@@ -0,0 +1,56 @@
+---
+library_name: transformers
+pipeline_tag: text-generation
+model_name: dpo-tulu3-lr5e-7-tulu3sft-100B-no-bad-data-off-policy-if
+base_model: tulu3sft-normal-smollm-1p7b-100B-20n-2048sl-960gbsz-no-bad-data
+tags:
+- dpo
+- trl
+- smollm2
+- llama
+- conversational
+license: other
+---
+
+# Model Card for dpo-tulu3-lr5e-7-tulu3sft-100B-no-bad-data-off-policy-if
+
+This repository contains a DPO fine-tune of the local SFT checkpoint
+`tulu3sft-normal-smollm-1p7b-100B-20n-2048sl-960gbsz-no-bad-data`.
+
+The final model weights are stored at the repository root. Intermediate
+training checkpoints are also included under `checkpoint-500`, `checkpoint-1000`,
+and `checkpoint-1270`.
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline(
+    "text-generation",
+    model="Raghav-Singhal/dpo-tulu3-lr5e-7-tulu3sft-100B-no-bad-data-off-policy-if",
+    device="cuda",
+)
+output = generator(
+    [{"role": "user", "content": question}],
+    max_new_tokens=128,
+    return_full_text=False,
+)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/raghav_singhal/model-raising-dpo/runs/ykqcsy7f)
+
+This model was trained with DPO, a method introduced in
+[Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://huggingface.co/papers/2305.18290).
+
+### Framework versions
+
+- TRL: 1.0.0
+- Transformers: 4.57.6
+- Pytorch: 2.10.0a0+b4e4ee81d3.nv25.12
+- Datasets: 4.8.4
+- Tokenizers: 0.22.1