初始化项目，由ModelHub XC社区提供模型

Model: moushi21/dpo-qwen-cot-merged20
Source: Original Platform

README.md

@@ -0,0 +1,82 @@
+---
+base_model: unsloth/Qwen3-4B-Instruct-2507
+datasets:
+- u-10bei/structured_data_with_cot_dataset_512_v2
+- u-10bei/structured_data_with_cot_dataset_512_v4
+- u-10bei/structured_data_with_cot_dataset_512_v5
+- u-10bei/structured_data_with_cot_dataset_512
+- u-10bei/structured_data_with_cot_dataset_v2
+- u-10bei/structured_data_with_cot_dataset
+- u-10bei/dpo-dataset-qwen-cot
+language:
+- en
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- sft
+- dpo
+- unsloth
+- qwen
+- reasoning
+- structured-data
+---
+
+# Qwen3-4B-StructEval-Iterative-Alignment
+
+This model is a highly optimized version of **Qwen3-4B-Instruct-2507**, specifically developed for the **StructEval-T** task. It features a sophisticated iterative alignment process to maximize performance in structured data reasoning and Chain-of-Thought (CoT) generation.
+
+This repository contains **full-merged 16-bit weights**. No adapter loading is required.
+
+## Training Pipeline
+Unlike standard fine-tuning, this model has undergone a **four-stage iterative training process** to ensure precise alignment and deep reasoning capabilities:
+
+1. **Stage 1: SFT** - Foundation building with structured CoT trajectories.
+2. **Stage 2: DPO** - First alignment to preference reasoning paths.
+3. **Stage 3: SFT** - Knowledge reinforcement and format refinement.
+4. **Stage 4: DPO** - Final preference optimization for high-fidelity structured outputs.
+
+## Training Objective
+The model is engineered to excel in:
+- **Complex Reasoning**: Enhanced Chain-of-Thought processing for structural evaluation.
+- **Structural Integrity**: Strict adherence to complex data formats (JSON, Tables, etc.).
+- **Consistency**: High-reliability outputs across iterative multi-turn interactions.
+
+## Training Configuration (Final Stage)
+- **Method**: Iterative DPO (Direct Preference Optimization)
+- **Base model**: unsloth/Qwen3-4B-Instruct-2507
+- **Epochs**: 1
+- **Learning rate**: 3e-06
+- **Beta**: 0.05
+- **Max sequence length**: 2560
+- **Platform**: Trained with Unsloth
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_id = "moushi21/dpo-qwen-cot-merged20"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.float16,
+    device_map="auto"
+)
+```
+
+## Sources & Terms (IMPORTANT)
+
+Training data:
+- u-10bei/structured_data_with_cot_dataset_512_v2
+- u-10bei/structured_data_with_cot_dataset_512_v4
+- u-10bei/structured_data_with_cot_dataset_512_v5
+- u-10bei/structured_data_with_cot_dataset_512
+- u-10bei/structured_data_with_cot_dataset_v2
+- u-10bei/structured_data_with_cot_dataset
+- u-10bei/dpo-dataset-qwen-cot
+
+Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License.
+Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.