83 lines
2.9 KiB
Markdown
83 lines
2.9 KiB
Markdown
---
|
|
base_model: unsloth/Qwen3-4B-Instruct-2507
|
|
datasets:
|
|
- u-10bei/structured_data_with_cot_dataset_512_v2
|
|
- u-10bei/structured_data_with_cot_dataset_512_v4
|
|
- u-10bei/structured_data_with_cot_dataset_512_v5
|
|
- u-10bei/structured_data_with_cot_dataset_512
|
|
- u-10bei/structured_data_with_cot_dataset_v2
|
|
- u-10bei/structured_data_with_cot_dataset
|
|
- u-10bei/dpo-dataset-qwen-cot
|
|
language:
|
|
- en
|
|
license: apache-2.0
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- sft
|
|
- dpo
|
|
- unsloth
|
|
- qwen
|
|
- reasoning
|
|
- structured-data
|
|
---
|
|
|
|
# Qwen3-4B-StructEval-Iterative-Alignment
|
|
|
|
This model is a highly optimized version of **Qwen3-4B-Instruct-2507**, specifically developed for the **StructEval-T** task. It features a sophisticated iterative alignment process to maximize performance in structured data reasoning and Chain-of-Thought (CoT) generation.
|
|
|
|
This repository contains **full-merged 16-bit weights**. No adapter loading is required.
|
|
|
|
## Training Pipeline
|
|
Unlike standard fine-tuning, this model has undergone a **four-stage iterative training process** to ensure precise alignment and deep reasoning capabilities:
|
|
|
|
1. **Stage 1: SFT** - Foundation building with structured CoT trajectories.
|
|
2. **Stage 2: DPO** - First alignment to preference reasoning paths.
|
|
3. **Stage 3: SFT** - Knowledge reinforcement and format refinement.
|
|
4. **Stage 4: DPO** - Final preference optimization for high-fidelity structured outputs.
|
|
|
|
## Training Objective
|
|
The model is engineered to excel in:
|
|
- **Complex Reasoning**: Enhanced Chain-of-Thought processing for structural evaluation.
|
|
- **Structural Integrity**: Strict adherence to complex data formats (JSON, Tables, etc.).
|
|
- **Consistency**: High-reliability outputs across iterative multi-turn interactions.
|
|
|
|
## Training Configuration (Final Stage)
|
|
- **Method**: Iterative DPO (Direct Preference Optimization)
|
|
- **Base model**: unsloth/Qwen3-4B-Instruct-2507
|
|
- **Epochs**: 1
|
|
- **Learning rate**: 3e-06
|
|
- **Beta**: 0.05
|
|
- **Max sequence length**: 2560
|
|
- **Platform**: Trained with Unsloth
|
|
|
|
## Usage
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
import torch
|
|
|
|
model_id = "moushi21/dpo-qwen-cot-merged20"
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_id,
|
|
torch_dtype=torch.float16,
|
|
device_map="auto"
|
|
)
|
|
```
|
|
|
|
## Sources & Terms (IMPORTANT)
|
|
|
|
Training data:
|
|
- u-10bei/structured_data_with_cot_dataset_512_v2
|
|
- u-10bei/structured_data_with_cot_dataset_512_v4
|
|
- u-10bei/structured_data_with_cot_dataset_512_v5
|
|
- u-10bei/structured_data_with_cot_dataset_512
|
|
- u-10bei/structured_data_with_cot_dataset_v2
|
|
- u-10bei/structured_data_with_cot_dataset
|
|
- u-10bei/dpo-dataset-qwen-cot
|
|
|
|
Dataset License: MIT License. This dataset is used and distributed under the terms of the MIT License.
|
|
Compliance: Users must comply with the MIT license (including copyright notice) and the base model's original terms of use.
|