32 lines
945 B
Markdown
32 lines
945 B
Markdown
---
|
|
library_name: transformers
|
|
base_model: Qwen/Qwen3-8B
|
|
license: other
|
|
tags:
|
|
- ronpo
|
|
- mnpo
|
|
- dpo
|
|
- qwen3
|
|
- preference-optimization
|
|
datasets:
|
|
- UltraFeedback
|
|
---
|
|
|
|
# qwen3-8b-dpo-avg-beta0p05-s42
|
|
|
|
This is a research checkpoint for the RONPO AAAI revision experiments.
|
|
|
|
- Method: DPO-avg on the averaged three-reward oracle
|
|
- Base model: `Qwen/Qwen3-8B`, non-thinking mode
|
|
- Training seed: 42
|
|
- DPO beta: 0.05
|
|
- Data split: the existing MNPO/RONPO UltraFeedback split
|
|
- Oracle construction: per-prompt min-max normalization over
|
|
`Skywork/Skywork-Reward-V2-Llama-3.1-8B`,
|
|
`Nexusflow/Athene-RM-8B`, and `RLHFlow/ArmoRM-Llama3-8B-v0.1`,
|
|
followed by an unweighted average
|
|
- Local source checkpoint at upload time: `/ext_hdd/sjkim/mnpo/revision_qwen3_8b/full_iter1/train/dpo_avg_beta0p05_s42_odin2`
|
|
|
|
Intended use: reproducibility and evaluation for the RONPO research paper.
|
|
This model is not intended as a general-purpose production assistant.
|