初始化项目,由ModelHub XC社区提供模型
Model: Thrillcrazyer/TACReward7B Source: Original Platform
This commit is contained in:
48
README.md
Normal file
48
README.md
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
|
||||
datasets: DeepMath-103k
|
||||
library_name: transformers
|
||||
model_name: TACReward7B
|
||||
licence: license
|
||||
---
|
||||
|
||||
<h1 align= "center"> Reasoning-Aware Proxy Reward Model using Process Mining </h1>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://pnubaelab.github.io/"><b>BAELAB</b></a>, Pusan National University, Busan, Korea
|
||||
</p>
|
||||
<p align="center">
|
||||
<a href="https://yongzzai.com/">Yongjae Lee</a><sup>*</sup>, <a href="https://thrillcrazyer.github.io/">Taekyhun Park</a><sup>*</sup> , Hyerim Bae<sup>†</sup>
|
||||
</p>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/Thrillcrazyer/TACReward"><b>🌟 Github</b></a> |
|
||||
<a href="https://huggingface.co/Thrillcrazyer/Qwen-1.5B_THIP"><b>📥 1.5B Download</b></a> |
|
||||
<a href="https://huggingface.co/Thrillcrazyer/TACReward7B"><b>📥 7B Download</b></a> |
|
||||
<a href="https://arxiv.org/abs/2510.25065"><b>📄 Arxiv Paper Link</b></a> |
|
||||
</p>
|
||||
|
||||
# Abstract
|
||||
|
||||
Recent advances in sparse reward policy gradient methods have enabled effective reinforcement learning (LR)
|
||||
fine-tuning for post-training language models. However, for reasoning tasks such as mathematical problem solving,
|
||||
binarized outcome rewards provide limited feedback on intermediate reasoning steps. While some studies have attempted
|
||||
to address this issue by estimating **overall** reasoning quality, it remains unclear whether these rewards are
|
||||
reliable proxies for the quality of stepwise reasoning. In this study, we consider reasoning as a structured process and
|
||||
propose **TACReward** reward model. The model can be seamlessly integrated into sparse reward frameworks without
|
||||
additional human annotation costs or architectural modifications. TACReward aggregates stepwise structural deviations
|
||||
between teachers and policy reasoning using process mining techniques, producing a scalar output reward range of $[0, 1]$.
|
||||
Experiments on multiple mathematical reasoning benchmarks demonstrate that integrating the TACReward into sparse reward
|
||||
frameworks encourages the policy model to improve the structural quality of reasoning. Consequently, this leads to
|
||||
consistent performance improvements over existing sparse reward frameworks.
|
||||
|
||||
# Illustration of TACReward
|
||||
|
||||
<div align="center">
|
||||
<img src="https://arxiv.org/html/2510.25065v1/x1.png" width="600"/>
|
||||
</div>
|
||||
Reference in New Issue
Block a user