55 lines
1.6 KiB
Markdown
55 lines
1.6 KiB
Markdown
|
---
|
||
|
|
base_model: Qwen/Qwen3-0.6B
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
library_name: transformers
|
||
|
|
license: apache-2.0
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
datasets:
|
||
|
|
- Jingleqian/AAPA-data
|
||
|
|
tags:
|
||
|
|
- alignment
|
||
|
|
- grpo
|
||
|
|
- aapa
|
||
|
|
- qwen
|
||
|
|
- instruction-following
|
||
|
|
---
|
||
|
|
|
||
|
|
# AAPA-06B
|
||
|
|
|
||
|
|
This repository contains the 0.6B A-GRPO checkpoint released with **AAPA: Adversarially Anchored Preference Alignment for Post-Training of Large Language Models**.
|
||
|
|
|
||
|
|
AAPA is a plug-in framework that augments post-training objectives with a sentence-level adversarial anchoring signal. It compares policy rollouts with offline expert responses using a fixed lightweight discriminator, providing semantic grounding during preference optimization.
|
||
|
|
|
||
|
|
This checkpoint is trained from [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) using the AAPA code release.
|
||
|
|
|
||
|
|
## Resources
|
||
|
|
|
||
|
|
- Paper: https://arxiv.org/abs/2509.25148
|
||
|
|
- Hugging Face Paper: https://huggingface.co/papers/2509.25148
|
||
|
|
- Code: https://github.com/IsFaqq/AAPA
|
||
|
|
- Data: https://huggingface.co/datasets/Jingleqian/AAPA-data
|
||
|
|
- 8B checkpoint: https://huggingface.co/Jingleqian/AAPA-8B
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
model_id = "Jingleqian/AAPA-06B"
|
||
|
|
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id)
|
||
|
|
```
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@article{aapa2025,
|
||
|
|
title={AAPA: Adversarially Anchored Preference Alignment for Post-Training of Large Language Models},
|
||
|
|
author={Faqiang Qian and Kang An and Weikun Zhang and Ziliang Wang and Xuhui Zheng and Liangjian Wen and Yong Dai and Mengya Gao and Yichao Wu},
|
||
|
|
journal={arXiv preprint arXiv:2509.25148},
|
||
|
|
year={2025}
|
||
|
|
}
|
||
|
|
```
|