113 lines
4.5 KiB
Markdown
113 lines
4.5 KiB
Markdown
---
|
||
license: apache-2.0
|
||
language:
|
||
- en
|
||
base_model:
|
||
- Qwen/Qwen3-8B
|
||
datasets:
|
||
- SlowGuess/abforge-data
|
||
pipeline_tag: text-generation
|
||
library_name: transformers
|
||
tags:
|
||
- ablation-study
|
||
- scientific-reasoning
|
||
- post-training
|
||
- qwen3
|
||
---
|
||
|
||
# ABForge-Qwen3-8B-RL
|
||
|
||
<p align="center">
|
||
📄 <a href="https://arxiv.org/abs/XXXX.XXXXX">ArXiv</a>
|
||
 ｜  💻 <a href="https://github.com/SlowGuess/Abforge_1">Code</a>
|
||
 ｜  🤗 <a href="https://huggingface.co/collections/SlowGuess/abforge-6a2ac561d0e97f11e409dd75">Collection</a>
|
||
</p>
|
||
|
||
## About
|
||
|
||
This repository contains the **RL-only ablation** of **ABForge**, presented in *ABForge:
|
||
Post-Training for Paper-Grounded Ablation Design*. Given a paper's methodology with its
|
||
ablation content removed, ABForge proposes the ablation objectives the paper should investigate
|
||
and designs a rigorous experiment plan for each — both from a single checkpoint.
|
||
|
||
This model applies rubric-guided GRPO **directly to [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
|
||
for 200 updates with no SFT warm start**, on a 1:1 mixture of the two tasks with each rollout
|
||
routed to its task's reward by `data_source`. It is the *RL only* row of the paper's
|
||
post-training ablation; the released model
|
||
[ABForge-Qwen3-8B](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B) runs the same RL stage
|
||
from the SFT checkpoint instead.
|
||
|
||
## Links
|
||
|
||
| Resource | Link |
|
||
|---|---|
|
||
| Code | [SlowGuess/Abforge_1](https://github.com/SlowGuess/Abforge_1) |
|
||
| Training & evaluation data | [SlowGuess/abforge-data](https://huggingface.co/datasets/SlowGuess/abforge-data) |
|
||
| Released model (SFT → GRPO) | [SlowGuess/ABForge-Qwen3-8B](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B) |
|
||
| SFT checkpoint | [SlowGuess/ABForge-Qwen3-8B-SFT](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B-SFT) |
|
||
| Per-paper outputs & judge rationales | `outputs/task{1,2}/*/abforge-rl.jsonl` in the data repo |
|
||
|
||
## Training data
|
||
|
||
[SlowGuess/abforge-data](https://huggingface.co/datasets/SlowGuess/abforge-data) ships a
|
||
single table under `train/`, one row per paper, built by a semi-automated audit-in-the-loop
|
||
pipeline over research papers from major ML, NLP and CV venues. This model trains on the rows
|
||
flagged `in_rl_task1` and `in_rl_task2` — 30,000 papers each, disjoint from the SFT pool —
|
||
mixed 1:1. The benchmark papers carry no training flag, so they cannot leak in.
|
||
|
||
## Performance
|
||
|
||
AblationBench, automated rubric-based LLM-as-a-Judge evaluation (`eval/ablationbench_200.jsonl`,
|
||
200 papers, judge `claude-sonnet-4-6`). Task 1 is ablation objective identification
|
||
(`paper_score`); Task 2 is ablation plan synthesis (`design_score`, ×100).
|
||
|
||
| Model | Task 1 | Task 2 |
|
||
|---|---|---|
|
||
| [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) (base) | 44.4 | 43.4 |
|
||
| [ABForge-Qwen3-8B-SFT](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B-SFT) (SFT only) | 30.7 | 52.2 |
|
||
| **ABForge-Qwen3-8B-RL** (this model, RL only) | **52.2** | **54.9** |
|
||
| [ABForge-Qwen3-8B](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B) (SFT → GRPO) | 55.9 | 62.4 |
|
||
|
||
> GRPO alone already lifts both tasks over the base model, and unlike SFT it does not trade one
|
||
> for the other. Warm-starting the same RL stage from the SFT checkpoint is still worth +3.7 on
|
||
> Task 1 and +7.5 on Task 2 — SFT is an effective RL initialization even though it does not
|
||
> improve Task 1 on its own.
|
||
|
||
## Evaluation
|
||
|
||
Reproduce the numbers above with the [code release](https://github.com/SlowGuess/Abforge_1):
|
||
|
||
```bash
|
||
git clone https://github.com/SlowGuess/Abforge_1 && cd Abforge_1
|
||
huggingface-cli download SlowGuess/abforge-data --repo-type dataset \
|
||
--include "eval/*" --local-dir data
|
||
|
||
python run_inference_local.py --task 1 \
|
||
--input data/eval/ablationbench_200.jsonl \
|
||
--output outputs/task1_infer.jsonl \
|
||
--model-path SlowGuess/ABForge-Qwen3-8B-RL \
|
||
--dtype bf16 --device-map auto \
|
||
--max-new-tokens 5120 --temperature 0.0 --stop-on '</Result>'
|
||
|
||
export JUDGE_API_BASE=https://api.openai.com/v1
|
||
export JUDGE_API_KEY=...
|
||
export JUDGE_MODEL=...
|
||
scripts/evaluate_task1.sh outputs/task1_infer.jsonl
|
||
```
|
||
|
||
Swap `--task 2`, `--stop-on '</Proposed_Plan>'` and `scripts/evaluate_task2.sh` for Task 2. The
|
||
model is trained on the prompt templates in the code release and the rubric evaluator expects
|
||
the matching output structure, so use those templates and greedy decoding.
|
||
|
||
## Citation
|
||
|
||
<!-- TODO: replace with the final paper citation once available. -->
|
||
|
||
```bibtex
|
||
@misc{abforge2026,
|
||
title={ABForge: Post-Training for Paper-Grounded Ablation Design},
|
||
author={TODO},
|
||
year={2026},
|
||
}
|
||
```
|