初始化项目,由ModelHub XC社区提供模型
Model: SlowGuess/ABForge-Qwen3-8B-Task1-SFT Source: Original Platform
This commit is contained in:
73
README.md
Normal file
73
README.md
Normal file
@@ -0,0 +1,73 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
base_model: Qwen/Qwen3-8B
|
||||
pipeline_tag: text-generation
|
||||
library_name: transformers
|
||||
tags:
|
||||
- ablation-study
|
||||
- scientific-reasoning
|
||||
- post-training
|
||||
- qwen3
|
||||
---
|
||||
|
||||
# ABForge-Qwen3-8B-Task1-SFT
|
||||
|
||||
An **ABForge** model for **Task 1: Ablation Objective Generation**.
|
||||
|
||||
ABForge is a post-training pipeline for paper-grounded ablation design. This checkpoint is
|
||||
supervised fine-tuned from `Qwen/Qwen3-8B` (**SFT**).
|
||||
|
||||
## Task
|
||||
|
||||
Given the ablation-free context of a research paper, the model proposes candidate ablation objectives, each expressed as a **Target Module** (the component to ablate) paired with a **Research Question** it is meant to answer.
|
||||
|
||||
## Training data
|
||||
|
||||
SFT on `train/sft_task1_45961.jsonl`, from [`SlowGuess/abforge-data`](https://huggingface.co/datasets/SlowGuess/abforge-data)
|
||||
(derived from CC-licensed research papers). Evaluation uses the held-out **AblationBench** split
|
||||
(`eval/ablationbench_200.jsonl`) of the same dataset.
|
||||
|
||||
## Related models (Task 1)
|
||||
|
||||
- [`SlowGuess/ABForge-Qwen3-8B-Task1`](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B-Task1)
|
||||
- [`SlowGuess/ABForge-Qwen3-8B-Task1-SFT`](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B-Task1-SFT) (this model)
|
||||
- [`SlowGuess/ABForge-Qwen3-8B-Task1-RL`](https://huggingface.co/SlowGuess/ABForge-Qwen3-8B-Task1-RL)
|
||||
|
||||
## Evaluation
|
||||
|
||||
Reproduce AblationBench evaluation with the [`SlowGuess/Abforge_1`](https://github.com/SlowGuess/Abforge_1) code:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/SlowGuess/Abforge_1 && cd Abforge_1
|
||||
huggingface-cli download SlowGuess/abforge-data --repo-type dataset --local-dir data
|
||||
|
||||
export MODEL_PATH=SlowGuess/ABForge-Qwen3-8B-Task1-SFT
|
||||
|
||||
# 1. Generate predictions on AblationBench
|
||||
python run_inference_local.py --task 1 \
|
||||
--input data/eval/ablationbench_200.jsonl \
|
||||
--output preds.jsonl \
|
||||
--model-path "$MODEL_PATH" --dtype bf16 --max-new-tokens 4096
|
||||
|
||||
# 2. Score against the fixed AblationBench rubric (Claude judge)
|
||||
export ANTHROPIC_API_KEY=<your-key>
|
||||
python scripts/eval_task1_claude.py --input preds.jsonl --output scored.jsonl
|
||||
```
|
||||
|
||||
## Links
|
||||
|
||||
- Dataset: [`SlowGuess/abforge-data`](https://huggingface.co/datasets/SlowGuess/abforge-data)
|
||||
- Code: [`SlowGuess/Abforge_1`](https://github.com/SlowGuess/Abforge_1)
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@misc{abforge,
|
||||
title = {ABForge: A Post-Training Pipeline for Paper-Grounded Ablation Design},
|
||||
author = {ABForge authors},
|
||||
year = {2026},
|
||||
howpublished = {\url{https://github.com/SlowGuess/Abforge_1}}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user