15 KiB
base_model, library_name, pipeline_tag, datasets, tags, license, language, metrics
| base_model | library_name | pipeline_tag | datasets | tags | license | language | metrics | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen/Qwen3-8B-Base | transformers | text-generation |
|
|
cc-by-nc-4.0 |
|
|
Qwen3-8B-ODA-Math-460k
Qwen3-8B-ODA-Math-460k is a supervised fine-tuned (SFT) model built on top of Qwen3-8B-Base, trained with ODA-Math-460k.
ODA-Math-460k is a large-scale math reasoning dataset curated from top-performing open mathematics corpora (selected via the OpenDataArena leaderboard) and refined through deduplication, benchmark decontamination, LLM-based filtering, and verifier-backed response distillation.
It targets a “learnable but challenging” difficulty band: non-trivial for smaller models yet solvable by stronger reasoning models.
🧠 Model Summary
- Base Model:
Qwen/Qwen3-8B-Base - Training Data:
OpenDataArena/ODA-Math-460k - Domain Coverage: Mathematics (strictly filtered)
- Scale (selected training set): ~460K problems (after selection and verification pipeline)
- Goal: Efficiently improve mathematical reasoning and competition-style problem solving via high-quality, validated solutions.
⚙️ Training Data Curation Pipeline
ODA-Math-460k is constructed from an aggregated question pool and then progressively filtered and selected.
1️⃣ Data Collection
We prioritize source datasets based on their empirical impact on downstream model performance. Using the OpenDataArena leaderboard, we aggregate top-ranking math datasets that show strong efficacy for the Qwen and Llama model families. These sources form the initial pool for ODA-Math.
2️⃣ Deduplication & Decontamination
We first perform exact deduplication over all questions to remove identical items, and then run benchmark decontamination to reduce evaluation leakage by removing overlaps with standard and competition benchmarks.
3️⃣ Question Filtering (Quality & Suitability)
A multi-stage filtering pipeline refines domain specificity and usability by applying an LLM-based domain classifier (to remove out-of-domain items such as coding/general instruction tasks), an LLM-based validity validator (to remove ill-formed questions with missing premises or undefined notation), and problem-type filtering (via the Big Math toolkit) to exclude proof questions and guessing-prone formats like multiple-choice and true/false—leaving predominantly free-form problems with objectively verifiable answers.
📊 Filtration Statistics
| Pipeline Stage | Count | Percentage |
|---|---|---|
| Raw Collection | 11.4M | 100% |
| Dedup & Decontamination | 4.3M | 37.7% |
| Question Filtering | 3.3M | 28.9% |
| Stage-1 Filtering | 815.3K | 7.2% |
| Stage-2 Filtering | 459.6K | 4.0% |
🎯 Data Selection
Given the large curated pool, ODA-Math-460k retains problems that are hard for small models but solvable for stronger reasoning models.
Stage-1: Lower-Bound Filtering
Stage-1 removes trivial problems using Qwen3-8B in non-thinking mode: for each problem we sample k=4 responses, compute Pass@4 by matching each predicted final answer to y_gt, and keep the problem only if Pass@4(x) = 0 (i.e., none of four attempts is correct).
Stage-2: Upper-Bound Filtering
Stage-2 removes unsolvable or ambiguous problems using Qwen3-30B-A3B in thinking mode: we generate k=5 reasoning traces per problem, compute Pass@5, and keep the problem only if Pass@5(x) > 0 (i.e., at least one attempt solves it).
✅ Distillation & Verification
🧪 Response Synthesis
We distill solutions using AM-Thinking-v1 as the teacher, generating k=5 candidate reasoning traces (step-by-step solution + final answer) for each selected problem.
🔍 Response Verification
We verify generated responses with Compass-Verifier-7B, which takes (problem x, generated response y_gen, ground-truth answer y_gt) and outputs a binary correctness decision (correct / incorrect). We keep only the (problem, response) pairs judged correct, and discard the rest—so the released dataset contains verified solutions only.
📚 Training Data Source Composition
ODA-Math-460k is a mixture of multiple high-quality math datasets to avoid domination by a single style/annotation protocol. Top contributors:
| Source | Count | Percentage |
|---|---|---|
| ScaleQuest-Math | 87,755 | 19.09% |
| NuminaMath-CoT | 75,971 | 16.53% |
| OpenMathInstruct-2 | 65,688 | 14.29% |
| MegaScience (math) | 54,904 | 11.94% |
| OpenMathReasoning | 49,463 | 10.76% |
| AM-Thinking-Distilled | 38,375 | 8.35% |
| MiroMind-M1-SFT-719K | 23,417 | 5.09% |
| SCP-116K | 16,066 | 3.50% |
| DeepMath-309K | 11,956 | 2.60% |
| math-gpt-4o-200k | 8,355 | 1.82% |
| OpenR1-Math-220k | 7,999 | 1.74% |
| MathFusionQA | 6,510 | 1.42% |
🔬 Content Characteristics
📘 Subject Distribution
ODA-Math-460k maintains a more balanced subject composition than several peers:
- Algebra remains substantial (~44.8%),
- Geometry roughly 20–22%,
- Calculus, Discrete Math & Probability, and Number Theory each around ~11%.
This mitigates subject bias and reduces performance drops on underrepresented topics.
📉 Difficulty Distribution
Apart from model-based pass rate, we also adopt LLM-as-Judge difficulty estimation on a 1-10 scale, mapped to the AoPS ratings.
| Level | Equivalent Competition Tier | Description |
|---|---|---|
| 1 | Elementary / Middle School | MOEMS, AMC 8 (Early Qs). Standard word problems. |
| 2 | Junior High | AMC 8 (Hard), AMC 10 (Early). Complex word problems. |
| 3 | High School Beginner | AMC 10 (Mid), AMC 12 (Early). Requires creative thinking. |
| 4 | High School Intermediate | AMC 12 (Mid), AIME (Early). Intermediate complexity. |
| 5 | Advanced High School | AIME (Mid), JBMO. Simple proof-based Olympiad style. |
| 6 | Pre-Olympiad | AIME (Hard), USAJMO. Introductory Olympiad level. |
| 7 | Olympiad (Entry) | IMO (Easy/Medium), USAMO. Requires technical knowledge. |
| 8 | Olympiad (Medium) | IMO (Medium/Hard). High-level competition problems. |
| 9 | Olympiad (Expert) | IMO (Hard). Expert-level constructions/proofs. |
| 10 | Historically Hard | Outliers. Exceedingly tedious or difficult even for Olympians. |
ODA-Math-460k features a balanced mix of fundamental and intermediate reasoning tasks:
- Primary Mode: Difficulty 1 (~110k samples), providing a dense foundation of basic mathematical concepts.
- Secondary Mode: Difficulty 6 (~72k samples), offering a significant concentration of intermediate-level challenges.
- Tail: A steady decline toward Difficulty 10, maintaining a specialized set of high-complexity queries.
📈 Performance
ODA-Math-460k is evaluated as an SFT corpus for Qwen3-8B-Base.
Results show consistent gains over base checkpoints, with particularly strong improvements on competition-style benchmarks.
| Dataset | Size | GSM8K | Math500 | Omni-Math | Olympiad | AIME'24 | AIME'25 | CMIMC'25 | HMMT'25 | BRUMO'25 | AVG |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-8B-Base | |||||||||||
| Qwen3-8B-Base | - | 92.0 | 79.6 | 30.6 | 47.2 | 6.7 | 10.8 | 4.7 | 0.0 | 16.7 | 32.0 |
| LIMO | 817 | 83.9 | 69.0 | 21.8 | 31.3 | 12.5 | 8.8 | 2.2 | 1.7 | 13.8 | 27.2 |
| MegaScience (math) | 414k | 93.4 | 84.8 | 35.8 | 57.6 | 25.4 | 17.9 | 11.3 | 12.1 | 33.8 | 41.3 |
| Fast-Math-R1-SFT | 8k | 92.8 | 86.6 | 39.6 | 61.0 | 28.8 | 25.8 | 14.1 | 13.3 | 34.2 | 44.0 |
| Light-R1-SFT | 79k | 93.8 | 92.6 | 48.5 | 69.7 | 54.6 | 31.3 | 22.8 | 25.0 | 48.8 | 54.1 |
| SYNTHETIC-2 (math) | 50k | 93.9 | 93.8 | 58.8 | 71.5 | 58.8 | 45.8 | 28.4 | 32.9 | 54.2 | 59.8 |
| MiroMind-M1-SFT | 719k | 94.8 | 96.8 | 54.5 | 77.0 | 62.9 | 47.5 | 25.6 | 27.5 | 60.4 | 60.8 |
| OmniThought-0528 | 365k | 94.2 | 95.4 | 59.0 | 74.9 | 67.9 | 45.4 | 31.3 | 35.8 | 52.5 | 61.8 |
| AM-Thinking (math) | 558k | 95.2 | 95.6 | 64.5 | 77.5 | 65.8 | 54.6 | 36.3 | 41.3 | 62.5 | 65.9 |
| ODA-Math | 460k | 94.3 | 96.0 | 66.9 | 76.3 | 67.9 | 63.3 | 41.6 | 45.4 | 67.5 | 68.8 |
🌐 About OpenDataArena
OpenDataArena is an open research platform dedicated to discovering, evaluating, and advancing high-quality datasets for AI post-training. It provides a transparent, data-centric ecosystem to support reproducible dataset evaluation and sharing.
Key Features:
- 🏆 Dataset Leaderboard — helps researchers identify the most valuable and high-quality datasets across different domains.
- 📊 Detailed Evaluation Scores — provides comprehensive metrics to assess data quality, complexity, difficulty etc.
- 🧰 Data Processing Toolkit — OpenDataArena-Tool offers an open-source pipeline for dataset curation and scoring.
If you find our work helpful, please consider ⭐ starring and subscribing to support our research.
🚀 Usage
Model repo: OpenDataArena/Qwen3-8B-ODA-Math-460k. Below is a minimal runnable example for loading and inference:
from transformers import AutoModelForCausalLM, AutoTokenizer
MODEL_ID = "OpenDataArena/Qwen3-8B-ODA-Math-460k"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, device_map="auto", trust_remote_code=True)
messages = [
{"role": "user", "content": "Solve: If f(x)=x^2+1, what is f(3)?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
do_sample=True,
temperature=0.7,
top_p=0.9,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
📚 Citation
@article{gao2025closing,
title={Closing the Data Loop: Using OpenDataArena to Engineer Superior Training Datasets},
author={Gao, Xin and Wang, Xiaoyang and Zhu, Yun and Cai, Mengzhang and He, Conghui and Wu, Lijun},
journal={arXiv preprint arXiv:2601.09733},
year={2025}
}
@article{cai2025opendataarena,
title={OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value},
author={Cai, Mengzhang and Gao, Xin and Li, Yu and Lin, Honglin and Liu, Zheng and Pan, Zhuoshi and Pei, Qizhi and Shang, Xiaoran and Sun, Mengyuan and Tang, Zinan and others},
journal={arXiv preprint arXiv:2512.14051},
year={2025}
}