BikoRiko/GPT-2.5-Math
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1 Commit 1 Branch 0 Tags
7b2cbf9f5ac0ec563d152fe1246bbe7b21a5c0c5
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

language, license, base_model, tags, pipeline_tag
language license base_model tags pipeline_tag
en mit BikoRiko/GPT-2.4-High-Pro
gpt2
math
fine-tuned
Pro
Math
text-generation

GPT-2.5-Math

GPT-2.5-Math is an upgraded version of BikoRiko/GPT-2.4-High-Pro, featuring an expanded architecture and specialized fine-tuning on mathematical reasoning.

Model Details

  • Architecture: GPT-2 with 6 additional layers (Total parameters ~0.2B).
  • Training Hardware: NVIDIA H100 (via Modal.com).
  • Dataset: 5% subset of microsoft/orca-math-word-problems-200k.
  • Objective: Fine-tuned to solve math word problems and logical queries.

Performance

The model is trained for mathematical reasoning. While it is a 0.2B parameter model, it demonstrates the beginning of logical grounding for basic word problems.

Training Details

  • Optimizer: AdamW
  • Precision: Mixed Precision (torch.amp)
  • Epochs: 3
  • Learning Rate: 5e-5
Description
Model synced from source: BikoRiko/GPT-2.5-Math
Readme 767 KiB