gemma-3-1b-it-sft-metamathqa-modelmerge/README.md

library_name, base_model, tags, license

library_name	base_model	tags	license
transformers	google/gemma-3-1b-it	gemma3 math merged	gemma

Gemma 3 1B IT — MetaMathQA Merged (α=0.5)

A merged model created by interpolating the weights of a MetaMathQA-finetuned Gemma 3 1B IT with the original base model.

Method

1. Fine-tune google/gemma-3-1b-it on 7,000 samples from MetaMathQA using SFT.
2. Merge the fine-tuned weights back into the base model via linear interpolation with α=0.5:

\theta_{\text{merged}} = \alpha \cdot \theta_{\text{FT}} + (1 - \alpha) \cdot \theta_{\text{base}}

This simple averaging actually improves task-specific gain from fine-tuning while retaining more of the base model's instruction following that pure FT degrades.

Results

Method	MMLU Redux	GSM8K	IFEval
Base	39.79	33.66	40.48
FT	41.02	37.15	28.84
Merged	40.53	39.58	36.41