初始化项目,由ModelHub XC社区提供模型

Model: locailabs/gemma-3-1b-it-sft-metamathqa-modelmerge
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-26 22:36:27 +08:00
commit 2ac1fe09b7
9 changed files with 51583 additions and 0 deletions

31
README.md Normal file
View File

@@ -0,0 +1,31 @@
---
library_name: transformers
base_model:
- google/gemma-3-1b-it
tags:
- gemma3
- math
- merged
license: gemma
---
# Gemma 3 1B IT — MetaMathQA Merged (α=0.5)
A merged model created by interpolating the weights of a MetaMathQA-finetuned Gemma 3 1B IT with the original base model.
## Method
1. **Fine-tune** `google/gemma-3-1b-it` on 7,000 samples from [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA) using SFT.
2. **Merge** the fine-tuned weights back into the base model via linear interpolation with α=0.5:
$$\theta_{\text{merged}} = \alpha \cdot \theta_{\text{FT}} + (1 - \alpha) \cdot \theta_{\text{base}}$$
This simple averaging actually improves task-specific gain from fine-tuning while retaining more of the base model's instruction following that pure FT degrades.
## Results
| Method | MMLU Redux | GSM8K | IFEval |
|---|---|---|---|
| Base | 39.79 | 33.66 | **40.48** |
| FT | **41.02** | 37.15 | 28.84 |
| **Merged** | 40.53 | **39.58** | 36.41 |