43 lines
1.4 KiB
Markdown
43 lines
1.4 KiB
Markdown
---
|
|
license: gemma
|
|
datasets:
|
|
- pkupie/mc2_corpus
|
|
language:
|
|
- kk
|
|
base_model:
|
|
- google/gemma-3-4b-pt
|
|
pipeline_tag: text-generation
|
|
---
|
|
# Gemma 3 PT 4B Continually Pretrained on Kazakh (Arabic Script)
|
|
|
|
This model is a continual pretraining (CPT) checkpoint built by further pretraining Gemma 3 PT 4B on the Kazakh (Arabic Script) portion of the [MC^2 Corpus](https://huggingface.co/datasets/pkupie/mc2_corpus).
|
|
|
|
The model is intended to improve Kazakh (Arabic Script) language modeling and to support research on low-resource language adaptation.
|
|
|
|
Training details and methodology are described in: ["Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion"](https://arxiv.org/abs/2604.18106) (ACL 2026).
|
|
|
|
## Training Data
|
|
|
|
* **Corpus:** Kazakh (Arabic Script) subset of MC^2 Corpus
|
|
* **Language:** Kazakh (`kk`, Arabic Script)
|
|
* **Training paradigm:** Continual pretraining (CPT) starting from Gemma 3 PT 4B
|
|
|
|
|
|
## Intended Use
|
|
|
|
This checkpoint is released primarily for research purposes. Researchers are welcome to use this CPT checkpoint as a base model for future work, particularly in model merging and logit fusion.
|
|
|
|
|
|
## Citation
|
|
|
|
If you use this model, please cite:
|
|
|
|
```bibtex
|
|
@article{zhang2026efficient,
|
|
title={Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion},
|
|
author={Zhang, Chen and Lin, Jiuheng and Liao, Zhiyuan and Feng, Yansong},
|
|
journal={arXiv preprint arXiv:2604.18106},
|
|
year={2026}
|
|
}
|
|
```
|