---
language:
- mk
license: cc-by-nc-4.0
base_model: CohereLabs/aya-expanse-8b
tags:
- subnetwork-enhancement
- low-resource
- language-specific
pipeline_tag: text-generation
---

# Aya Expanse 8B Mkd_Cyrl

Language-enhanced Aya-Expanse-8b model for **Macedonian** using sparse subnetwork fine-tuning.

## Method
- Training approach: Language-specific neuron identification + subnetwork fine-tuning
- Parameters trained: <1% of total model parameters
- Framework: [Language Subnetwork Enhancement](https://github.com/d-gurgurov/Language-Subnetwork-Enhancement-LLMs)

## Performance
Enhanced monolingual capabilities in Macedonian while preserving multilingual performance.

## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("DGurgurov/aya-expanse-8b-mkd_cyrl")
tokenizer = AutoTokenizer.from_pretrained("DGurgurov/aya-expanse-8b-mkd_cyrl")

prompt = "Your Macedonian prompt here"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0]))
```

## Citation
```bibtex
@misc{gurgurov2025sparsesubnetworkenhancement,
    title={Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models}, 
    author={Daniil Gurgurov and Josef van Genabith and Simon Ostermann},
    year={2025},
    eprint={2510.13580},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2510.13580}
}

@misc{gurgurov2025languagearithmeticssystematiclanguage,
      title={Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation}, 
      author={Daniil Gurgurov and Katharina Trinley and Yusser Al Ghussin and Tanja Baeumel and Josef van Genabith and Simon Ostermann},
      year={2025},
      eprint={2507.22608},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2507.22608}, 
}
```