初始化项目,由ModelHub XC社区提供模型
Model: DGurgurov/aya-expanse-8b-mkd_cyrl Source: Original Platform
This commit is contained in:
59
README.md
Normal file
59
README.md
Normal file
@@ -0,0 +1,59 @@
|
||||
---
|
||||
language:
|
||||
- mk
|
||||
license: cc-by-nc-4.0
|
||||
base_model: CohereLabs/aya-expanse-8b
|
||||
tags:
|
||||
- subnetwork-enhancement
|
||||
- low-resource
|
||||
- language-specific
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# Aya Expanse 8B Mkd_Cyrl
|
||||
|
||||
Language-enhanced Aya-Expanse-8b model for **Macedonian** using sparse subnetwork fine-tuning.
|
||||
|
||||
## Method
|
||||
- Training approach: Language-specific neuron identification + subnetwork fine-tuning
|
||||
- Parameters trained: <1% of total model parameters
|
||||
- Framework: [Language Subnetwork Enhancement](https://github.com/d-gurgurov/Language-Subnetwork-Enhancement-LLMs)
|
||||
|
||||
## Performance
|
||||
Enhanced monolingual capabilities in Macedonian while preserving multilingual performance.
|
||||
|
||||
## Usage
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained("DGurgurov/aya-expanse-8b-mkd_cyrl")
|
||||
tokenizer = AutoTokenizer.from_pretrained("DGurgurov/aya-expanse-8b-mkd_cyrl")
|
||||
|
||||
prompt = "Your Macedonian prompt here"
|
||||
inputs = tokenizer(prompt, return_tensors="pt")
|
||||
outputs = model.generate(**inputs, max_length=100)
|
||||
print(tokenizer.decode(outputs[0]))
|
||||
```
|
||||
|
||||
## Citation
|
||||
```bibtex
|
||||
@misc{gurgurov2025sparsesubnetworkenhancement,
|
||||
title={Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models},
|
||||
author={Daniil Gurgurov and Josef van Genabith and Simon Ostermann},
|
||||
year={2025},
|
||||
eprint={2510.13580},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CL},
|
||||
url={https://arxiv.org/abs/2510.13580}
|
||||
}
|
||||
|
||||
@misc{gurgurov2025languagearithmeticssystematiclanguage,
|
||||
title={Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation},
|
||||
author={Daniil Gurgurov and Katharina Trinley and Yusser Al Ghussin and Tanja Baeumel and Josef van Genabith and Simon Ostermann},
|
||||
year={2025},
|
||||
eprint={2507.22608},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CL},
|
||||
url={https://arxiv.org/abs/2507.22608},
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user