初始化项目,由ModelHub XC社区提供模型
Model: oguzatas/mamba-tr-project-transformer Source: Original Platform
This commit is contained in:
45
README.md
Normal file
45
README.md
Normal file
@@ -0,0 +1,45 @@
|
||||
---
|
||||
license: mit
|
||||
language:
|
||||
- tr
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- transformer
|
||||
- gpt2
|
||||
- turkish
|
||||
- baseline
|
||||
links:
|
||||
- label: "Experiment Paper"
|
||||
url: "https://huggingface.co/oguzatas/mamba-tr-project-transformer/resolve/main/OguzhanAtasHW%20state-space-models%20on%20Morphologically%20Rich%20Languages.pdf"
|
||||
---
|
||||
---
|
||||
|
||||
[](https://huggingface.co/oguzatas/mamba-tr-project-transformer/resolve/main/OguzhanAtasHW%20state-space-models%20on%20Morphologically%20Rich%20Languages.pdf)
|
||||
[](https://colab.research.google.com/drive/1UbkR-i3P6X2cFVWXlntySjki_WBdg0ac?usp=sharing)
|
||||
# Transformer Baseline (GPT-2 Style) - Turkish
|
||||
|
||||
This model serves as the **baseline** for a comparative study between Transformer and Mamba architectures on agglutinative languages (specifically Turkish). It is a decoder-only Transformer model (~111M parameters) trained on the Turkish Wikipedia dataset.
|
||||
|
||||
## Model Description
|
||||
- **Architecture:** GPT-2 (Small)
|
||||
- **Parameters:** ~111 Million
|
||||
- **Context Length:** 1024
|
||||
- **Training Data:** Turkish Wikipedia (Nov 2023)
|
||||
- **Purpose:** To provide a performance benchmark for the Mamba architecture.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast
|
||||
|
||||
model_id = "oguzatas/mamba-tr-project-transformer"
|
||||
tokenizer_id = "oguzatas/mamba-tr-project-tokenizer"
|
||||
|
||||
tokenizer = PreTrainedTokenizerFast.from_pretrained(tokenizer_id)
|
||||
model = GPT2LMHeadModel.from_pretrained(model_id)
|
||||
|
||||
text = "Türkiye'nin başkenti"
|
||||
inputs = tokenizer(text, return_tensors="pt")
|
||||
outputs = model.generate(**inputs, max_new_tokens=20)
|
||||
|
||||
print(tokenizer.decode(outputs[0]))
|
||||
Reference in New Issue
Block a user