初始化项目，由ModelHub XC社区提供模型

Model: KK100000000000000/FEMA
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,177 @@
+---
+language:
+- en
+- fi
+base_model:
+- google/mt5-small
+pipeline_tag: text-generation
+---
+# FEMA (finnish_english_museum_announcements) - Multilingual Museum Announcement Generator
+---
+language:
+- en
+- fi
+pipeline_tag: text2text-generation
+tags:
+- museum
+- announcement
+- mT5
+- finnish
+- english
+- low-resource
+license: apache-2.0
+datasets:
+- KK1000000000000000/museum-announcements-multilingual
+metrics:
+- bleu
+- rouge
+- perplexity
+library_name: transformers
+---
+
+# mT5-small для генерации музейных анонсов (английский и финский)
+
+## Описание модели
+
+Данная модель — дообученная версия **`google/mt5-small`** для автоматической генерации музейных анонсов на **английском и финском языках**. Модель создана в рамках магистерской диссертации "Обучение языковой модели на материале текстов музейных анонсов" по профилю "цифровая лингвистика" (РГПУ им. А. И. Герцена, 2026).
+
+Анонс генерируется в структурированном виде, содержащем:
+- заголовок (Title)
+- дату (Date)
+- основной текст (Text)
+
+Модель учитывает жанровые особенности музейного анонса как **малоформатного гибридного текста** (новостное сообщение + рекламный текст), включая:
+- краткость (1–5 предложений)
+- отсутствие узкой терминологии
+- наличие императивных конструкций и оценочной лексики
+- высокую информационную насыщенность
+
+## Базовая модель и архитектура
+
+- **Архитектура:** Transformer (encoder-decoder)
+- **Базовая модель:** [google/mt5-small](https://huggingface.co/google/mt5-small) (300 млн параметров)
+- **Целевые языки:** английский (en), финский (fi)
+
+## Данные обучения
+
+### Оригинальные данные
+- **Объём:** 275 текстов (180 английских, 95 финских)
+- **Источники:** официальные сайты музеев Tate Modern, Saatchi Gallery, The National Gallery (англ.); Ateneum, Kiasma, HAM (фин.)
+- **Тексты были систематизированы по единой структуре:** Query ‘запрос для генерации’, Text_id ‘уникальный идентификатор текста’, Title ‘заголовок анонса’, Date ‘дата проведения мероприятия’, Text ‘полный текст анонса’, Source ‘название музея-источника’, Category ‘тематическая категория’ (перевод мой – К. П.). Также были выделены 10 тематических категорий, которые отражают все виды мероприятий, встречающихся в анонсах на сайтах музеев. Категории разделены на три группы: музейные мероприятия (Exhibition/Näyttely ‘вытавка’, Workshop/Taidepaja ‘мастер-класс’ (перевод мой – К. П.)), жанры искусства (Art ‘изобразительное искусство’, Photography ‘фотоискусство’, Video art ‘видео-арт’, Cinematography ‘киноискусство’, Music ‘музыка’, Dance art ‘танцевальное искусство’ и формы представления арт-объектов (Installation ‘инсталляция’, Performance ‘перформанс’ 
+
+### Синтетические данные (2-й этап обучения)
+- **Объём:** 500 английских + 800 финских текстов
+- **Метод:** контрастное обучение с автоматической системой вознаграждения (LUG-система)
+
+## Методология обучения
+
+Обучение проходило в **два этапа**:
+
+### Этап 1 – Адаптация (supervised fine-tuning)
+- Методы: градиентный спуск, параметрически-эффективная настройка **LoRA**, языковые маркеры (`[EN]`, `[FI]`)
+- Гиперпараметры (после оптимизации):
+  - learning rate = 1e-5
+  - температура = 0.3
+  - эпохи = 100
+  - batch size = 2
+
+### Этап 2 – Оптимизация (contrastive learning + reward system)
+- Генерация синтетического датасета с помощью модели 1-го этапа
+- Автоматическая оценка каждого текста по трём параметрам (**LUG-система**):
+  - **Language** – грамматическая и лексическая правильность
+  - **Usable** – структурная корректность (наличие заголовка, даты, основного текста, длина)
+  - **Good** – стилистическое качество (оценочная лексика, разнообразие, сходство с эталоном)
+- Контрастное дообучение на отфильтрованных примерах (оценка ≥24 из 30 – положительные, ≤17 – отрицательные)
+
+## Оценка качества
+
+### Автоматические метрики (на тестовой выборке)
+
+| Язык      | BLEU (avg) | ROUGE-L F1 | Perplexity |
+|-----------|------------|------------|------------|
+| Английский| 0.87       | 0.95       | 0.10       |
+| Финский   | 0.80       | 0.76       | 0.50       |
+
+### Экспертная оценка
+Среднее расхождение между автоматической оценкой (LUG-система) и оценкой музейных сотрудников составило **1.2 балла из 30 (4%)**, что подтверждает практическую пригодность системы.
+
+## Как использовать модель
+
+### Установка зависимостей
+### Version_1
+```bash
+pip install transformers torch sentencepiece
+from transformers import MT5ForConditionalGeneration, MT5Tokenizer
+
+model_name = "KK100000000000000/FEMA"
+
+tokenizer = MT5Tokenizer.from_pretrained("./FEMA-final")
+model = MT5ForConditionalGeneration.from_pretrained("./FEMA-final")
+
+# перенос на GPU, если доступен
+import torch
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+model.to(device)
+
+def generate_announcement(text, language="en"):
+    """
+    Генерирует музейный анонс.
+    language: "en" или "fi"
+    text: запрос, например "Announcement of the exhibition"
+    """
+    input_text = f"[{language.upper()}] {text}"
+    inputs = tokenizer(input_text, return_tensors="pt", truncation=True, max_length=64)
+    
+    inputs = {k: v.to(device) for k, v in inputs.items()}
+    
+    outputs = model.generate(
+        **inputs,
+        max_length=128,
+        num_beams=5,
+        temperature=0.7,
+        do_sample=True,
+        early_stopping=True,
+        no_repeat_ngram_size=2
+    )
+    
+    generated = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    
+# Примеры использования
+print("=== Английский ===")
+print(generate_announcement("Announcement of the exhibition", "en"))
+print("\n=== Финский ===")
+print(generate_announcement("Ilmoitus taidepajasta", "fi"))
+
+### Version_2
+```bash
+pip install transformers torch sentencepiece
+
+## Использование
+```python
+from transformers import GPT2LMHeadModel, GPT2Tokenizer
+
+model = GPT2LMHeadModel.from_pretrained("./FEMA-final")
+tokenizer = GPT2Tokenizer.from_pretrained("./FEMA-final")
+tokenizer.pad_token = tokenizer.eos_token
+
+def generate_announcement(text, language="en"):
+    input_text = f"{language}: {text} [SEP]"
+    inputs = tokenizer(input_text, return_tensors="pt", max_length=256, truncation=True)
+    outputs = model.generate(**inputs, max_new_tokens=150, num_beams=5)
+    return tokenizer.decode(outputs[0], skip_special_tokens=True).replace(input_text, "").strip()
+```
+
+## Датасеты
+### Version_1
+- ENG.xlsx
+- FI.xlsx
+- announcement_examples_reward_0-6_20251201_184730
+### Version_2
+- ENG.tsv
+- FI.tsv
+- English_Announcements.tsv
+- Finnish_Announcements.tsv
+
+### "LUG"-СИСТЕМА ОЦЕНИВАНИЯ
+### интерфейс: https://huggingface.co/spaces/KK100000000000000/lug-museum-evaluator/tree/main 
+![image](https://cdn-uploads.huggingface.co/production/uploads/66e60387c3b3a128bdab903e/Su0pJxUvUb5bDkpaoPUqv.png)

config.json

@@ -0,0 +1,41 @@
+{
+  "activation_function": "gelu_new",
+  "add_cross_attention": false,
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50256,
+  "dtype": "float32",
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50256,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": null,
+  "n_layer": 12,
+  "n_positions": 1024,
+  "pad_token_id": null,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 50
+    }
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.0.0",
+  "use_cache": false,
+  "vocab_size": 50258
+}

config_summary.json

@@ -0,0 +1,190 @@
+{
+  "model_config": {
+    "add_cross_attention": false,
+    "tie_word_embeddings": true,
+    "vocab_size": 50258,
+    "n_positions": 1024,
+    "n_embd": 768,
+    "n_layer": 12,
+    "n_head": 12,
+    "n_inner": null,
+    "activation_function": "gelu_new",
+    "resid_pdrop": 0.1,
+    "embd_pdrop": 0.1,
+    "attn_pdrop": 0.1,
+    "layer_norm_epsilon": 1e-05,
+    "initializer_range": 0.02,
+    "summary_type": "cls_index",
+    "summary_use_proj": true,
+    "summary_activation": null,
+    "summary_first_dropout": 0.1,
+    "summary_proj_to_labels": true,
+    "scale_attn_weights": true,
+    "use_cache": false,
+    "scale_attn_by_inverse_layer_idx": false,
+    "reorder_and_upcast_attn": false,
+    "bos_token_id": 50256,
+    "eos_token_id": 50256,
+    "pad_token_id": null,
+    "return_dict": true,
+    "output_hidden_states": false,
+    "dtype": "float32",
+    "chunk_size_feed_forward": 0,
+    "is_encoder_decoder": false,
+    "architectures": [
+      "GPT2LMHeadModel"
+    ],
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "problem_type": null,
+    "_name_or_path": "gpt2",
+    "transformers_version": "5.0.0",
+    "model_type": "gpt2",
+    "n_ctx": 1024,
+    "task_specific_params": {
+      "text-generation": {
+        "do_sample": true,
+        "max_length": 50
+      }
+    },
+    "output_attentions": false
+  },
+  "tokenizer_config": {
+    "vocab_size": 50258,
+    "special_tokens": {
+      "pad_token": "<|endoftext|>",
+      "eos_token": "<|endoftext|>",
+      "bos_token": "<|endoftext|>",
+      "additional_special_tokens": []
+    }
+  },
+  "training_args": {
+    "output_dir": "./FEMA-checkpoints",
+    "do_train": false,
+    "do_eval": true,
+    "do_predict": false,
+    "eval_strategy": "epoch",
+    "prediction_loss_only": false,
+    "per_device_train_batch_size": 2,
+    "per_device_eval_batch_size": 2,
+    "gradient_accumulation_steps": 2,
+    "eval_accumulation_steps": null,
+    "eval_delay": 0,
+    "torch_empty_cache_steps": null,
+    "learning_rate": 5e-05,
+    "weight_decay": 0.01,
+    "adam_beta1": 0.9,
+    "adam_beta2": 0.999,
+    "adam_epsilon": 1e-08,
+    "max_grad_norm": 1.0,
+    "num_train_epochs": 3,
+    "max_steps": -1,
+    "lr_scheduler_type": "linear",
+    "lr_scheduler_kwargs": null,
+    "warmup_ratio": null,
+    "warmup_steps": 100,
+    "log_level": "passive",
+    "log_level_replica": "warning",
+    "log_on_each_node": true,
+    "logging_dir": "./logs",
+    "logging_strategy": "steps",
+    "logging_first_step": false,
+    "logging_steps": 10,
+    "logging_nan_inf_filter": true,
+    "save_strategy": "epoch",
+    "save_steps": 500,
+    "save_total_limit": 3,
+    "enable_jit_checkpoint": false,
+    "save_on_each_node": false,
+    "save_only_model": false,
+    "restore_callback_states_from_checkpoint": false,
+    "use_cpu": false,
+    "seed": 42,
+    "data_seed": null,
+    "bf16": false,
+    "fp16": true,
+    "bf16_full_eval": false,
+    "fp16_full_eval": false,
+    "tf32": null,
+    "local_rank": -1,
+    "ddp_backend": null,
+    "debug": [],
+    "dataloader_drop_last": false,
+    "eval_steps": null,
+    "dataloader_num_workers": 0,
+    "dataloader_prefetch_factor": null,
+    "run_name": null,
+    "disable_tqdm": false,
+    "remove_unused_columns": true,
+    "label_names": null,
+    "load_best_model_at_end": true,
+    "metric_for_best_model": "eval_loss",
+    "greater_is_better": false,
+    "ignore_data_skip": false,
+    "fsdp": [],
+    "fsdp_config": {
+      "min_num_params": 0,
+      "xla": false,
+      "xla_fsdp_v2": false,
+      "xla_fsdp_grad_ckpt": false
+    },
+    "accelerator_config": {
+      "split_batches": false,
+      "dispatch_batches": null,
+      "even_batches": true,
+      "use_seedable_sampler": true,
+      "non_blocking": false,
+      "gradient_accumulation_kwargs": null
+    },
+    "parallelism_config": null,
+    "deepspeed": null,
+    "label_smoothing_factor": 0.0,
+    "optim": "adamw_torch_fused",
+    "optim_args": null,
+    "group_by_length": false,
+    "length_column_name": "length",
+    "report_to": [],
+    "project": "huggingface",
+    "trackio_space_id": "trackio",
+    "ddp_find_unused_parameters": null,
+    "ddp_bucket_cap_mb": null,
+    "ddp_broadcast_buffers": null,
+    "dataloader_pin_memory": true,
+    "dataloader_persistent_workers": false,
+    "skip_memory_metrics": true,
+    "push_to_hub": false,
+    "resume_from_checkpoint": null,
+    "hub_model_id": null,
+    "hub_strategy": "every_save",
+    "hub_token": "<HUB_TOKEN>",
+    "hub_private_repo": null,
+    "hub_always_push": false,
+    "hub_revision": null,
+    "gradient_checkpointing": false,
+    "gradient_checkpointing_kwargs": null,
+    "include_for_metrics": [],
+    "eval_do_concat_batches": true,
+    "auto_find_batch_size": false,
+    "full_determinism": false,
+    "ddp_timeout": 1800,
+    "torch_compile": false,
+    "torch_compile_backend": null,
+    "torch_compile_mode": null,
+    "include_num_input_tokens_seen": "no",
+    "neftune_noise_alpha": null,
+    "optim_target_modules": null,
+    "batch_eval_metrics": false,
+    "eval_on_start": false,
+    "use_liger_kernel": false,
+    "liger_kernel_config": null,
+    "eval_use_gather_object": false,
+    "average_tokens_across_devices": true,
+    "use_cache": false
+  }
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "transformers_version": "5.0.0"
+}

generation_examples.json

@@ -0,0 +1,32 @@
+[
+  {
+    "input": "Announcement of the exhibition",
+    "language": "en",
+    "output": "en: Announcement of the exhibition  Text_id: 037, Title: BRITISH ART FAIR, Date: Until 11 Sep 2023, Text: This exhibition presents a unique perspective on contemporary art with works spanning multiple decades. Featuring works from international artists., Source: Saatchi Gallery, Category: Exhibition, Event: Bridget Riley Gallery's 25th Anniversary Celebrating 25 Years of British Art, Group: Tate Modern, Address: 001 Victoria and Albert St., Text(s): Explore the intersection of art and politics through this innovative exhibition showcasing 25 years of French-American art practice., Data: yksityinen virasto tapahtumasta nykytaiteen käytännön linssin läpi. Source"
+  },
+  {
+    "input": "kirjoita ilmoitus taidepajasta",
+    "language": "fi",
+    "output": "fi: kirjoita ilmoitus taidepajasta  Title: Digitaaliset työkimukset\nDate: 13.maaliskuuta - 19.lokakuuta 2024\nText: Kriittinen tarkastelu digitän liikeen käytännön linssin läpi. Dokumenta nykytaiteen tapahtumasta esityksestä kokeellä. Esitykeset teokset yhteistyöpääivät ja digitaalauksia. Kokeelä sekoiden tanssitaalinen teoksassa., saakka aussiin taiteilijat. Digitat"
+  },
+  {
+    "input": "Music event announcement",
+    "language": "en",
+    "output": "en: Music event announcement  Title: Experimental Sounds\nDate: 26 April - 12 August 2024\nText: Discover the world of experimental sounds through this innovative exhibition showcasing contemporary approaches. Featuring interactive elements, this immersive exhibition presents a unique perspective on experimental sound with works spanning multiple decades. Powered by the artist-curator Saatchi Gallery, the exhibition showcases works from international artists, including Guillermo Delacroix, Guillaume Van Gogh, Piet Mondrian, and many more. Explore the intersection of contemporary art practice with contemporary music practice through immersive installations and interactive experiences., opened at Tate Modern on 18 April 2024"
+  },
+  {
+    "input": "Näyttelyilmoitus",
+    "language": "fi",
+    "output": "fi: Näyttelyilmoitus  Title: Dokumentaarinen taide\nDate: 26.toukokuuta - 21.syyskuuta 2024\nText:  Institutiona nykytaiteen kirjoita ja tarkastelu digitaanen taidetta. Esityksestä kokeellän yhteistyönnön linssin läpi. Sijaintispesoiden tutkimukset ovat kuuttaan. Kokeelleen vuoden maalikkaan käympäiväänn ensiivat sekä., valokuvataiteilijat yksityssä. Jotk"
+  },
+  {
+    "input": "Announcement of the workshop",
+    "language": "en",
+    "output": "en: Announcement of the workshop  Title: Traditional Methods\nDate: 5 April - 12 August 2024\nText: This exhibition presents a unique perspective on traditional methods with works spanning multiple decades. The exhibition focuses on contemporary approaches to traditional approaches, including works by internationally renowned artists. Explore the intersection of contemporary art practice with contemporary innovation in this immersive exhibition featuring interactive elements. Source: Saatchi Gallery, Category: Exhibition materials, Text: A comprehensive exploration of traditional techniques, featuring works from international artists, will be on display for the first time on Friday 25 April 2024 at 18.00–22.30 p.m.\nLab materials: Worksheets, hands-on activities, materials for personal and professional study, and networking opportunities. Materials include: paper, glue sticks,"
+  },
+  {
+    "input": "Ilmoitus esityksestä",
+    "language": "fi",
+    "output": "fi: Ilmoitus esityksestä  Title: Kokeellinen tarkastelukset\nDate: 26.tammikuuta - 20.syyskuuta 2024\nText:  Institutiona tapahtumasta nykytaiteen käytännön linssin läpi. Sijaintispesoiden tanssitaan taidettaan kanssi. Esitykeset kirjoisten ensiintyöksen liikeen jotniikki olemmaan. Dokumentaarinen taiteilijat ja tauksitysta nähdän vuoden ovat tutkimuusta. Yhteistä k"
+  }
+]

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e95f6ad4bb3d9c70d4c5493520ca1a2286816bb9de0fe47d2280e4648c293624
+size 497777280

tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "[SEP]"
+  ],
+  "is_local": false,
+  "model_max_length": 1024,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4100cc67ff8a5de8997ebebda91d61cbf179e7f6ad6abd28def6f30cd316404
+size 5201

training_history.json

@@ -0,0 +1,749 @@
+{
+  "train_losses": [
+    {
+      "step": 10,
+      "loss": 6.137748718261719
+    },
+    {
+      "step": 20,
+      "loss": 5.469404220581055
+    },
+    {
+      "step": 30,
+      "loss": 5.144061279296875
+    },
+    {
+      "step": 40,
+      "loss": 4.876457977294922
+    },
+    {
+      "step": 50,
+      "loss": 4.415411376953125
+    },
+    {
+      "step": 60,
+      "loss": 3.700760269165039
+    },
+    {
+      "step": 70,
+      "loss": 3.0622364044189454
+    },
+    {
+      "step": 80,
+      "loss": 3.438877487182617
+    },
+    {
+      "step": 90,
+      "loss": 2.777506637573242
+    },
+    {
+      "step": 100,
+      "loss": 2.799178695678711
+    },
+    {
+      "step": 110,
+      "loss": 2.521131706237793
+    },
+    {
+      "step": 120,
+      "loss": 2.195684242248535
+    },
+    {
+      "step": 130,
+      "loss": 2.079251289367676
+    },
+    {
+      "step": 140,
+      "loss": 2.136846733093262
+    },
+    {
+      "step": 150,
+      "loss": 2.282297706604004
+    },
+    {
+      "step": 160,
+      "loss": 2.790775108337402
+    },
+    {
+      "step": 170,
+      "loss": 1.9956499099731446
+    },
+    {
+      "step": 180,
+      "loss": 1.890342903137207
+    },
+    {
+      "step": 190,
+      "loss": 1.692027473449707
+    },
+    {
+      "step": 200,
+      "loss": 1.9817930221557618
+    },
+    {
+      "step": 210,
+      "loss": 1.7466983795166016
+    },
+    {
+      "step": 220,
+      "loss": 2.121308135986328
+    },
+    {
+      "step": 230,
+      "loss": 1.948258399963379
+    },
+    {
+      "step": 240,
+      "loss": 1.3363015174865722
+    },
+    {
+      "step": 250,
+      "loss": 1.5559185028076172
+    },
+    {
+      "step": 260,
+      "loss": 1.568422794342041
+    },
+    {
+      "step": 270,
+      "loss": 1.237374210357666
+    },
+    {
+      "step": 280,
+      "loss": 1.8047700881958009
+    },
+    {
+      "step": 290,
+      "loss": 1.533273410797119
+    },
+    {
+      "step": 300,
+      "loss": 1.2513471603393556
+    },
+    {
+      "step": 310,
+      "loss": 1.269908618927002
+    },
+    {
+      "step": 320,
+      "loss": 1.5620304107666017
+    },
+    {
+      "step": 330,
+      "loss": 1.448842716217041
+    },
+    {
+      "step": 340,
+      "loss": 1.5316163063049317
+    },
+    {
+      "step": 350,
+      "loss": 1.1749078750610351
+    },
+    {
+      "step": 360,
+      "loss": 1.7663183212280273
+    },
+    {
+      "step": 370,
+      "loss": 1.4664443969726562
+    },
+    {
+      "step": 380,
+      "loss": 1.3086597442626953
+    },
+    {
+      "step": 390,
+      "loss": 1.3566680908203126
+    },
+    {
+      "step": 400,
+      "loss": 1.6223735809326172
+    },
+    {
+      "step": 410,
+      "loss": 1.3682662010192872
+    },
+    {
+      "step": 420,
+      "loss": 1.4399548530578614
+    },
+    {
+      "step": 430,
+      "loss": 1.344713306427002
+    },
+    {
+      "step": 440,
+      "loss": 1.2974468231201173
+    },
+    {
+      "step": 450,
+      "loss": 1.3712547302246094
+    },
+    {
+      "step": 460,
+      "loss": 1.325570011138916
+    },
+    {
+      "step": 470,
+      "loss": 1.1255184173583985
+    },
+    {
+      "step": 480,
+      "loss": 1.4051344871520997
+    },
+    {
+      "step": 490,
+      "loss": 1.2838367462158202
+    },
+    {
+      "step": 500,
+      "loss": 1.6373512268066406
+    },
+    {
+      "step": 510,
+      "loss": 1.2228617668151855
+    },
+    {
+      "step": 520,
+      "loss": 1.3670686721801757
+    },
+    {
+      "step": 530,
+      "loss": 1.3356054306030274
+    },
+    {
+      "step": 540,
+      "loss": 0.6968086242675782
+    },
+    {
+      "step": 550,
+      "loss": 1.4029299736022949
+    },
+    {
+      "step": 560,
+      "loss": 1.0916556358337401
+    },
+    {
+      "step": 570,
+      "loss": 1.1958709716796876
+    },
+    {
+      "step": 580,
+      "loss": 1.1345650672912597
+    },
+    {
+      "step": 590,
+      "loss": 1.4366456031799317
+    },
+    {
+      "step": 600,
+      "loss": 1.3605615615844726
+    },
+    {
+      "step": 610,
+      "loss": 1.2108879089355469
+    },
+    {
+      "step": 620,
+      "loss": 1.1100153923034668
+    },
+    {
+      "step": 630,
+      "loss": 0.9917799949645996
+    }
+  ],
+  "eval_losses": [
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.2418931722640991,
+      "eval_runtime": 1.2417
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.9693114161491394,
+      "eval_runtime": 1.1335
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 0.899703323841095,
+      "eval_runtime": 1.1268
+    }
+  ],
+  "full_logs": [
+    {
+      "loss": 6.137748718261719,
+      "grad_norm": 23.5301456451416,
+      "learning_rate": 4.5e-06,
+      "epoch": 0.04739336492890995,
+      "step": 10
+    },
+    {
+      "loss": 5.469404220581055,
+      "grad_norm": 18.555707931518555,
+      "learning_rate": 9.5e-06,
+      "epoch": 0.0947867298578199,
+      "step": 20
+    },
+    {
+      "loss": 5.144061279296875,
+      "grad_norm": 15.515172004699707,
+      "learning_rate": 1.45e-05,
+      "epoch": 0.14218009478672985,
+      "step": 30
+    },
+    {
+      "loss": 4.876457977294922,
+      "grad_norm": 18.302534103393555,
+      "learning_rate": 1.9500000000000003e-05,
+      "epoch": 0.1895734597156398,
+      "step": 40
+    },
+    {
+      "loss": 4.415411376953125,
+      "grad_norm": 16.696500778198242,
+      "learning_rate": 2.45e-05,
+      "epoch": 0.23696682464454977,
+      "step": 50
+    },
+    {
+      "loss": 3.700760269165039,
+      "grad_norm": 16.752397537231445,
+      "learning_rate": 2.95e-05,
+      "epoch": 0.2843601895734597,
+      "step": 60
+    },
+    {
+      "loss": 3.0622364044189454,
+      "grad_norm": 15.191258430480957,
+      "learning_rate": 3.45e-05,
+      "epoch": 0.33175355450236965,
+      "step": 70
+    },
+    {
+      "loss": 3.438877487182617,
+      "grad_norm": 16.608257293701172,
+      "learning_rate": 3.9500000000000005e-05,
+      "epoch": 0.3791469194312796,
+      "step": 80
+    },
+    {
+      "loss": 2.777506637573242,
+      "grad_norm": 15.705228805541992,
+      "learning_rate": 4.4500000000000004e-05,
+      "epoch": 0.4265402843601896,
+      "step": 90
+    },
+    {
+      "loss": 2.799178695678711,
+      "grad_norm": 13.66840648651123,
+      "learning_rate": 4.9500000000000004e-05,
+      "epoch": 0.47393364928909953,
+      "step": 100
+    },
+    {
+      "loss": 2.521131706237793,
+      "grad_norm": 14.546332359313965,
+      "learning_rate": 4.9155722326454034e-05,
+      "epoch": 0.5213270142180095,
+      "step": 110
+    },
+    {
+      "loss": 2.195684242248535,
+      "grad_norm": 19.163509368896484,
+      "learning_rate": 4.821763602251407e-05,
+      "epoch": 0.5687203791469194,
+      "step": 120
+    },
+    {
+      "loss": 2.079251289367676,
+      "grad_norm": 21.224340438842773,
+      "learning_rate": 4.727954971857411e-05,
+      "epoch": 0.6161137440758294,
+      "step": 130
+    },
+    {
+      "loss": 2.136846733093262,
+      "grad_norm": 15.907443046569824,
+      "learning_rate": 4.634146341463415e-05,
+      "epoch": 0.6635071090047393,
+      "step": 140
+    },
+    {
+      "loss": 2.282297706604004,
+      "grad_norm": 17.09271240234375,
+      "learning_rate": 4.5403377110694186e-05,
+      "epoch": 0.7109004739336493,
+      "step": 150
+    },
+    {
+      "loss": 2.790775108337402,
+      "grad_norm": 13.511648178100586,
+      "learning_rate": 4.446529080675422e-05,
+      "epoch": 0.7582938388625592,
+      "step": 160
+    },
+    {
+      "loss": 1.9956499099731446,
+      "grad_norm": 16.616485595703125,
+      "learning_rate": 4.3527204502814265e-05,
+      "epoch": 0.8056872037914692,
+      "step": 170
+    },
+    {
+      "loss": 1.890342903137207,
+      "grad_norm": 16.081674575805664,
+      "learning_rate": 4.25891181988743e-05,
+      "epoch": 0.8530805687203792,
+      "step": 180
+    },
+    {
+      "loss": 1.692027473449707,
+      "grad_norm": 13.304547309875488,
+      "learning_rate": 4.165103189493433e-05,
+      "epoch": 0.9004739336492891,
+      "step": 190
+    },
+    {
+      "loss": 1.9817930221557618,
+      "grad_norm": 18.04024887084961,
+      "learning_rate": 4.071294559099437e-05,
+      "epoch": 0.9478672985781991,
+      "step": 200
+    },
+    {
+      "loss": 1.7466983795166016,
+      "grad_norm": 15.37869644165039,
+      "learning_rate": 3.977485928705441e-05,
+      "epoch": 0.995260663507109,
+      "step": 210
+    },
+    {
+      "eval_loss": 1.2418931722640991,
+      "eval_runtime": 1.2417,
+      "eval_samples_per_second": 75.705,
+      "eval_steps_per_second": 37.853,
+      "epoch": 1.0,
+      "step": 211
+    },
+    {
+      "loss": 2.121308135986328,
+      "grad_norm": 13.085193634033203,
+      "learning_rate": 3.883677298311445e-05,
+      "epoch": 1.042654028436019,
+      "step": 220
+    },
+    {
+      "loss": 1.948258399963379,
+      "grad_norm": 12.731958389282227,
+      "learning_rate": 3.789868667917448e-05,
+      "epoch": 1.0900473933649288,
+      "step": 230
+    },
+    {
+      "loss": 1.3363015174865722,
+      "grad_norm": 16.21705436706543,
+      "learning_rate": 3.6960600375234525e-05,
+      "epoch": 1.1374407582938388,
+      "step": 240
+    },
+    {
+      "loss": 1.5559185028076172,
+      "grad_norm": 14.514242172241211,
+      "learning_rate": 3.602251407129456e-05,
+      "epoch": 1.1848341232227488,
+      "step": 250
+    },
+    {
+      "loss": 1.568422794342041,
+      "grad_norm": 12.71330738067627,
+      "learning_rate": 3.50844277673546e-05,
+      "epoch": 1.2322274881516588,
+      "step": 260
+    },
+    {
+      "loss": 1.237374210357666,
+      "grad_norm": 14.786351203918457,
+      "learning_rate": 3.414634146341464e-05,
+      "epoch": 1.2796208530805688,
+      "step": 270
+    },
+    {
+      "loss": 1.8047700881958009,
+      "grad_norm": 16.485517501831055,
+      "learning_rate": 3.320825515947467e-05,
+      "epoch": 1.3270142180094786,
+      "step": 280
+    },
+    {
+      "loss": 1.533273410797119,
+      "grad_norm": 14.662726402282715,
+      "learning_rate": 3.227016885553471e-05,
+      "epoch": 1.3744075829383886,
+      "step": 290
+    },
+    {
+      "loss": 1.2513471603393556,
+      "grad_norm": 14.021987915039062,
+      "learning_rate": 3.133208255159475e-05,
+      "epoch": 1.4218009478672986,
+      "step": 300
+    },
+    {
+      "loss": 1.269908618927002,
+      "grad_norm": 15.776110649108887,
+      "learning_rate": 3.0393996247654788e-05,
+      "epoch": 1.4691943127962086,
+      "step": 310
+    },
+    {
+      "loss": 1.5620304107666017,
+      "grad_norm": 14.94189453125,
+      "learning_rate": 2.945590994371482e-05,
+      "epoch": 1.5165876777251186,
+      "step": 320
+    },
+    {
+      "loss": 1.448842716217041,
+      "grad_norm": 17.744874954223633,
+      "learning_rate": 2.851782363977486e-05,
+      "epoch": 1.5639810426540284,
+      "step": 330
+    },
+    {
+      "loss": 1.5316163063049317,
+      "grad_norm": 16.207765579223633,
+      "learning_rate": 2.75797373358349e-05,
+      "epoch": 1.6113744075829384,
+      "step": 340
+    },
+    {
+      "loss": 1.1749078750610351,
+      "grad_norm": 12.924609184265137,
+      "learning_rate": 2.6641651031894936e-05,
+      "epoch": 1.6587677725118484,
+      "step": 350
+    },
+    {
+      "loss": 1.7663183212280273,
+      "grad_norm": 11.12903881072998,
+      "learning_rate": 2.5703564727954972e-05,
+      "epoch": 1.7061611374407581,
+      "step": 360
+    },
+    {
+      "loss": 1.4664443969726562,
+      "grad_norm": 16.2852783203125,
+      "learning_rate": 2.476547842401501e-05,
+      "epoch": 1.7535545023696684,
+      "step": 370
+    },
+    {
+      "loss": 1.3086597442626953,
+      "grad_norm": 15.057639122009277,
+      "learning_rate": 2.3827392120075048e-05,
+      "epoch": 1.8009478672985781,
+      "step": 380
+    },
+    {
+      "loss": 1.3566680908203126,
+      "grad_norm": 14.29211139678955,
+      "learning_rate": 2.2889305816135084e-05,
+      "epoch": 1.8483412322274881,
+      "step": 390
+    },
+    {
+      "loss": 1.6223735809326172,
+      "grad_norm": 11.776235580444336,
+      "learning_rate": 2.1951219512195124e-05,
+      "epoch": 1.8957345971563981,
+      "step": 400
+    },
+    {
+      "loss": 1.3682662010192872,
+      "grad_norm": 13.214008331298828,
+      "learning_rate": 2.1013133208255163e-05,
+      "epoch": 1.943127962085308,
+      "step": 410
+    },
+    {
+      "loss": 1.4399548530578614,
+      "grad_norm": 15.142778396606445,
+      "learning_rate": 2.0075046904315196e-05,
+      "epoch": 1.9905213270142181,
+      "step": 420
+    },
+    {
+      "eval_loss": 0.9693114161491394,
+      "eval_runtime": 1.1335,
+      "eval_samples_per_second": 82.931,
+      "eval_steps_per_second": 41.466,
+      "epoch": 2.0,
+      "step": 422
+    },
+    {
+      "loss": 1.344713306427002,
+      "grad_norm": 14.642666816711426,
+      "learning_rate": 1.9136960600375236e-05,
+      "epoch": 2.037914691943128,
+      "step": 430
+    },
+    {
+      "loss": 1.2974468231201173,
+      "grad_norm": 13.815317153930664,
+      "learning_rate": 1.8198874296435272e-05,
+      "epoch": 2.085308056872038,
+      "step": 440
+    },
+    {
+      "loss": 1.3712547302246094,
+      "grad_norm": 14.043903350830078,
+      "learning_rate": 1.726078799249531e-05,
+      "epoch": 2.132701421800948,
+      "step": 450
+    },
+    {
+      "loss": 1.325570011138916,
+      "grad_norm": 13.121524810791016,
+      "learning_rate": 1.6322701688555348e-05,
+      "epoch": 2.1800947867298577,
+      "step": 460
+    },
+    {
+      "loss": 1.1255184173583985,
+      "grad_norm": 18.340551376342773,
+      "learning_rate": 1.5384615384615387e-05,
+      "epoch": 2.227488151658768,
+      "step": 470
+    },
+    {
+      "loss": 1.4051344871520997,
+      "grad_norm": 15.610395431518555,
+      "learning_rate": 1.4446529080675422e-05,
+      "epoch": 2.2748815165876777,
+      "step": 480
+    },
+    {
+      "loss": 1.2838367462158202,
+      "grad_norm": 12.021589279174805,
+      "learning_rate": 1.3508442776735461e-05,
+      "epoch": 2.322274881516588,
+      "step": 490
+    },
+    {
+      "loss": 1.6373512268066406,
+      "grad_norm": 11.820240020751953,
+      "learning_rate": 1.2570356472795497e-05,
+      "epoch": 2.3696682464454977,
+      "step": 500
+    },
+    {
+      "loss": 1.2228617668151855,
+      "grad_norm": 14.760135650634766,
+      "learning_rate": 1.1632270168855535e-05,
+      "epoch": 2.4170616113744074,
+      "step": 510
+    },
+    {
+      "loss": 1.3670686721801757,
+      "grad_norm": 12.443073272705078,
+      "learning_rate": 1.0694183864915573e-05,
+      "epoch": 2.4644549763033177,
+      "step": 520
+    },
+    {
+      "loss": 1.3356054306030274,
+      "grad_norm": 15.922508239746094,
+      "learning_rate": 9.756097560975611e-06,
+      "epoch": 2.5118483412322274,
+      "step": 530
+    },
+    {
+      "loss": 0.6968086242675782,
+      "grad_norm": 12.17047119140625,
+      "learning_rate": 8.818011257035647e-06,
+      "epoch": 2.5592417061611377,
+      "step": 540
+    },
+    {
+      "loss": 1.4029299736022949,
+      "grad_norm": 12.573970794677734,
+      "learning_rate": 7.879924953095685e-06,
+      "epoch": 2.6066350710900474,
+      "step": 550
+    },
+    {
+      "loss": 1.0916556358337401,
+      "grad_norm": 13.132047653198242,
+      "learning_rate": 6.941838649155723e-06,
+      "epoch": 2.654028436018957,
+      "step": 560
+    },
+    {
+      "loss": 1.1958709716796876,
+      "grad_norm": 12.399245262145996,
+      "learning_rate": 6.00375234521576e-06,
+      "epoch": 2.7014218009478674,
+      "step": 570
+    },
+    {
+      "loss": 1.1345650672912597,
+      "grad_norm": 12.746565818786621,
+      "learning_rate": 5.065666041275798e-06,
+      "epoch": 2.748815165876777,
+      "step": 580
+    },
+    {
+      "loss": 1.4366456031799317,
+      "grad_norm": 13.03442668914795,
+      "learning_rate": 4.127579737335835e-06,
+      "epoch": 2.7962085308056874,
+      "step": 590
+    },
+    {
+      "loss": 1.3605615615844726,
+      "grad_norm": 13.585875511169434,
+      "learning_rate": 3.189493433395872e-06,
+      "epoch": 2.843601895734597,
+      "step": 600
+    },
+    {
+      "loss": 1.2108879089355469,
+      "grad_norm": 13.078906059265137,
+      "learning_rate": 2.25140712945591e-06,
+      "epoch": 2.890995260663507,
+      "step": 610
+    },
+    {
+      "loss": 1.1100153923034668,
+      "grad_norm": 12.80677318572998,
+      "learning_rate": 1.3133208255159477e-06,
+      "epoch": 2.938388625592417,
+      "step": 620
+    },
+    {
+      "loss": 0.9917799949645996,
+      "grad_norm": 10.720328330993652,
+      "learning_rate": 3.75234521575985e-07,
+      "epoch": 2.985781990521327,
+      "step": 630
+    },
+    {
+      "eval_loss": 0.899703323841095,
+      "eval_runtime": 1.1268,
+      "eval_samples_per_second": 83.423,
+      "eval_steps_per_second": 41.711,
+      "epoch": 3.0,
+      "step": 633
+    },
+    {
+      "train_runtime": 268.2648,
+      "train_samples_per_second": 9.427,
+      "train_steps_per_second": 2.36,
+      "total_flos": 330403774464000.0,
+      "train_loss": 1.9539168042970871,
+      "epoch": 3.0,
+      "step": 633
+    }
+  ]
+}

training_metadata.json

@@ -0,0 +1,21 @@
+{
+  "model_name": "FEMA",
+  "base_model": "gpt2",
+  "training_date": "2026-05-14 12:54:25.432374",
+  "epochs": 3,
+  "batch_size": 2,
+  "max_length": 256,
+  "train_size": 843,
+  "test_size": 94,
+  "final_train_loss": 1.9539168042970871,
+  "final_eval_loss": 0.899703323841095,
+  "datasets_used": [
+    "ENG.tsv",
+    "FI.tsv",
+    "English_Announcements.tsv",
+    "Finnish_Announcements.tsv"
+  ],
+  "model_parameters": 124440576,
+  "trainable_parameters": 124440576,
+  "device": "cuda:0"
+}