初始化项目，由ModelHub XC社区提供模型

Model: IlyaGusev/rugpt3medium_sum_gazeta Source: Original Platform
2026-09-02 22:57:19 +08:00
commit aa63abb58b
10 changed files with 186 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,31 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text 
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+pytorch_model.bin filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+vocab.json filter=lfs diff=lfs merge=lfs -text
+merges.txt filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,100 @@
+---
+language:
+- ru
+tags:
+- causal-lm
+- summarization
+datasets:
+- IlyaGusev/gazeta
+license:
+- apache-2.0
+inference: false
+widget:
+- text: "Высота башни составляет 324 метра (1063 фута), примерно такая же высота, как у 81-этажного здания, и самое высокое сооружение в Париже. Его основание квадратно, размером 125 метров (410 футов) с любой стороны. Во время строительства Эйфелева башня превзошла монумент Вашингтона, став самым высоким искусственным сооружением в мире, и этот титул она удерживала в течение 41 года до завершения строительство здания Крайслер в Нью-Йорке в 1930 году. Это первое сооружение которое достигло высоты 300 метров. Из-за добавления вещательной антенны на вершине башни в 1957 году она сейчас выше здания Крайслер на 5,2 метра (17 футов). За исключением передатчиков, Эйфелева башня является второй самой высокой отдельно стоящей структурой во Франции после виадука Мийо.<s>"
+  example_title: "Википедия"
+---
+
+# RuGPT3MediumSumGazeta
+
+## Model description
+
+This is the model for abstractive summarization for Russian based on [rugpt3medium_based_on_gpt2](https://huggingface.co/sberbank-ai/rugpt3medium_based_on_gpt2).
+
+
+## Intended uses & limitations
+
+#### How to use
+
+Colab: [link](https://colab.research.google.com/drive/1eR-ev0Y5ISWIwGnzYYoHyGMaSIUz8GTN)
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_name = "IlyaGusev/rugpt3medium_sum_gazeta"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+article_text = "..."
+
+text_tokens = tokenizer(
+    article_text,
+    max_length=600,
+    add_special_tokens=False, 
+    padding=False,
+    truncation=True
+)["input_ids"]
+input_ids = text_tokens + [tokenizer.sep_token_id]
+input_ids = torch.LongTensor([input_ids])
+
+output_ids = model.generate(
+    input_ids=input_ids,
+    no_repeat_ngram_size=4
+)
+
+summary = tokenizer.decode(output_ids[0], skip_special_tokens=False)
+summary = summary.split(tokenizer.sep_token)[1]
+summary = summary.split(tokenizer.eos_token)[0]
+print(summary)
+```
+
+## Training data
+
+- Dataset: [Gazeta](https://huggingface.co/datasets/IlyaGusev/gazeta)
+
+## Training procedure
+
+- Training script: [train.py](https://github.com/IlyaGusev/summarus/blob/master/external/hf_scripts/train.py)
+- Config: [gpt_training_config.json](https://github.com/IlyaGusev/summarus/blob/master/external/hf_scripts/configs/gpt_training_config.json)
+
+## Eval results
+
+* Train dataset: **Gazeta v1 train**
+* Test dataset: **Gazeta v1 test**
+* Source max_length: **600**
+* Target max_length: **200**
+* no_repeat_ngram_size: **4**
+* num_beams: **5**
+
+| Model                     | R-1-f | R-2-f | R-L-f | chrF | METEOR | BLEU | Avg char length |
+|:--------------------------|:------|:------|:------|:-------|:-------|:-----|:-----|
+| [mbart_ru_sum_gazeta](https://huggingface.co/IlyaGusev/mbart_ru_sum_gazeta)       | **32.4**  | 14.3  | 28.0  | 39.7 | **26.4** | 12.1 | 371 |
+| [rut5_base_sum_gazeta](https://huggingface.co/IlyaGusev/rut5_base_sum_gazeta)      | 32.2  | **14.4**  | **28.1** | **39.8** | 25.7 | **12.3** | 330 |
+| [rugpt3medium_sum_gazeta](https://huggingface.co/IlyaGusev/rugpt3medium_sum_gazeta) | 26.2 | 7.7 | 21.7 | 33.8 | 18.2 | 4.3 | 244 |
+
+* Train dataset: **Gazeta v1 train**
+* Test dataset: **Gazeta v2 test**
+* Source max_length: **600**
+* Target max_length: **200**
+* no_repeat_ngram_size: **4**
+* num_beams: **5**
+
+| Model                     | R-1-f | R-2-f | R-L-f | chrF | METEOR | BLEU | Avg char length |
+|:--------------------------|:------|:------|:------|:-------|:-------|:-----|:-----|
+| [mbart_ru_sum_gazeta](https://huggingface.co/IlyaGusev/mbart_ru_sum_gazeta)        | **28.7**  | **11.1**  | 24.4  | **37.3** | **22.7**  | **9.4** | 373 |
+| [rut5_base_sum_gazeta](https://huggingface.co/IlyaGusev/rut5_base_sum_gazeta)      | 28.6 | **11.1** | **24.5** | 37.2 | 22.0 | **9.4** | 331 |
+| [rugpt3medium_sum_gazeta](https://huggingface.co/IlyaGusev/rugpt3medium_sum_gazeta) | 24.1 | 6.5 | 19.8 | 32.1 | 16.3 | 3.6 | 242 |
+
+Evaluation script: [evaluate.py](https://github.com/IlyaGusev/summarus/blob/master/evaluate.py)
+
+Flags: --language ru --tokenize-after --lower
--- a/added_tokens.json
+++ b/added_tokens.json
@@ -0,0 +1 @@
+{"<|endoftext|>": 50257}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,40 @@
+{
+  "_name_or_path": "sberbank-ai/rugpt3_medium_gpt2_based",
+  "activation_function": "gelu_new",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 1,
+  "decoder_start_token_id": 1,
+  "embd_pdrop": 0.1,
+  "eos_token_id": 2,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "max_length": 800,
+  "model_type": "gpt2",
+  "n_ctx": 2048,
+  "n_embd": 1024,
+  "n_head": 16,
+  "n_inner": null,
+  "n_layer": 24,
+  "n_positions": 2048,
+  "n_special": 0,
+  "num_beams": 5,
+  "output_past": true,
+  "pad_token_id": 0,
+  "predict_special_tokens": true,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.12.5",
+  "use_cache": true,
+  "vocab_size": 50257
+}
--- a/merges.txt
+++ b/merges.txt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8202ee0696fad7c18b47df6df2b0a7800eeca5a7c98b8d181020a02cb564ac3
+size 1270963
--- a/pytorch_model.bin
+++ b/pytorch_model.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fbe433a71aa72c3b63eb1264ae1569d2be57e695c6c743a431e50307aacbcc4
+size 1524273113
--- a/special_tokens_map.json
+++ b/special_tokens_map.json
@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "<s>", "pad_token": "<pad>"}
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38bff9ece1cbc8b9b2d5d67146cf83c9f644a6e83c5e57f2a23129cb41ac9b37
+size 2984643
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1 @@
+{"unk_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "bos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "<|endoftext|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": false, "do_lower_case": false, "strip_accents": false, "special_tokens_map_file": null, "name_or_path": "/slot/sandbox/d/in/data/0_data_unpacked/rugpt3_medium_gpt2_based", "errors": "replace", "tokenizer_class": "GPT2Tokenizer"}
--- a/vocab.json
+++ b/vocab.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7be5387908a52936262a09514bf0a9327ff17981097b6b2225c67120fd905a5
+size 1612610
				`@@ -0,0 +1 @@`
				`{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "<s>", "pad_token": "<pad>"}`
				`@@ -0,0 +1 @@`
				{"unk_token": {"content": "<\|endoftext\|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "bos_token": {"content": "<\|endoftext\|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "<\|endoftext\|>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": false, "do_lower_case": false, "strip_accents": false, "special_tokens_map_file": null, "name_or_path": "/slot/sandbox/d/in/data/0_data_unpacked/rugpt3_medium_gpt2_based", "errors": "replace", "tokenizer_class": "GPT2Tokenizer"}