初始化项目，由ModelHub XC社区提供模型

Model: PORTULAN/gervasio-7b-portuguese-ptpt-decoder
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024 Rodrigo Santos, João Silva, Luís Gomes, João Rodrigues, 
+António Branco
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,165 @@
+---
+license: mit
+language:
+- pt
+tags:
+- gervasio-pt*
+- gervasio-ptpt
+- gervasio-ptbr
+- gervasio-7b-portuguese-ptpt-decoder
+- gervasio-7b-portuguese-ptbr-decoder
+- portulan
+- albertina-pt*
+- clm
+- gpt
+- portuguese
+- decoder
+- foundation model
+datasets:
+- PORTULAN/extraglue
+- PORTULAN/extraglue-instruct
+---
+</br>
+</br>
+<img align="left" width="40" height="40" src="https://github.githubassets.com/images/icons/emoji/unicode/1f917.png">
+<p style="text-align: center;">&nbsp;&nbsp;&nbsp;&nbsp;This is the model card for Gervásio 7B PTPT Decoder. 
+  You may be interested in some of the other models in the <a href="https://huggingface.co/PORTULAN">Albertina (encoders), Gervásio (decoders) and Serafim (sentence encoder) families</a>.
+</p>
+</br>
+</br>
+
+# Gervásio 7B PTPT
+
+</br>
+
+This model has been **deprecated**.
+
+We recommend you use the improved [**gervasio-8b-portuguese-ptpt-decoder**](https://huggingface.co/PORTULAN/gervasio-8b-portuguese-ptpt-decoder).
+
+
+<!--
+**Gervásio PT*** is a **fully open** decoder for the **Portuguese language**. 
+
+It is a **decoder** of the LLaMA family, based on the neural architecture Transformer and developed over the LLaMA-2 7B model.
+Its further improvement through additional training was done over language resources that include new instruction data sets of Portuguese prepared for this purpose ([extraGLUE-Instruct
+](https://huggingface.co/datasets/PORTULAN/extraglue-instruct)).
+
+It has different versions that were trained for different variants of Portuguese (PT), 
+namely for the European variant, spoken in Portugal ([**gervasio-7b-portuguese-ptpt-decoder**](https://huggingface.co/PORTULAN/gervasio-7b-portuguese-ptpt-decoder)), and for the American variant, spoken in Brazil ([**gervasio-7b-portuguese-ptbr-decoder**](https://huggingface.co/PORTULAN/gervasio-7b-portuguese-ptbr-decoder)).
+
+All versions of Gervásio are **openly distributed for free under an open license**, including thus for research and commercial purposes, and given its size, can 
+be run on consumer-grade hardware.
+
+**Gervásio 7B PTPT** is developed by NLX-Natural Language and Speech Group, at the University of Lisbon, Faculty of Sciences, Department of Informatics, Portugal.
+
+For the record, its full name is **Gervásio Produz Textos em Português**, to which corresponds the natural acronym **GPT PT**, 
+and which is known more shortly as **Gervásio PT*** or, even more briefly, just as **Gervásio**, among its acquaintances.
+
+Gervásio 7B PTPT is developed by a team from the University of Lisbon, Portugal. 
+For a fully detailed description, check the respective [publication](https://arxiv.org/abs/2402.18766):
+
+``` latex
+@misc{gervasio,
+      title={Advancing Generative AI for Portuguese with
+             Open Decoder Gervásio PT-*}, 
+      author={Rodrigo Santos, João Silva, Luís Gomes,
+              João Rodrigues, António Branco},
+      year={2024},
+      eprint={2402.18766},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```
+
+Please use the above cannonical reference when using or citing this model.
+
+
+<br>
+
+
+# Model Description
+
+**This model card is for Gervásio 7B PTPT**, with 7 billion parameters, a hidden size of 4,096 units, an intermediate size of 11,008 units, 32 attention heads, 32 hidden layers, and a tokenizer obtained using the Byte-Pair Encoding (BPE) algorithm implemented with SentencePiece, featuring a vocabulary size of 32,000.
+
+Gervásio 7B PTPT is distributed under an [MIT license](https://huggingface.co/PORTULAN/gervasio-7b-portuguese-ptpt-decoder/blob/main/LICENSE).
+
+
+<br>
+
+# Training Data
+
+**Gervásio 7B PTPT** was trained over standard supervised fine-tuning, and to keep some alignment with mainstream benchmarks for English, we resorted to tasks and respective datasets in the GLUE and the SuperGLUE collections.
+
+
+We selected those datasets where the outcome of their machine translation into European Portuguese could preserve, in the target language, the linguistic properties at stake.
+
+From GLUE, we resorted to the following four tasks:
+- MRPC (paraphrase Detection).
+- RTE (recognizing Textual Entailment).
+- STS-B (semantic textual similarity).
+- WNLI (coreference and natural language inference).
+
+And from SuperGLUE, we included these other four tasks: 
+- BoolQ (yes/no question answering).
+- CB (inference with 3 labels).
+- COPA (reasoning)
+- MultiRC (question answering).
+
+
+
+These datasets were machine translated into European Portuguese and from the [extraGLUE](https://huggingface.co/datasets/PORTULAN/extraglue) dataset.
+
+
+Furthermore, instruction templates have been manually crafted for each task.
+These take the various fields in the dataset and arrange them into prompts, which were collected into the [extraGLUE-instruct](https://huggingface.co/datasets/PORTULAN/extraglue-instruct) dataset.
+
+We also employed data augmentation techniques to enhance the size and diversity of our dataset.
+This involved repurposing the tasks in various ways, such as generation of answers from MultiRC, question generation from BoolQ, and other relevant modifications.
+
+
+# Training Details
+
+We applied supervised fine-tuning with a causal language modeling training objective following a zero-out technique during the fine-tuning process.
+Specifically, while the entire prompt received attention during fine-tuning, only the response tokens were subjected to back-propagation.
+
+In terms of hyper-parameters, the model was trained with a learning rate of 2 * 10^-5, a weight decay of 0.1, a two-epoch training regime without warm-up, and to ensure the same number of tokens back-propagated per step, we employed an input sequence of 512 tokens with a batch size of 16 and 16 accumulation steps.
+
+Due to hardware limitations that imposed a shorter sequence length (512) compared to the base model (4096), instead of the typical practice of concatenating all training examples and then dividing them into batches with the same input sequence length, we separated each example individually.
+In other words, each example occupies the full input sequence length.
+
+
+
+# Performance
+
+For testing, we reserved the translated datasets MRPC (similarity) and RTE (inference), from GLUE, and COPA (reasoning/qa), from SuperGLUE, which were taking as representatives of three major types of tasks, and were not seen during training.
+
+
+| Model                    | MRPC (F1)      | RTE (F1)       | COPA (F1) |
+|--------------------------|----------------|----------------|-----------|
+| **Gervásio 7B PTPT**    | **0.7273**     | **0.8291**     | **0.5459**| 
+| **LLaMA-2 (English)**              | 0.0328         | 0.0482         | 0.3844    |
+| **LLaMA-2 Chat (English)**         | 0.5703         | 0.4697         | 0.4737    |
+<br>
+
+# How to use
+
+You can use this model directly with a pipeline for causal language modeling:
+
+```python3
+>>> from transformers import pipeline
+>>> generator = pipeline(model='PORTULAN/gervasio-7b-portuguese-ptpt-decoder')
+>>> generator("A comida portuguesa é", max_new_tokens=10)
+
+```
+<br>
+
+# Acknowledgments
+
+The research reported here was partially supported by: PORTULAN CLARIN—Research Infrastructure for the Science and Technology of Language,
+funded by Lisboa 2020, Alentejo 2020 and FCT—Fundação para a Ciência e Tecnologia under the
+grant PINFRA/22117/2016; research project GPT-PT - Transformer-based Decoder for the Portuguese Language, funded by FCT—Fundação para a Ciência e Tecnologia under the
+grant CPCA-IAC/AV/478395/2022; innovation project 
+ACCELERAT.AI - Multilingual Intelligent Contact Centers, funded by IAPMEI, I.P. - Agência para a Competitividade e Inovação 
+under the grant C625734525-00462629, of Plano de Recuperação e Resiliência, 
+call RE-C05-i01.01 – Agendas/Alianças Mobilizadoras para a Reindustrialização.
+-->

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "</s>": 2,
+  "<s>": 1,
+  "<unk>": 0
+}

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "meta-llama/Llama-2-7b-hf",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.35.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "max_length": 4096,
+  "pad_token_id": 0,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.35.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe9ce8d1f8a90bf71c289968bf2d64d593327757ba7bef974866c899a47d7bc8
+size 13476865232

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17072797126a9ab898f7e651a323641c4275fef6243d468a3c9d1748b5282531
+size 13476871854

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,40 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}