language, license, library_name, tags, datasets, metrics, pipeline_tag, widget, inference, co2_eq_emissions, model-index

language	license	library_name	tags	datasets	metrics	pipeline_tag	widget	inference	co2_eq_emissions	model-index
pt	apache-2.0	transformers	text-generation-inference	TucanoBR/GigaVerbo	perplexity	text-generation	text example_title A floresta da Amazônia é conhecida por sua Exemplo text example_title Uma das coisas que Portugal, Angola, Brasil e Moçambique tem em comum é o Exemplo text example_title O Carnaval do Rio de Janeiro é Exemplo	parameters repetition_penalty temperature top_k top_p max_new_tokens 1.2 0.1 50 1.0 150	emissions source training_type geographical_location hardware_used 960000 CodeCarbon pre-training Germany NVIDIA A100-SXM4-80GB	name results Tucano-1b1 task dataset metrics source type name text-generation Text Generation name type split args CALAME-PT NOVA-vision-language/calame-pt all num_few_shot 0 type value name acc 58.24 accuracy url name https://huggingface.co/datasets/NOVA-vision-language/calame-pt Context-Aware LAnguage Modeling Evaluation for Portuguese task dataset metrics source type name text-generation Text Generation name type split args LAMBADA-PT TucanoBR/lambada-pt train num_few_shot 0 type value name acc 34.7 accuracy url name https://huggingface.co/datasets/TucanoBR/lambada-pt LAMBADA-PT task dataset metrics source type name text-generation Text Generation name type split args ENEM Challenge (No Images) eduagarcia/enem_challenge train num_few_shot 3 type value name acc 21.41 accuracy url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args BLUEX (No Images) eduagarcia-temp/BLUEX_without_images train num_few_shot 3 type value name acc 23.37 accuracy url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args OAB Exams eduagarcia/oab_exams train num_few_shot 3 type value name acc 25.97 accuracy url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args Assin2 RTE assin2 test num_few_shot 15 type value name f1_macro 60.82 f1-macro url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args Assin2 STS eduagarcia/portuguese_benchmark test num_few_shot 10 type value name pearson 24.63 pearson url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args FaQuAD NLI ruanchaves/faquad-nli test num_few_shot 15 type value name f1_macro 43.97 f1-macro url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args HateBR Binary ruanchaves/hatebr test num_few_shot 25 type value name f1_macro 29.0 f1-macro url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args PT Hate Speech Binary hate_speech_portuguese test num_few_shot 25 type value name f1_macro 41.19 f1-macro url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type split args tweetSentBR eduagarcia-temp/tweetsentbr test num_few_shot 25 type value name f1_macro 32.18 f1-macro url name https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard Open Portuguese LLM Leaderboard task dataset metrics source type name text-generation Text Generation name type args ARC-Challenge (PT) arc_pt num_few_shot 25 type value name acc_norm 30.43 normalized accuracy url name https://github.com/nlp-uoregon/mlmm-evaluation Evaluation Framework for Multilingual Large Language Models task dataset metrics source type name text-generation Text Generation name type args HellaSwag (PT) hellaswag_pt num_few_shot 10 type value name acc_norm 42.84 normalized accuracy url name https://github.com/nlp-uoregon/mlmm-evaluation Evaluation Framework for Multilingual Large Language Models task dataset metrics source type name text-generation Text Generation name type args TruthfulQA truthfulqa_pt num_few_shot 0 type value name mc2 41.59 bleurt url name https://github.com/nlp-uoregon/mlmm-evaluation Evaluation Framework for Multilingual Large Language Models

Tucano-1b1

Model Summary

Tucano is a series of decoder-transformers natively pretrained in Portuguese. All Tucano models were trained on GigaVerbo, a concatenation of deduplicated Portuguese text corpora amounting to 200 billion tokens.

Read our preprint here.

Details

• Architecture: a Transformer-based model pre-trained via causal language modeling
• Size: 1,100,048,384 parameters
• Context length: 2048 tokens
• Dataset: TucanoBR/GigaVerbo
• Language: Portuguese
• Number of steps: 480,000
• GPU: 16 NVIDIA A100-SXM4-80GB
• Training time: ~ 180 hours
• Emissions: 962 KgCO2 (Germany)
• Total energy consumption: 2524 kWh

This repository has the source code used to train this model. The main libraries used are:

• PyTorch
• Transformers
• Datasets
• Tokenizers
• Sentencepiece
• Accelerate
• FlashAttention
• Liger Kernel
• Codecarbon
• TRL

Intended Uses

The primary intended use of the Tucano models is to serve as foundations for research and development involving native Portuguese language modeling. Checkpoints saved during training are designed to provide a controlled setting for performing comparative experiments, specifically regarding the effects of active pretraining on the performance of currently available benchmarks. You may also fine-tune and adapt Tucano models for deployment if your use follows the Apache 2.0 license. If you decide to use the Tucano models as a basis for your fine-tuned model, please conduct your own risk and bias assessment.

Out-of-scope Use

• Tucano models are not intended for deployment. They are not an out-of-the-box product and should not be used for human-facing interactions.

• Tucano models are for the Portuguese language only and are unsuitable for text generation tasks in other languages.

• Tucano models have not been fine-tuned for downstream tasks.

Basic usage

Using the pipeline:

from transformers import pipeline

generator = pipeline("text-generation", model="TucanoBR/Tucano-1b1")

completions  = generator("A floresta da Amazônia é conhecida por sua", num_return_sequences=2, max_new_tokens=100)

for comp in completions:
  print(f"🤖 {comp['generated_text']}")

Using the AutoTokenizer and AutoModelForCausalLM:

from transformers import GenerationConfig, TextGenerationPipeline, AutoTokenizer, AutoModelForCausalLM
import torch

# Specify the model and tokenizer
model_id = "TucanoBR/Tucano-1b1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Specify the generation parameters as you like
generation_config = GenerationConfig(
    **{
    "do_sample": True,
    "max_new_tokens": 2048,
    "renormalize_logits": True,
    "repetition_penalty": 1.2,
    "temperature": 0.1,
    "top_k": 50,
    "top_p": 1.0,
    "use_cache": True, 
  }
)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
generator = TextGenerationPipeline(model=model, task="text-generation", tokenizer=tokenizer, device=device)

# Generate text
prompt = "A floresta da Amazônia é conhecida por sua"
completion = generator(prompt, generation_config=generation_config)
print(completion[0]['generated_text'])

Limitations

Like almost all other language models trained on large text datasets scraped from the web, the Tucano models show behavior that does not make them an out-of-the-box solution to many real-world applications, especially those requiring factual, reliable, and nontoxic text generation. Tucano models are all subject to the following:

• Hallucinations: Tucano models can produce content that can be mistaken as true facts, but are misleading or entirely false, i.e., hallucination.

• Biases and Toxicity: Tucano models inherit the social and historical stereotypes from the data used to train them. Given these biases, the model can produce toxic content, i.e., harmful, offensive, or detrimental to individuals, groups, or communities.

• Unreliable Code: Tucano models may produce incorrect code snippets and statements. These code generations should not be treated as suggestions or accurate solutions.

• Language Limitations: Tucano models are primarily designed to interact with Portuguese. Other languages might challenge its comprehension, leading to potential misinterpretations or errors in response.

• Repetition and Verbosity: Tucano models may get stuck on repetition loops (especially if the repetition penalty during generations is set to a meager value) or produce verbose responses unrelated to the prompt it was given.

Hence, even though our models are released with a permissive license, we urge users to perform their risk analysis on them if they intend to use them for real-world applications.

Evaluations

The table below compares our models against several Portuguese and multilingual language models on the evaluation harness used in our study. More information on it can be found here. To learn more about our evaluation harness selection, read our preprint.

	Average	Calame-PT	Lambada-PT	ARC-PT	HellaSwag-PT
Llama-3.2-3B	52	58.43	49.1	43.25	57.2
Granite-3.0-2b	51.63	56.36	47.55	42.56	60.05
Tucano-2b4	43.58	59.06	37.67	30.43	47.17
Llama-3.2-1B	42.95	51.83	41.02	33.5	45.44
Tucano-1b1	41.55	58.24	34.7	30.43	42.84
Gemma-2b	40.38	51.16	39.88	37.95	32.53
Bloom-1b7	40.37	55.64	31.98	30.34	43.52
Tucano-630m	39.5	56.55	33.13	28.89	39.41
Gemma-2-2b	39.21	56.7	47.1	24.19	28.85
Bloom-1b1	38.18	52.94	30.22	29.83	39.74
GlórIA-1b3	36.05	52.79	27.71	26.67	37.04
Tucano-160m	35.14	52.31	28.16	27.01	33.07
Xglm-564m	34.55	50.58	27.42	25.56	34.64
Bloom-560m	34.32	49.95	25.44	24.74	37.15
TTL-460m	33.78	49.42	23.29	29.4	33
mGPT-1b3	31.81	47.14	29.92	23.81	26.37
TTL-160m	30.78	46.72	20.98	26.15	29.29
Lola-v1	30.19	26.4	18.32	30.42	45.61
GPorTuguese	28.92	40.61	22.98	22.48	29.62

Cite as 🤗

@misc{correa2024tucanoadvancingneuraltext,
      title={{Tucano: Advancing Neural Text Generation for Portuguese}}, 
      author={Corr{\^e}a, Nicholas Kluge and Sen, Aniket and Falk, Sophia and Fatimah, Shiza},
      year={2024},
      eprint={2411.07854},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2411.07854}, 
}

@article{correa2025tucanoadvancingneuraltext,
    title={{Tucano: Advancing Neural Text Generation for Portuguese}},
    author={Corr{\^e}a, Nicholas Kluge and Sen, Aniket and Falk, Sophia and Fatimah, Shiza},
    journal={Patterns},
    publisher={Elsevier},
    year={2025},
    doi={10.1016/j.patter.2025.101325},
    url={https://doi.org/10.1016/j.patter.2025.101325},
    issn={2666-3899}
}

Aknowlegments

We gratefully acknowledge the granted access to the Marvin cluster hosted by University of Bonn along with the support provided by its High Performance Computing & Analytics Lab.

License

Tucano is licensed under the Apache License, Version 2.0. For more details, see the LICENSE file.