---
library_name: transformers
license: llama3.2
base_model: meta-llama/Llama-3.2-1B-Instruct
tags:
- peft-factory
- full
- llama-factory
- generated_from_trainer
model-index:
- name: train_sst2_42_1779354538
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# train_sst2_42_1779354538

This model is a fine-tuned version of [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) on the sst2 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0936
- Num Input Tokens Seen: 3725120

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-06
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Input Tokens Seen |
|:-------------:|:------:|:----:|:---------------:|:-----------------:|
| 0.0578        | 0.0500 | 379  | 0.1688          | 187072            |
| 0.1097        | 0.1000 | 758  | 0.1367          | 373504            |
| 0.0564        | 0.1501 | 1137 | 0.1656          | 557824            |
| 0.1883        | 0.2001 | 1516 | 0.1097          | 743424            |
| 0.2857        | 0.2501 | 1895 | 0.1319          | 930944            |
| 0.1387        | 0.3001 | 2274 | 0.1241          | 1116800           |
| 0.0494        | 0.3501 | 2653 | 0.1384          | 1303872           |
| 0.1412        | 0.4002 | 3032 | 0.1309          | 1490688           |
| 0.0509        | 0.4502 | 3411 | 0.1158          | 1678208           |
| 0.179         | 0.5002 | 3790 | 0.1276          | 1864128           |
| 0.1555        | 0.5502 | 4169 | 0.0994          | 2047552           |
| 0.1894        | 0.6002 | 4548 | 0.1099          | 2232448           |
| 0.1297        | 0.6503 | 4927 | 0.0997          | 2420096           |
| 0.1829        | 0.7003 | 5306 | 0.0936          | 2605504           |
| 0.0314        | 0.7503 | 5685 | 0.1006          | 2790656           |
| 0.1396        | 0.8003 | 6064 | 0.0963          | 2979456           |
| 0.171         | 0.8503 | 6443 | 0.1007          | 3167488           |
| 0.0013        | 0.9004 | 6822 | 0.0976          | 3355520           |
| 0.0683        | 0.9504 | 7201 | 0.0981          | 3541632           |


### Framework versions

- Transformers 4.51.3
- Pytorch 2.10.0+cu128
- Datasets 4.0.0
- Tokenizers 0.21.4