train_sst2_42_1779354537/README.md

library_name, license, base_model, tags, model-index

library_name	license	base_model	tags	model-index
transformers	llama3.2	meta-llama/Llama-3.2-1B-Instruct	peft-factory freeze llama-factory generated_from_trainer	name results train_sst2_42_1779354537

train_sst2_42_1779354537

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the sst2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0908
• Num Input Tokens Seen: 3725120

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0584	0.0500	379	0.1753	187072
0.1154	0.1000	758	0.1295	373504
0.0745	0.1501	1137	0.1949	557824
0.1712	0.2001	1516	0.1069	743424
0.2865	0.2501	1895	0.1277	930944
0.1225	0.3001	2274	0.1098	1116800
0.1152	0.3501	2653	0.1235	1303872
0.1615	0.4002	3032	0.1323	1490688
0.0698	0.4502	3411	0.1182	1678208
0.3465	0.5002	3790	0.1325	1864128
0.1538	0.5502	4169	0.0976	2047552
0.1911	0.6002	4548	0.1150	2232448
0.1499	0.6503	4927	0.0984	2420096
0.2014	0.7003	5306	0.0908	2605504
0.0014	0.7503	5685	0.0957	2790656
0.1294	0.8003	6064	0.0955	2979456
0.1202	0.8503	6443	0.0970	3167488
0.0013	0.9004	6822	0.0957	3355520
0.05	0.9504	7201	0.0956	3541632

Framework versions

• Transformers 4.51.3
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4