library_name, license, base_model, tags, model-index

library_name	license	base_model	tags	model-index
transformers	llama3.2	meta-llama/Llama-3.2-1B-Instruct	peft-factory full llama-factory generated_from_trainer	name results train_rte_42_1776331559

train_rte_42_1776331559

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.1189
• Num Input Tokens Seen: 2035272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2309	0.2527	71	0.1802	105024
0.1861	0.5053	142	0.2462	209536
0.0658	0.7580	213	0.1589	312576
0.0765	1.0107	284	0.1189	414040
0.1848	1.2633	355	0.2128	517656
0.0306	1.5160	426	0.1791	624344
0.1029	1.7687	497	0.1360	725656
0.1868	2.0214	568	0.1606	821416
0.0259	2.2740	639	0.2542	926760
0.029	2.5267	710	0.2361	1025320
0.0005	2.7794	781	0.2352	1128104
0.0001	3.0320	852	0.2580	1229440
0.0001	3.2847	923	0.2295	1332544
0.0001	3.5374	994	0.2405	1438336
0.0	3.7900	1065	0.2512	1539072
0.0	4.0427	1136	0.2552	1642696
0.0	4.2954	1207	0.2572	1743624
0.0	4.5480	1278	0.2590	1849416
0.0	4.8007	1349	0.2602	1954568

Framework versions

• Transformers 4.51.3
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4