--- library_name: transformers tags: - generated_from_trainer metrics: - accuracy model-index: - name: 100M_low_1000_6910 results: [] --- # 100M_low_1000_6910 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 3.3009 - Accuracy: 0.3946 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0006 - train_batch_size: 32 - eval_batch_size: 16 - seed: 6910 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 100 - num_epochs: 10 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:------:|:-----:|:---------------:|:--------:| | 5.1184 | 0.1078 | 1000 | 5.0307 | 0.2266 | | 4.5882 | 0.2156 | 2000 | 4.5119 | 0.2696 | | 4.3179 | 0.3235 | 3000 | 4.2384 | 0.2980 | | 4.1615 | 0.4313 | 4000 | 4.0961 | 0.3116 | | 4.0664 | 0.5391 | 5000 | 3.9981 | 0.3208 | | 4.0053 | 0.6469 | 6000 | 3.9227 | 0.3275 | | 3.9311 | 0.7547 | 7000 | 3.8684 | 0.3327 | | 3.8888 | 0.8625 | 8000 | 3.8206 | 0.3374 | | 3.8417 | 0.9704 | 9000 | 3.7826 | 0.3407 | | 3.7812 | 1.0782 | 10000 | 3.7592 | 0.3442 | | 3.7435 | 1.1860 | 11000 | 3.7252 | 0.3471 | | 3.7391 | 1.2938 | 12000 | 3.7055 | 0.3491 | | 3.7123 | 1.4016 | 13000 | 3.6798 | 0.3514 | | 3.7017 | 1.5094 | 14000 | 3.6590 | 0.3535 | | 3.6819 | 1.6173 | 15000 | 3.6405 | 0.3555 | | 3.6808 | 1.7251 | 16000 | 3.6213 | 0.3568 | | 3.6642 | 1.8329 | 17000 | 3.6082 | 0.3583 | | 3.6455 | 1.9407 | 18000 | 3.5910 | 0.3600 | | 3.5613 | 2.0485 | 19000 | 3.5819 | 0.3616 | | 3.5589 | 2.1563 | 20000 | 3.5736 | 0.3625 | | 3.5597 | 2.2642 | 21000 | 3.5627 | 0.3635 | | 3.5455 | 2.3720 | 22000 | 3.5519 | 0.3648 | | 3.5547 | 2.4798 | 23000 | 3.5428 | 0.3656 | | 3.5427 | 2.5876 | 24000 | 3.5324 | 0.3670 | | 3.5285 | 2.6954 | 25000 | 3.5216 | 0.3680 | | 3.5395 | 2.8032 | 26000 | 3.5136 | 0.3686 | | 3.523 | 2.9111 | 27000 | 3.5060 | 0.3693 | | 3.4475 | 3.0189 | 28000 | 3.5024 | 0.3704 | | 3.4418 | 3.1267 | 29000 | 3.4971 | 0.3705 | | 3.47 | 3.2345 | 30000 | 3.4905 | 0.3717 | | 3.4761 | 3.3423 | 31000 | 3.4836 | 0.3726 | | 3.4649 | 3.4501 | 32000 | 3.4763 | 0.3731 | | 3.456 | 3.5580 | 33000 | 3.4716 | 0.3732 | | 3.4552 | 3.6658 | 34000 | 3.4656 | 0.3743 | | 3.4719 | 3.7736 | 35000 | 3.4591 | 0.3749 | | 3.4387 | 3.8814 | 36000 | 3.4521 | 0.3754 | | 3.4191 | 3.9892 | 37000 | 3.4474 | 0.3759 | | 3.3624 | 4.0970 | 38000 | 3.4529 | 0.3762 | | 3.3861 | 4.2049 | 39000 | 3.4477 | 0.3769 | | 3.4024 | 4.3127 | 40000 | 3.4409 | 0.3772 | | 3.4166 | 4.4205 | 41000 | 3.4363 | 0.3781 | | 3.3894 | 4.5283 | 42000 | 3.4303 | 0.3787 | | 3.4099 | 4.6361 | 43000 | 3.4254 | 0.3790 | | 3.394 | 4.7439 | 44000 | 3.4206 | 0.3794 | | 3.3911 | 4.8518 | 45000 | 3.4143 | 0.3802 | | 3.3895 | 4.9596 | 46000 | 3.4123 | 0.3804 | | 3.3172 | 5.0674 | 47000 | 3.4136 | 0.3808 | | 3.3081 | 5.1752 | 48000 | 3.4135 | 0.3811 | | 3.3303 | 5.2830 | 49000 | 3.4095 | 0.3813 | | 3.3431 | 5.3908 | 50000 | 3.4027 | 0.3819 | | 3.3191 | 5.4987 | 51000 | 3.3983 | 0.3822 | | 3.3286 | 5.6065 | 52000 | 3.3952 | 0.3826 | | 3.3427 | 5.7143 | 53000 | 3.3901 | 0.3830 | | 3.3263 | 5.8221 | 54000 | 3.3863 | 0.3835 | | 3.3294 | 5.9299 | 55000 | 3.3817 | 0.3840 | | 3.24 | 6.0377 | 56000 | 3.3839 | 0.3839 | | 3.2721 | 6.1456 | 57000 | 3.3864 | 0.3839 | | 3.2775 | 6.2534 | 58000 | 3.3806 | 0.3846 | | 3.2946 | 6.3612 | 59000 | 3.3778 | 0.3849 | | 3.2855 | 6.4690 | 60000 | 3.3749 | 0.3852 | | 3.2779 | 6.5768 | 61000 | 3.3691 | 0.3860 | | 3.2812 | 6.6846 | 62000 | 3.3644 | 0.3863 | | 3.2898 | 6.7925 | 63000 | 3.3619 | 0.3868 | | 3.3025 | 6.9003 | 64000 | 3.3567 | 0.3871 | | 3.2126 | 7.0081 | 65000 | 3.3610 | 0.3870 | | 3.2238 | 7.1159 | 66000 | 3.3621 | 0.3874 | | 3.2352 | 7.2237 | 67000 | 3.3576 | 0.3876 | | 3.2156 | 7.3315 | 68000 | 3.3556 | 0.3880 | | 3.2497 | 7.4394 | 69000 | 3.3526 | 0.3882 | | 3.2359 | 7.5472 | 70000 | 3.3483 | 0.3887 | | 3.2385 | 7.6550 | 71000 | 3.3431 | 0.3892 | | 3.2328 | 7.7628 | 72000 | 3.3404 | 0.3895 | | 3.2317 | 7.8706 | 73000 | 3.3351 | 0.3898 | | 3.2461 | 7.9784 | 74000 | 3.3336 | 0.3901 | | 3.162 | 8.0863 | 75000 | 3.3396 | 0.3898 | | 3.1882 | 8.1941 | 76000 | 3.3383 | 0.3901 | | 3.1773 | 8.3019 | 77000 | 3.3338 | 0.3906 | | 3.1951 | 8.4097 | 78000 | 3.3299 | 0.3909 | | 3.1813 | 8.5175 | 79000 | 3.3278 | 0.3911 | | 3.177 | 8.6253 | 80000 | 3.3245 | 0.3918 | | 3.1908 | 8.7332 | 81000 | 3.3191 | 0.3922 | | 3.1958 | 8.8410 | 82000 | 3.3159 | 0.3925 | | 3.1736 | 8.9488 | 83000 | 3.3149 | 0.3926 | | 3.1096 | 9.0566 | 84000 | 3.3174 | 0.3926 | | 3.147 | 9.1644 | 85000 | 3.3151 | 0.3928 | | 3.1339 | 9.2722 | 86000 | 3.3137 | 0.3932 | | 3.1222 | 9.3801 | 87000 | 3.3120 | 0.3933 | | 3.1379 | 9.4879 | 88000 | 3.3084 | 0.3938 | | 3.1324 | 9.5957 | 89000 | 3.3067 | 0.3939 | | 3.1352 | 9.7035 | 90000 | 3.3044 | 0.3943 | | 3.1327 | 9.8113 | 91000 | 3.3029 | 0.3944 | | 3.1428 | 9.9191 | 92000 | 3.3009 | 0.3946 | ### Framework versions - Transformers 4.47.0.dev0 - Pytorch 2.5.0+cu124 - Datasets 3.0.2 - Tokenizers 0.20.1