--- library_name: transformers license: apache-2.0 base_model: google/vit-large-patch16-224-in21k tags: - generated_from_trainer metrics: - accuracy - f1 model-index: - name: vit-transformer3 results: [] --- # vit-transformer3 This model is a fine-tuned version of [google/vit-large-patch16-224-in21k](https://huggingface.co/google/vit-large-patch16-224-in21k) on the None dataset. It achieves the following results on the evaluation set: - Loss: 1.8890 - Accuracy: 0.6833 - F1: 0.6840 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 32 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 100 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | |:-------------:|:-------:|:----:|:---------------:|:--------:|:------:| | 1.8752 | 0.9552 | 16 | 0.9886 | 0.6667 | 0.6484 | | 0.7728 | 1.9701 | 33 | 0.6862 | 0.5667 | 0.4099 | | 0.7065 | 2.9851 | 50 | 0.6627 | 0.6333 | 0.6132 | | 0.6845 | 4.0 | 67 | 0.7065 | 0.55 | 0.4922 | | 0.6513 | 4.9552 | 83 | 0.7202 | 0.4667 | 0.3905 | | 0.6567 | 5.9701 | 100 | 0.7677 | 0.5333 | 0.4667 | | 0.6539 | 6.9851 | 117 | 0.6269 | 0.6167 | 0.6047 | | 0.7025 | 8.0 | 134 | 0.6838 | 0.65 | 0.6107 | | 0.6698 | 8.9552 | 150 | 0.6313 | 0.6667 | 0.6337 | | 0.6986 | 9.9701 | 167 | 0.6200 | 0.6667 | 0.6484 | | 0.6811 | 10.9851 | 184 | 0.5869 | 0.6833 | 0.6840 | | 0.6132 | 12.0 | 201 | 0.5881 | 0.6833 | 0.6687 | | 0.7235 | 12.9552 | 217 | 0.5732 | 0.65 | 0.6274 | | 0.5768 | 13.9701 | 234 | 0.5802 | 0.6833 | 0.6825 | | 0.5307 | 14.9851 | 251 | 0.6610 | 0.7 | 0.7010 | | 0.552 | 16.0 | 268 | 0.6229 | 0.7333 | 0.7296 | | 0.5548 | 16.9552 | 284 | 0.6186 | 0.7167 | 0.7036 | | 0.4863 | 17.9701 | 301 | 0.8409 | 0.5667 | 0.5366 | | 0.5048 | 18.9851 | 318 | 1.0019 | 0.4833 | 0.4015 | | 0.4919 | 20.0 | 335 | 0.6475 | 0.7333 | 0.7333 | | 0.4788 | 20.9552 | 351 | 0.6931 | 0.6333 | 0.6282 | | 0.5076 | 21.9701 | 368 | 0.6798 | 0.7 | 0.6983 | | 0.5047 | 22.9851 | 385 | 0.6784 | 0.7 | 0.7 | | 0.3477 | 24.0 | 402 | 0.8261 | 0.7 | 0.6983 | | 0.4508 | 24.9552 | 418 | 0.6846 | 0.6833 | 0.6825 | | 0.4948 | 25.9701 | 435 | 0.7509 | 0.6833 | 0.6804 | | 0.3661 | 26.9851 | 452 | 0.7321 | 0.6667 | 0.6678 | | 0.3072 | 28.0 | 469 | 0.8338 | 0.6833 | 0.6839 | | 0.3573 | 28.9552 | 485 | 0.9031 | 0.65 | 0.6434 | | 0.3828 | 29.9701 | 502 | 0.8582 | 0.6667 | 0.6667 | | 0.2931 | 30.9851 | 519 | 0.7648 | 0.65 | 0.6515 | | 0.3193 | 32.0 | 536 | 0.9218 | 0.6333 | 0.6333 | | 0.2783 | 32.9552 | 552 | 0.8452 | 0.7 | 0.7013 | | 0.2816 | 33.9701 | 569 | 0.8310 | 0.6833 | 0.6735 | | 0.3018 | 34.9851 | 586 | 0.8437 | 0.7 | 0.6960 | | 0.2256 | 36.0 | 603 | 1.0684 | 0.65 | 0.6507 | | 0.2609 | 36.9552 | 619 | 0.9117 | 0.65 | 0.6491 | | 0.2198 | 37.9701 | 636 | 1.1688 | 0.5833 | 0.5652 | | 0.306 | 38.9851 | 653 | 0.9001 | 0.6167 | 0.6130 | | 0.2243 | 40.0 | 670 | 1.2253 | 0.6333 | 0.6313 | | 0.3482 | 40.9552 | 686 | 1.0028 | 0.65 | 0.6491 | | 0.196 | 41.9701 | 703 | 0.8747 | 0.6667 | 0.6682 | | 0.2261 | 42.9851 | 720 | 1.3642 | 0.65 | 0.6468 | | 0.2802 | 44.0 | 737 | 1.3271 | 0.5833 | 0.5704 | | 0.1965 | 44.9552 | 753 | 1.3784 | 0.6 | 0.6018 | | 0.2198 | 45.9701 | 770 | 1.3224 | 0.6667 | 0.6682 | | 0.1852 | 46.9851 | 787 | 1.5364 | 0.6333 | 0.6243 | | 0.197 | 48.0 | 804 | 1.5706 | 0.6167 | 0.6174 | | 0.1932 | 48.9552 | 820 | 1.3610 | 0.6667 | 0.6648 | | 0.1495 | 49.9701 | 837 | 1.4687 | 0.6167 | 0.6174 | | 0.1404 | 50.9851 | 854 | 1.3438 | 0.7 | 0.6983 | | 0.1275 | 52.0 | 871 | 1.4674 | 0.6 | 0.5978 | | 0.1545 | 52.9552 | 887 | 1.3120 | 0.6167 | 0.6183 | | 0.147 | 53.9701 | 904 | 1.5816 | 0.6167 | 0.6183 | | 0.1541 | 54.9851 | 921 | 1.5117 | 0.6667 | 0.6678 | | 0.1283 | 56.0 | 938 | 1.5965 | 0.6667 | 0.6678 | | 0.1715 | 56.9552 | 954 | 1.6750 | 0.65 | 0.6491 | | 0.1513 | 57.9701 | 971 | 1.9170 | 0.5333 | 0.5164 | | 0.2349 | 58.9851 | 988 | 1.5358 | 0.6333 | 0.6346 | | 0.1248 | 60.0 | 1005 | 1.6686 | 0.6833 | 0.6840 | | 0.1076 | 60.9552 | 1021 | 1.7018 | 0.6333 | 0.6346 | | 0.1431 | 61.9701 | 1038 | 1.9088 | 0.6333 | 0.6333 | | 0.0838 | 62.9851 | 1055 | 1.8821 | 0.6333 | 0.6346 | | 0.0989 | 64.0 | 1072 | 1.6053 | 0.65 | 0.6491 | | 0.1323 | 64.9552 | 1088 | 1.7114 | 0.6333 | 0.6312 | | 0.0908 | 65.9701 | 1105 | 1.7326 | 0.65 | 0.6491 | | 0.2056 | 66.9851 | 1122 | 1.7166 | 0.6167 | 0.6130 | | 0.0752 | 68.0 | 1139 | 1.8009 | 0.65 | 0.6467 | | 0.1116 | 68.9552 | 1155 | 1.6964 | 0.6667 | 0.6678 | | 0.0821 | 69.9701 | 1172 | 1.7557 | 0.6167 | 0.6094 | | 0.1284 | 70.9851 | 1189 | 1.8039 | 0.65 | 0.6491 | | 0.1905 | 72.0 | 1206 | 1.7951 | 0.6167 | 0.6094 | | 0.1031 | 72.9552 | 1222 | 1.6888 | 0.6667 | 0.6648 | | 0.0706 | 73.9701 | 1239 | 1.8992 | 0.65 | 0.6467 | | 0.0944 | 74.9851 | 1256 | 1.6965 | 0.6833 | 0.6840 | | 0.1042 | 76.0 | 1273 | 1.6756 | 0.6833 | 0.6825 | | 0.1599 | 76.9552 | 1289 | 1.4360 | 0.7333 | 0.7342 | | 0.0896 | 77.9701 | 1306 | 1.5759 | 0.65 | 0.6467 | | 0.0674 | 78.9851 | 1323 | 1.7071 | 0.7 | 0.7010 | | 0.1133 | 80.0 | 1340 | 1.6499 | 0.6833 | 0.6840 | | 0.0506 | 80.9552 | 1356 | 1.6546 | 0.6833 | 0.6825 | | 0.1015 | 81.9701 | 1373 | 1.6468 | 0.7 | 0.7013 | | 0.0923 | 82.9851 | 1390 | 1.8567 | 0.6667 | 0.6622 | | 0.0752 | 84.0 | 1407 | 1.8140 | 0.7 | 0.7010 | | 0.0768 | 84.9552 | 1423 | 1.8225 | 0.6667 | 0.6678 | | 0.0683 | 85.9701 | 1440 | 1.8094 | 0.6833 | 0.6840 | | 0.0454 | 86.9851 | 1457 | 1.8892 | 0.65 | 0.6491 | | 0.054 | 88.0 | 1474 | 1.8180 | 0.7 | 0.7010 | | 0.0449 | 88.9552 | 1490 | 1.7891 | 0.7333 | 0.7345 | | 0.0645 | 89.9701 | 1507 | 1.8262 | 0.7 | 0.7010 | | 0.0632 | 90.9851 | 1524 | 1.8187 | 0.7167 | 0.7179 | | 0.0795 | 92.0 | 1541 | 1.7941 | 0.7333 | 0.7345 | | 0.0923 | 92.9552 | 1557 | 1.8340 | 0.6833 | 0.6840 | | 0.0486 | 93.9701 | 1574 | 1.8843 | 0.6667 | 0.6667 | | 0.0821 | 94.9851 | 1591 | 1.8907 | 0.6667 | 0.6667 | | 0.0384 | 95.5224 | 1600 | 1.8890 | 0.6833 | 0.6840 | ### Framework versions - Transformers 4.45.2 - Pytorch 2.4.0+cu121 - Datasets 3.1.0 - Tokenizers 0.20.1