hushem_1x_deit_tiny_adamax_lr001_fold4

This model is a fine-tuned version of facebook/deit-tiny-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.7197
• Accuracy: 0.7619

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.67	1	4.5038	0.2619
No log	2.0	3	1.5021	0.2381
No log	2.67	4	1.6655	0.2619
No log	4.0	6	1.3927	0.2381
No log	4.67	7	1.4664	0.2381
No log	6.0	9	1.4341	0.2381
1.9815	6.67	10	1.3866	0.5238
1.9815	8.0	12	1.4168	0.2381
1.9815	8.67	13	1.3770	0.2381
1.9815	10.0	15	1.3099	0.2619
1.9815	10.67	16	1.3229	0.2381
1.9815	12.0	18	1.2134	0.5
1.9815	12.67	19	1.1451	0.5238
1.3526	14.0	21	1.1341	0.6429
1.3526	14.67	22	0.9936	0.5952
1.3526	16.0	24	0.8768	0.6905
1.3526	16.67	25	0.9003	0.7143
1.3526	18.0	27	0.7438	0.7857
1.3526	18.67	28	0.6744	0.7143
1.0291	20.0	30	0.6946	0.7381
1.0291	20.67	31	0.6723	0.7381
1.0291	22.0	33	0.7030	0.7619
1.0291	22.67	34	0.6565	0.7857
1.0291	24.0	36	0.6394	0.7619
1.0291	24.67	37	0.7519	0.7143
1.0291	26.0	39	0.7489	0.6667
0.712	26.67	40	0.5267	0.8095
0.712	28.0	42	0.6166	0.7619
0.712	28.67	43	0.7873	0.7143
0.712	30.0	45	0.8388	0.7619
0.712	30.67	46	0.7831	0.7381
0.712	32.0	48	0.7151	0.7619
0.712	32.67	49	0.7126	0.7619
0.4557	33.33	50	0.7197	0.7619

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1