LamaAldakhil/ScCvT_K-fold

ScCvT_fold_5

This model is a fine-tuned version of microsoft/cvt-13 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3026
• F1: 0.9100
• Roc Auc: 0.9850
• Accuracy: 0.9103

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 65

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc	Accuracy
1.8858	1.0	60	1.8336	0.3756	0.6770	0.3062
1.7573	2.0	120	1.5748	0.5904	0.8013	0.5342
1.6042	3.0	180	1.2448	0.6648	0.8782	0.6270
1.4011	4.0	240	1.0728	0.6856	0.9162	0.6458
1.3588	5.0	300	0.8680	0.7432	0.9329	0.7131
1.2291	6.0	360	0.8431	0.7378	0.9400	0.7053
1.145	7.0	420	0.8501	0.7355	0.9453	0.6974
1.0652	8.0	480	0.7471	0.7559	0.9533	0.7251
1.0174	9.0	540	0.5592	0.8140	0.9634	0.8002
0.892	10.0	600	0.6785	0.7726	0.9614	0.7480
0.8584	11.0	660	0.5690	0.8088	0.9676	0.7898
0.8662	12.0	720	0.6049	0.7911	0.9696	0.7679
0.9131	13.0	780	0.4984	0.8295	0.9717	0.8179
0.8616	14.0	840	0.4755	0.8301	0.9722	0.8190
0.8398	15.0	900	0.5121	0.8237	0.9721	0.8101
0.6746	16.0	960	0.4823	0.8322	0.9742	0.8185
0.779	17.0	1020	0.5121	0.8193	0.9749	0.8002
0.7436	18.0	1080	0.4911	0.8270	0.9740	0.8153
0.7586	19.0	1140	0.4376	0.8439	0.9769	0.8346
0.688	20.0	1200	0.4732	0.8377	0.9784	0.8247
0.8294	21.0	1260	0.4889	0.8323	0.9777	0.8138
0.7451	22.0	1320	0.4015	0.8605	0.9787	0.8545
0.708	23.0	1380	0.3818	0.8677	0.9800	0.8612
0.6112	24.0	1440	0.4229	0.8651	0.9781	0.8565
0.6936	25.0	1500	0.3678	0.8714	0.9809	0.8670
0.6486	26.0	1560	0.3565	0.8722	0.9807	0.8685
0.6078	27.0	1620	0.3489	0.8760	0.9815	0.8722
0.6513	28.0	1680	0.3546	0.8774	0.9821	0.8727
0.6562	29.0	1740	0.3420	0.8816	0.9817	0.8774
0.6561	30.0	1800	0.3649	0.8745	0.9818	0.8691
0.6056	31.0	1860	0.3692	0.8730	0.9823	0.8670
0.5927	32.0	1920	0.3584	0.8825	0.9822	0.8779
0.6021	33.0	1980	0.3345	0.8870	0.9821	0.8842
0.6052	34.0	2040	0.3388	0.8887	0.9820	0.8868
0.6026	35.0	2100	0.3161	0.8938	0.9831	0.8925
0.5735	36.0	2160	0.3324	0.8952	0.9825	0.8957
0.6058	37.0	2220	0.3262	0.8932	0.9829	0.8905
0.5648	38.0	2280	0.3243	0.8931	0.9834	0.8899
0.6248	39.0	2340	0.3499	0.8825	0.9837	0.8769
0.4926	40.0	2400	0.3066	0.9045	0.9837	0.9045
0.5341	41.0	2460	0.3037	0.9042	0.9835	0.9061
0.5215	42.0	2520	0.3193	0.8968	0.9832	0.8967
0.5892	43.0	2580	0.3276	0.8910	0.9837	0.8884
0.559	44.0	2640	0.3129	0.9006	0.9843	0.8988
0.5306	45.0	2700	0.3137	0.9024	0.9839	0.9035
0.4789	46.0	2760	0.3128	0.8984	0.9842	0.8972
0.5518	47.0	2820	0.3117	0.9057	0.9838	0.9051
0.5201	48.0	2880	0.3110	0.9023	0.9842	0.9014
0.5698	49.0	2940	0.3031	0.9074	0.9841	0.9071
0.5227	50.0	3000	0.3343	0.9007	0.9834	0.8988
0.5416	51.0	3060	0.3117	0.9078	0.9837	0.9082
0.5882	52.0	3120	0.3132	0.9049	0.9835	0.9051
0.4286	53.0	3180	0.3133	0.9064	0.9837	0.9066
0.5278	54.0	3240	0.3050	0.9080	0.9847	0.9077
0.586	55.0	3300	0.3063	0.9065	0.9842	0.9082
0.4708	56.0	3360	0.3119	0.9058	0.9840	0.9056
0.5512	57.0	3420	0.3140	0.9023	0.9842	0.9014
0.5535	58.0	3480	0.3079	0.9046	0.9847	0.9045
0.4817	59.0	3540	0.3077	0.9061	0.9845	0.9061
0.5381	60.0	3600	0.3122	0.9015	0.9848	0.9009
0.6305	61.0	3660	0.3026	0.9100	0.9850	0.9103
0.4658	62.0	3720	0.3044	0.9093	0.9851	0.9092
0.4873	63.0	3780	0.3076	0.9074	0.9849	0.9071
0.5791	64.0	3840	0.3045	0.9077	0.9850	0.9077
0.5871	65.0	3900	0.3028	0.9076	0.9852	0.9071

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3