hubert-classifier-aug-fold-2

This model is a fine-tuned version of facebook/hubert-base-ls960 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7842
• Accuracy: 0.8652
• Precision: 0.8801
• Recall: 0.8652
• F1: 0.8644
• Binary: 0.9058

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1	Binary
No log	0.24	50	4.4178	0.0232	0.0049	0.0232	0.0073	0.1830
No log	0.48	100	4.3135	0.0787	0.0707	0.0787	0.0397	0.2956
No log	0.72	150	3.9934	0.1124	0.0716	0.1124	0.0606	0.3658
No log	0.96	200	3.6655	0.1506	0.1302	0.1506	0.0909	0.3913
4.2392	1.2	250	3.3326	0.2262	0.1653	0.2262	0.1582	0.4506
4.2392	1.44	300	2.9883	0.3401	0.2439	0.3401	0.2525	0.5362
4.2392	1.68	350	2.6683	0.3700	0.3123	0.3700	0.2952	0.5566
4.2392	1.92	400	2.4107	0.4052	0.3684	0.4052	0.3301	0.5785
3.1531	2.16	450	2.0225	0.4966	0.4535	0.4966	0.4251	0.6475
3.1531	2.4	500	1.7604	0.5618	0.5154	0.5618	0.4974	0.6929
3.1531	2.63	550	1.5309	0.6090	0.6103	0.6090	0.5608	0.7267
3.1531	2.87	600	1.3523	0.6599	0.6317	0.6599	0.6202	0.7609
2.1046	3.11	650	1.2380	0.6742	0.6824	0.6742	0.6423	0.7706
2.1046	3.35	700	1.1780	0.7004	0.7396	0.7004	0.6836	0.7886
2.1046	3.59	750	0.9850	0.7483	0.7666	0.7483	0.7323	0.8229
2.1046	3.83	800	0.9012	0.7715	0.7962	0.7715	0.7633	0.8399
1.4918	4.07	850	0.8522	0.7700	0.7916	0.7700	0.7603	0.8387
1.4918	4.31	900	0.7412	0.7948	0.8061	0.7948	0.7856	0.8564
1.4918	4.55	950	0.7650	0.8015	0.8137	0.8015	0.7906	0.8622
1.4918	4.79	1000	0.7356	0.8082	0.8232	0.8082	0.8019	0.8652
1.1749	5.03	1050	0.6688	0.8097	0.8209	0.8097	0.8051	0.8678
1.1749	5.27	1100	0.6470	0.8082	0.8308	0.8082	0.8040	0.8661
1.1749	5.51	1150	0.6289	0.8262	0.8388	0.8262	0.8228	0.8781
1.1749	5.75	1200	0.7243	0.8045	0.8206	0.8045	0.7992	0.8633
1.1749	5.99	1250	0.5897	0.8367	0.8500	0.8367	0.8356	0.8870
0.9784	6.23	1300	0.6412	0.8255	0.8397	0.8255	0.8249	0.8789
0.9784	6.47	1350	0.5967	0.8367	0.8462	0.8367	0.8331	0.8860
0.9784	6.71	1400	0.6342	0.8165	0.8356	0.8165	0.8142	0.8724
0.9784	6.95	1450	0.6780	0.8232	0.8386	0.8232	0.8207	0.8776
0.8541	7.19	1500	0.6008	0.8404	0.8532	0.8404	0.8382	0.8892
0.8541	7.43	1550	0.5507	0.8524	0.8615	0.8524	0.8505	0.8964
0.8541	7.66	1600	0.6125	0.8292	0.8435	0.8292	0.8267	0.8810
0.8541	7.9	1650	0.5411	0.8577	0.8701	0.8577	0.8575	0.9013
0.7624	8.14	1700	0.5236	0.8614	0.8693	0.8614	0.8603	0.9044
0.7624	8.38	1750	0.5491	0.8472	0.8562	0.8472	0.8450	0.8948
0.7624	8.62	1800	0.5398	0.8614	0.8686	0.8614	0.8583	0.9042
0.7624	8.86	1850	0.5902	0.8509	0.8627	0.8509	0.8497	0.8969
0.6922	9.1	1900	0.5808	0.8547	0.8686	0.8547	0.8543	0.8989
0.6922	9.34	1950	0.6115	0.8390	0.8513	0.8390	0.8368	0.8886
0.6922	9.58	2000	0.5479	0.8667	0.8759	0.8667	0.8645	0.9073
0.6922	9.82	2050	0.6070	0.8487	0.8610	0.8487	0.8476	0.8953
0.6251	10.06	2100	0.5700	0.8592	0.8724	0.8592	0.8589	0.9013
0.6251	10.3	2150	0.5362	0.8742	0.8846	0.8742	0.8742	0.9122
0.6251	10.54	2200	0.6288	0.8494	0.8669	0.8494	0.8499	0.8952
0.6251	10.78	2250	0.5886	0.8569	0.8692	0.8569	0.8555	0.8995
0.6044	11.02	2300	0.6453	0.8577	0.8712	0.8577	0.8556	0.9021
0.6044	11.26	2350	0.6322	0.8479	0.8609	0.8479	0.8461	0.8950
0.6044	11.5	2400	0.5856	0.8629	0.8713	0.8629	0.8619	0.9052
0.6044	11.74	2450	0.5970	0.8569	0.8670	0.8569	0.8561	0.9013
0.6044	11.98	2500	0.5703	0.8689	0.8789	0.8689	0.8676	0.9091
0.5484	12.22	2550	0.6249	0.8629	0.8739	0.8629	0.8627	0.9043
0.5484	12.46	2600	0.6848	0.8434	0.8565	0.8434	0.8404	0.8918
0.5484	12.69	2650	0.5845	0.8659	0.8766	0.8659	0.8658	0.9072
0.5484	12.93	2700	0.6151	0.8592	0.8695	0.8592	0.8579	0.9028
0.5297	13.17	2750	0.5739	0.8734	0.8820	0.8734	0.8728	0.9122
0.5297	13.41	2800	0.5720	0.8682	0.8797	0.8682	0.8671	0.9088
0.5297	13.65	2850	0.5494	0.8779	0.8874	0.8779	0.8764	0.9158
0.5297	13.89	2900	0.5730	0.8727	0.8813	0.8727	0.8719	0.9112
0.5084	14.13	2950	0.6109	0.8652	0.8757	0.8652	0.8629	0.9073
0.5084	14.37	3000	0.6417	0.8652	0.8762	0.8652	0.8642	0.9067
0.5084	14.61	3050	0.5735	0.8712	0.8788	0.8712	0.8701	0.9101
0.5084	14.85	3100	0.5614	0.8757	0.8835	0.8757	0.8743	0.9139
0.4813	15.09	3150	0.6592	0.8644	0.8735	0.8644	0.8632	0.9064
0.4813	15.33	3200	0.5960	0.8719	0.8786	0.8719	0.8702	0.9112
0.4813	15.57	3250	0.5824	0.8742	0.8815	0.8742	0.8735	0.9127
0.4813	15.81	3300	0.6188	0.8674	0.8767	0.8674	0.8668	0.9082
0.4615	16.05	3350	0.5480	0.8749	0.8832	0.8749	0.8742	0.9131
0.4615	16.29	3400	0.5980	0.8764	0.8844	0.8764	0.8760	0.9140
0.4615	16.53	3450	0.5855	0.8742	0.8815	0.8742	0.8732	0.9120
0.4615	16.77	3500	0.5869	0.8719	0.8832	0.8719	0.8717	0.9114
0.4421	17.01	3550	0.6259	0.8674	0.8760	0.8674	0.8657	0.9077
0.4421	17.25	3600	0.6427	0.8577	0.8684	0.8577	0.8569	0.9012
0.4421	17.49	3650	0.6241	0.8749	0.8847	0.8749	0.8739	0.9133
0.4421	17.72	3700	0.6669	0.8592	0.8688	0.8592	0.8574	0.9026
0.4421	17.96	3750	0.5774	0.8757	0.8834	0.8757	0.8741	0.9140
0.4298	18.2	3800	0.6578	0.8704	0.8766	0.8704	0.8681	0.9100
0.4298	18.44	3850	0.6365	0.8712	0.8796	0.8712	0.8712	0.9088
0.4298	18.68	3900	0.5537	0.8779	0.8845	0.8779	0.8759	0.9155
0.4298	18.92	3950	0.6618	0.8757	0.8848	0.8757	0.8740	0.9129
0.4154	19.16	4000	0.6952	0.8569	0.8681	0.8569	0.8555	0.9005
0.4154	19.4	4050	0.6206	0.8727	0.8797	0.8727	0.8717	0.9116
0.4154	19.64	4100	0.6469	0.8734	0.8846	0.8734	0.8726	0.9121
0.4154	19.88	4150	0.6405	0.8674	0.8788	0.8674	0.8665	0.9081
0.39	20.12	4200	0.6393	0.8809	0.8888	0.8809	0.8796	0.9176
0.39	20.36	4250	0.6617	0.8779	0.8881	0.8779	0.8777	0.9145
0.39	20.6	4300	0.6272	0.8697	0.8820	0.8697	0.8696	0.9098
0.39	20.84	4350	0.6635	0.8704	0.8818	0.8704	0.8707	0.9098
0.4019	21.08	4400	0.5965	0.8846	0.8943	0.8846	0.8849	0.9203
0.4019	21.32	4450	0.6427	0.8764	0.8872	0.8764	0.8764	0.9145
0.4019	21.56	4500	0.6726	0.8712	0.8799	0.8712	0.8699	0.9101
0.4019	21.8	4550	0.5973	0.8772	0.8854	0.8772	0.8766	0.9159
0.3718	22.04	4600	0.6342	0.8764	0.8868	0.8764	0.8759	0.9139
0.3718	22.28	4650	0.6081	0.8846	0.8945	0.8846	0.8848	0.9193
0.3718	22.51	4700	0.6140	0.8779	0.8900	0.8779	0.8767	0.9150
0.3718	22.75	4750	0.6561	0.8719	0.8840	0.8719	0.8715	0.9116
0.3718	22.99	4800	0.5921	0.8757	0.8853	0.8757	0.8746	0.9139
0.3638	23.23	4850	0.6855	0.8719	0.8826	0.8719	0.8702	0.9106
0.3638	23.47	4900	0.5923	0.8816	0.8933	0.8816	0.8813	0.9170
0.3638	23.71	4950	0.6988	0.8629	0.8761	0.8629	0.8608	0.9049
0.3638	23.95	5000	0.7042	0.8734	0.8840	0.8734	0.8730	0.9124
0.3447	24.19	5050	0.7146	0.8667	0.8755	0.8667	0.8647	0.9071
0.3447	24.43	5100	0.7134	0.8659	0.8754	0.8659	0.8647	0.9066
0.3447	24.67	5150	0.6893	0.8682	0.8784	0.8682	0.8672	0.9083
0.3447	24.91	5200	0.6617	0.8757	0.8887	0.8757	0.8750	0.9140
0.3387	25.15	5250	0.6747	0.8652	0.8772	0.8652	0.8636	0.9050
0.3387	25.39	5300	0.6693	0.8697	0.8794	0.8697	0.8684	0.9099
0.3387	25.63	5350	0.7019	0.8727	0.8863	0.8727	0.8720	0.9121
0.3387	25.87	5400	0.7221	0.8637	0.8793	0.8637	0.8626	0.9057
0.3303	26.11	5450	0.6852	0.8734	0.8821	0.8734	0.8723	0.9130
0.3303	26.35	5500	0.6092	0.8801	0.8883	0.8801	0.8796	0.9166
0.3303	26.59	5550	0.6416	0.8801	0.8881	0.8801	0.8795	0.9170
0.3303	26.83	5600	0.6762	0.8667	0.8763	0.8667	0.8646	0.9077
0.3253	27.07	5650	0.6886	0.8742	0.8841	0.8742	0.8737	0.9121
0.3253	27.31	5700	0.7574	0.8629	0.8742	0.8629	0.8602	0.9049
0.3253	27.54	5750	0.6952	0.8749	0.8836	0.8749	0.8745	0.9143
0.3253	27.78	5800	0.7068	0.8667	0.8786	0.8667	0.8663	0.9095
0.3233	28.02	5850	0.6912	0.8749	0.8867	0.8749	0.8741	0.9141

Framework versions

• Transformers 4.38.2
• Pytorch 2.3.0
• Datasets 2.19.1
• Tokenizers 0.15.1