predict-perception-bertino-cause-none

This model is a fine-tuned version of indigo-ai/BERTino on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1988
• R2: 0.4467

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 20
• eval_batch_size: 8
• seed: 1996
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 47

Training results

Training Loss	Epoch	Step	Validation Loss	R2
0.56	1.0	14	0.3460	0.0372
0.3752	2.0	28	0.3082	0.1423
0.147	3.0	42	0.2299	0.3603
0.0961	4.0	56	0.3254	0.0944
0.0859	5.0	70	0.2650	0.2625
0.0735	6.0	84	0.2430	0.3237
0.042	7.0	98	0.2567	0.2856
0.0328	8.0	112	0.2092	0.4180
0.028	9.0	126	0.2262	0.3706
0.0237	10.0	140	0.2170	0.3960
0.0235	11.0	154	0.2137	0.4054
0.0195	12.0	168	0.2009	0.4409
0.0217	13.0	182	0.2001	0.4431
0.0176	14.0	196	0.2123	0.4091
0.0226	15.0	210	0.2076	0.4224
0.019	16.0	224	0.1920	0.4657
0.0122	17.0	238	0.2301	0.3598
0.0121	18.0	252	0.2092	0.4178
0.0112	19.0	266	0.2038	0.4329
0.0081	20.0	280	0.2008	0.4411
0.0079	21.0	294	0.1930	0.4631
0.0083	22.0	308	0.2076	0.4222
0.0061	23.0	322	0.2036	0.4334
0.0057	24.0	336	0.1986	0.4472
0.0059	25.0	350	0.2079	0.4215
0.0082	26.0	364	0.2125	0.4087
0.0093	27.0	378	0.2096	0.4168
0.0061	28.0	392	0.2129	0.4076
0.005	29.0	406	0.2054	0.4284
0.0058	30.0	420	0.2024	0.4368
0.006	31.0	434	0.1999	0.4437
0.0047	32.0	448	0.1917	0.4666
0.0046	33.0	462	0.2000	0.4435
0.005	34.0	476	0.2003	0.4425
0.0041	35.0	490	0.2057	0.4276
0.0037	36.0	504	0.1985	0.4476
0.0049	37.0	518	0.2029	0.4353
0.0031	38.0	532	0.1963	0.4539
0.0031	39.0	546	0.1957	0.4554
0.0031	40.0	560	0.1962	0.4540
0.0029	41.0	574	0.2000	0.4433
0.0028	42.0	588	0.1986	0.4473
0.0035	43.0	602	0.1972	0.4514
0.0029	44.0	616	0.1984	0.4479
0.0036	45.0	630	0.2005	0.4422
0.0033	46.0	644	0.1994	0.4452
0.0029	47.0	658	0.1988	0.4467

Framework versions

• Transformers 4.16.2
• Pytorch 1.10.2+cu113
• Datasets 1.18.3
• Tokenizers 0.11.0