smol-finetuned-SC

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7200

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 5
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 40
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0846	100	0.8014
No log	0.1691	200	0.8304
No log	0.2537	300	0.7394
No log	0.3382	400	0.8970
0.7898	0.4228	500	0.7392
0.7898	0.5073	600	0.7310
0.7898	0.5919	700	0.8090
0.7898	0.6765	800	0.7245
0.7898	0.7610	900	0.7344
0.7284	0.8456	1000	0.7322
0.7284	0.9301	1100	0.7280
0.7284	1.0147	1200	0.7254
0.7284	1.0992	1300	0.7273
0.7284	1.1838	1400	0.7366
0.7173	1.2684	1500	0.7314
0.7173	1.3529	1600	0.7186
0.7173	1.4375	1700	0.7164
0.7173	1.5220	1800	0.7190
0.7173	1.6066	1900	0.7159
0.7196	1.6912	2000	0.7262
0.7196	1.7757	2100	0.7288
0.7196	1.8603	2200	0.7892
0.7196	1.9448	2300	0.7237
0.7196	2.0294	2400	0.7182
0.7166	2.1139	2500	0.7238
0.7166	2.1985	2600	0.7181
0.7166	2.2831	2700	0.7149
0.7166	2.3676	2800	0.7224
0.7166	2.4522	2900	0.7151
0.7047	2.5367	3000	0.7209
0.7047	2.6213	3100	0.7171
0.7047	2.7058	3200	0.7155
0.7047	2.7904	3300	0.7157
0.7047	2.8750	3400	0.7175
0.7081	2.9595	3500	0.7194
0.7081	3.0441	3600	0.7164
0.7081	3.1286	3700	0.7179
0.7081	3.2132	3800	0.7186
0.7081	3.2977	3900	0.7223
0.694	3.3823	4000	0.7195
0.694	3.4669	4100	0.7209
0.694	3.5514	4200	0.7181
0.694	3.6360	4300	0.7194
0.694	3.7205	4400	0.7184
0.6989	3.8051	4500	0.7203
0.6989	3.8897	4600	0.7192
0.6989	3.9742	4700	0.7200

Framework versions

• Transformers 4.43.3
• Pytorch 2.1.0+cu118
• Datasets 2.14.5
• Tokenizers 0.19.1