twitter-roberta-base-sentiment-latest_12112024T123630

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-sentiment-latest on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5160
• F1: 0.8689
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 600
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Rate
No log	0.9942	43	1.7742	0.1656	7e-07
No log	1.9884	86	1.7368	0.2208	0.0000
No log	2.9827	129	1.6531	0.3182	0.0000
No log	4.0	173	1.5111	0.4169	0.0000
No log	4.9942	216	1.3427	0.4913	0.0000
No log	5.9884	259	1.1750	0.5379	0.0000
No log	6.9827	302	1.0970	0.5486	5e-06
No log	8.0	346	1.0081	0.5856	0.0000
No log	8.9942	389	0.9728	0.5991	0.0000
No log	9.9884	432	0.9005	0.6481	0.0000
No log	10.9827	475	0.8614	0.6640	0.0000
1.2671	12.0	519	0.7905	0.7202	0.0000
1.2671	12.9942	562	0.7560	0.7367	0.0000
1.2671	13.9884	605	0.7399	0.7421	1e-05
1.2671	14.9827	648	0.6596	0.7804	0.0000
1.2671	16.0	692	0.6331	0.7966	0.0000
1.2671	16.9942	735	0.6272	0.7994	0.0000
1.2671	17.9884	778	0.5878	0.8249	0.0000
1.2671	18.9827	821	0.5564	0.8386	0.0000
1.2671	20.0	865	0.5482	0.8474	0.0000
1.2671	20.9942	908	0.5523	0.8501	0.0000
1.2671	21.9884	951	0.5309	0.8534	0.0000
1.2671	22.9827	994	0.5364	0.8582	4e-06
0.4473	24.0	1038	0.5176	0.8638	3e-06
0.4473	24.9942	1081	0.5256	0.8663	0.0000
0.4473	25.9884	1124	0.5182	0.8691	0.0000
0.4473	26.9827	1167	0.5237	0.8680	8e-07
0.4473	28.0	1211	0.5160	0.8689	3e-07
0.4473	28.9942	1254	0.5216	0.8673	1e-07
0.4473	29.8266	1290	0.5220	0.8670	0.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.19.1