bert-base-uncased_11112024T103209

This model is a fine-tuned version of google-bert/bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4280
• F1: 0.8712
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 600
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Rate
No log	0.9942	86	1.7522	0.1396	0.0000
No log	2.0	173	1.5504	0.3793	0.0000
No log	2.9942	259	1.3168	0.5063	0.0000
No log	4.0	346	1.0578	0.5762	0.0000
No log	4.9942	432	0.8963	0.6332	0.0000
1.3438	6.0	519	0.7904	0.6792	0.0000
1.3438	6.9942	605	0.6959	0.7280	2e-05
1.3438	8.0	692	0.5408	0.8100	2e-05
1.3438	8.9942	778	0.4754	0.8469	0.0000
1.3438	10.0	865	0.4280	0.8712	0.0000
1.3438	10.9942	951	0.4683	0.8750	0.0000
0.4057	12.0	1038	0.5107	0.8769	0.0000
0.4057	12.9942	1124	0.5242	0.8879	0.0000
0.4057	14.0	1211	0.6143	0.8807	0.0000
0.4057	14.9942	1297	0.6044	0.8844	0.0000
0.4057	16.0	1384	0.5825	0.8942	0.0000
0.4057	16.9942	1470	0.6377	0.8896	0.0000
0.0457	18.0	1557	0.7469	0.8774	0.0000
0.0457	18.9942	1643	0.7769	0.8818	0.0000
0.0457	20.0	1730	0.6606	0.8943	0.0000
0.0457	20.9942	1816	0.7124	0.8915	0.0000
0.0457	22.0	1903	0.7385	0.8879	0.0000
0.0457	22.9942	1989	0.6596	0.8977	0.0000
0.0106	24.0	2076	0.7477	0.8887	0.0000
0.0106	24.9942	2162	0.6636	0.8990	0.0000
0.0106	26.0	2249	0.7530	0.8924	0.0000
0.0106	26.9942	2335	0.7221	0.8944	0.0000
0.0106	28.0	2422	0.7504	0.8931	0.0000
0.0051	28.9942	2508	0.7383	0.8951	0.0000
0.0051	30.0	2595	0.7678	0.8904	0.0000
0.0051	30.9942	2681	0.7626	0.8903	0.0000
0.0051	32.0	2768	0.7509	0.8915	0.0000
0.0051	32.9942	2854	0.7659	0.8915	2e-06
0.0051	34.0	2941	0.7721	0.8905	0.0000
0.0032	34.9942	3027	0.7705	0.8904	1e-06
0.0032	36.0	3114	0.7724	0.8893	7e-07
0.0032	36.9942	3200	0.7740	0.8895	4e-07
0.0032	38.0	3287	0.7749	0.8892	1e-07
0.0032	38.9942	3373	0.7746	0.8889	0.0
0.0032	39.7688	3440	0.7747	0.8889	0.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.19.1