BERT-TextClassification

This model is a fine-tuned version of bert-base-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3769
• Accuracy: 0.841

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	125	0.6928	0.518
No log	2.0	250	0.6834	0.573
No log	3.0	375	0.6808	0.534
0.6958	4.0	500	0.6763	0.533
0.6958	5.0	625	0.6564	0.639
0.6958	6.0	750	0.6368	0.672
0.6958	7.0	875	0.6091	0.699
0.6446	8.0	1000	0.5769	0.713
0.6446	9.0	1125	0.5434	0.73
0.6446	10.0	1250	0.5142	0.748
0.6446	11.0	1375	0.4820	0.757
0.5224	12.0	1500	0.4638	0.785
0.5224	13.0	1625	0.4383	0.792
0.5224	14.0	1750	0.4222	0.804
0.5224	15.0	1875	0.4121	0.816
0.4233	16.0	2000	0.3995	0.826
0.4233	17.0	2125	0.3958	0.822
0.4233	18.0	2250	0.3886	0.833
0.4233	19.0	2375	0.3843	0.832
0.3784	20.0	2500	0.3820	0.835
0.3784	21.0	2625	0.3804	0.834
0.3784	22.0	2750	0.3784	0.836
0.3784	23.0	2875	0.3773	0.84
0.3621	24.0	3000	0.3771	0.841
0.3621	25.0	3125	0.3769	0.841

Framework versions

• PEFT 0.10.0
• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2