metadata

license: apache-2.0
library_name: peft
tags:
  - generated_from_trainer
metrics:
  - accuracy
base_model: bert-base-cased
model-index:
  - name: BERT-TextClassification
    results: []

BERT-TextClassification

This model is a fine-tuned version of bert-base-cased on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.3769
Accuracy: 0.841

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	125	0.6928	0.518
No log	2.0	250	0.6834	0.573
No log	3.0	375	0.6808	0.534
0.6958	4.0	500	0.6763	0.533
0.6958	5.0	625	0.6564	0.639
0.6958	6.0	750	0.6368	0.672
0.6958	7.0	875	0.6091	0.699
0.6446	8.0	1000	0.5769	0.713
0.6446	9.0	1125	0.5434	0.73
0.6446	10.0	1250	0.5142	0.748
0.6446	11.0	1375	0.4820	0.757
0.5224	12.0	1500	0.4638	0.785
0.5224	13.0	1625	0.4383	0.792
0.5224	14.0	1750	0.4222	0.804
0.5224	15.0	1875	0.4121	0.816
0.4233	16.0	2000	0.3995	0.826
0.4233	17.0	2125	0.3958	0.822
0.4233	18.0	2250	0.3886	0.833
0.4233	19.0	2375	0.3843	0.832
0.3784	20.0	2500	0.3820	0.835
0.3784	21.0	2625	0.3804	0.834
0.3784	22.0	2750	0.3784	0.836
0.3784	23.0	2875	0.3773	0.84
0.3621	24.0	3000	0.3771	0.841
0.3621	25.0	3125	0.3769	0.841

Framework versions

PEFT 0.10.0
Transformers 4.38.2
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2