metadata
library_name: transformers
license: apache-2.0
base_model: distilbert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: d_bert_v1
results: []
d_bert_v1
This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.3069
- Accuracy: 0.8929
- F1: 0.8931
- Precision: 0.8942
- Recall: 0.8929
Model description
Discriminator model for semantically similar classes
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.5
- num_epochs: 3
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Precision | Recall |
---|---|---|---|---|---|---|---|
1.5322 | 0.16 | 500 | 1.1796 | 0.6822 | 0.6712 | 0.6833 | 0.6822 |
0.7762 | 0.32 | 1000 | 0.5988 | 0.8002 | 0.7985 | 0.7973 | 0.8002 |
0.5433 | 0.48 | 1500 | 0.4945 | 0.8290 | 0.8288 | 0.8295 | 0.8290 |
0.4879 | 0.64 | 2000 | 0.4819 | 0.8301 | 0.8319 | 0.8407 | 0.8301 |
0.4447 | 0.8 | 2500 | 0.4223 | 0.8496 | 0.8511 | 0.8542 | 0.8496 |
0.4187 | 0.96 | 3000 | 0.4062 | 0.8525 | 0.8541 | 0.8594 | 0.8525 |
0.3746 | 1.12 | 3500 | 0.3892 | 0.8657 | 0.8650 | 0.8654 | 0.8657 |
0.3615 | 1.28 | 4000 | 0.3829 | 0.8637 | 0.8656 | 0.8694 | 0.8637 |
0.3507 | 1.44 | 4500 | 0.3501 | 0.8735 | 0.8748 | 0.8784 | 0.8735 |
0.3369 | 1.6 | 5000 | 0.3900 | 0.8567 | 0.8601 | 0.8759 | 0.8567 |
0.332 | 1.76 | 5500 | 0.3247 | 0.8842 | 0.8850 | 0.8867 | 0.8842 |
0.3316 | 1.92 | 6000 | 0.3280 | 0.8807 | 0.8803 | 0.8816 | 0.8807 |
0.2858 | 2.08 | 6500 | 0.3257 | 0.8881 | 0.8879 | 0.8881 | 0.8881 |
0.2613 | 2.24 | 7000 | 0.3282 | 0.8850 | 0.8861 | 0.8889 | 0.8850 |
0.2575 | 2.4 | 7500 | 0.3209 | 0.8875 | 0.8881 | 0.8913 | 0.8875 |
0.241 | 2.56 | 8000 | 0.3204 | 0.8896 | 0.8905 | 0.8930 | 0.8896 |
0.2431 | 2.7200 | 8500 | 0.3225 | 0.8851 | 0.8862 | 0.8903 | 0.8851 |
0.2248 | 2.88 | 9000 | 0.3069 | 0.8929 | 0.8931 | 0.8942 | 0.8929 |
Framework versions
- Transformers 4.44.2
- Pytorch 2.4.0
- Datasets 3.0.0
- Tokenizers 0.19.1