yemen2016's picture
End of training
7bd3337 verified
metadata
license: cc-by-4.0
base_model: vesteinn/DanskBERT
tags:
  - generated_from_trainer
model-index:
  - name: MeMo_BERT-SA_DanskBERT
    results: []

MeMo_BERT-SA_DanskBERT

This model is a fine-tuned version of vesteinn/DanskBERT on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0740
  • F1-score: 0.7498

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss F1-score
No log 1.0 297 0.6212 0.7242
0.6865 2.0 594 0.8063 0.7421
0.6865 3.0 891 0.8167 0.7366
0.4433 4.0 1188 1.3513 0.7479
0.4433 5.0 1485 1.0740 0.7498
0.2387 6.0 1782 1.6338 0.7128
0.0864 7.0 2079 1.7501 0.7032
0.0864 8.0 2376 2.1446 0.7137
0.0505 9.0 2673 2.2850 0.7117
0.0505 10.0 2970 2.2474 0.7430
0.0183 11.0 3267 2.2403 0.7315
0.023 12.0 3564 2.2304 0.7274
0.023 13.0 3861 2.3297 0.7304
0.0227 14.0 4158 2.3744 0.7338
0.0227 15.0 4455 2.3618 0.7383
0.0113 16.0 4752 2.2777 0.7457
0.0058 17.0 5049 2.3752 0.7440
0.0058 18.0 5346 2.4774 0.7406
0.0019 19.0 5643 2.3590 0.7451
0.0019 20.0 5940 2.3930 0.7477

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2