Hugging Face's logo

BramVanroy
/
mbart-large-cc25-ft-amr30-es_nl-stratified

Text2Text Generation
Transformers
Safetensors
Dutch
Spanish
mbart
Inference Endpoints
Model card Files Community
mbart-large-cc25-ft-amr30-es_nl-stratified / README.md
BramVanroy's picture
BramVanroy
Update README.md
f60d104
preview code
|
raw
history blame contribute delete
5.47 kB
metadata
base_model: facebook/mbart-large-cc25
language:
  - nl
  - es

ES and NL to AMR parsing (stratified)

This version was trained on a subselection of the data. The AMR 3.0 corpus was translated to all the relevant languages. We then divided the dataset so that in total we only see half of each language's dataset (so that in total we only see the full AMR 3.0 corpus in size once). In other words, all languages were undersampled for research purposes.

This model is a fine-tuned version of facebook/mbart-large-cc25 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6212
  • Smatch Precision: 72.94
  • Smatch Recall: 75.83
  • Smatch Fscore: 74.36
  • Smatch Unparsable: 0
  • Percent Not Recoverable: 0.4065

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 25

Training results

Training Loss Epoch Step Validation Loss Smatch Precision Smatch Recall Smatch Fscore Smatch Unparsable Percent Not Recoverable
0.4098 1.0 3477 1.3168 17.61 63.89 27.62 0 0.0
0.3307 2.0 6954 1.0109 21.08 68.69 32.26 0 0.0581
0.1253 3.0 10431 0.9193 32.88 71.46 45.04 0 0.0
0.1665 4.0 13908 0.7549 35.07 72.54 47.29 0 0.0
0.0435 5.0 17385 0.8298 40.25 74.91 52.37 0 0.0581
0.2156 6.0 20862 0.6525 45.7 75.11 56.82 0 0.0
0.133 7.0 24339 0.6548 47.7 75.36 58.42 0 0.0
0.0624 8.0 27817 0.6054 53.59 75.18 62.57 0 0.0
0.0841 9.0 31294 0.6496 54.68 75.01 63.25 0 0.0581
0.1073 10.0 34771 0.5960 55.76 76.35 64.45 0 0.0
0.048 11.0 38248 0.5924 60.99 76.4 67.83 0 0.0
0.0341 12.0 41725 0.5880 60.39 76.31 67.42 0 0.0581
0.0079 13.0 45202 0.6117 61.61 76.52 68.26 0 0.0
0.0244 14.0 48679 0.6191 63.78 76.44 69.54 0 0.0581
0.0575 15.0 52156 0.6320 66.27 76.71 71.11 0 0.1161
0.0204 16.0 55634 0.6126 67.51 76.48 71.72 0 0.0
0.0278 17.0 59111 0.6114 67.6 76.8 71.91 0 0.0581
0.0219 18.0 62588 0.6184 68.84 77.14 72.75 0 0.0581
0.01 19.0 66065 0.6197 69.62 76.77 73.02 0 0.0
0.0423 20.0 69542 0.6204 71.01 76.89 73.83 0 0.0581
0.0095 21.0 73019 0.6309 70.76 76.53 73.53 0 0.0581
0.0132 22.0 76496 0.6208 71.97 76.41 74.12 0 0.2904
0.0148 23.0 79973 0.6307 71.86 76.61 74.16 0 0.0581
0.0034 24.0 83451 0.6258 72.41 76.24 74.28 0 0.3484
0.0527 25.0 86925 0.6212 72.94 75.83 74.36 0 0.4065

Framework versions

  • Transformers 4.34.0.dev0
  • Pytorch 2.0.1+cu117
  • Datasets 2.14.2
  • Tokenizers 0.13.3