BramVanroy/mbart-large-cc25-ft-amr30-es_nl-stratified

ES and NL to AMR parsing (stratified)

This version was trained on a subselection of the data. The AMR 3.0 corpus was translated to all the relevant languages. We then divided the dataset so that in total we only see half of each language's dataset (so that in total we only see the full AMR 3.0 corpus in size once). In other words, all languages were undersampled for research purposes.

This model is a fine-tuned version of facebook/mbart-large-cc25 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6212
• Smatch Precision: 72.94
• Smatch Recall: 75.83
• Smatch Fscore: 74.36
• Smatch Unparsable: 0
• Percent Not Recoverable: 0.4065

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Smatch Precision	Smatch Recall	Smatch Fscore	Smatch Unparsable	Percent Not Recoverable
0.4098	1.0	3477	1.3168	17.61	63.89	27.62	0	0.0
0.3307	2.0	6954	1.0109	21.08	68.69	32.26	0	0.0581
0.1253	3.0	10431	0.9193	32.88	71.46	45.04	0	0.0
0.1665	4.0	13908	0.7549	35.07	72.54	47.29	0	0.0
0.0435	5.0	17385	0.8298	40.25	74.91	52.37	0	0.0581
0.2156	6.0	20862	0.6525	45.7	75.11	56.82	0	0.0
0.133	7.0	24339	0.6548	47.7	75.36	58.42	0	0.0
0.0624	8.0	27817	0.6054	53.59	75.18	62.57	0	0.0
0.0841	9.0	31294	0.6496	54.68	75.01	63.25	0	0.0581
0.1073	10.0	34771	0.5960	55.76	76.35	64.45	0	0.0
0.048	11.0	38248	0.5924	60.99	76.4	67.83	0	0.0
0.0341	12.0	41725	0.5880	60.39	76.31	67.42	0	0.0581
0.0079	13.0	45202	0.6117	61.61	76.52	68.26	0	0.0
0.0244	14.0	48679	0.6191	63.78	76.44	69.54	0	0.0581
0.0575	15.0	52156	0.6320	66.27	76.71	71.11	0	0.1161
0.0204	16.0	55634	0.6126	67.51	76.48	71.72	0	0.0
0.0278	17.0	59111	0.6114	67.6	76.8	71.91	0	0.0581
0.0219	18.0	62588	0.6184	68.84	77.14	72.75	0	0.0581
0.01	19.0	66065	0.6197	69.62	76.77	73.02	0	0.0
0.0423	20.0	69542	0.6204	71.01	76.89	73.83	0	0.0581
0.0095	21.0	73019	0.6309	70.76	76.53	73.53	0	0.0581
0.0132	22.0	76496	0.6208	71.97	76.41	74.12	0	0.2904
0.0148	23.0	79973	0.6307	71.86	76.61	74.16	0	0.0581
0.0034	24.0	83451	0.6258	72.41	76.24	74.28	0	0.3484
0.0527	25.0	86925	0.6212	72.94	75.83	74.36	0	0.4065

Framework versions

• Transformers 4.34.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.2
• Tokenizers 0.13.3