esp-to-lsm-barto-model

This model is a fine-tuned version of vgaraujov/bart-base-spanish on Spanish-Mexican Sign Language (MSL) glosses dataset. It achieves the following results on the evaluation set:

• Loss: 0.0118
• Bleu: 82.2615
• Rouge: {'rouge1': 0.9459411340293693, 'rouge2': 0.8725612535612537, 'rougeL': 0.9409690603514131, 'rougeLsum': 0.9414154570919278}
• Ter: 7.9703

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.5e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Rouge	Ter
0.0862	1.0	85	0.0441	48.0832	{'rouge1': 0.7799851292498354, 'rouge2': 0.6094051319051321, 'rougeL': 0.7634691389764923, 'rougeLsum': 0.7637001348324879}	34.4764
0.0323	2.0	170	0.0214	72.9928	{'rouge1': 0.884157046828874, 'rouge2': 0.7702001337442517, 'rougeL': 0.8722237432043938, 'rougeLsum': 0.8720308750417114}	16.6821
0.0198	3.0	255	0.0161	78.5669	{'rouge1': 0.9199390239390244, 'rouge2': 0.82409139009139, 'rougeL': 0.911472143869203, 'rougeLsum': 0.9116302602626135}	12.0482
0.0162	4.0	340	0.0143	79.2390	{'rouge1': 0.9243402735608619, 'rouge2': 0.8460535205535207, 'rougeL': 0.9158039177598002, 'rougeLsum': 0.9159157335039689}	10.7507
0.0137	5.0	425	0.0137	82.3938	{'rouge1': 0.9334139504286565, 'rouge2': 0.8579696784696786, 'rougeL': 0.9274591149591148, 'rougeLsum': 0.927894385026738}	9.6386
0.0108	6.0	510	0.0128	84.1329	{'rouge1': 0.9350887445887449, 'rouge2': 0.8754486161986161, 'rougeL': 0.9311620617944146, 'rougeLsum': 0.9313348612172142}	9.0825
0.0098	7.0	595	0.0129	79.7416	{'rouge1': 0.9399191766838828, 'rouge2': 0.8716096403596405, 'rougeL': 0.9330582073155609, 'rougeLsum': 0.933733249865603}	9.4532
0.009	8.0	680	0.0125	82.9321	{'rouge1': 0.9443956476530007, 'rouge2': 0.8689144281644281, 'rougeL': 0.9390896358543419, 'rougeLsum': 0.9394144809438929}	8.8971
0.0084	9.0	765	0.0122	81.9698	{'rouge1': 0.946071417961124, 'rouge2': 0.8742369759869761, 'rougeL': 0.9409199134199135, 'rougeLsum': 0.9414803284950346}	9.0825
0.0068	10.0	850	0.0121	81.9526	{'rouge1': 0.9484588107970461, 'rouge2': 0.8778730158730159, 'rougeL': 0.9433170783464899, 'rougeLsum': 0.9437279305661661}	8.4337
0.0078	11.0	935	0.0118	82.4911	{'rouge1': 0.9460536750830865, 'rouge2': 0.8745218762718765, 'rougeL': 0.9401823225793814, 'rougeLsum': 0.9404524821583646}	8.8044
0.0063	12.0	1020	0.0120	81.7252	{'rouge1': 0.9465396825396828, 'rouge2': 0.8755185185185186, 'rougeL': 0.9404898777692895, 'rougeLsum': 0.941089275103981}	8.8044
0.0069	13.0	1105	0.0121	81.7348	{'rouge1': 0.9456640068308027, 'rouge2': 0.8716636381048146, 'rougeL': 0.940350419274568, 'rougeLsum': 0.941292909747631}	8.5264
0.0059	14.0	1190	0.0120	82.7243	{'rouge1': 0.9473343307019777, 'rouge2': 0.8731392958892958, 'rougeL': 0.9422385620915033, 'rougeLsum': 0.9425819221628045}	8.4337
0.006	15.0	1275	0.0118	81.2037	{'rouge1': 0.9470927234530175, 'rouge2': 0.8718730158730159, 'rougeL': 0.942004562431033, 'rougeLsum': 0.9425554706731177}	8.3411
0.0055	16.0	1360	0.0119	82.1601	{'rouge1': 0.9435703038791275, 'rouge2': 0.8706992266992267, 'rougeL': 0.938448826500297, 'rougeLsum': 0.9388509252185724}	8.1557
0.0055	17.0	1445	0.0119	82.0465	{'rouge1': 0.9453517120564336, 'rouge2': 0.8718517740429506, 'rougeL': 0.9403101408825094, 'rougeLsum': 0.940731923391366}	8.0630
0.0051	18.0	1530	0.0118	82.1849	{'rouge1': 0.9452478036669215, 'rouge2': 0.8716373626373629, 'rougeL': 0.9402017337237925, 'rougeLsum': 0.9406384008148714}	8.0630
0.0055	19.0	1615	0.0118	82.0985	{'rouge1': 0.9452005559799677, 'rouge2': 0.8723565323565323, 'rougeL': 0.9399868644427471, 'rougeLsum': 0.9405265469824293}	8.0630
0.0052	20.0	1700	0.0118	82.2615	{'rouge1': 0.9459411340293693, 'rouge2': 0.8725612535612537, 'rougeL': 0.9409690603514131, 'rougeLsum': 0.9414154570919278}	7.9703

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1