keyword-masked-model

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.6588
• Validation Loss: 0.5614
• Train Rouge1: 81.6702
• Train Rouge2: 69.0116
• Train Rougel: 81.6273
• Train Rougelsum: 81.5364
• Epoch: 9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rouge1	Train Rouge2	Train Rougel	Train Rougelsum	Epoch
0.9026	0.7256	78.4320	65.5502	78.2535	78.1327	0
0.8436	0.6875	79.2603	66.4389	79.1002	79.0620	1
0.7989	0.6597	79.8406	66.7444	79.5641	79.5095	2
0.7739	0.6403	81.0719	68.0576	80.8293	80.7287	3
0.7439	0.6246	81.0565	68.0129	80.7808	80.6909	4
0.7209	0.6135	81.1721	68.2028	80.9586	80.8343	5
0.6962	0.5982	81.6791	68.9723	81.5971	81.5262	6
0.6922	0.5822	81.7266	69.0548	81.6877	81.6085	7
0.6657	0.5696	82.0421	69.3520	81.9003	81.8580	8
0.6588	0.5614	81.6702	69.0116	81.6273	81.5364	9

Framework versions

• Transformers 4.31.0
• TensorFlow 2.12.0
• Datasets 2.13.1
• Tokenizers 0.13.3