hubert2BertMusic100

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4721
• Rouge1: 26.5915
• Rouge2: 7.5185
• Rougel: 18.8807
• Rougelsum: 18.8279
• Gen Len: 56.16

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.9005	1.0	1361	1.6875	27.9936	8.03	19.6327	19.583	52.7
1.7493	2.0	2722	1.6272	26.1413	7.7635	18.9764	18.8667	53.6
1.7037	3.0	4083	1.5873	27.1884	7.3279	19.607	19.6344	56.55
1.6621	4.0	5444	1.5505	31.0103	8.788	22.4097	22.3902	60.45
1.6824	5.0	6805	1.5272	26.0334	7.709	18.5954	18.5607	54.87
1.6451	6.0	8166	1.5073	30.49	7.7309	21.4853	21.537	65.1
1.6502	7.0	9527	1.4947	30.1318	8.4307	21.6758	21.7107	61.48
1.6588	8.0	10888	1.4809	27.3167	7.7378	19.4021	19.3359	57.38
1.6896	9.0	12249	1.4737	26.7199	7.5034	18.9846	18.9279	55.96
1.7313	10.0	13610	1.4721	26.5915	7.5185	18.8807	18.8279	56.16

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.2
• Tokenizers 0.13.3