text_shortening_model_v13

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0548
• Rouge1: 0.5772
• Rouge2: 0.3353
• Rougel: 0.5189
• Rougelsum: 0.5189
• Bert precision: 0.8941
• Bert recall: 0.8987
• Average word count: 11.2143
• Max word count: 15
• Min word count: 6
• Average token count: 16.5071
• % shortened texts with length > 12: 30.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.1544	1.0	62	1.6383	0.5496	0.3173	0.5011	0.5016	0.8821	0.8901	11.7214	17	4	17.0214	50.0
1.0203	2.0	124	1.5337	0.569	0.3214	0.5114	0.5112	0.8863	0.8968	11.9857	17	5	17.2214	50.7143
0.9474	3.0	186	1.5169	0.5754	0.3283	0.5196	0.5186	0.8874	0.8985	12.0857	17	6	17.2571	53.5714
0.8615	4.0	248	1.5058	0.5785	0.3368	0.5211	0.5198	0.8917	0.9006	11.7	17	6	16.8714	47.8571
0.8182	5.0	310	1.4855	0.5817	0.3284	0.5203	0.5195	0.8907	0.9	11.7357	17	6	16.9357	46.4286
0.784	6.0	372	1.4813	0.5862	0.3398	0.5242	0.5242	0.8918	0.9016	11.7	16	6	17.0	45.0
0.7749	7.0	434	1.4723	0.581	0.334	0.5241	0.5233	0.8951	0.8984	11.1929	16	6	16.3286	32.1429
0.7396	8.0	496	1.4936	0.5791	0.3402	0.5184	0.5183	0.8933	0.8992	11.4786	17	6	16.5571	34.2857
0.6856	9.0	558	1.5083	0.5757	0.3364	0.5174	0.5172	0.8944	0.8979	11.2	16	6	16.2357	30.7143
0.6679	10.0	620	1.5295	0.5814	0.3399	0.5271	0.5276	0.8915	0.9	11.7786	16	7	16.9143	40.0
0.6506	11.0	682	1.5363	0.5829	0.3491	0.5282	0.5283	0.8953	0.8994	11.3786	16	6	16.5286	33.5714
0.6521	12.0	744	1.5526	0.5645	0.3303	0.5095	0.5096	0.8914	0.8951	11.2286	16	5	16.4929	30.7143
0.6125	13.0	806	1.5787	0.5709	0.324	0.5097	0.5108	0.8906	0.8953	11.4214	16	6	16.6571	35.0
0.5915	14.0	868	1.5946	0.5757	0.3373	0.5152	0.5159	0.8926	0.8969	11.4071	16	6	16.5571	32.8571
0.5737	15.0	930	1.6204	0.577	0.3322	0.5219	0.5223	0.8918	0.8986	11.5929	16	6	16.8214	35.7143
0.5812	16.0	992	1.6372	0.5748	0.3243	0.52	0.5203	0.891	0.8977	11.6071	16	7	16.8214	37.8571
0.5468	17.0	1054	1.6514	0.5673	0.3304	0.5152	0.5152	0.895	0.8954	11.0	15	5	15.9929	26.4286
0.56	18.0	1116	1.6630	0.576	0.3273	0.5229	0.5228	0.8907	0.898	11.5786	16	6	16.8429	35.0
0.5548	19.0	1178	1.6868	0.5739	0.3262	0.5139	0.5135	0.8923	0.8972	11.3429	16	6	16.5929	33.5714
0.5338	20.0	1240	1.6954	0.5702	0.3295	0.518	0.5182	0.8914	0.8975	11.6	16	6	16.7429	37.8571
0.5323	21.0	1302	1.7255	0.585	0.3376	0.5262	0.5266	0.8938	0.9007	11.5643	16	6	16.7429	35.0
0.5075	22.0	1364	1.7320	0.5708	0.3272	0.5137	0.5144	0.8929	0.8953	11.3286	16	6	16.4143	32.1429
0.4916	23.0	1426	1.7601	0.5724	0.3276	0.5161	0.5171	0.8928	0.8965	11.3357	16	6	16.6	31.4286
0.4789	24.0	1488	1.7779	0.5726	0.3253	0.5128	0.513	0.8934	0.8964	11.4143	16	6	16.5	35.0
0.4851	25.0	1550	1.7970	0.575	0.3318	0.5204	0.521	0.8935	0.8982	11.4429	16	6	16.6714	31.4286
0.4682	26.0	1612	1.8094	0.5783	0.3376	0.5203	0.5213	0.8937	0.8984	11.4714	16	6	16.7571	30.0
0.4703	27.0	1674	1.8299	0.5814	0.3383	0.5208	0.5215	0.8934	0.8982	11.3929	16	6	16.6643	30.0
0.483	28.0	1736	1.8396	0.576	0.3394	0.5155	0.5162	0.8945	0.8975	11.3357	16	6	16.3857	28.5714
0.4712	29.0	1798	1.8567	0.5741	0.326	0.5125	0.5129	0.893	0.8981	11.5	16	6	16.5786	35.7143
0.4679	30.0	1860	1.8818	0.5855	0.3416	0.5239	0.5242	0.895	0.9005	11.4429	16	6	16.7143	33.5714
0.4653	31.0	1922	1.8758	0.5805	0.3378	0.5217	0.5222	0.894	0.8986	11.3357	16	6	16.4857	30.0
0.4484	32.0	1984	1.8920	0.5812	0.3363	0.5207	0.5206	0.8946	0.8991	11.3357	16	6	16.5143	30.0
0.4428	33.0	2046	1.8925	0.5832	0.3372	0.5195	0.5203	0.8968	0.8987	11.1286	16	6	16.2214	26.4286
0.4266	34.0	2108	1.9185	0.5736	0.3322	0.517	0.518	0.8952	0.8974	11.0214	15	6	16.1714	25.0
0.429	35.0	2170	1.9366	0.5829	0.3371	0.5224	0.5231	0.8965	0.8988	11.1643	16	6	16.25	27.1429
0.4034	36.0	2232	1.9510	0.5823	0.3392	0.5288	0.5288	0.8963	0.8986	11.1143	15	6	16.1214	30.0
0.4111	37.0	2294	1.9517	0.587	0.3426	0.529	0.5296	0.8959	0.9011	11.3857	16	6	16.55	31.4286
0.4318	38.0	2356	1.9450	0.5851	0.3444	0.5262	0.5268	0.8963	0.9009	11.2714	16	6	16.4571	30.0
0.4399	39.0	2418	1.9539	0.5772	0.3339	0.5164	0.5169	0.8958	0.8995	11.0929	15	6	16.2929	25.0
0.4268	40.0	2480	1.9620	0.5806	0.3319	0.5187	0.5188	0.8962	0.8983	11.0214	16	6	16.0643	26.4286
0.4119	41.0	2542	1.9939	0.5819	0.3408	0.5239	0.5238	0.8945	0.8992	11.3	16	6	16.4929	30.0
0.4061	42.0	2604	1.9714	0.5813	0.338	0.5214	0.5228	0.897	0.8997	11.05	16	6	16.2429	25.7143
0.4176	43.0	2666	1.9911	0.5847	0.3388	0.5266	0.5265	0.8951	0.9003	11.1929	16	6	16.4643	28.5714
0.4041	44.0	2728	2.0105	0.5844	0.3468	0.5257	0.5256	0.8957	0.901	11.1786	15	6	16.5357	29.2857
0.3925	45.0	2790	2.0220	0.5787	0.3423	0.5179	0.5185	0.8936	0.8992	11.25	16	6	16.5143	32.1429
0.4095	46.0	2852	2.0179	0.581	0.3404	0.5197	0.5202	0.8957	0.8998	11.2143	16	6	16.4357	29.2857
0.397	47.0	2914	2.0124	0.5803	0.3385	0.5188	0.5193	0.8952	0.899	11.2357	16	6	16.2786	32.1429
0.3801	48.0	2976	2.0186	0.5778	0.3359	0.518	0.518	0.8944	0.8986	11.2143	16	6	16.4	32.1429
0.3966	49.0	3038	2.0234	0.5807	0.337	0.5185	0.5192	0.8953	0.9001	11.2571	16	6	16.4929	30.0
0.3838	50.0	3100	2.0317	0.5807	0.3427	0.523	0.5234	0.8954	0.8989	11.0571	16	6	16.2786	26.4286
0.3818	51.0	3162	2.0281	0.5811	0.3428	0.5238	0.5242	0.8956	0.9001	11.1643	16	6	16.3643	30.7143
0.3793	52.0	3224	2.0399	0.5824	0.3438	0.5214	0.522	0.8947	0.9003	11.2071	16	6	16.4714	30.7143
0.3734	53.0	3286	2.0470	0.5811	0.3413	0.5222	0.5227	0.8952	0.9	11.1643	15	6	16.4214	29.2857
0.3876	54.0	3348	2.0509	0.5764	0.3382	0.515	0.5156	0.8948	0.8983	11.1071	15	6	16.2643	28.5714
0.3736	55.0	3410	2.0498	0.5722	0.3331	0.5135	0.514	0.8937	0.8972	11.1357	16	6	16.3071	27.8571
0.3981	56.0	3472	2.0499	0.5726	0.3337	0.5133	0.5138	0.8939	0.8977	11.1286	15	6	16.3357	29.2857
0.3731	57.0	3534	2.0500	0.5767	0.3353	0.5173	0.5176	0.8946	0.8984	11.1286	15	6	16.3643	27.8571
0.3786	58.0	3596	2.0529	0.5779	0.3377	0.5199	0.5208	0.895	0.8994	11.1929	16	6	16.4357	28.5714
0.3648	59.0	3658	2.0545	0.5766	0.3348	0.518	0.5181	0.8939	0.8985	11.2143	15	6	16.5143	30.0
0.373	60.0	3720	2.0548	0.5772	0.3353	0.5189	0.5189	0.8941	0.8987	11.2143	15	6	16.5071	30.0

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cpu
• Datasets 2.14.5
• Tokenizers 0.13.3