text_shortening_model_v27

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.1933
• Rouge1: 0.4266
• Rouge2: 0.2061
• Rougel: 0.38
• Rougelsum: 0.3804
• Bert precision: 0.8628
• Bert recall: 0.8555
• Average word count: 8.003
• Max word count: 16
• Min word count: 3
• Average token count: 12.3784
• % shortened texts with length > 12: 3.003

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
2.4306	1.0	145	1.8708	0.4779	0.2499	0.4349	0.4355	0.8758	0.866	7.9099	16	3	12.3093	5.1051
1.7537	2.0	290	1.8412	0.4532	0.2437	0.4165	0.4174	0.8687	0.8604	8.4775	19	3	12.8859	6.9069
1.4338	3.0	435	1.7898	0.4365	0.219	0.4002	0.4007	0.868	0.856	7.6637	14	3	11.8919	2.1021
1.2645	4.0	580	1.8826	0.4609	0.238	0.4158	0.4159	0.8711	0.8637	8.4655	16	4	12.8228	6.006
1.1208	5.0	725	1.9741	0.4389	0.2351	0.4038	0.4051	0.8719	0.8568	7.5886	18	3	12.1231	2.4024
1.0057	6.0	870	1.9700	0.4658	0.2526	0.4275	0.4276	0.8728	0.8646	8.0841	19	2	12.3634	7.8078
0.973	7.0	1015	2.0594	0.4488	0.2358	0.4085	0.4093	0.8735	0.8591	7.3063	14	4	11.6757	0.9009
0.9018	8.0	1160	2.0945	0.4362	0.2229	0.4006	0.4005	0.8654	0.8568	8.1411	19	4	12.5435	8.4084
0.8608	9.0	1305	2.1088	0.4096	0.1926	0.372	0.372	0.8603	0.8514	8.0661	19	2	12.7297	3.6036
0.8243	10.0	1450	2.2384	0.4237	0.2089	0.3876	0.3891	0.8688	0.8548	7.4775	18	3	11.8228	2.1021
0.7966	11.0	1595	2.2565	0.418	0.2104	0.3823	0.3824	0.8673	0.847	7.2402	19	2	11.4024	2.4024
0.7687	12.0	1740	2.3329	0.4238	0.2061	0.3819	0.383	0.8649	0.8518	8.0721	19	2	12.4715	6.006
0.7548	13.0	1885	2.2799	0.4253	0.2129	0.3822	0.3835	0.8642	0.8532	7.9069	17	4	12.2733	4.2042
0.7301	14.0	2030	2.4219	0.4066	0.1904	0.3715	0.3728	0.8629	0.8478	7.4324	18	4	11.6697	3.6036
0.7011	15.0	2175	2.3663	0.4463	0.2222	0.4042	0.4052	0.8655	0.8606	8.3634	16	4	12.955	6.9069
0.6667	16.0	2320	2.5128	0.4238	0.1918	0.3835	0.3843	0.8631	0.8522	7.6456	15	3	12.0841	2.4024
0.6854	17.0	2465	2.3646	0.4202	0.2011	0.3774	0.3776	0.861	0.8543	8.3664	17	2	13.033	8.4084
0.648	18.0	2610	2.5636	0.4159	0.2074	0.3753	0.3751	0.8562	0.8525	8.5135	19	4	13.024	6.006
0.6346	19.0	2755	2.5641	0.4173	0.1937	0.3732	0.3735	0.8592	0.8549	8.8078	19	3	13.0931	12.3123
0.6223	20.0	2900	2.5289	0.4268	0.2164	0.3904	0.3897	0.8617	0.8574	8.2372	17	4	12.9099	5.4054
0.6127	21.0	3045	2.4946	0.427	0.2022	0.3844	0.3842	0.8645	0.8575	8.0511	16	3	12.8108	5.7057
0.6209	22.0	3190	2.6277	0.3987	0.1934	0.3657	0.3657	0.8584	0.8506	7.8859	18	3	12.1742	5.4054
0.5752	23.0	3335	2.7998	0.4019	0.1954	0.3648	0.3646	0.8576	0.8511	8.3904	17	3	12.7057	7.5075
0.5588	24.0	3480	2.6732	0.4039	0.1948	0.3649	0.3652	0.8594	0.8492	7.8829	15	3	12.0901	6.006
0.5641	25.0	3625	2.6012	0.419	0.2091	0.376	0.3765	0.8588	0.8523	8.03	16	3	12.2763	3.003
0.5525	26.0	3770	2.6587	0.418	0.1929	0.3722	0.3726	0.8577	0.8545	8.5345	17	4	13.0961	8.1081
0.5372	27.0	3915	2.7572	0.4104	0.1895	0.366	0.3671	0.8583	0.8495	7.8949	17	3	12.1862	4.8048
0.5105	28.0	4060	2.7023	0.4319	0.2127	0.3884	0.3891	0.8636	0.8571	8.2553	16	3	12.5495	6.6066
0.5026	29.0	4205	2.6991	0.4252	0.2222	0.3899	0.3903	0.867	0.8543	7.7898	19	4	12.2643	4.2042
0.4956	30.0	4350	2.7064	0.4066	0.1974	0.3726	0.3735	0.8568	0.8523	8.4985	18	3	13.021	8.7087
0.5064	31.0	4495	2.7564	0.4159	0.205	0.3763	0.3765	0.8613	0.8523	7.6877	16	3	12.3393	3.003
0.4932	32.0	4640	2.6909	0.394	0.1866	0.3564	0.3573	0.8574	0.8496	7.8378	16	2	12.4715	3.6036
0.4757	33.0	4785	2.7851	0.4117	0.1932	0.3719	0.3728	0.8582	0.8534	8.5946	18	3	12.973	8.1081
0.4753	34.0	4930	2.7823	0.3814	0.1747	0.3466	0.3464	0.8555	0.8459	7.7357	18	3	12.0721	3.3033
0.4603	35.0	5075	2.7607	0.4135	0.2003	0.3777	0.3781	0.8616	0.8538	8.0601	19	3	12.3183	5.4054
0.4645	36.0	5220	2.8364	0.4073	0.1957	0.3643	0.3652	0.8544	0.8524	8.8529	19	2	13.1982	12.012
0.4377	37.0	5365	2.7809	0.3965	0.192	0.357	0.3573	0.858	0.8442	7.4384	19	2	11.5495	2.4024
0.4287	38.0	5510	2.7801	0.4191	0.1984	0.3774	0.3779	0.8593	0.8533	8.2462	16	2	12.5015	6.3063
0.4295	39.0	5655	2.7206	0.4281	0.2104	0.3851	0.3861	0.8634	0.856	8.1922	16	4	12.5826	5.7057
0.4121	40.0	5800	2.8363	0.4049	0.1916	0.3614	0.3624	0.8553	0.8516	8.5435	19	4	12.7928	9.6096
0.3893	41.0	5945	2.7785	0.4255	0.2086	0.3858	0.3864	0.8601	0.8574	8.3964	17	4	13.0541	4.5045
0.3786	42.0	6090	2.8752	0.3908	0.1775	0.3497	0.3509	0.851	0.8463	8.2853	17	2	12.8679	7.8078
0.3703	43.0	6235	2.8799	0.4148	0.1894	0.3719	0.3727	0.8606	0.8519	8.1502	18	3	12.4745	3.9039
0.3636	44.0	6380	2.8542	0.4043	0.1922	0.3631	0.3635	0.8554	0.8504	8.2883	19	4	12.7147	4.5045
0.3438	45.0	6525	2.8282	0.4218	0.2022	0.3792	0.3802	0.861	0.8528	8.2072	16	4	12.6486	6.3063
0.3511	46.0	6670	2.9184	0.405	0.1934	0.3652	0.3658	0.8572	0.8487	8.2372	18	3	12.5526	7.5075
0.3453	47.0	6815	2.8649	0.4064	0.1956	0.3681	0.3686	0.8601	0.8508	8.0871	16	3	12.3604	5.7057
0.3299	48.0	6960	2.9183	0.4266	0.202	0.3777	0.3787	0.8591	0.8578	8.6216	17	4	13.2402	9.009
0.3132	49.0	7105	2.9077	0.4242	0.2021	0.3784	0.3793	0.861	0.8562	8.4354	19	4	12.6877	5.1051
0.3031	50.0	7250	2.9042	0.4177	0.1977	0.3741	0.3752	0.8584	0.8522	8.006	15	4	12.4565	2.7027
0.2974	51.0	7395	2.8820	0.4318	0.2087	0.3849	0.3854	0.8605	0.857	8.2613	16	3	12.8769	6.9069
0.2873	52.0	7540	2.8622	0.4194	0.2023	0.3786	0.3782	0.8626	0.8542	8.021	18	3	12.3243	3.003
0.2718	53.0	7685	2.8665	0.4128	0.2043	0.3716	0.3717	0.8592	0.8541	8.2643	16	3	12.8348	6.006
0.2598	54.0	7830	2.9774	0.4177	0.1983	0.3794	0.3797	0.8612	0.8511	7.8709	19	3	12.048	4.2042
0.2532	55.0	7975	2.8569	0.4111	0.1959	0.3717	0.3723	0.8612	0.8531	7.9399	16	3	12.5315	3.6036
0.2363	56.0	8120	2.9634	0.4253	0.2111	0.385	0.386	0.8657	0.8543	7.8438	14	3	12.3153	3.003
0.2323	57.0	8265	2.9573	0.418	0.1924	0.3771	0.3781	0.8573	0.854	8.4234	16	3	13.1261	6.3063
0.2223	58.0	8410	2.9307	0.4276	0.2079	0.3847	0.3854	0.8651	0.8545	7.7688	16	3	11.97	2.1021
0.2101	59.0	8555	2.9953	0.4114	0.1928	0.3673	0.3681	0.8562	0.8502	8.3814	19	4	12.7087	5.7057
0.2069	60.0	8700	2.9768	0.4154	0.1921	0.3718	0.3725	0.8619	0.8538	7.97	16	4	12.2072	3.9039
0.1971	61.0	8845	2.9218	0.4276	0.2046	0.3847	0.3854	0.8609	0.8568	8.4414	18	4	12.8949	6.3063
0.1873	62.0	8990	2.9857	0.4068	0.191	0.3606	0.3609	0.8559	0.8496	8.2583	16	4	12.5646	5.1051
0.1815	63.0	9135	2.8995	0.417	0.1981	0.3722	0.3723	0.8624	0.8528	8.003	15	4	12.2583	5.7057
0.1807	64.0	9280	2.9644	0.415	0.1933	0.3694	0.3693	0.8585	0.8541	8.4024	17	3	12.7688	7.5075
0.1729	65.0	9425	2.9640	0.4138	0.1965	0.3692	0.3698	0.8576	0.8515	8.042	16	3	12.6036	4.2042
0.1609	66.0	9570	2.9912	0.4255	0.2051	0.3816	0.3826	0.8632	0.8554	8.0751	16	4	12.2733	5.1051
0.1621	67.0	9715	3.0527	0.4249	0.2033	0.3786	0.3793	0.862	0.8544	8.0631	15	2	12.4925	3.3033
0.1468	68.0	9860	3.0214	0.4274	0.2053	0.3822	0.3824	0.861	0.8552	8.4204	18	4	12.7447	7.8078
0.1334	69.0	10005	3.1114	0.4116	0.1911	0.3698	0.3695	0.8601	0.8515	7.9099	14	3	12.0961	3.9039
0.1261	70.0	10150	2.9442	0.4226	0.2032	0.3783	0.3785	0.8625	0.854	8.033	16	3	12.4384	4.5045
0.1137	71.0	10295	3.0685	0.422	0.2035	0.375	0.3757	0.8621	0.8543	8.0541	16	2	12.3904	3.9039
0.1078	72.0	10440	2.9759	0.4198	0.1981	0.3759	0.3767	0.8602	0.8544	8.1712	16	2	12.7297	4.5045
0.1074	73.0	10585	2.9892	0.4226	0.2082	0.3835	0.3841	0.8621	0.8556	8.0661	14	2	12.5195	4.2042
0.105	74.0	10730	3.0216	0.427	0.1997	0.379	0.3801	0.8611	0.8562	8.3093	17	3	12.8108	5.1051
0.0944	75.0	10875	3.0108	0.4169	0.1956	0.3714	0.3721	0.8582	0.8523	8.1231	14	4	12.7568	3.003
0.0871	76.0	11020	3.0794	0.4246	0.2007	0.3739	0.3756	0.8593	0.8556	8.3063	14	4	12.7598	4.8048
0.0739	77.0	11165	3.0940	0.4205	0.1974	0.3776	0.3792	0.8629	0.8532	7.9189	15	2	12.0961	3.003
0.066	78.0	11310	3.0764	0.4234	0.201	0.3781	0.3785	0.8603	0.8559	8.2913	16	3	12.8198	4.8048
0.0641	79.0	11455	3.0736	0.4299	0.2067	0.3831	0.3835	0.8622	0.8568	8.018	15	4	12.4835	3.003
0.0633	80.0	11600	3.0852	0.4191	0.2007	0.3741	0.3741	0.86	0.8537	8.1742	19	3	12.5556	4.8048
0.0625	81.0	11745	3.0450	0.4153	0.1989	0.3734	0.374	0.8583	0.8524	8.1321	16	4	12.5826	3.9039
0.0624	82.0	11890	3.1202	0.4286	0.209	0.385	0.3851	0.8642	0.8557	8.0	16	4	12.3003	3.003
0.0593	83.0	12035	3.0514	0.4319	0.2159	0.3887	0.3899	0.8653	0.8587	8.0601	14	4	12.4805	1.8018
0.0562	84.0	12180	3.0821	0.4362	0.2166	0.3924	0.3925	0.8656	0.8576	8.1051	15	4	12.5736	4.5045
0.0586	85.0	12325	3.0843	0.4297	0.2061	0.3861	0.3865	0.8649	0.856	8.1051	15	3	12.3964	5.1051
0.0528	86.0	12470	3.0610	0.4209	0.2034	0.3752	0.3755	0.8606	0.8542	8.2162	16	4	12.6817	5.1051
0.0478	87.0	12615	3.0935	0.4244	0.2076	0.382	0.3815	0.8596	0.8553	8.3243	15	2	12.9009	6.006
0.0431	88.0	12760	3.0865	0.429	0.2092	0.3847	0.3843	0.8645	0.855	7.964	15	4	12.2312	3.003
0.0453	89.0	12905	3.0960	0.4147	0.1984	0.3718	0.3722	0.8619	0.8528	7.9219	14	3	12.2973	3.3033
0.0429	90.0	13050	3.1163	0.4237	0.205	0.3776	0.3776	0.8622	0.8552	8.1231	16	4	12.4985	3.003
0.0381	91.0	13195	3.0962	0.427	0.2089	0.3814	0.3817	0.8624	0.8547	8.006	14	4	12.3664	2.4024
0.0374	92.0	13340	3.1022	0.4275	0.2031	0.3818	0.3823	0.8636	0.8574	8.2042	15	3	12.5646	4.2042
0.0357	93.0	13485	3.1479	0.4282	0.2089	0.3855	0.3865	0.8637	0.8559	8.009	17	3	12.2492	3.003
0.0329	94.0	13630	3.1188	0.4311	0.2086	0.3858	0.3861	0.8646	0.8559	7.8949	15	3	12.2703	2.4024
0.0307	95.0	13775	3.1409	0.4284	0.2099	0.3825	0.3828	0.8633	0.8562	7.994	17	3	12.3153	2.4024
0.0291	96.0	13920	3.1605	0.4292	0.2074	0.3831	0.3833	0.8635	0.8554	7.8979	14	4	12.3243	1.5015
0.0299	97.0	14065	3.1838	0.4274	0.2022	0.3791	0.3792	0.863	0.8552	7.9489	16	4	12.3303	2.1021
0.0264	98.0	14210	3.1810	0.4224	0.201	0.3762	0.3773	0.8624	0.8544	7.9309	16	3	12.2372	2.4024
0.0257	99.0	14355	3.1893	0.4241	0.2056	0.3785	0.3796	0.8624	0.855	7.985	16	3	12.3874	2.4024
0.0244	100.0	14500	3.1933	0.4266	0.2061	0.38	0.3804	0.8628	0.8555	8.003	16	3	12.3784	3.003

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3