text_shortening_model_v31

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.7416
• Rouge1: 0.4961
• Rouge2: 0.2712
• Rougel: 0.4388
• Rougelsum: 0.4386
• Bert precision: 0.8749
• Bert recall: 0.8711
• Average word count: 8.5135
• Max word count: 16
• Min word count: 3
• Average token count: 13.1592
• % shortened texts with length > 12: 10.2102

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.1978	1.0	145	1.5250	0.4953	0.2842	0.4528	0.4524	0.8806	0.8681	7.8919	18	3	12.4234	4.2042
1.0092	2.0	290	1.4421	0.5257	0.3053	0.4698	0.4689	0.875	0.8809	9.6006	18	4	14.3574	19.2192
0.8932	3.0	435	1.4060	0.5266	0.3045	0.4728	0.472	0.8766	0.8776	9.0841	18	4	13.6366	14.7147
0.79	4.0	580	1.4022	0.5329	0.3136	0.4714	0.4714	0.8802	0.8797	8.952	16	4	13.6036	12.9129
0.7506	5.0	725	1.4514	0.5145	0.2935	0.4485	0.4485	0.8745	0.8726	8.97	18	4	13.6096	12.012
0.6981	6.0	870	1.4602	0.5146	0.2914	0.4566	0.4559	0.8778	0.8762	8.958	18	3	13.5195	15.3153
0.6426	7.0	1015	1.4745	0.5196	0.2973	0.4596	0.4593	0.8759	0.8788	9.1802	16	4	13.9159	14.1141
0.6251	8.0	1160	1.5026	0.5217	0.2965	0.461	0.4611	0.8802	0.8775	8.8198	16	4	13.3393	12.012
0.5901	9.0	1305	1.5890	0.5156	0.2967	0.4606	0.4609	0.8773	0.876	8.7718	17	3	13.4655	9.6096
0.5544	10.0	1450	1.6294	0.5172	0.287	0.4562	0.4559	0.8779	0.876	8.7688	18	4	13.5195	11.7117
0.5354	11.0	1595	1.6805	0.5169	0.2871	0.457	0.4571	0.8768	0.8774	8.994	17	4	13.6486	14.1141
0.5103	12.0	1740	1.7334	0.5121	0.2824	0.4556	0.455	0.8785	0.8745	8.5465	16	3	13.1021	8.1081
0.4796	13.0	1885	1.7767	0.499	0.2763	0.442	0.4418	0.8754	0.8739	8.6396	17	4	13.3183	11.4114
0.4825	14.0	2030	1.8319	0.5114	0.2849	0.4497	0.4501	0.8746	0.8758	8.994	17	4	13.6667	12.9129
0.4572	15.0	2175	1.8613	0.5129	0.2884	0.4546	0.4549	0.8785	0.8757	8.6877	17	3	13.3784	10.5105
0.4489	16.0	2320	1.8790	0.5144	0.2829	0.4533	0.4536	0.8777	0.8754	8.8078	16	3	13.4955	13.2132
0.4211	17.0	2465	1.9604	0.4936	0.2641	0.4322	0.4326	0.8735	0.8696	8.4985	17	3	13.1892	9.009
0.4246	18.0	2610	2.0639	0.4951	0.2634	0.4331	0.4334	0.8721	0.8703	8.7538	16	4	13.3453	12.6126
0.4063	19.0	2755	2.0587	0.499	0.2685	0.4378	0.4383	0.8741	0.8707	8.5916	16	3	13.3003	9.9099
0.3912	20.0	2900	2.1089	0.5068	0.2727	0.4471	0.4469	0.8764	0.8744	8.7538	18	3	13.4625	11.1111
0.3855	21.0	3045	2.1048	0.5022	0.2704	0.4473	0.4478	0.875	0.8728	8.6847	16	4	13.3483	9.3093
0.3808	22.0	3190	2.1804	0.4977	0.2722	0.4414	0.4412	0.875	0.8711	8.5315	17	4	13.0631	10.8108
0.3851	23.0	3335	2.1740	0.4993	0.2696	0.4442	0.4443	0.8742	0.8719	8.5676	15	3	13.2252	9.009
0.3741	24.0	3480	2.1872	0.4921	0.2683	0.4365	0.4369	0.8728	0.8692	8.5195	17	3	13.2192	8.4084
0.3604	25.0	3625	2.2617	0.4988	0.2681	0.4421	0.4426	0.8747	0.8705	8.5255	17	3	13.2492	8.1081
0.3676	26.0	3770	2.2561	0.4931	0.2603	0.4328	0.4331	0.874	0.8711	8.6276	15	3	13.3363	11.7117
0.3799	27.0	3915	2.2404	0.4912	0.2652	0.4329	0.433	0.8729	0.8702	8.6517	17	3	13.4414	8.1081
0.3617	28.0	4060	2.2728	0.4983	0.2704	0.4424	0.4427	0.8756	0.8734	8.7568	17	3	13.5225	11.4114
0.3588	29.0	4205	2.2695	0.4904	0.2601	0.4331	0.4328	0.8743	0.87	8.4775	18	3	13.1592	9.009
0.3567	30.0	4350	2.3006	0.4993	0.2693	0.4419	0.4417	0.8747	0.8737	8.8529	17	3	13.5976	12.012
0.3573	31.0	4495	2.3257	0.4979	0.2669	0.4378	0.4379	0.8743	0.8735	8.9069	18	3	13.6697	12.9129
0.3471	32.0	4640	2.3513	0.4989	0.2723	0.441	0.4405	0.8758	0.8728	8.6246	17	3	13.3063	10.8108
0.3591	33.0	4785	2.3467	0.4972	0.2751	0.4415	0.4413	0.8742	0.8727	8.8078	17	3	13.5616	10.5105
0.3401	34.0	4930	2.4229	0.4854	0.2661	0.4313	0.4318	0.8737	0.8701	8.5826	17	3	13.2673	8.7087
0.3476	35.0	5075	2.3804	0.4895	0.2602	0.4322	0.4326	0.874	0.8712	8.6577	17	3	13.2883	9.3093
0.3473	36.0	5220	2.4242	0.4938	0.2689	0.438	0.4387	0.8745	0.8713	8.5976	17	3	13.2432	9.3093
0.3415	37.0	5365	2.3836	0.4943	0.2617	0.4351	0.4351	0.8751	0.8711	8.4054	17	3	13.0571	8.1081
0.3549	38.0	5510	2.4110	0.501	0.2696	0.4402	0.4406	0.8765	0.8713	8.2282	17	3	12.9459	6.6066
0.3432	39.0	5655	2.4016	0.4999	0.27	0.4387	0.4393	0.8751	0.8712	8.5285	17	3	13.2402	8.4084
0.3387	40.0	5800	2.4546	0.4986	0.2718	0.4417	0.4422	0.8742	0.871	8.5766	17	3	13.2312	9.3093
0.3351	41.0	5945	2.4478	0.4981	0.2714	0.4367	0.4372	0.8756	0.8722	8.4775	15	3	13.1411	8.7087
0.3366	42.0	6090	2.4447	0.4961	0.2703	0.4359	0.437	0.8746	0.8699	8.4745	16	3	13.1231	9.3093
0.3324	43.0	6235	2.4974	0.4989	0.2809	0.4428	0.4432	0.8747	0.873	8.7147	16	3	13.4565	10.5105
0.3306	44.0	6380	2.4938	0.4902	0.2657	0.4301	0.4306	0.8733	0.8692	8.4925	15	3	13.1622	8.4084
0.3388	45.0	6525	2.5098	0.4788	0.2616	0.4246	0.4245	0.8734	0.8662	8.2162	16	3	12.7538	8.1081
0.346	46.0	6670	2.4595	0.4987	0.2689	0.438	0.4389	0.875	0.8718	8.5676	16	3	13.2252	9.9099
0.3401	47.0	6815	2.5098	0.4934	0.2653	0.4353	0.4356	0.8744	0.87	8.3934	15	3	13.048	8.1081
0.3271	48.0	6960	2.5204	0.4951	0.2674	0.4373	0.4372	0.8749	0.8703	8.4625	16	3	13.024	9.009
0.3267	49.0	7105	2.5291	0.4887	0.2605	0.428	0.4284	0.8728	0.8702	8.7057	18	3	13.3363	11.1111
0.3382	50.0	7250	2.5422	0.4899	0.2666	0.4354	0.4356	0.8755	0.8707	8.4505	16	3	13.0931	8.1081
0.3255	51.0	7395	2.5254	0.4921	0.2634	0.4346	0.4352	0.8738	0.8691	8.4535	16	3	13.027	10.2102
0.32	52.0	7540	2.5460	0.4991	0.2727	0.4423	0.4421	0.8745	0.873	8.8919	16	3	13.5736	11.7117
0.3154	53.0	7685	2.5446	0.5027	0.2712	0.4463	0.4463	0.8768	0.8734	8.6426	16	3	13.2973	11.1111
0.3293	54.0	7830	2.5378	0.4928	0.2669	0.4352	0.4354	0.8736	0.869	8.5225	16	3	13.1291	10.2102
0.3231	55.0	7975	2.5905	0.4949	0.2678	0.4378	0.4375	0.8743	0.8714	8.6426	15	3	13.3003	9.009
0.3239	56.0	8120	2.5884	0.4969	0.2697	0.4399	0.4399	0.8737	0.8712	8.6697	16	3	13.3754	10.5105
0.3174	57.0	8265	2.5500	0.4958	0.267	0.4389	0.4386	0.8739	0.8715	8.7327	16	4	13.3844	10.5105
0.3209	58.0	8410	2.5804	0.4989	0.2706	0.442	0.4426	0.8751	0.8717	8.5766	15	3	13.1952	9.3093
0.3297	59.0	8555	2.5909	0.494	0.2622	0.4343	0.4338	0.8733	0.8698	8.5976	16	3	13.1652	11.7117
0.3226	60.0	8700	2.5857	0.4976	0.2639	0.4377	0.438	0.8753	0.8701	8.3904	17	3	12.973	7.8078
0.3241	61.0	8845	2.5824	0.5011	0.2698	0.4428	0.4436	0.8764	0.8725	8.5345	16	3	13.1502	10.5105
0.3201	62.0	8990	2.6156	0.4968	0.2673	0.4371	0.4372	0.8755	0.8702	8.3904	16	3	12.979	6.9069
0.3234	63.0	9135	2.6374	0.4945	0.2677	0.4387	0.4388	0.8744	0.8693	8.4444	17	3	12.958	8.1081
0.3246	64.0	9280	2.6338	0.4912	0.2672	0.4396	0.4402	0.8738	0.8698	8.4955	17	3	13.1021	8.1081
0.3188	65.0	9425	2.6206	0.4999	0.2739	0.4443	0.4444	0.8763	0.8726	8.6006	17	3	13.2042	10.5105
0.3186	66.0	9570	2.6499	0.5007	0.2771	0.4462	0.4463	0.8765	0.8729	8.5375	17	3	13.2162	9.3093
0.319	67.0	9715	2.6488	0.5023	0.2715	0.4452	0.4454	0.8761	0.8736	8.6817	17	3	13.3904	10.2102
0.3328	68.0	9860	2.6238	0.5002	0.2696	0.4408	0.4411	0.8755	0.8717	8.5075	17	3	13.1081	9.009
0.3068	69.0	10005	2.6525	0.4971	0.2684	0.4391	0.4397	0.8755	0.8712	8.5045	17	3	13.1411	11.4114
0.3192	70.0	10150	2.6494	0.4976	0.2722	0.4395	0.4405	0.8762	0.8714	8.3964	17	3	13.033	8.4084
0.3232	71.0	10295	2.6642	0.4976	0.2717	0.4412	0.4411	0.8756	0.8717	8.5075	17	3	13.1622	9.9099
0.3084	72.0	10440	2.6596	0.4931	0.2669	0.4352	0.4354	0.8734	0.8696	8.4865	17	3	13.1682	9.009
0.313	73.0	10585	2.6551	0.4942	0.2699	0.4363	0.4368	0.8742	0.8699	8.4715	16	3	13.1201	9.6096
0.3194	74.0	10730	2.6769	0.4962	0.2689	0.4388	0.4389	0.874	0.8715	8.5976	17	3	13.2763	10.5105
0.3143	75.0	10875	2.6860	0.493	0.2652	0.4335	0.4343	0.8734	0.8702	8.5706	16	3	13.2462	9.3093
0.3209	76.0	11020	2.6777	0.4893	0.2592	0.4325	0.4324	0.8726	0.869	8.5225	16	3	13.2012	9.3093
0.3078	77.0	11165	2.6797	0.4877	0.261	0.4321	0.4323	0.8724	0.8693	8.5796	16	3	13.2402	9.6096
0.3192	78.0	11310	2.6812	0.495	0.2677	0.4382	0.4383	0.8739	0.871	8.5706	18	3	13.2523	10.8108
0.3147	79.0	11455	2.6777	0.495	0.2693	0.4371	0.4374	0.874	0.8707	8.5015	16	3	13.1471	9.3093
0.3049	80.0	11600	2.6767	0.4917	0.2647	0.4344	0.4346	0.8723	0.8696	8.5616	16	3	13.2162	9.9099
0.3191	81.0	11745	2.6932	0.4929	0.2683	0.4392	0.4392	0.8737	0.8707	8.5676	16	3	13.2342	9.6096
0.3073	82.0	11890	2.7036	0.4959	0.2699	0.4389	0.4393	0.8738	0.8722	8.6547	17	3	13.3964	10.2102
0.3129	83.0	12035	2.6941	0.4918	0.2657	0.4341	0.434	0.8742	0.8703	8.4985	16	3	13.1411	9.3093
0.3308	84.0	12180	2.6968	0.4927	0.2659	0.4335	0.4337	0.8737	0.8698	8.4955	16	3	13.1652	9.3093
0.3221	85.0	12325	2.6966	0.4903	0.2606	0.4306	0.4306	0.8726	0.8698	8.5766	16	3	13.2823	9.6096
0.3085	86.0	12470	2.7123	0.4862	0.2608	0.4288	0.4286	0.8723	0.8688	8.4595	16	3	13.0901	8.7087
0.3281	87.0	12615	2.7101	0.4918	0.2638	0.4322	0.4328	0.8731	0.8695	8.4775	16	3	13.1291	9.009
0.3183	88.0	12760	2.7102	0.4902	0.2649	0.4294	0.4301	0.873	0.8688	8.4955	16	3	13.0901	9.6096
0.3063	89.0	12905	2.7198	0.4934	0.2676	0.4338	0.4344	0.8734	0.8692	8.4565	17	3	13.0751	9.009
0.3123	90.0	13050	2.7228	0.492	0.2676	0.4338	0.4343	0.8732	0.8692	8.4535	17	3	13.0931	9.3093
0.3163	91.0	13195	2.7264	0.4953	0.2702	0.4357	0.4358	0.874	0.8693	8.4625	17	3	13.033	9.3093
0.3085	92.0	13340	2.7236	0.4934	0.2702	0.4369	0.4369	0.8738	0.8695	8.4925	17	3	13.0721	9.9099
0.3257	93.0	13485	2.7202	0.4953	0.2706	0.4368	0.4368	0.8746	0.8699	8.4595	16	3	13.0571	10.2102
0.3092	94.0	13630	2.7261	0.4988	0.2748	0.4415	0.4419	0.8755	0.8708	8.4535	16	3	13.0751	9.9099
0.3187	95.0	13775	2.7248	0.4968	0.2727	0.4383	0.4389	0.8751	0.8709	8.5075	16	3	13.1321	9.9099
0.3155	96.0	13920	2.7335	0.4962	0.2686	0.4372	0.4373	0.8749	0.8712	8.5135	16	3	13.1772	10.2102
0.3271	97.0	14065	2.7384	0.4971	0.2721	0.4396	0.4397	0.8749	0.8711	8.5135	16	3	13.1832	10.5105
0.3096	98.0	14210	2.7400	0.496	0.2712	0.4386	0.4385	0.8748	0.8711	8.5225	16	3	13.1682	10.2102
0.3116	99.0	14355	2.7411	0.4961	0.2712	0.4388	0.4386	0.8749	0.8711	8.5135	16	3	13.1592	10.2102
0.3102	100.0	14500	2.7416	0.4961	0.2712	0.4388	0.4386	0.8749	0.8711	8.5135	16	3	13.1592	10.2102

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3