text_shortening_model_v28

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9239
• Rouge1: 0.4821
• Rouge2: 0.2554
• Rougel: 0.4273
• Rougelsum: 0.4271
• Bert precision: 0.8753
• Bert recall: 0.8686
• Average word count: 8.1231
• Max word count: 14
• Min word count: 4
• Average token count: 12.4805
• % shortened texts with length > 12: 2.7027

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.005
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
2.1941	1.0	37	1.6111	0.5035	0.2708	0.4544	0.4553	0.8744	0.8772	9.2643	18	3	13.5826	12.9129
1.4552	2.0	74	1.5748	0.4724	0.2457	0.4313	0.4313	0.8737	0.8635	8.009	17	3	12.024	6.006
1.096	3.0	111	1.5283	0.5016	0.282	0.4515	0.4519	0.8774	0.8757	8.7748	16	4	13.0811	7.2072
0.8801	4.0	148	1.5903	0.4848	0.2549	0.4372	0.4375	0.88	0.8713	8.3544	16	4	12.6607	6.9069
0.7226	5.0	185	1.6953	0.4557	0.2378	0.408	0.4086	0.8752	0.8603	7.3904	13	1	11.6697	0.9009
0.6003	6.0	222	1.8416	0.4935	0.2616	0.4327	0.4339	0.8758	0.8712	8.5736	17	3	12.8498	8.7087
0.4852	7.0	259	1.8375	0.4662	0.2428	0.4147	0.4161	0.8703	0.8653	8.3904	16	2	12.7147	7.5075
0.4469	8.0	296	1.9116	0.4617	0.2433	0.41	0.4117	0.8724	0.8649	8.0781	17	2	12.5495	2.7027
0.4025	9.0	333	1.9871	0.4716	0.2443	0.4161	0.4164	0.8691	0.8662	9.006	19	4	13.4204	15.3153
0.3568	10.0	370	2.0547	0.4864	0.2649	0.4377	0.4377	0.8742	0.8724	8.6396	16	3	13.1381	7.8078
0.3071	11.0	407	2.1554	0.4582	0.2388	0.405	0.4053	0.8712	0.8595	7.7087	14	4	12.033	2.1021
0.2794	12.0	444	2.1352	0.4768	0.2567	0.4341	0.4344	0.8757	0.8705	8.4655	16	4	12.8949	8.1081
0.2627	13.0	481	2.1300	0.4703	0.2518	0.4227	0.4227	0.876	0.8674	8.015	17	3	12.2342	4.5045
0.2251	14.0	518	2.2319	0.4887	0.2623	0.4335	0.4336	0.8757	0.8704	8.3544	15	4	12.7357	5.7057
0.217	15.0	555	2.2311	0.4709	0.2523	0.4196	0.4196	0.8739	0.8683	8.2613	17	4	12.6276	5.4054
0.2097	16.0	592	2.2460	0.471	0.2463	0.4137	0.4147	0.8732	0.8649	8.1682	15	3	12.4384	4.5045
0.1841	17.0	629	2.3917	0.4564	0.229	0.4072	0.4076	0.8709	0.8663	8.3934	16	3	12.7027	5.4054
0.176	18.0	666	2.3731	0.4644	0.2408	0.4093	0.4103	0.87	0.8633	8.1712	16	4	12.5495	4.5045
0.1531	19.0	703	2.3836	0.4925	0.2727	0.439	0.439	0.879	0.8711	8.1111	16	3	12.2703	3.9039
0.1599	20.0	740	2.3611	0.4731	0.2575	0.4199	0.4202	0.8743	0.8669	8.1141	16	3	12.5315	4.8048
0.1469	21.0	777	2.4164	0.4774	0.2515	0.4295	0.4302	0.876	0.8709	8.3754	15	3	12.8348	6.3063
0.1449	22.0	814	2.4769	0.4702	0.2461	0.4205	0.421	0.874	0.8688	8.4054	16	4	12.7508	7.8078
0.1417	23.0	851	2.5470	0.4669	0.2438	0.4163	0.4163	0.8733	0.8649	7.9339	14	3	12.1922	2.7027
0.1255	24.0	888	2.5590	0.4642	0.2379	0.4127	0.4136	0.8736	0.8621	7.6517	14	4	11.7057	2.1021
0.1281	25.0	925	2.4347	0.4707	0.2571	0.4227	0.4233	0.8734	0.8675	8.2492	15	3	12.6937	4.5045
0.1399	26.0	962	2.5391	0.4649	0.2454	0.4132	0.414	0.8703	0.8684	8.6547	17	4	13.1982	8.4084
0.1279	27.0	999	2.5712	0.4723	0.2526	0.4208	0.4207	0.8729	0.8682	8.3393	17	4	12.6547	6.3063
0.1224	28.0	1036	2.5410	0.466	0.2485	0.4159	0.4156	0.8743	0.8663	7.955	15	3	12.2643	3.3033
0.1095	29.0	1073	2.6742	0.4647	0.2382	0.4094	0.4098	0.873	0.8641	8.033	16	4	12.3243	5.1051
0.1202	30.0	1110	2.5533	0.4748	0.2495	0.4225	0.4234	0.8757	0.8676	8.1562	16	4	12.4204	4.8048
0.1236	31.0	1147	2.5441	0.4709	0.2444	0.418	0.4187	0.87	0.8659	8.4144	17	4	12.8228	5.7057
0.1074	32.0	1184	2.6271	0.4845	0.2619	0.4291	0.4301	0.8758	0.8684	8.1502	15	3	12.4985	4.5045
0.0939	33.0	1221	2.6391	0.4806	0.2549	0.4251	0.4261	0.8722	0.869	8.6486	16	3	13.1592	7.8078
0.0976	34.0	1258	2.6159	0.4798	0.2582	0.4264	0.4268	0.8738	0.8701	8.6096	17	3	13.0931	8.4084
0.1042	35.0	1295	2.6224	0.4849	0.2557	0.428	0.4284	0.876	0.8705	8.2673	16	3	12.6757	4.5045
0.094	36.0	1332	2.5925	0.4742	0.2542	0.4289	0.4296	0.8754	0.8683	8.033	15	3	12.3483	3.6036
0.0794	37.0	1369	2.5782	0.4897	0.262	0.4354	0.4364	0.8762	0.8723	8.3153	16	3	12.7538	6.9069
0.0823	38.0	1406	2.6590	0.4752	0.2486	0.4222	0.423	0.8737	0.8651	8.027	15	3	12.3123	6.3063
0.0813	39.0	1443	2.6823	0.4817	0.2605	0.427	0.4276	0.8763	0.8696	8.1532	15	2	12.6006	5.1051
0.0868	40.0	1480	2.6642	0.4827	0.2572	0.4308	0.4314	0.8757	0.8702	8.3964	16	3	12.6877	8.1081
0.0786	41.0	1517	2.7908	0.4623	0.24	0.4086	0.4096	0.8704	0.863	8.1351	16	4	12.7447	7.2072
0.0901	42.0	1554	2.7242	0.4613	0.2405	0.4115	0.413	0.8716	0.8636	8.2523	18	4	12.5465	6.9069
0.0912	43.0	1591	2.7376	0.474	0.2446	0.4194	0.4191	0.8707	0.8694	8.6877	16	3	13.2282	10.5105
0.0887	44.0	1628	2.7192	0.479	0.2539	0.4266	0.4268	0.874	0.8703	8.4865	15	4	13.1321	7.2072
0.0807	45.0	1665	2.6935	0.4738	0.2501	0.4213	0.4223	0.874	0.8675	8.2042	16	2	12.6787	6.006
0.0801	46.0	1702	2.7149	0.4662	0.2443	0.4229	0.4236	0.8745	0.8659	8.033	15	3	12.3453	4.2042
0.0764	47.0	1739	2.6544	0.4697	0.249	0.4206	0.4202	0.8726	0.8668	8.2432	16	4	12.6637	6.9069
0.0765	48.0	1776	2.7157	0.4764	0.2535	0.4234	0.4236	0.8762	0.8676	8.021	15	3	12.3544	4.5045
0.065	49.0	1813	2.8051	0.4666	0.2452	0.4161	0.4165	0.8728	0.8665	8.2673	16	2	12.6246	5.1051
0.0626	50.0	1850	2.7845	0.4781	0.2519	0.4254	0.4253	0.8746	0.8688	8.2192	16	3	12.5796	5.4054
0.0608	51.0	1887	2.7371	0.4745	0.2456	0.4213	0.4208	0.8751	0.866	8.0871	17	2	12.3063	6.9069
0.0599	52.0	1924	2.7620	0.474	0.2515	0.419	0.4204	0.8718	0.8667	8.1381	15	4	12.8979	3.9039
0.0625	53.0	1961	2.8097	0.4646	0.2481	0.4137	0.4146	0.8706	0.8663	8.2733	15	3	12.7237	4.5045
0.0529	54.0	1998	2.8677	0.4714	0.2436	0.4142	0.4147	0.8745	0.8651	7.8709	16	2	12.2012	4.2042
0.05	55.0	2035	2.7892	0.467	0.2465	0.4152	0.4159	0.8739	0.8668	8.1712	17	2	12.4925	3.3033
0.047	56.0	2072	2.7682	0.4719	0.2451	0.4223	0.423	0.8717	0.866	8.1802	15	3	12.5826	5.1051
0.0504	57.0	2109	2.7897	0.4823	0.2555	0.427	0.4276	0.8754	0.8717	8.5345	15	3	12.9249	7.5075
0.0463	58.0	2146	2.8505	0.471	0.2513	0.42	0.4204	0.8748	0.8683	8.2132	15	3	12.6426	5.7057
0.0487	59.0	2183	2.7699	0.4658	0.2472	0.4156	0.4166	0.8726	0.8667	8.1231	15	3	12.5465	3.9039
0.045	60.0	2220	2.7589	0.4718	0.2495	0.4211	0.4216	0.8741	0.8676	8.2432	17	4	12.5556	5.1051
0.047	61.0	2257	2.8092	0.4814	0.2517	0.4253	0.4257	0.8759	0.8687	7.997	17	3	12.3393	3.003
0.0415	62.0	2294	2.8059	0.4689	0.2494	0.4183	0.4191	0.8767	0.8655	7.7538	17	2	12.1381	2.4024
0.0429	63.0	2331	2.8317	0.4783	0.2516	0.4248	0.4252	0.8764	0.8689	8.0811	17	3	12.4234	3.003
0.0383	64.0	2368	2.8147	0.4728	0.2547	0.4189	0.4193	0.8732	0.867	8.1622	18	3	12.5916	4.2042
0.039	65.0	2405	2.8237	0.4638	0.2401	0.414	0.4145	0.871	0.8654	8.3183	15	3	12.7057	5.7057
0.0417	66.0	2442	2.8289	0.4726	0.2532	0.4242	0.4243	0.8746	0.8667	7.9159	15	3	12.3363	3.003
0.0365	67.0	2479	2.8272	0.4752	0.2506	0.4222	0.4218	0.8732	0.8673	8.1652	15	3	12.5165	4.8048
0.0372	68.0	2516	2.8469	0.4726	0.2491	0.4225	0.423	0.873	0.8665	8.1802	17	4	12.5796	5.4054
0.0363	69.0	2553	2.8233	0.4745	0.2554	0.4244	0.4239	0.8751	0.8672	8.0601	16	4	12.2342	3.6036
0.0356	70.0	2590	2.8652	0.4737	0.2471	0.4169	0.4167	0.8735	0.8658	8.03	17	4	12.3273	5.1051
0.0366	71.0	2627	2.8722	0.4838	0.2598	0.4274	0.428	0.8767	0.869	8.0541	14	4	12.2913	3.003
0.0334	72.0	2664	2.8650	0.4708	0.2508	0.4194	0.4195	0.873	0.8674	8.2252	16	4	12.6426	4.2042
0.0328	73.0	2701	2.8827	0.479	0.2498	0.4221	0.422	0.8753	0.8683	8.1802	16	4	12.4835	3.3033
0.0322	74.0	2738	2.8599	0.479	0.2524	0.4295	0.43	0.8746	0.8689	8.2583	17	4	12.6727	4.2042
0.0308	75.0	2775	2.8559	0.4781	0.255	0.4279	0.4292	0.8766	0.8687	8.042	14	4	12.3033	2.4024
0.0304	76.0	2812	2.8364	0.4779	0.2581	0.4286	0.4287	0.8759	0.8682	7.994	17	3	12.3063	3.3033
0.0322	77.0	2849	2.8167	0.472	0.2489	0.4222	0.4225	0.8746	0.8673	8.003	17	4	12.3754	4.8048
0.0296	78.0	2886	2.8835	0.4716	0.2541	0.4217	0.4219	0.8734	0.8679	8.2252	17	4	12.6787	4.5045
0.0284	79.0	2923	2.8712	0.4729	0.2526	0.4228	0.4229	0.874	0.8672	8.1772	18	4	12.5495	3.6036
0.0286	80.0	2960	2.8709	0.4826	0.2596	0.4328	0.4324	0.877	0.8705	8.1592	18	4	12.4234	4.2042
0.0287	81.0	2997	2.8556	0.4746	0.2558	0.4228	0.4236	0.8747	0.8681	8.1381	17	4	12.3994	3.003
0.0287	82.0	3034	2.8867	0.4788	0.2617	0.429	0.4291	0.8757	0.8701	8.3273	17	4	12.6997	5.1051
0.0298	83.0	3071	2.8793	0.4828	0.2609	0.4306	0.4295	0.8757	0.8702	8.2673	17	4	12.6066	4.5045
0.0266	84.0	3108	2.8795	0.472	0.2499	0.4208	0.4207	0.8742	0.8677	8.1742	15	4	12.5345	3.6036
0.0257	85.0	3145	2.8788	0.48	0.2543	0.4244	0.4247	0.876	0.8686	8.1321	14	4	12.3874	2.7027
0.0255	86.0	3182	2.9130	0.4868	0.266	0.4307	0.4304	0.8762	0.8702	8.1652	15	4	12.5285	3.3033
0.0254	87.0	3219	2.9050	0.4847	0.2627	0.4327	0.432	0.877	0.8702	8.042	16	4	12.4324	3.3033
0.0233	88.0	3256	2.9014	0.4855	0.2615	0.433	0.4328	0.8758	0.8701	8.2613	16	4	12.5706	3.9039
0.0268	89.0	3293	2.8937	0.487	0.2586	0.4316	0.4317	0.8763	0.8707	8.2402	15	4	12.5616	4.2042
0.0243	90.0	3330	2.8926	0.4838	0.2584	0.4271	0.4268	0.8765	0.8695	8.1171	14	4	12.3483	3.3033
0.0248	91.0	3367	2.8870	0.4775	0.2503	0.4223	0.4222	0.8748	0.8678	8.1201	14	4	12.4354	3.3033
0.0237	92.0	3404	2.8978	0.4816	0.2556	0.4275	0.4275	0.8752	0.8688	8.0991	14	4	12.4685	2.7027
0.0244	93.0	3441	2.9025	0.4778	0.2506	0.4246	0.4249	0.8747	0.868	8.039	14	4	12.4174	3.003
0.0227	94.0	3478	2.9164	0.4733	0.2486	0.4199	0.4204	0.8745	0.8669	7.973	14	4	12.3123	2.7027
0.0215	95.0	3515	2.9183	0.4795	0.2495	0.4233	0.4231	0.8751	0.8682	8.03	14	4	12.4084	3.003
0.0225	96.0	3552	2.9207	0.4763	0.2463	0.4204	0.4206	0.8752	0.8677	8.0511	14	4	12.3934	2.7027
0.0208	97.0	3589	2.9226	0.4815	0.2556	0.4271	0.4276	0.8758	0.869	8.0871	14	4	12.4144	2.7027
0.0225	98.0	3626	2.9234	0.4832	0.2576	0.4285	0.4281	0.8762	0.8693	8.1351	16	4	12.4595	3.003
0.0219	99.0	3663	2.9243	0.4809	0.2543	0.4249	0.4249	0.8754	0.8686	8.1141	14	4	12.4775	2.7027
0.0214	100.0	3700	2.9239	0.4821	0.2554	0.4273	0.4271	0.8753	0.8686	8.1231	14	4	12.4805	2.7027

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3