text_shortening_model_v63

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9523
• Rouge1: 0.6828
• Rouge2: 0.4806
• Rougel: 0.6361
• Rougelsum: 0.6363
• Bert precision: 0.9094
• Bert recall: 0.9135
• Bert f1-score: 0.9109
• Average word count: 7.798
• Max word count: 16
• Min word count: 2
• Average token count: 12.0077
• % shortened texts with length > 12: 3.3248

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Bert f1-score	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.8123	1.0	57	1.0664	0.6167	0.412	0.5601	0.5595	0.8975	0.8952	0.8956	7.5908	15	0	11.6752	4.0921
1.1902	2.0	114	0.9103	0.6564	0.4627	0.6141	0.6137	0.9076	0.9085	0.9074	7.4476	17	2	11.578	1.7903
1.0198	3.0	171	0.8631	0.6718	0.481	0.6327	0.6328	0.907	0.9121	0.9089	7.8363	17	2	11.9744	3.8363
0.9149	4.0	228	0.8266	0.674	0.4873	0.6365	0.6361	0.9089	0.9123	0.9101	7.6061	14	2	11.8312	2.5575
0.8256	5.0	285	0.8033	0.68	0.4876	0.643	0.6432	0.9107	0.9121	0.9109	7.5013	14	2	11.6317	2.5575
0.773	6.0	342	0.8007	0.6839	0.4923	0.6546	0.6546	0.9105	0.9141	0.9117	7.6752	16	2	11.9105	2.8133
0.7243	7.0	399	0.7875	0.6775	0.4878	0.6457	0.6465	0.9101	0.9119	0.9104	7.5831	16	2	11.711	2.3018
0.6609	8.0	456	0.7780	0.6833	0.4876	0.6432	0.6432	0.9122	0.9136	0.9124	7.5575	16	2	11.6522	2.3018
0.6402	9.0	513	0.7823	0.6872	0.4871	0.6424	0.6421	0.9107	0.9143	0.912	7.7187	16	2	11.844	3.8363
0.5944	10.0	570	0.7878	0.6795	0.4827	0.6395	0.6387	0.9082	0.913	0.91	7.7161	16	2	11.9591	3.0691
0.5638	11.0	627	0.7889	0.6802	0.4805	0.64	0.6407	0.9111	0.9124	0.9112	7.6368	16	2	11.7749	2.5575
0.5474	12.0	684	0.7987	0.6736	0.4712	0.6295	0.6293	0.9082	0.9121	0.9096	7.6471	16	2	11.8389	2.3018
0.5249	13.0	741	0.7942	0.6859	0.4823	0.6432	0.643	0.9113	0.915	0.9126	7.7545	16	2	11.9488	3.3248
0.4936	14.0	798	0.8077	0.6777	0.4786	0.6369	0.6365	0.9097	0.9122	0.9104	7.6368	16	2	11.7852	2.3018
0.4705	15.0	855	0.8099	0.6809	0.4753	0.6388	0.639	0.9088	0.9123	0.91	7.6624	16	2	11.9156	2.8133
0.4558	16.0	912	0.8154	0.6813	0.4783	0.6389	0.6398	0.9092	0.9135	0.9108	7.7775	16	2	11.9335	3.8363
0.4352	17.0	969	0.8138	0.6897	0.4911	0.6493	0.6496	0.9121	0.9151	0.9131	7.6624	16	2	11.8747	2.8133
0.421	18.0	1026	0.8274	0.6902	0.4868	0.652	0.6517	0.9114	0.9161	0.9132	7.798	16	2	12.0256	3.3248
0.4137	19.0	1083	0.8238	0.6894	0.4902	0.6491	0.6494	0.9118	0.9164	0.9136	7.798	16	2	12.0332	3.0691
0.4026	20.0	1140	0.8385	0.6846	0.4841	0.6428	0.643	0.9098	0.9147	0.9117	7.821	16	2	12.0281	4.3478
0.3866	21.0	1197	0.8393	0.6894	0.4866	0.6469	0.6472	0.9117	0.9166	0.9136	7.8107	16	2	12.0281	4.3478
0.3762	22.0	1254	0.8501	0.691	0.4882	0.6484	0.649	0.9118	0.9175	0.9141	7.8951	16	2	12.133	4.3478
0.3592	23.0	1311	0.8486	0.6906	0.4834	0.6452	0.6458	0.9119	0.9166	0.9137	7.7647	16	2	11.9821	3.3248
0.3532	24.0	1368	0.8530	0.6858	0.4825	0.6425	0.6429	0.9124	0.9157	0.9135	7.7366	16	2	11.9974	3.0691
0.3318	25.0	1425	0.8625	0.6886	0.4867	0.6486	0.6486	0.9111	0.9175	0.9138	7.8414	16	2	12.1765	3.8363
0.3427	26.0	1482	0.8727	0.6879	0.4879	0.6459	0.6464	0.9118	0.9166	0.9137	7.7852	16	2	12.0614	3.3248
0.3245	27.0	1539	0.8885	0.6845	0.4808	0.6381	0.6384	0.9107	0.9152	0.9124	7.7775	16	2	11.9463	3.0691
0.3189	28.0	1596	0.8864	0.6828	0.4769	0.6392	0.6395	0.911	0.9137	0.9119	7.7059	16	2	11.8389	2.5575
0.3069	29.0	1653	0.8970	0.6806	0.4768	0.6374	0.6378	0.91	0.9132	0.9111	7.7289	16	2	11.9437	2.8133
0.3041	30.0	1710	0.8942	0.6802	0.4743	0.6354	0.6361	0.9107	0.9128	0.9113	7.6292	16	2	11.7954	2.8133
0.302	31.0	1767	0.9005	0.6801	0.4785	0.6373	0.6376	0.9095	0.9137	0.9111	7.7698	16	2	11.9923	3.3248
0.2912	32.0	1824	0.9060	0.6806	0.4792	0.6377	0.6374	0.9096	0.913	0.9107	7.6982	16	2	11.9156	3.3248
0.2843	33.0	1881	0.9129	0.6838	0.4801	0.6395	0.6394	0.9101	0.9142	0.9116	7.757	16	2	11.9079	4.3478
0.2833	34.0	1938	0.9175	0.6861	0.4846	0.6408	0.6413	0.9106	0.9142	0.9118	7.7494	16	2	11.9309	3.8363
0.2751	35.0	1995	0.9189	0.6886	0.4831	0.6442	0.6447	0.9121	0.9149	0.913	7.665	16	2	11.9028	2.5575
0.2713	36.0	2052	0.9234	0.6868	0.4882	0.6439	0.6437	0.9114	0.9155	0.9129	7.7903	16	2	12.023	2.8133
0.2587	37.0	2109	0.9345	0.6813	0.4829	0.6387	0.638	0.9102	0.914	0.9115	7.7673	16	2	11.9514	3.5806
0.2646	38.0	2166	0.9315	0.6841	0.4829	0.6387	0.6386	0.9106	0.9135	0.9115	7.7161	16	2	11.9182	3.5806
0.2583	39.0	2223	0.9359	0.6833	0.4799	0.6375	0.6379	0.9104	0.9137	0.9115	7.757	16	2	11.9591	2.5575
0.2518	40.0	2280	0.9392	0.6877	0.4851	0.6395	0.6403	0.9107	0.9141	0.9118	7.798	16	2	12.0051	3.3248
0.2453	41.0	2337	0.9420	0.6885	0.4835	0.6405	0.6412	0.9109	0.9141	0.912	7.7494	16	2	11.954	3.5806
0.251	42.0	2394	0.9427	0.6852	0.4798	0.636	0.6367	0.9108	0.9136	0.9116	7.7647	16	2	11.9488	3.5806
0.2495	43.0	2451	0.9445	0.6821	0.4792	0.6342	0.6351	0.9099	0.913	0.9109	7.7596	16	2	11.9565	3.5806
0.248	44.0	2508	0.9448	0.681	0.4782	0.6336	0.6342	0.9091	0.9132	0.9106	7.7928	16	2	12.0179	3.3248
0.2516	45.0	2565	0.9472	0.6839	0.4852	0.6387	0.6388	0.91	0.914	0.9114	7.8465	16	2	12.0537	3.8363
0.2475	46.0	2622	0.9523	0.6812	0.4814	0.6357	0.6361	0.909	0.9137	0.9108	7.867	16	2	12.0972	3.8363
0.241	47.0	2679	0.9518	0.6801	0.4809	0.6337	0.6338	0.909	0.9132	0.9106	7.8286	16	2	12.046	3.8363
0.2386	48.0	2736	0.9519	0.6801	0.4783	0.633	0.6332	0.9084	0.9129	0.9101	7.8363	16	2	12.0537	4.0921
0.2398	49.0	2793	0.9521	0.6816	0.48	0.6349	0.635	0.9093	0.9132	0.9107	7.7775	16	2	11.9847	3.3248
0.2323	50.0	2850	0.9523	0.6828	0.4806	0.6361	0.6363	0.9094	0.9135	0.9109	7.798	16	2	12.0077	3.3248

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3