text_shortening_model_v78

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2534
• Bert precision: 0.8992
• Bert recall: 0.8971
• Bert f1-score: 0.8977
• Average word count: 6.6982
• Max word count: 16
• Min word count: 2
• Average token count: 10.8994
• % shortened texts with length > 12: 1.4724

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Bert precision	Bert recall	Bert f1-score	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.7828	1.0	30	1.2876	0.8895	0.8866	0.8876	6.7595	16	2	10.6969	1.3497
1.3539	2.0	60	1.1852	0.8938	0.8896	0.8912	6.6074	16	2	10.6442	1.227
1.2004	3.0	90	1.1374	0.8942	0.891	0.8921	6.6896	16	2	10.7534	1.3497
1.1048	4.0	120	1.1165	0.8929	0.8931	0.8926	6.8025	16	2	10.9178	1.7178
1.0322	5.0	150	1.0995	0.8953	0.896	0.8952	6.827	17	2	11.0037	1.9632
0.9687	6.0	180	1.0822	0.894	0.8947	0.8939	6.9043	17	3	11.0147	2.2086
0.9076	7.0	210	1.0858	0.8976	0.8965	0.8966	6.7951	17	2	10.9804	1.8405
0.8602	8.0	240	1.0894	0.8977	0.897	0.8969	6.7902	17	2	10.9558	2.0859
0.8076	9.0	270	1.0945	0.8982	0.8974	0.8974	6.7669	17	2	10.9779	1.9632
0.7739	10.0	300	1.0843	0.8974	0.8959	0.8962	6.719	17	2	10.8564	1.9632
0.7361	11.0	330	1.0982	0.8972	0.8967	0.8965	6.789	17	3	10.9779	1.5951
0.7052	12.0	360	1.0953	0.8977	0.8958	0.8963	6.7055	17	2	10.8798	1.8405
0.6749	13.0	390	1.1098	0.8981	0.8967	0.897	6.7325	17	2	10.908	2.0859
0.6441	14.0	420	1.1158	0.8996	0.8981	0.8984	6.7043	17	2	10.8761	1.5951
0.623	15.0	450	1.1146	0.9009	0.8989	0.8994	6.7006	17	2	10.9104	1.3497
0.604	16.0	480	1.1267	0.902	0.8988	0.9	6.7104	17	2	10.8491	1.3497
0.583	17.0	510	1.1357	0.8999	0.8989	0.8989	6.7706	17	2	10.9767	2.0859
0.5605	18.0	540	1.1513	0.8996	0.899	0.8989	6.7534	17	2	11.0356	2.0859
0.5439	19.0	570	1.1643	0.8993	0.8988	0.8986	6.816	17	2	11.0638	1.8405
0.5281	20.0	600	1.1626	0.8998	0.8985	0.8987	6.7399	16	2	11.0025	1.4724
0.5165	21.0	630	1.1720	0.9002	0.8974	0.8983	6.6417	16	2	10.7816	1.5951
0.5002	22.0	660	1.1834	0.9005	0.8995	0.8996	6.7607	17	2	10.9693	2.2086
0.486	23.0	690	1.2005	0.9027	0.8979	0.8999	6.5853	17	2	10.7264	1.8405
0.4757	24.0	720	1.1916	0.9008	0.8991	0.8995	6.6994	16	2	10.8859	2.0859
0.4679	25.0	750	1.2023	0.8988	0.8985	0.8982	6.7853	16	2	10.984	2.0859
0.4502	26.0	780	1.2108	0.9012	0.8991	0.8997	6.7141	16	2	10.8577	2.3313
0.4452	27.0	810	1.2188	0.9015	0.8987	0.8996	6.6098	14	2	10.7853	1.8405
0.4351	28.0	840	1.2187	0.9005	0.8997	0.8996	6.7681	16	2	10.9877	2.0859
0.4223	29.0	870	1.2319	0.9001	0.8985	0.8989	6.6798	16	2	10.9031	1.7178
0.4186	30.0	900	1.2386	0.8987	0.8982	0.898	6.7509	16	2	10.9718	1.8405
0.4146	31.0	930	1.2399	0.9001	0.8989	0.8991	6.719	16	2	10.9067	1.8405
0.4039	32.0	960	1.2467	0.8997	0.8992	0.899	6.7632	16	2	11.0074	2.0859
0.3998	33.0	990	1.2425	0.8997	0.899	0.8989	6.7497	16	2	10.946	1.8405
0.396	34.0	1020	1.2474	0.9001	0.8982	0.8987	6.6945	16	2	10.8773	1.7178
0.3938	35.0	1050	1.2497	0.9003	0.8977	0.8985	6.6454	16	2	10.8245	0.9816
0.3828	36.0	1080	1.2511	0.9002	0.8978	0.8986	6.6798	16	2	10.8663	1.4724
0.3842	37.0	1110	1.2482	0.9002	0.898	0.8987	6.6883	16	2	10.8883	1.7178
0.391	38.0	1140	1.2507	0.9001	0.8974	0.8983	6.6577	16	2	10.8528	1.3497
0.3796	39.0	1170	1.2529	0.8994	0.8972	0.8979	6.6945	16	2	10.8933	1.5951
0.3807	40.0	1200	1.2534	0.8992	0.8971	0.8977	6.6982	16	2	10.8994	1.4724

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3