text_shortening_model_v57

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8302
• Rouge1: 0.676
• Rouge2: 0.4816
• Rougel: 0.6172
• Rougelsum: 0.6179
• Bert precision: 0.9135
• Bert recall: 0.9082
• Bert f1-score: 0.9104
• Average word count: 8.1161
• Max word count: 16
• Min word count: 4
• Average token count: 12.2946
• % shortened texts with length > 12: 6.25

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Bert f1-score	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
2.6374	1.0	49	2.0882	0.3209	0.1826	0.2886	0.2875	0.7596	0.789	0.7725	9.6562	17	0	15.9018	30.3571
2.1652	2.0	98	1.7029	0.3216	0.1848	0.2939	0.2932	0.7531	0.7906	0.77	9.7054	17	0	16.0402	25.0
1.8731	3.0	147	1.4874	0.4234	0.277	0.3972	0.3951	0.8117	0.8225	0.8158	8.7455	17	0	14.567	21.4286
1.6683	4.0	196	1.3346	0.5198	0.3525	0.4889	0.4888	0.8491	0.8338	0.8405	7.4598	17	0	12.7188	13.3929
1.5287	5.0	245	1.2321	0.5535	0.377	0.5101	0.5109	0.8713	0.8588	0.8643	7.7143	17	0	12.6384	12.0536
1.4303	6.0	294	1.1638	0.5773	0.3941	0.529	0.529	0.8848	0.8703	0.8769	7.5	17	0	12.2009	9.375
1.3696	7.0	343	1.1091	0.5945	0.4125	0.5426	0.5421	0.8946	0.8814	0.8874	7.6473	16	0	12.1875	9.375
1.305	8.0	392	1.0707	0.597	0.4082	0.5434	0.5432	0.8959	0.883	0.8889	7.6741	16	0	12.2143	8.0357
1.2575	9.0	441	1.0384	0.6094	0.4193	0.5521	0.553	0.8993	0.8868	0.8925	7.6786	17	0	12.125	7.1429
1.241	10.0	490	1.0125	0.617	0.423	0.5595	0.5601	0.9038	0.8918	0.8973	7.817	17	2	12.1027	7.5893
1.17	11.0	539	0.9912	0.6173	0.4236	0.5593	0.5591	0.9052	0.892	0.8981	7.7455	17	2	11.9911	6.25
1.1413	12.0	588	0.9750	0.6253	0.4322	0.5661	0.5663	0.9049	0.8935	0.8986	7.9286	17	2	12.1786	7.1429
1.1367	13.0	637	0.9586	0.63	0.4356	0.5704	0.5704	0.9068	0.8943	0.9	7.8705	17	2	12.0357	6.6964
1.1101	14.0	686	0.9458	0.6273	0.4355	0.5665	0.5671	0.9057	0.8949	0.8998	7.942	17	2	12.1384	6.6964
1.0711	15.0	735	0.9374	0.6357	0.4424	0.5718	0.5721	0.9068	0.8969	0.9013	8.0179	17	2	12.1875	6.6964
1.0553	16.0	784	0.9282	0.6378	0.4455	0.5752	0.5756	0.9084	0.8969	0.9022	7.8571	16	2	12.0536	4.9107
1.047	17.0	833	0.9188	0.6439	0.4525	0.5821	0.5825	0.9085	0.8996	0.9035	7.9955	16	2	12.1741	5.3571
1.0201	18.0	882	0.9104	0.643	0.4536	0.5832	0.5836	0.9083	0.8997	0.9035	8.0134	16	2	12.2098	5.8036
1.0228	19.0	931	0.9023	0.6471	0.4601	0.5862	0.5865	0.9101	0.902	0.9056	8.0	16	2	12.1652	4.9107
0.9896	20.0	980	0.8936	0.6497	0.463	0.5882	0.5888	0.9103	0.9017	0.9055	8.0491	16	2	12.1741	5.3571
0.9815	21.0	1029	0.8873	0.6555	0.4659	0.5937	0.5948	0.9106	0.9025	0.9061	8.0402	16	2	12.2411	5.8036
0.9877	22.0	1078	0.8828	0.6618	0.4728	0.6005	0.6007	0.9125	0.9047	0.9081	8.1205	16	2	12.308	6.25
0.9696	23.0	1127	0.8774	0.661	0.4679	0.6	0.5994	0.9128	0.9046	0.9082	8.0938	16	3	12.2902	6.25
0.9556	24.0	1176	0.8737	0.6613	0.4717	0.6023	0.6022	0.913	0.9052	0.9086	8.0893	16	3	12.3036	6.25
0.95	25.0	1225	0.8703	0.6636	0.4725	0.6044	0.6041	0.913	0.9055	0.9088	8.1384	16	3	12.3616	6.25
0.9464	26.0	1274	0.8660	0.6629	0.4723	0.6057	0.6052	0.9125	0.9053	0.9085	8.1562	16	3	12.3482	6.25
0.9189	27.0	1323	0.8605	0.6633	0.4746	0.6084	0.6079	0.9124	0.9052	0.9083	8.0848	16	3	12.2723	6.25
0.9277	28.0	1372	0.8583	0.662	0.4731	0.6059	0.6057	0.9118	0.9059	0.9084	8.1607	16	3	12.3304	6.25
0.9142	29.0	1421	0.8550	0.6663	0.4784	0.6106	0.6104	0.9126	0.9073	0.9095	8.1786	16	4	12.3482	6.6964
0.913	30.0	1470	0.8529	0.6656	0.477	0.6073	0.607	0.9123	0.9073	0.9093	8.2589	16	4	12.4241	7.1429
0.8984	31.0	1519	0.8507	0.6708	0.4804	0.6114	0.6116	0.9128	0.9083	0.9101	8.2098	16	4	12.3973	6.6964
0.903	32.0	1568	0.8479	0.6728	0.4777	0.6096	0.6096	0.9133	0.9081	0.9103	8.2232	16	4	12.3973	7.1429
0.8947	33.0	1617	0.8452	0.6741	0.4785	0.6119	0.6116	0.9132	0.9081	0.9101	8.1741	16	4	12.3616	7.1429
0.8883	34.0	1666	0.8424	0.6733	0.4766	0.6108	0.6107	0.9125	0.9072	0.9094	8.1607	16	4	12.3438	7.1429
0.877	35.0	1715	0.8403	0.6742	0.4799	0.6141	0.6145	0.9133	0.908	0.9102	8.1429	16	4	12.3304	6.6964
0.8612	36.0	1764	0.8393	0.6737	0.4808	0.6141	0.6143	0.9133	0.908	0.9102	8.1384	16	4	12.3259	6.6964
0.8848	37.0	1813	0.8363	0.673	0.478	0.6131	0.6133	0.9124	0.9074	0.9095	8.1384	16	4	12.3214	6.6964
0.8717	38.0	1862	0.8363	0.6729	0.478	0.613	0.6132	0.9129	0.9075	0.9097	8.0848	16	4	12.2545	5.8036
0.8739	39.0	1911	0.8355	0.6711	0.4775	0.6115	0.6118	0.913	0.9072	0.9096	8.0714	16	4	12.2366	5.8036
0.8569	40.0	1960	0.8343	0.672	0.4772	0.6125	0.6128	0.9132	0.9074	0.9098	8.0804	16	4	12.2366	5.8036
0.8601	41.0	2009	0.8342	0.675	0.4831	0.6163	0.6165	0.9139	0.9081	0.9105	8.0982	16	4	12.2634	5.8036
0.8519	42.0	2058	0.8330	0.6743	0.481	0.6147	0.6152	0.9137	0.908	0.9104	8.1027	16	4	12.2723	5.8036
0.8713	43.0	2107	0.8322	0.6757	0.4844	0.617	0.6172	0.9133	0.9079	0.9102	8.125	16	4	12.2857	6.25
0.8554	44.0	2156	0.8313	0.6746	0.4809	0.6151	0.6154	0.9132	0.9079	0.9101	8.1384	16	4	12.2857	6.25
0.8559	45.0	2205	0.8314	0.6773	0.4849	0.6184	0.6189	0.9137	0.9085	0.9106	8.125	16	4	12.2857	6.25
0.847	46.0	2254	0.8312	0.6767	0.4829	0.6175	0.6176	0.9136	0.9084	0.9105	8.125	16	4	12.2857	6.25
0.8588	47.0	2303	0.8306	0.6754	0.4814	0.6163	0.6163	0.9131	0.9082	0.9102	8.1473	16	4	12.3214	6.6964
0.8484	48.0	2352	0.8304	0.676	0.4816	0.6172	0.6179	0.9135	0.9082	0.9104	8.1161	16	4	12.2946	6.25
0.8514	49.0	2401	0.8303	0.676	0.4816	0.6172	0.6179	0.9135	0.9082	0.9104	8.1161	16	4	12.2946	6.25
0.8562	50.0	2450	0.8302	0.676	0.4816	0.6172	0.6179	0.9135	0.9082	0.9104	8.1161	16	4	12.2946	6.25

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3