text_shortening_model_v69

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5824
• Bert precision: 0.8984
• Bert recall: 0.9007
• Bert f1-score: 0.899
• Average word count: 6.3944
• Max word count: 16
• Min word count: 2
• Average token count: 10.5305
• % shortened texts with length > 12: 0.8008

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Bert precision	Bert recall	Bert f1-score	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.6663	1.0	37	1.2012	0.8867	0.8851	0.8853	6.2943	15	1	10.2032	0.7007
1.228	2.0	74	1.1078	0.8922	0.8923	0.8918	6.3453	16	2	10.2863	1.5015
1.0592	3.0	111	1.0686	0.8922	0.8955	0.8933	6.5656	16	2	10.5856	0.8008
0.9195	4.0	148	1.0606	0.8958	0.8952	0.895	6.3323	16	1	10.3564	0.9009
0.8261	5.0	185	1.0630	0.897	0.8972	0.8967	6.3413	16	2	10.4134	0.7007
0.7476	6.0	222	1.0865	0.8946	0.8987	0.8962	6.5626	16	2	10.6527	1.1011
0.6731	7.0	259	1.1104	0.8962	0.8957	0.8955	6.3073	16	2	10.3063	0.9009
0.6166	8.0	296	1.1198	0.8976	0.9003	0.8985	6.4505	16	2	10.5255	0.8008
0.5618	9.0	333	1.1591	0.8959	0.9007	0.8978	6.5355	16	2	10.6957	1.7017
0.5272	10.0	370	1.1745	0.8982	0.9023	0.8998	6.5586	16	1	10.7227	1.7017
0.4796	11.0	407	1.1884	0.8971	0.899	0.8976	6.4244	16	1	10.5215	1.001
0.452	12.0	444	1.2263	0.9009	0.9004	0.9002	6.2342	16	1	10.3213	0.8008
0.4163	13.0	481	1.2370	0.8979	0.8987	0.8978	6.3113	16	1	10.3564	1.2012
0.3837	14.0	518	1.2830	0.8986	0.902	0.8998	6.4725	16	1	10.5786	1.7017
0.3544	15.0	555	1.2913	0.8991	0.8997	0.8989	6.3524	16	1	10.3984	1.001
0.3319	16.0	592	1.3335	0.8977	0.8987	0.8977	6.3423	16	2	10.4354	1.2012
0.313	17.0	629	1.3357	0.8996	0.9014	0.9	6.3854	15	2	10.5445	1.001
0.3005	18.0	666	1.3610	0.8999	0.8994	0.8991	6.1962	15	2	10.2753	0.6006
0.2789	19.0	703	1.3865	0.8991	0.9028	0.9005	6.4314	16	2	10.6406	1.001
0.2706	20.0	740	1.3929	0.8983	0.9015	0.8994	6.4244	15	2	10.5986	0.9009
0.2476	21.0	777	1.4228	0.8998	0.9004	0.8996	6.3023	16	2	10.4264	0.9009
0.2434	22.0	814	1.4307	0.8985	0.9002	0.8988	6.3223	16	2	10.4094	0.6006
0.2327	23.0	851	1.4522	0.8977	0.8995	0.8981	6.3473	16	2	10.4885	0.7007
0.2164	24.0	888	1.4538	0.8995	0.9006	0.8995	6.3203	16	2	10.4014	0.8008
0.2123	25.0	925	1.4741	0.8979	0.9006	0.8987	6.3934	16	2	10.6116	0.7007
0.2054	26.0	962	1.5038	0.8971	0.9006	0.8984	6.4314	16	2	10.6106	0.7007
0.1996	27.0	999	1.4962	0.8982	0.9015	0.8994	6.4525	16	2	10.6316	0.9009
0.1881	28.0	1036	1.5320	0.8993	0.9021	0.9002	6.4054	15	2	10.5876	0.7007
0.1814	29.0	1073	1.5209	0.8978	0.9007	0.8987	6.4535	16	2	10.6116	0.9009
0.1738	30.0	1110	1.5377	0.9003	0.9022	0.9008	6.3784	16	2	10.5315	0.7007
0.1761	31.0	1147	1.5387	0.8996	0.9021	0.9004	6.3744	16	2	10.5946	0.8008
0.1632	32.0	1184	1.5566	0.8995	0.9013	0.8999	6.3654	16	2	10.5105	1.001
0.1613	33.0	1221	1.5549	0.8994	0.9019	0.9001	6.3944	16	2	10.5696	1.001
0.1555	34.0	1258	1.5728	0.8993	0.9021	0.9002	6.4054	16	2	10.5936	1.1011
0.1604	35.0	1295	1.5671	0.8987	0.9018	0.8998	6.4274	16	2	10.5836	1.1011
0.1525	36.0	1332	1.5748	0.8991	0.901	0.8995	6.3664	16	2	10.5205	1.1011
0.1534	37.0	1369	1.5709	0.8993	0.9009	0.8996	6.3493	16	2	10.4775	0.8008
0.1478	38.0	1406	1.5777	0.8986	0.9008	0.8992	6.3894	16	2	10.5095	0.8008
0.1423	39.0	1443	1.5815	0.8986	0.9008	0.8992	6.3934	16	2	10.5295	0.8008
0.1474	40.0	1480	1.5824	0.8984	0.9007	0.899	6.3944	16	2	10.5305	0.8008

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3