Hugging Face's logo

ldos
/
text_shortening_model_v31

Text2Text Generation
Transformers
PyTorch
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
text_shortening_model_v31 / README.md
ldos's picture
ldos
End of training
04a91c8
preview code
|
raw
history blame contribute delete
25.5 kB
metadata
license: apache-2.0
base_model: t5-small
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: text_shortening_model_v31
    results: []

text_shortening_model_v31

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 2.7416
  • Rouge1: 0.4961
  • Rouge2: 0.2712
  • Rougel: 0.4388
  • Rougelsum: 0.4386
  • Bert precision: 0.8749
  • Bert recall: 0.8711
  • Average word count: 8.5135
  • Max word count: 16
  • Min word count: 3
  • Average token count: 13.1592
  • % shortened texts with length > 12: 10.2102

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Bert precision Bert recall Average word count Max word count Min word count Average token count % shortened texts with length > 12
1.1978 1.0 145 1.5250 0.4953 0.2842 0.4528 0.4524 0.8806 0.8681 7.8919 18 3 12.4234 4.2042
1.0092 2.0 290 1.4421 0.5257 0.3053 0.4698 0.4689 0.875 0.8809 9.6006 18 4 14.3574 19.2192
0.8932 3.0 435 1.4060 0.5266 0.3045 0.4728 0.472 0.8766 0.8776 9.0841 18 4 13.6366 14.7147
0.79 4.0 580 1.4022 0.5329 0.3136 0.4714 0.4714 0.8802 0.8797 8.952 16 4 13.6036 12.9129
0.7506 5.0 725 1.4514 0.5145 0.2935 0.4485 0.4485 0.8745 0.8726 8.97 18 4 13.6096 12.012
0.6981 6.0 870 1.4602 0.5146 0.2914 0.4566 0.4559 0.8778 0.8762 8.958 18 3 13.5195 15.3153
0.6426 7.0 1015 1.4745 0.5196 0.2973 0.4596 0.4593 0.8759 0.8788 9.1802 16 4 13.9159 14.1141
0.6251 8.0 1160 1.5026 0.5217 0.2965 0.461 0.4611 0.8802 0.8775 8.8198 16 4 13.3393 12.012
0.5901 9.0 1305 1.5890 0.5156 0.2967 0.4606 0.4609 0.8773 0.876 8.7718 17 3 13.4655 9.6096
0.5544 10.0 1450 1.6294 0.5172 0.287 0.4562 0.4559 0.8779 0.876 8.7688 18 4 13.5195 11.7117
0.5354 11.0 1595 1.6805 0.5169 0.2871 0.457 0.4571 0.8768 0.8774 8.994 17 4 13.6486 14.1141
0.5103 12.0 1740 1.7334 0.5121 0.2824 0.4556 0.455 0.8785 0.8745 8.5465 16 3 13.1021 8.1081
0.4796 13.0 1885 1.7767 0.499 0.2763 0.442 0.4418 0.8754 0.8739 8.6396 17 4 13.3183 11.4114
0.4825 14.0 2030 1.8319 0.5114 0.2849 0.4497 0.4501 0.8746 0.8758 8.994 17 4 13.6667 12.9129
0.4572 15.0 2175 1.8613 0.5129 0.2884 0.4546 0.4549 0.8785 0.8757 8.6877 17 3 13.3784 10.5105
0.4489 16.0 2320 1.8790 0.5144 0.2829 0.4533 0.4536 0.8777 0.8754 8.8078 16 3 13.4955 13.2132
0.4211 17.0 2465 1.9604 0.4936 0.2641 0.4322 0.4326 0.8735 0.8696 8.4985 17 3 13.1892 9.009
0.4246 18.0 2610 2.0639 0.4951 0.2634 0.4331 0.4334 0.8721 0.8703 8.7538 16 4 13.3453 12.6126
0.4063 19.0 2755 2.0587 0.499 0.2685 0.4378 0.4383 0.8741 0.8707 8.5916 16 3 13.3003 9.9099
0.3912 20.0 2900 2.1089 0.5068 0.2727 0.4471 0.4469 0.8764 0.8744 8.7538 18 3 13.4625 11.1111
0.3855 21.0 3045 2.1048 0.5022 0.2704 0.4473 0.4478 0.875 0.8728 8.6847 16 4 13.3483 9.3093
0.3808 22.0 3190 2.1804 0.4977 0.2722 0.4414 0.4412 0.875 0.8711 8.5315 17 4 13.0631 10.8108
0.3851 23.0 3335 2.1740 0.4993 0.2696 0.4442 0.4443 0.8742 0.8719 8.5676 15 3 13.2252 9.009
0.3741 24.0 3480 2.1872 0.4921 0.2683 0.4365 0.4369 0.8728 0.8692 8.5195 17 3 13.2192 8.4084
0.3604 25.0 3625 2.2617 0.4988 0.2681 0.4421 0.4426 0.8747 0.8705 8.5255 17 3 13.2492 8.1081
0.3676 26.0 3770 2.2561 0.4931 0.2603 0.4328 0.4331 0.874 0.8711 8.6276 15 3 13.3363 11.7117
0.3799 27.0 3915 2.2404 0.4912 0.2652 0.4329 0.433 0.8729 0.8702 8.6517 17 3 13.4414 8.1081
0.3617 28.0 4060 2.2728 0.4983 0.2704 0.4424 0.4427 0.8756 0.8734 8.7568 17 3 13.5225 11.4114
0.3588 29.0 4205 2.2695 0.4904 0.2601 0.4331 0.4328 0.8743 0.87 8.4775 18 3 13.1592 9.009
0.3567 30.0 4350 2.3006 0.4993 0.2693 0.4419 0.4417 0.8747 0.8737 8.8529 17 3 13.5976 12.012
0.3573 31.0 4495 2.3257 0.4979 0.2669 0.4378 0.4379 0.8743 0.8735 8.9069 18 3 13.6697 12.9129
0.3471 32.0 4640 2.3513 0.4989 0.2723 0.441 0.4405 0.8758 0.8728 8.6246 17 3 13.3063 10.8108
0.3591 33.0 4785 2.3467 0.4972 0.2751 0.4415 0.4413 0.8742 0.8727 8.8078 17 3 13.5616 10.5105
0.3401 34.0 4930 2.4229 0.4854 0.2661 0.4313 0.4318 0.8737 0.8701 8.5826 17 3 13.2673 8.7087
0.3476 35.0 5075 2.3804 0.4895 0.2602 0.4322 0.4326 0.874 0.8712 8.6577 17 3 13.2883 9.3093
0.3473 36.0 5220 2.4242 0.4938 0.2689 0.438 0.4387 0.8745 0.8713 8.5976 17 3 13.2432 9.3093
0.3415 37.0 5365 2.3836 0.4943 0.2617 0.4351 0.4351 0.8751 0.8711 8.4054 17 3 13.0571 8.1081
0.3549 38.0 5510 2.4110 0.501 0.2696 0.4402 0.4406 0.8765 0.8713 8.2282 17 3 12.9459 6.6066
0.3432 39.0 5655 2.4016 0.4999 0.27 0.4387 0.4393 0.8751 0.8712 8.5285 17 3 13.2402 8.4084
0.3387 40.0 5800 2.4546 0.4986 0.2718 0.4417 0.4422 0.8742 0.871 8.5766 17 3 13.2312 9.3093
0.3351 41.0 5945 2.4478 0.4981 0.2714 0.4367 0.4372 0.8756 0.8722 8.4775 15 3 13.1411 8.7087
0.3366 42.0 6090 2.4447 0.4961 0.2703 0.4359 0.437 0.8746 0.8699 8.4745 16 3 13.1231 9.3093
0.3324 43.0 6235 2.4974 0.4989 0.2809 0.4428 0.4432 0.8747 0.873 8.7147 16 3 13.4565 10.5105
0.3306 44.0 6380 2.4938 0.4902 0.2657 0.4301 0.4306 0.8733 0.8692 8.4925 15 3 13.1622 8.4084
0.3388 45.0 6525 2.5098 0.4788 0.2616 0.4246 0.4245 0.8734 0.8662 8.2162 16 3 12.7538 8.1081
0.346 46.0 6670 2.4595 0.4987 0.2689 0.438 0.4389 0.875 0.8718 8.5676 16 3 13.2252 9.9099
0.3401 47.0 6815 2.5098 0.4934 0.2653 0.4353 0.4356 0.8744 0.87 8.3934 15 3 13.048 8.1081
0.3271 48.0 6960 2.5204 0.4951 0.2674 0.4373 0.4372 0.8749 0.8703 8.4625 16 3 13.024 9.009
0.3267 49.0 7105 2.5291 0.4887 0.2605 0.428 0.4284 0.8728 0.8702 8.7057 18 3 13.3363 11.1111
0.3382 50.0 7250 2.5422 0.4899 0.2666 0.4354 0.4356 0.8755 0.8707 8.4505 16 3 13.0931 8.1081
0.3255 51.0 7395 2.5254 0.4921 0.2634 0.4346 0.4352 0.8738 0.8691 8.4535 16 3 13.027 10.2102
0.32 52.0 7540 2.5460 0.4991 0.2727 0.4423 0.4421 0.8745 0.873 8.8919 16 3 13.5736 11.7117
0.3154 53.0 7685 2.5446 0.5027 0.2712 0.4463 0.4463 0.8768 0.8734 8.6426 16 3 13.2973 11.1111
0.3293 54.0 7830 2.5378 0.4928 0.2669 0.4352 0.4354 0.8736 0.869 8.5225 16 3 13.1291 10.2102
0.3231 55.0 7975 2.5905 0.4949 0.2678 0.4378 0.4375 0.8743 0.8714 8.6426 15 3 13.3003 9.009
0.3239 56.0 8120 2.5884 0.4969 0.2697 0.4399 0.4399 0.8737 0.8712 8.6697 16 3 13.3754 10.5105
0.3174 57.0 8265 2.5500 0.4958 0.267 0.4389 0.4386 0.8739 0.8715 8.7327 16 4 13.3844 10.5105
0.3209 58.0 8410 2.5804 0.4989 0.2706 0.442 0.4426 0.8751 0.8717 8.5766 15 3 13.1952 9.3093
0.3297 59.0 8555 2.5909 0.494 0.2622 0.4343 0.4338 0.8733 0.8698 8.5976 16 3 13.1652 11.7117
0.3226 60.0 8700 2.5857 0.4976 0.2639 0.4377 0.438 0.8753 0.8701 8.3904 17 3 12.973 7.8078
0.3241 61.0 8845 2.5824 0.5011 0.2698 0.4428 0.4436 0.8764 0.8725 8.5345 16 3 13.1502 10.5105
0.3201 62.0 8990 2.6156 0.4968 0.2673 0.4371 0.4372 0.8755 0.8702 8.3904 16 3 12.979 6.9069
0.3234 63.0 9135 2.6374 0.4945 0.2677 0.4387 0.4388 0.8744 0.8693 8.4444 17 3 12.958 8.1081
0.3246 64.0 9280 2.6338 0.4912 0.2672 0.4396 0.4402 0.8738 0.8698 8.4955 17 3 13.1021 8.1081
0.3188 65.0 9425 2.6206 0.4999 0.2739 0.4443 0.4444 0.8763 0.8726 8.6006 17 3 13.2042 10.5105
0.3186 66.0 9570 2.6499 0.5007 0.2771 0.4462 0.4463 0.8765 0.8729 8.5375 17 3 13.2162 9.3093
0.319 67.0 9715 2.6488 0.5023 0.2715 0.4452 0.4454 0.8761 0.8736 8.6817 17 3 13.3904 10.2102
0.3328 68.0 9860 2.6238 0.5002 0.2696 0.4408 0.4411 0.8755 0.8717 8.5075 17 3 13.1081 9.009
0.3068 69.0 10005 2.6525 0.4971 0.2684 0.4391 0.4397 0.8755 0.8712 8.5045 17 3 13.1411 11.4114
0.3192 70.0 10150 2.6494 0.4976 0.2722 0.4395 0.4405 0.8762 0.8714 8.3964 17 3 13.033 8.4084
0.3232 71.0 10295 2.6642 0.4976 0.2717 0.4412 0.4411 0.8756 0.8717 8.5075 17 3 13.1622 9.9099
0.3084 72.0 10440 2.6596 0.4931 0.2669 0.4352 0.4354 0.8734 0.8696 8.4865 17 3 13.1682 9.009
0.313 73.0 10585 2.6551 0.4942 0.2699 0.4363 0.4368 0.8742 0.8699 8.4715 16 3 13.1201 9.6096
0.3194 74.0 10730 2.6769 0.4962 0.2689 0.4388 0.4389 0.874 0.8715 8.5976 17 3 13.2763 10.5105
0.3143 75.0 10875 2.6860 0.493 0.2652 0.4335 0.4343 0.8734 0.8702 8.5706 16 3 13.2462 9.3093
0.3209 76.0 11020 2.6777 0.4893 0.2592 0.4325 0.4324 0.8726 0.869 8.5225 16 3 13.2012 9.3093
0.3078 77.0 11165 2.6797 0.4877 0.261 0.4321 0.4323 0.8724 0.8693 8.5796 16 3 13.2402 9.6096
0.3192 78.0 11310 2.6812 0.495 0.2677 0.4382 0.4383 0.8739 0.871 8.5706 18 3 13.2523 10.8108
0.3147 79.0 11455 2.6777 0.495 0.2693 0.4371 0.4374 0.874 0.8707 8.5015 16 3 13.1471 9.3093
0.3049 80.0 11600 2.6767 0.4917 0.2647 0.4344 0.4346 0.8723 0.8696 8.5616 16 3 13.2162 9.9099
0.3191 81.0 11745 2.6932 0.4929 0.2683 0.4392 0.4392 0.8737 0.8707 8.5676 16 3 13.2342 9.6096
0.3073 82.0 11890 2.7036 0.4959 0.2699 0.4389 0.4393 0.8738 0.8722 8.6547 17 3 13.3964 10.2102
0.3129 83.0 12035 2.6941 0.4918 0.2657 0.4341 0.434 0.8742 0.8703 8.4985 16 3 13.1411 9.3093
0.3308 84.0 12180 2.6968 0.4927 0.2659 0.4335 0.4337 0.8737 0.8698 8.4955 16 3 13.1652 9.3093
0.3221 85.0 12325 2.6966 0.4903 0.2606 0.4306 0.4306 0.8726 0.8698 8.5766 16 3 13.2823 9.6096
0.3085 86.0 12470 2.7123 0.4862 0.2608 0.4288 0.4286 0.8723 0.8688 8.4595 16 3 13.0901 8.7087
0.3281 87.0 12615 2.7101 0.4918 0.2638 0.4322 0.4328 0.8731 0.8695 8.4775 16 3 13.1291 9.009
0.3183 88.0 12760 2.7102 0.4902 0.2649 0.4294 0.4301 0.873 0.8688 8.4955 16 3 13.0901 9.6096
0.3063 89.0 12905 2.7198 0.4934 0.2676 0.4338 0.4344 0.8734 0.8692 8.4565 17 3 13.0751 9.009
0.3123 90.0 13050 2.7228 0.492 0.2676 0.4338 0.4343 0.8732 0.8692 8.4535 17 3 13.0931 9.3093
0.3163 91.0 13195 2.7264 0.4953 0.2702 0.4357 0.4358 0.874 0.8693 8.4625 17 3 13.033 9.3093
0.3085 92.0 13340 2.7236 0.4934 0.2702 0.4369 0.4369 0.8738 0.8695 8.4925 17 3 13.0721 9.9099
0.3257 93.0 13485 2.7202 0.4953 0.2706 0.4368 0.4368 0.8746 0.8699 8.4595 16 3 13.0571 10.2102
0.3092 94.0 13630 2.7261 0.4988 0.2748 0.4415 0.4419 0.8755 0.8708 8.4535 16 3 13.0751 9.9099
0.3187 95.0 13775 2.7248 0.4968 0.2727 0.4383 0.4389 0.8751 0.8709 8.5075 16 3 13.1321 9.9099
0.3155 96.0 13920 2.7335 0.4962 0.2686 0.4372 0.4373 0.8749 0.8712 8.5135 16 3 13.1772 10.2102
0.3271 97.0 14065 2.7384 0.4971 0.2721 0.4396 0.4397 0.8749 0.8711 8.5135 16 3 13.1832 10.5105
0.3096 98.0 14210 2.7400 0.496 0.2712 0.4386 0.4385 0.8748 0.8711 8.5225 16 3 13.1682 10.2102
0.3116 99.0 14355 2.7411 0.4961 0.2712 0.4388 0.4386 0.8749 0.8711 8.5135 16 3 13.1592 10.2102
0.3102 100.0 14500 2.7416 0.4961 0.2712 0.4388 0.4386 0.8749 0.8711 8.5135 16 3 13.1592 10.2102

Framework versions

  • Transformers 4.33.1
  • Pytorch 2.0.1+cu118
  • Datasets 2.14.5
  • Tokenizers 0.13.3