---
license: apache-2.0
base_model: t5-small
tags:
- generated_from_trainer
datasets:
- samsum
metrics:
- rouge
model-index:
- name: t5-small-t5-dialogue-summarizer
  results:
  - task:
      name: Sequence-to-sequence Language Modeling
      type: text2text-generation
    dataset:
      name: samsum
      type: samsum
      config: samsum
      split: validation
      args: samsum
    metrics:
    - name: Rouge1
      type: rouge
      value: 43.3371
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-small-t5-dialogue-summarizer

This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the samsum dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7032
- Rouge1: 43.3371
- Rouge2: 20.6294
- Rougel: 36.6607
- Rougelsum: 40.209
- Gen Len: 16.698

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
| No log        | 1.0   | 460  | 1.8115          | 41.2589 | 18.3552 | 34.5107 | 38.2488   | 16.8068 |
| 1.9846        | 2.0   | 921  | 1.7892          | 41.1617 | 18.4345 | 34.745  | 38.2061   | 16.6247 |
| 1.9568        | 3.0   | 1381 | 1.7757          | 41.7317 | 19.0104 | 35.2965 | 38.6958   | 16.4059 |
| 1.9298        | 4.0   | 1842 | 1.7573          | 42.0478 | 19.1229 | 35.4855 | 39.0882   | 16.6235 |
| 1.9049        | 5.0   | 2302 | 1.7496          | 42.4985 | 19.5594 | 35.9228 | 39.4201   | 16.5416 |
| 1.8852        | 6.0   | 2763 | 1.7411          | 42.3214 | 19.6152 | 35.7488 | 39.3079   | 16.7139 |
| 1.8674        | 7.0   | 3223 | 1.7335          | 42.3206 | 19.7528 | 35.9918 | 39.2783   | 16.5073 |
| 1.855         | 8.0   | 3684 | 1.7300          | 42.9099 | 20.2273 | 36.4393 | 39.8506   | 16.61   |
| 1.8435        | 9.0   | 4144 | 1.7225          | 42.9661 | 20.3074 | 36.3468 | 39.8945   | 16.7103 |
| 1.8342        | 10.0  | 4605 | 1.7198          | 43.0181 | 20.2982 | 36.4202 | 39.9022   | 16.7726 |
| 1.8216        | 11.0  | 5065 | 1.7169          | 43.0296 | 20.5422 | 36.6314 | 40.111    | 16.6883 |
| 1.8168        | 12.0  | 5526 | 1.7144          | 43.3035 | 20.7167 | 36.7924 | 40.2953   | 16.7787 |
| 1.8168        | 13.0  | 5986 | 1.7104          | 43.2258 | 20.7416 | 36.7823 | 40.2551   | 16.7286 |
| 1.8088        | 14.0  | 6447 | 1.7075          | 43.3982 | 20.8281 | 36.8254 | 40.3198   | 16.7384 |
| 1.8008        | 15.0  | 6907 | 1.7079          | 43.3077 | 20.7164 | 36.6791 | 40.2372   | 16.687  |
| 1.8014        | 16.0  | 7368 | 1.7047          | 43.1989 | 20.6984 | 36.7104 | 40.2285   | 16.6479 |
| 1.7934        | 17.0  | 7828 | 1.7034          | 43.4149 | 20.7879 | 36.7308 | 40.3556   | 16.7922 |
| 1.7894        | 18.0  | 8289 | 1.7041          | 43.2962 | 20.7667 | 36.7017 | 40.28     | 16.6883 |
| 1.7914        | 19.0  | 8749 | 1.7037          | 43.2489 | 20.6943 | 36.676  | 40.1802   | 16.6932 |
| 1.7827        | 19.98 | 9200 | 1.7032          | 43.3371 | 20.6294 | 36.6607 | 40.209    | 16.698  |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0