metadata

library_name: transformers
license: apache-2.0
base_model: t5-small
tags:
  - generated_from_trainer
datasets:
  - samsum
metrics:
  - rouge
model-index:
  - name: t5-small-transcript-summarizer
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: validation
          args: samsum
        metrics:
          - name: Rouge1
            type: rouge
            value: 41.4993

t5-small-transcript-summarizer

This model is a fine-tuned version of t5-small on the samsum dataset. It achieves the following results on the evaluation set:

Loss: 1.7347
Rouge1: 41.4993
Rouge2: 18.6768
Rougel: 34.8901
Rougelsum: 38.434
Gen Len: 16.533

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 4
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.9691	1.0	3683	1.7864	40.9391	18.2339	34.4773	38.0777	16.7017
1.9213	2.0	7366	1.7533	41.4658	18.8086	34.8947	38.3528	16.5208
1.8757	3.0	11049	1.7367	41.641	18.7355	35.0432	38.5403	16.544
1.8484	4.0	14732	1.7347	41.4993	18.6768	34.8901	38.434	16.533

Framework versions

Transformers 4.44.2
Pytorch 2.4.1+cu121
Datasets 3.0.1
Tokenizers 0.19.1