schlager-bot-004

This model is a fine-tuned version of LeoLM/leo-hessianai-7b on a dataset of 1048 schlager song lyrics. Schlager songs form a genre of German music and, therefore, the input should be in German as well to ensure best results.

Model description

The model takes a verse (in German) and uses it to generate the text to a schlager song.

Intended uses & limitations

This model is not intended for commercial use of any kind as the dataset used for fine-tuning contains propietary information.

Training and evaluation data

The training and evaluation data was extracted from Spotify via an API.

Training procedure

The model was fine-tuned using a Google colab notebook with a V100 GPU and High RAM. For details, please see the Github repository (https://github.com/NiclasFenton-Wiegleb/schlager-lyrics-bot/tree/main)

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0005
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 8
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 10

Framework versions

  • Transformers 4.34.0
  • Pytorch 2.1.0+cu118
  • Datasets 2.13.0
  • Tokenizers 0.14.1
Downloads last month
26
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for niclasfw/schlager-bot-004

Finetuned
(1)
this model

Space using niclasfw/schlager-bot-004 1