license: gemma
library_name: peft
tags:
- trl
- sft
- generated_from_trainer
base_model: google/gemma-2b
model-index:
- name: gemma-prompt
results: []
gemma-prompt
This model is a fine-tuned version of google/gemma-2b on 2 datasets- dolly-15k for general knowledge and a curated m-a-p/MusicPile
Model description
This model is a completed trained model used for music knowledge and prompt automation from musical vibes.
Intended uses & limitations
Intended use for the model is to have it generate prompts for music that takes into account elements of the surrounding environment, such as the types of buildings nearby, the weather, time of day, and nearby landmarks.
Training and evaluation data
The datasets used to help train the model are the dolly-15k dataset for general purpose of answering questions and following commands, and a second curated m-a-p/MusicPile data used to fine-tune the model specifically for musical vibe and descriptions of different objects, places, and things. We evaluate our model using a portion of the m-a-p/MusicPile.
Training procedure
Split dataset from MusicPile to focus on distilled music knowledge Used dolly for general finetuning
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- training_steps: 888
Framework versions
- PEFT 0.10.0
- Transformers 4.40.0
- Pytorch 2.0.1a0+cxx11.abi
- Datasets 2.19.0
- Tokenizers 0.19.1