Create README.md

README.md

@@ -0,0 +1,43 @@
+---
+tags:
+- gpt2
+- text-generation
+- music-modeling
+- music-generation
+widget:
+- text: PIECE_START
+- text: >-
+    PIECE_START PIECE_START TRACK_START INST=34 DENSITY=8
+- text: >-
+    PIECE_START TRACK_START INST=1
+license: apache-2.0
+---
+
+
+# GPT-2 for Music
+
+Language Models such as GPT-2 can be used for Music Generation. The idea is to represent pieces of music as texts, effectively reducing the task to Language Generation.
+
+This model is a rather small instance of GPT-2 trained the [Lakhclean dataset](https://colinraffel.com/projects/lmd/). The model generates 4 bars at a time at a 16th note resolution with 4/4 meter.
+
+If you want to contribute, if you want to say hello, if you want to know more, find me on [LinkedIn](https://www.linkedin.com/in/dr-tristan-behrens-734967a2/)
+
+## Model description
+
+The model is GPT-2 with 6 decoders and 8 attention-heads each. The context length is 2048. The embedding dimensions are 512 as well.
+
+## Intended uses & limitations
+
+This model is just a proof of concept. It shows that HuggingFace can be used to compose music.
+
+### How to use
+
+There is a notebook in the repo that you can use to generate symbolic music and then render it.
+
+### Limitations and bias
+
+Since this model has been trained on a very small corpus of music, it is overfitting heavily. 
+
+### Acknowledgements
+
+This model has been created with support from NVIDIA. I am very grateful for the GPU compute they provided!