|
--- |
|
library_name: diffusers |
|
tags: |
|
- music |
|
pipeline_tag: text-to-audio |
|
--- |
|
|
|
# Official Hugging Face Diffusers Implementation of QA-MDT |
|
**QAMDT: Quality-Aware Diffusion for Text-to-Music 🎶** |
|
|
|
QADMT brings a new approach to text-to-music generation by using quality-aware training to tackle issues like low-fidelity audio and weak labeling in datasets. |
|
|
|
With a masked diffusion transformer (MDT), QADMT delivers SOTA results on MusicCaps and Song-Describer, enhancing both quality and musicality. |
|
|
|
It follows from [this paper](https://arxiv.org/pdf/2405.15863) by the University of Science and Technology of China, authored by [@changli](https://github.com/ivcylc) *et al.*. |
|
## Usage: |
|
|
|
```bash |
|
!git lfs install |
|
!git clone https://huggingface.co/jadechoghari/openmusic |
|
``` |
|
|
|
Manually change the folder name from `openmusic` to `qa_mdt` |
|
|
|
```bash |
|
pip install -r qa_mdt/requirements.txt |
|
pip install xformers==0.0.26.post1 |
|
pip install torchlibrosa==0.0.9 librosa==0.9.2 |
|
pip install -q pytorch_lightning==2.1.3 torchlibrosa==0.0.9 librosa==0.9.2 ftfy==6.1.1 braceexpand |
|
pip install torch==2.3.0+cu121 torchvision==0.18.0+cu121 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu121 |
|
``` |
|
|
|
```python |
|
from qa_mdt.pipeline import MOSDiffusionPipeline |
|
|
|
pipe = MOSDiffusionPipeline() |
|
pipe("A modern synthesizer creating futuristic soundscapes.") |
|
``` |
|
|
|
# Enjoy the music!! 🎶 |
