Edit model card

The model was trained on MSP-Podcast for the Odyssey 2024 Emotion Recognition competition baseline
This particular model is the multi-attributed based model which predict arousal, dominance and valence in a range of approximately 0...1.

Benchmarks

CCC based on Test3 and Development sets of the Odyssey Competition

Multi-Task Setup
Test 3Development
Val Dom Aro Val Dom Aro
0.577 0.577 0.405 0.652 0.688 0.579

For more details: demo, paper, and GitHub.

@InProceedings{Goncalves_2024,
            author={L. Goncalves and A. N. Salman and A. {Reddy Naini} and L. Moro-Velazquez and T. Thebaud and L. {Paola Garcia} and N. Dehak and B. Sisman and C. Busso},
            title={Odyssey2024 - Speech Emotion Recognition Challenge: Dataset, Baseline Framework, and Results},
            booktitle={Odyssey 2024: The Speaker and Language Recognition Workshop)},
            volume={To appear},
            year={2024},
            month={June},
            address =  {Quebec, Canada},
}

Usage

from transformers import AutoModelForAudioClassification
import librosa, torch

#load model
model = AutoModelForAudioClassification.from_pretrained("3loi/SER-Odyssey-Baseline-WavLM-Multi-Attributes", trust_remote_code=True)

#get mean/std
mean = model.config.mean
std = model.config.std


#load an audio file
audio_path = "/path/to/audio.wav"
raw_wav, _ = librosa.load(audio_path, sr=model.config.sampling_rate)

#normalize the audio by mean/std
norm_wav = (raw_wav - mean) / (std+0.000001)

#generate the mask
mask = torch.ones(1, len(norm_wav))

#batch it (add dim)
wavs = torch.tensor(norm_wav).unsqueeze(0)


#predict
with torch.no_grad():
    pred = model(wavs, mask)

print(model.config.id2label) 
print(pred)
#{0: 'arousal', 1: 'dominance', 2: 'valence'}
#tensor([[0.3670, 0.4553, 0.4240]])
Downloads last month
50,062
Safetensors
Model size
319M params
Tensor type
F32
ยท
Inference Examples
Inference API (serverless) does not yet support model repos that contain custom code.

Space using 3loi/SER-Odyssey-Baseline-WavLM-Multi-Attributes 1