|
--- |
|
library_name: transformers |
|
tags: |
|
- audio |
|
- automatic-speech-recognition |
|
license: mit |
|
language: |
|
- ar |
|
--- |
|
|
|
# ArTST (ASR task) |
|
|
|
ArTST model finetuned for automatic speech recognition (speech-to-text) on MGB2. |
|
|
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. |
|
|
|
- **Developed by:** Speech Lab, MBZUAI |
|
- **Model type:** SpeechT5 |
|
- **Language:** Arabic |
|
- **Finetuned from:** (ArTST pretrained)[https://github.com/mbzuai-nlp/ArTST] |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
```python |
|
import soundfile as sf |
|
from transformers import ( |
|
SpeechT5Config, |
|
SpeechT5FeatureExtractor, |
|
SpeechT5ForSpeechToText, |
|
SpeechT5Processor, |
|
SpeechT5Tokenizer, |
|
) |
|
|
|
from custom_tokenizer import CustomTextTokenizer |
|
|
|
device = "cuda" if torch.cuda.is_available() else "cpu" |
|
|
|
tokenizer = SpeechT5Tokenizer.from_pretrained("mbzuai/artst_asr") |
|
processor = SpeechT5Processor.from_pretrained("mbzuai/artst_asr" , tokenizer=tokenizer) |
|
model = SpeechT5ForSpeechToText.from_pretrained("mbzuai/artst_asr").to(device) |
|
|
|
audio, sr = sf.read("audio.wav") |
|
|
|
inputs = processor(audio=audio, sampling_rate=sr, return_tensors="pt") |
|
predicted_ids = model.generate(**inputs.to(device), max_length=150) |
|
|
|
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True) |
|
print(transcription[0]) |
|
``` |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Model Sources [optional] |
|
- **Repository:** [github](https://github.com/mbzuai-nlp/ArTST) |
|
- **Paper :** [ArabicNLP2023](https://aclanthology.org/2023.arabicnlp-1.5/) |
|
<!-- - **Demo [optional]:** [More Information Needed] --> |
|
|
|
|
|
## Citation [optional] |
|
|
|
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
|
|
|
**BibTeX:** |
|
``` |
|
@inproceedings{toyin-etal-2023-artst, |
|
title = "{A}r{TST}: {A}rabic Text and Speech Transformer", |
|
author = "Toyin, Hawau and |
|
Djanibekov, Amirbek and |
|
Kulkarni, Ajinkya and |
|
Aldarmaki, Hanan", |
|
booktitle = "Proceedings of ArabicNLP 2023", |
|
month = dec, |
|
year = "2023", |
|
address = "Singapore (Hybrid)", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://aclanthology.org/2023.arabicnlp-1.5", |
|
doi = "10.18653/v1/2023.arabicnlp-1.5", |
|
pages = "41--51", |
|
} |
|
``` |
|
<!-- **APA:** |
|
|
|
[More Information Needed] |
|
|
|
## Glossary [optional] |
|
|
|
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. --> |
|
|
|
<!-- [More Information Needed] |
|
|
|
## More Information [optional] |
|
|
|
[More Information Needed] |
|
|
|
## Model Card Authors [optional] |
|
|
|
[More Information Needed] |
|
|
|
## Model Card Contact |
|
|
|
[More Information Needed] --> |
