Simonlob
/

TTS_Akyl-AI_alpha

@@ -1,199 +1,128 @@
 ---
-library_name: transformers
 pipeline_tag: text-to-speech
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]

 ---
+license: cc-by-nc-4.0
+inference: true
+tags:
+- mms
+- vits
 pipeline_tag: text-to-speech
+language:
+- ky
 ---
+# Introduction
+This repository contains a text-to-speech (TTS) model fine-tuned on data consisting of sentences in the Kyrgyz language with audio examples voiced by a single speaker. The audio is provided at a sample rate of 16 kHz. The dataset comprises 5000 examples and 7 hours of audio. The model is based on the facebook/mms-tts-kir model pre-trained on the Kyrgyz language. The code for fine-tuning the model was based on the code from this [GitHub repository](https://github.com/ylacombe/finetune-hf-vits). Experimental findings concluded that the best results are achieved through two-stage fine-tuning:
+* Training with Learning Rate 1e-4 and 4 epochs,
+* Training with Learning Rate 5e-7 and 80 epochs.
+# MMS: Scaling Speech Technology to 1000+ languages
+The Massively Multilingual Speech (MMS) project expands speech technology from about 100 languages to over 1,000 by building a single multilingual speech recognition model supporting over 1,100 languages (more than 10 times as many as before), language identification models able to identify over [4,000 languages](https://dl.fbaipublicfiles.com/mms/misc/language_coverage_mms.html) (40 times more than before), pretrained models supporting over 1,400 languages, and text-to-speech models for over 1,100 languages. Our goal is to make it easier for people to access information and to use devices in their preferred language.
+You can find details in the paper [Scaling Speech Technology to 1000+ languages](https://research.facebook.com/publications/scaling-speech-technology-to-1000-languages/) and the [blog post](https://ai.facebook.com/blog/multilingual-model-speech-recognition/).
+An overview of the languages covered by MMS can be found [here](https://dl.fbaipublicfiles.com/mms/misc/language_coverage_mms.html).
+##  Transformers
+MMS has been added to Transformers. For more information, please refer to [Transformers' MMS docs](https://huggingface.co/docs/transformers/main/en/model_doc/mms).
+[Click here](https://huggingface.co/models?other=mms) to find all MMS checkpoints on the Hub.
+Checkout the demo here [![Open In HF Spaces](https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm-dark.svg)](https://huggingface.co/spaces/facebook/MMS)
+##
+# Inference
+The model takes Cyrillic text in the Kyrgyz language as input and preprocesses it by removing punctuation marks (periods, commas, colons, exclamation and question marks) as well as words written in Latin script. Therefore, it is not advisable to feed multiple sentences into the model at once as they will be vocalized without intonational pauses, indicating the end of one and the beginning of a new sentence. Words written in Latin script will be skipped in the generated speech.
+For example:
+```
+text = 'Кандай улут болбосун кыргызча жооп кайтарышыбыз керек.'
+```
+You can use this model by executing the code provided below.
+```
+import subprocess
+from transformers import pipeline
+from IPython.display import Audio
+import numpy as np
+import torch
+import scipy
+model_id = "Simonlob/simonlob_akylay"
+synthesiser = pipeline("text-to-speech", model_id) # add device=0 if you want to use a GPU
+```
+```
+text = 'Кандай улут болбосун кыргызча жооп кайтарышыбыз керек.'
+speech = synthesiser(text)
+```
+The output of the model looks as follows:
+```
+{'audio': array([[-1.7045566e-04,  8.9107212e-05,  2.8329418e-04, ...,
+          8.0898666e-08,  4.8763245e-06,  5.4663483e-06]], dtype=float32),
+ 'sampling_rate': 16000}
+```
+Listen to the result:
+```
+Audio(speech['audio'], rate=speech['sampling_rate'])
+```
+Save the audio as a file:
+```
+scipy.io.wavfile.write("<OUTPUT PATH>.wav", rate=speech["sampling_rate"], data=speech["audio"][0])
+```
+</details>
+## Model details
+- **Model type:** Text-to-speech model
+- **License:** CC-BY-NC 4.0 license
+- **Cite as:**
+      @article{pratap2023mms,
+        title={Scaling Speech Technology to 1,000+ Languages},
+        author={Vineel Pratap and Andros Tjandra and Bowen Shi and Paden Tomasello and Arun Babu and Sayani Kundu and Ali Elkahky and Zhaoheng Ni and Apoorv Vyas and Maryam Fazel-Zarandi and Alexei Baevski and Yossi Adi and Xiaohui Zhang and Wei-Ning Hsu and Alexis Conneau and Michael Auli},
+      journal={arXiv},
+      year={2023}
+      }
+## Credits
+- Facebook AI Research ([Official Space](https://huggingface.co/spaces/facebook/MMS))
+- Yoach Lacombe (Research) [GitHub](https://github.com/ylacombe/finetune-hf-vits)
+- The Cramer Project (Data collection and preprocessing)[Official Space](https://thecramer.com/), [Akyl_AI](https://github.com/Akyl-AI)
+- Amantur Amatov (Expert)
+- Timur Turatali (Expert, Research) [GitHub](https://github.com/golden-ratio)
+- Den Pavlov (Research, Data preprocessing and fine-tuning) [GitHub](https://github.com/simonlobgromov/finetune-hf-vits)
+- Ulan Abdurazakov (Environment Developer)
+- Nursultan Bakashov (CEO)
+## Additional Links
+- [Blog post](https://ai.facebook.com/blog/multilingual-model-speech-recognition/)
+- [Transformers documentation](https://huggingface.co/docs/transformers/main/en/model_doc/mms).
+- [Paper](https://arxiv.org/abs/2305.13516)
+- [GitHub Repository for fine tuning](https://github.com/ylacombe/finetune-hf-vits)
+- [GitHub Repository](https://github.com/facebookresearch/fairseq/tree/main/examples/mms#asr)
+- [Other **MMS** checkpoints](https://huggingface.co/models?other=mms)
+- MMS base checkpoints:
+  - [facebook/mms-1b](https://huggingface.co/facebook/mms-1b)
+  - [facebook/mms-300m](https://huggingface.co/facebook/mms-300m)