Hugging Face's logo

michaelfeil
/
ct2fast-opus-mt-es-en

Translation
Transformers
Spanish
English
ctranslate2
Inference Endpoints
Model card Files Community
Edit model card

# Fast-Inference with Ctranslate2

Speedup inference by 2x-8x using int8 inference in C++

quantized version of Helsinki-NLP/opus-mt-es-en

pip install hf-hub-ctranslate2>=1.0.0 ctranslate2>=3.13.0

Converted using

ct2-transformers-converter --model Helsinki-NLP/opus-mt-es-en --output_dir /home/michael/tmp-ct2fast-opus-mt-es-en --force --copy_files README.md generation_config.json tokenizer_config.json vocab.json source.spm .gitattributes target.spm --quantization float16

Checkpoint compatible to ctranslate2 and hf-hub-ctranslate2

  • compute_type=int8_float16 for device="cuda"
  • compute_type=int8 for device="cpu"
from hf_hub_ctranslate2 import TranslatorCT2fromHfHub, GeneratorCT2fromHfHub
from transformers import AutoTokenizer

model_name = "michaelfeil/ct2fast-opus-mt-es-en"
# use either TranslatorCT2fromHfHub or GeneratorCT2fromHfHub here, depending on model.
model = TranslatorCT2fromHfHub(
        # load in int8 on CUDA
        model_name_or_path=model_name, 
        device="cuda",
        compute_type="int8_float16",
        tokenizer=AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-es-en")
)
outputs = model.generate(
    text=["How do you call a fast Flan-ingo?", "User: How are you doing?"],
)
print(outputs)

Licence and other remarks:

This is just a quantized version. Licence conditions are intended to be idential to original huggingface repo.

Original description

spa-eng

Benchmarks

testset BLEU chr-F
newssyscomb2009-spaeng.spa.eng 30.6 0.570
news-test2008-spaeng.spa.eng 27.9 0.553
newstest2009-spaeng.spa.eng 30.4 0.572
newstest2010-spaeng.spa.eng 36.1 0.614
newstest2011-spaeng.spa.eng 34.2 0.599
newstest2012-spaeng.spa.eng 37.9 0.624
newstest2013-spaeng.spa.eng 35.3 0.609
Tatoeba-test.spa.eng 59.6 0.739

System Info:

Downloads last month
41
Inference Examples
Translation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.