|
--- |
|
tags: |
|
- clip |
|
library_name: open_clip |
|
pipeline_tag: zero-shot-image-classification |
|
license: cc-by-nc-4.0 |
|
datasets: |
|
- visheratin/laion-coco-nllb |
|
--- |
|
|
|
## Model Summary |
|
|
|
NLLB-CLIP is a model that combines a text encoder from the [NLLB model](https://huggingface.co/facebook/nllb-200-distilled-1.3B) and an image encoder from the |
|
LAION [CLIP](https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K). This allows us to extend the model capabilities |
|
to 201 languages of the Flores-200. NLLB-CLIP sets state-of-the-art on the [Crossmodal-3600](https://google.github.io/crossmodal-3600/) dataset by performing very |
|
well on low-resource languages. You can find more details about the model in the [paper](https://arxiv.org/abs/2309.01859). |
|
|
|
## Acknowledgements |
|
|
|
I thank [ML Collective](https://mlcollective.org/) for providing Google Cloud compute resources to train the OpenCLIP-compatible version of NLLB-CLIP. |
