Update README.md

README.md

@@ -9,17 +9,35 @@ Without Convolution or Region Supervision](https://arxiv.org/abs/2102.03334) by
 
 Disclaimer: The team releasing ViLT did not write a model card for this model so this model card has been written by the Hugging Face team.
 
-## Model description
-
-(to do)
-
 ## Intended uses & limitations
 
-You can use the raw model for natural language visual reasoning. 
+You can use the model to determine whether a sentence is true or false given 2 images.
 
 ### How to use
 
-(to do)
+Here is how to use the model in PyTorch:
+
+```
+from transformers import ViltProcessor, ViltForImagesAndTextClassification
+import requests
+from PIL import Image
+
+image1 = Image.open(requests.get("https://lil.nlp.cornell.edu/nlvr/exs/ex0_0.jpg", stream=True).raw)
+image2 = Image.open(requests.get("https://lil.nlp.cornell.edu/nlvr/exs/ex0_1.jpg", stream=True).raw)
+text = "The left image contains twice the number of dogs as the right image."
+
+processor = ViltProcessor.from_pretrained("dandelin/vilt-b32-finetuned-nlvr2")
+model = ViltForImagesAndTextClassification.from_pretrained("dandelin/vilt-b32-finetuned-nlvr2")
+
+# prepare inputs
+encoding = processor([image1, image2], text, return_tensors="pt")
+
+# forward pass
+outputs = model(input_ids=encoding.input_ids, pixel_values=encoding.pixel_values.unsqueeze(0))
+logits = outputs.logits
+idx = logits.argmax(-1).item()
+print("Predicted answer:", model.config.id2label[idx])
+```
 
 ## Training data