---
license: mit
language:
- en
widget:
- text: >
    In a busy city street, an autonomous car navigates around pedestrians and cyclists, making a series of complex decisions to ensure safety.
  example_title: Complex VQA Scenario
- text: >
    On a clear highway, an autonomous vehicle adjusts its speed based on the flow of traffic and the presence of a slower-moving truck ahead.
  example_title: Simple VQA Scenario
tags:
- vision-language
- autonomous-driving
---

### What is this?
Lingo-Judge, a novel evaluation metric that aligns closely with human judgment on the LingoQA evaluation suits.

### How to use
```python
# Import necessary libraries
from transformers import pipeline

# Define the model name to be used in the pipeline
model_name = 'wayveai/Lingo-Judge'

# Define the question and its corresponding answer and prediction
question = "Are there any pedestrians crossing the road? If yes, how many?"
answer = "1"
prediction = "Yes, there is one"

# Initialize the pipeline with the specified model, device, and other parameters
pipe = pipeline("text-classification", model=model_name)
# Format the input string with the question, answer, and prediction
input = f"[CLS]\nQuestion: {question}\nAnswer: {answer}\nStudent: {prediction}"

# Pass the input through the pipeline to get the result
result = pipe(input)

# Print the result and score
score = result[0]['score']
print(score > 0.5, score)