Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,44 @@
|
|
1 |
---
|
2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
license: mit
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
widget:
|
6 |
+
- text: >
|
7 |
+
In a busy city street, an autonomous car navigates around pedestrians and cyclists, making a series of complex decisions to ensure safety.
|
8 |
+
example_title: Complex VQA Scenario
|
9 |
+
- text: >
|
10 |
+
On a clear highway, an autonomous vehicle adjusts its speed based on the flow of traffic and the presence of a slower-moving truck ahead.
|
11 |
+
example_title: Simple VQA Scenario
|
12 |
+
tags:
|
13 |
+
- vision-language
|
14 |
+
- autonomous-driving
|
15 |
---
|
16 |
+
|
17 |
+
### What is this?
|
18 |
+
Lingo-Judge, a novel evaluation metric that aligns closely with human judgment on the LingoQA evaluation suits.
|
19 |
+
|
20 |
+
### How to use
|
21 |
+
```python
|
22 |
+
# Import necessary libraries
|
23 |
+
from transformers import pipeline
|
24 |
+
|
25 |
+
# Define the model name to be used in the pipeline
|
26 |
+
model_name = 'wayveai/Lingo-Judge'
|
27 |
+
|
28 |
+
# Define the question and its corresponding answer and prediction
|
29 |
+
question = "Are there any pedestrians crossing the road? If yes, how many?"
|
30 |
+
answer = "1"
|
31 |
+
prediction = "Yes, there is one"
|
32 |
+
|
33 |
+
# Initialize the pipeline with the specified model, device, and other parameters
|
34 |
+
pipe = pipeline("text-classification", model=model_name)
|
35 |
+
# Format the input string with the question, answer, and prediction
|
36 |
+
input = f"[CLS]\nQuestion: {question}\nAnswer: {answer}\nStudent: {prediction}"
|
37 |
+
|
38 |
+
# Pass the input through the pipeline to get the result
|
39 |
+
result = pipe(input)
|
40 |
+
|
41 |
+
# Print the result and score
|
42 |
+
score = result[0]['score']
|
43 |
+
print(score > 0.5, score)
|
44 |
+
|