Spaces:
Sleeping
Sleeping
Create readme.txt
Browse files- my_model/results/readme.txt +32 -0
my_model/results/readme.txt
ADDED
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
|
2 |
+
Directory Overview: This directory contains all the evaluation modules and results:
|
3 |
+
|
4 |
+
################################################################################################################################
|
5 |
+
|
6 |
+
## 1. evaluation.py
|
7 |
+
The `evaluation.py` module provides a comprehensive framework for evaluating Knowledge-Based Visual Question
|
8 |
+
Answering (KB-VQA) models. This module includes methods for syntactic and semantic evaluation, leveraging both
|
9 |
+
exact match and VQA scores. The evaluation results can be saved to an Excel file for further analysis.
|
10 |
+
|
11 |
+
Notes
|
12 |
+
Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config" file.
|
13 |
+
The module requires the following dependencies: pandas, fuzzywuzzy, nltk, ast, typing, streamlit, and openai.
|
14 |
+
|
15 |
+
################################################################################################################################
|
16 |
+
|
17 |
+
## 2. demo.py
|
18 |
+
The `demo.py` module includes the `ResultDemonstrator` class, which provides a comprehensive framework for visualizing
|
19 |
+
and demonstrating the results of Knowledge-Based Visual Question Answering (KB-VQA) models.
|
20 |
+
The class utilizes Streamlit for displaying data and Altair for creating interactive visualizations.
|
21 |
+
this class is the backbone for the results demo tool of the application.
|
22 |
+
|
23 |
+
Notes
|
24 |
+
Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config".
|
25 |
+
The module requires the following dependencies: os, altair, streamlit, PIL, pandas, random.
|
26 |
+
|
27 |
+
################################################################################################################################
|
28 |
+
|
29 |
+
## 3. evaluation_results.xlsx
|
30 |
+
This file contains all the evaluation results based on the evaluation data (~ 1,000 samples), the results are in the sheet names "Main Data".
|
31 |
+
running the evaluation module will read the main columns (Model Answers for all model's configurations) from this sheet and regenerate and save the evaluation rsults.
|
32 |
+
This Main Data sheet is the result of running the evaluation module (with the same file name chosen for save).
|