m7mdal7aj commited on
Commit
8cb43ef
1 Parent(s): e5940c3

Update my_model/tabs/model_arch.py

Browse files
Files changed (1) hide show
  1. my_model/tabs/model_arch.py +15 -9
my_model/tabs/model_arch.py CHANGED
@@ -2,8 +2,19 @@ import streamlit as st
2
  import streamlit.components.v1 as components
3
 
4
 
5
- def run_model_arch():
 
 
 
6
 
 
 
 
 
 
 
 
 
7
  with open("Files/Model Arch.html", 'r', encoding='utf-8') as f:
8
  model_arch_html = f.read()
9
 
@@ -20,11 +31,6 @@ def run_model_arch():
20
  \nThe evaluation results underscore the developed model’s competent and competitive performance. It achieves a VQA score of 63.57% under syntactic evaluation and excels with an Exact Match (EM) score of 68.36%. Further, semantic evaluations yield even more impressive outcomes, with VQA and EM scores of 71.09% and 72.55%, respectively. These results demonstrate that the model effectively applies reasoning over the visual context and successfully retrieves the necessary knowledge to answer visual questions.""")
21
 
22
 
23
- st.header("Acknowledgement")
24
- st.write("""I am profoundly grateful for the support and guidance I have received throughout the course of my dissertation. I would like to extend my deepest appreciation to the following individuals:
25
- \nTo my supervisor, [Dr. Andreas Theophilou](https://researchportal.bath.ac.uk/en/persons/andreas-theophilou), whose expertise, and insightful guidance have been instrumental in the completion of this research. Your mentorship has not only profoundly shaped my work but also my future endeavours in the field of computer science.
26
- Special mention must be made of my mentors at the University of Bath— [Dr. Ben Ralph](https://researchportal.bath.ac.uk/en/persons/ben-ralph), [Dr. Hongping Cai](https://researchportal.bath.ac.uk/en/persons/hongping-cai), and [Dr. Nadejda Roubtsova](https://researchportal.bath.ac.uk/en/persons/nadejda-roubtsova). The wealth of knowledge and insights I have gained from you has been indispensable. Your unwavering dedication to academic excellence and steadfast support have been crucial in navigating my academic journey.
27
- \nMy colleagues deserve equal gratitude, for their camaraderie and collaborative spirit have not only made this journey feasible but also deeply enjoyable. The shared experiences and the challenges we have overcome together have been integral to my personal and professional growth.
28
- \nLastly, my heartfelt thanks are extended to my family, whose unyielding love and encouragement have been my steadfast anchor. Your belief in my abilities has consistently inspired me and bolstered my strength throughout this process.
29
- This dissertation is not merely a reflection of my individual efforts but stands as a testament to the collective support and wisdom of each individual mentioned above. I am honoured and privileged to be part of such a supportive and enriching academic community.
30
- """)
 
2
  import streamlit.components.v1 as components
3
 
4
 
5
+ def run_model_arch() -> None:
6
+ """
7
+ Displays the model architecture and accompanying abstract and design details for the Knowledge-Based Visual Question
8
+ Answering (KB-VQA) model.
9
 
10
+ This function reads an HTML file containing the model architecture and renders it in a Streamlit application.
11
+ It also provides detailed descriptions of the research, abstract, and design of the KB-VQA model.
12
+
13
+ Returns:
14
+ None
15
+ """
16
+
17
+ # Read the model architecture HTML file
18
  with open("Files/Model Arch.html", 'r', encoding='utf-8') as f:
19
  model_arch_html = f.read()
20
 
 
31
  \nThe evaluation results underscore the developed model’s competent and competitive performance. It achieves a VQA score of 63.57% under syntactic evaluation and excels with an Exact Match (EM) score of 68.36%. Further, semantic evaluations yield even more impressive outcomes, with VQA and EM scores of 71.09% and 72.55%, respectively. These results demonstrate that the model effectively applies reasoning over the visual context and successfully retrieves the necessary knowledge to answer visual questions.""")
32
 
33
 
34
+ st.header("Design")
35
+ st.write("""As illustrated in architecture, the model operates through a sequential pipeline, beginning with the Image to Language Transformation Module, in this module, the image undergoes simultaneous processing via image captioning and object detection frozen models, aiming to comprehensively capture the visual context and cues. These models, selected for their initial effectiveness, are designed to be pluggable, allowing for easy replacement with more advanced models as new technologies develop, thus ensuring the module remains at the forefront of technological advancement. Following this, the Prompt Engineering Module processes the generated captions and the list of detected objects, along with their bounding boxes and confidence levels, merging these elements with the question at hand utilizing a meticulously crafted prompting template. The pipeline ends with a Fine-tuned Pre-Trained Large Language Model (PT-LLMs), which is responsible for performing reasoning and deriving the required knowledge to formulate an informed response to the question.
36
+ """)