File size: 1,059 Bytes
1247a15
 
02162cc
 
 
 
 
 
 
 
 
 
 
 
 
 
1247a15
 
 
 
 
 
 
402ed7d
1247a15
d159d67
 
 
402ed7d
 
 
b2122db
402ed7d
 
 
 
d159d67
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
---
license: apache-2.0
datasets:
- HuggingFaceFW/fineweb
- PleIAs/YouTube-Commons
- allenai/WildChat-1M
language:
- de
- en
- ja
- fr
library_name: mlx
tags:
- moe
- multimodal
- j.o.s.i.e.
---

# This will be the repo for J.O.S.I.E.v4o

Like **OpenAIs GPT-4o**, it's natively Multimodal, based on the **NExT-GPT** combined with **ROPE**, **RMS Normalisation**, and **MoE**, parred with the **GPT-4o Tokenizer** from OpenAI.
This is a *future project* and will take it's time.

Further more, I will probably make a **UI application** with that model too.

Further updates comming soon!!!


First architecture Overview:

First Beta will utilize the already pretrained ImageBind Model. The linear input Projection is because the outputs of the ImageBind model are not in the correct dimensions.
Later on the input and output projections will be removed.

<img src="Architecture_overview_beta3.png" width="100%" height="auto"/>


Source code and more info will be available on my <a href="https://github.com/Goekdeniz-Guelmez/J.O.S.I.E.v4-o.git">GitHub Repo</a>