miguelcarv commited on
Commit
6100b6f
•
1 Parent(s): 019db21

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -13
README.md CHANGED
@@ -1,8 +1,3 @@
1
- ---
2
- language:
3
- - en
4
- pipeline_tag: image-text-to-text
5
- ---
6
  # Φ Pheye - a family of efficient small vision-language models
7
 
8
  - These models train a fraction of the number of parameters other models of similar sizes train
@@ -18,20 +13,22 @@ pipeline_tag: image-text-to-text
18
  | MoE-LLaVA-2.7B×4 | 336 | 5.3B | 5.9M | 77.1 | - | 50.2 | - |
19
  | moondream1 | 384 | 1.86B | 3.9M | 74.7 | - | 35.6 |
20
  | moondream2 | 384 | 1.86B | - | 77.7 | 92.5 | 49.7 | 120.2 |
21
- | [Pheye-x4 🤗](https://huggingface.co/miguelcarv/Pheye-x4-448) | 448 | 295M | 2.9M | 75.2 | 110.3 | 45.9 | 106.4 |
22
  | [Pheye-x4 🤗](https://huggingface.co/miguelcarv/Pheye-x4-672) | 672 | 295M | 2.9M | 75.5 | 110.8 | 49.2 | 111.9 |
23
  | [Pheye-x2 🤗](https://huggingface.co/miguelcarv/Pheye-x2-448) | 448 | 578M | 2.9M | 76.0 | 111.8 | 47.3 | 108.9 |
24
  | [Pheye-x2 🤗](https://huggingface.co/miguelcarv/Pheye-x2-672) | 672 | 578M | 2.9M | 76.4 | 110.5 | 50.5 | 115.9 |
25
 
26
- ## Examples
27
 
28
- | Image | Example |
29
- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
30
- | <img src="https://c5.staticflickr.com/6/5463/17191308944_ae0b20bb7e_o.jpg" width="500"/> | **How much do these popcorn packets weight?**<br>These popcorn packets weigh 90g each.<br><br>**What brand are these popcorn packets?**<br>These popcorn packets are from the brand "Magic Pop". |
31
- | <img src="https://farm2.staticflickr.com/2708/5836100440_6e1117d36f_o.jpg" width="500"/> | **Can I pet that dog?**<br>No, you cannot pet the dog in the image. The dog is a black bear, which is a wild animal and should not be approached or interacted with by humans. It is important to maintain a safe distance from wild animals to avoid any potential harm to both the animal and the person. |
32
- | <img src="https://farm4.staticflickr.com/3898/15152219465_4fb6d266ff_o.jpg" width="500"/> | **Why is the person wearing protective gear?**<br>The person in the image is wearing protective gear, such as a beekeeper's suit and a helmet, because they are engaged in beekeeping. Beekeeping involves handling bees and their hives, which can be potentially dangerous due to the risk of bee stings. The protective gear helps to minimize the risk of bee stings and ensures the beekeeper's safety while working with bees. |
33
  | |
34
 
 
 
35
  ## Usage
36
 
37
  To generate a sample response from a prompt use `generate.py`.
@@ -39,6 +36,7 @@ Use a Python version >= 3.11. Start by cloning the repo and create a virtual env
39
 
40
  ```bash
41
  git clone https://github.com/miguelscarv/pheye.git
 
42
  python3 -m venv venv
43
  source venv/bin/activate
44
  pip3 install -r requirements.txt
@@ -52,4 +50,4 @@ python3 generate.py --image_path images/dog_flower.jpg --prompt "What is the dog
52
 
53
  ## Acknowledgments
54
 
55
- This implementation was inspired by [OpenFlamingo](https://github.com/mlfoundations/open_flamingo)'s repository.
 
 
 
 
 
 
1
  # Φ Pheye - a family of efficient small vision-language models
2
 
3
  - These models train a fraction of the number of parameters other models of similar sizes train
 
13
  | MoE-LLaVA-2.7B×4 | 336 | 5.3B | 5.9M | 77.1 | - | 50.2 | - |
14
  | moondream1 | 384 | 1.86B | 3.9M | 74.7 | - | 35.6 |
15
  | moondream2 | 384 | 1.86B | - | 77.7 | 92.5 | 49.7 | 120.2 |
16
+ | [Pheye-x4 🤗](https://huggingface.co/miguelcarv/Pheye-x4-448) | 448 | 295M | 2.9M | 75.2 | 110.1 | 45.9 | 106.4 |
17
  | [Pheye-x4 🤗](https://huggingface.co/miguelcarv/Pheye-x4-672) | 672 | 295M | 2.9M | 75.5 | 110.8 | 49.2 | 111.9 |
18
  | [Pheye-x2 🤗](https://huggingface.co/miguelcarv/Pheye-x2-448) | 448 | 578M | 2.9M | 76.0 | 111.8 | 47.3 | 108.9 |
19
  | [Pheye-x2 🤗](https://huggingface.co/miguelcarv/Pheye-x2-672) | 672 | 578M | 2.9M | 76.4 | 110.5 | 50.5 | 115.9 |
20
 
21
+ ## Examples\*
22
 
23
+ | Image | Example |
24
+ | ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
25
+ | <img src="https://c5.staticflickr.com/6/5463/17191308944_ae0b20bb7e_o.jpg" width="500"/> | **How much do these popcorn packets weigh?**<br>These popcorn packets weigh 90g each.<br><br>**What brand are these popcorn packets?**<br>These popcorn packets are from the brand "Magic Pop". |
26
+ | <img src="https://farm2.staticflickr.com/2708/5836100440_6e1117d36f_o.jpg" width="500"/> | **Can I pet that dog?**<br>No, you cannot pet the dog in the image. It is a black bear, not a dog. |
27
+ | <img src="images/beekepper.jpeg" width="500"/> | **Why is the person wearing protective gear?**<br>The person in the image is wearing protective gear, such as a beekeeper's suit and a helmet, to ensure their safety while working with bees. Beekeeping involves handling bees and their hives, which can be potentially dangerous due to the risk of bee stings. The protective gear helps to minimize the risk of bee stings and other bee-related injuries, allowing the beekeeper to carry out their work safely and effectively. |
28
  | |
29
 
30
+ \* Generated by Pheye-x2-672
31
+
32
  ## Usage
33
 
34
  To generate a sample response from a prompt use `generate.py`.
 
36
 
37
  ```bash
38
  git clone https://github.com/miguelscarv/pheye.git
39
+ cd pheye
40
  python3 -m venv venv
41
  source venv/bin/activate
42
  pip3 install -r requirements.txt
 
50
 
51
  ## Acknowledgments
52
 
53
+ This implementation was inspired by [OpenFlamingo](https://github.com/mlfoundations/open_flamingo)'s repository.