ChihHsuan-Yang commited on
Commit
595ba8c
1 Parent(s): 824c735

updated readme

Browse files
Files changed (1) hide show
  1. README.md +12 -7
README.md CHANGED
@@ -6,6 +6,8 @@ tags:
6
  - zero-shot-image-classification
7
  - clip
8
  - biology
 
 
9
  - CV
10
  - images
11
  - animals
@@ -17,6 +19,7 @@ tags:
17
  - multimodal
18
  - knowledge-guided
19
  datasets:
 
20
  - imageomics/TreeOfLife-10M
21
  - iNat21
22
  - BIOSCAN-1M
@@ -46,7 +49,7 @@ ARBORCLIP is a new suite of vision-language foundation models for biodiversity.
46
  - **License:** MIT
47
  - **Fine-tuned from model:** [OpenAI CLIP](https://github.com/mlfoundations/open_clip), [MetaCLIP](https://github.com/facebookresearch/MetaCLIP), [BioCLIP](https://github.com/Imageomics/BioCLIP)
48
 
49
- These models were developed for the benefit of the AI community as an open-source product, thus we request that any derivative products are also open-source.
50
 
51
 
52
  ### Model Description
@@ -54,12 +57,15 @@ These models were developed for the benefit of the AI community as an open-sourc
54
  ArborCLIP is based on OpenAI's [CLIP](https://openai.com/research/clip) model.
55
  The models were trained on [ARBORETUM-40M](https://baskargroup.github.io/Arboretum/) for the following configurations:
56
 
57
- - **ARBORCLIP-O:** Trained a ViT-B/16 backbone initialized from the [OpenCLIP's](https://github.com/mlfoundations/open_clip) checkpoint. Training was conducted for 40 epochs.
58
- - **ARBORCLIP-B:** Trained a ViT-B/16 backbone initialized from the [BioCLIP's](https://github.com/Imageomics/BioCLIP) checkpoint. Training was conducted for 8 epochs.
59
- - **ARBORCLIP-M:** Trained a ViT-L/14 backbone initialized from the [MetaCLIP's](https://github.com/facebookresearch/MetaCLIP) checkpoint. Training was conducted for 12 epochs.
60
 
61
 
62
- To access the checkpoints of the above models, go to the `Files and versions` tab and download the weights. These weights can be directly used for zero-shot classification and finetuning. The filenames correspond to the specific model weights - `arborclip-vit-b-16-from-openai-epoch-40.pt` (**ARBORCLIP-O**), `arborclip-vit-b-16-from-bioclip-epoch-8.pt` (**ARBORCLIP-B**) and`arborclip-vit-l-14-from-metaclip-epoch-12.pt` (**ARBORCLIP-M**).
 
 
 
63
 
64
  ### Model Training
65
  **See the [Model Training](https://github.com/baskargroup/Arboretum?tab=readme-ov-file#model-training) section on the [Github](https://github.com/baskargroup/Arboretum) for examples of how to use ArborCLIP models in zero-shot image classification tasks.**
@@ -128,8 +134,7 @@ Additionally, models trained on web-scraped data excel at classifying at the hig
128
  level (kingdom), while models begin to benefit from specialist datasets like [ARBORETUM-40M](https://baskargroup.github.io/Arboretum/) and
129
  [Tree-of-Life-10M](https://huggingface.co/datasets/imageomics/TreeOfLife-10M) at the lower taxonomic levels (order and species). From a practical standpoint, `ArborCLIP` is highly accurate at the species level, and higher-level taxa can be deterministically derived from lower ones.
130
 
131
- Addressing these limitations will further enhance the applicability of models like `ArborCLIP` in
132
- real-world biodiversity monitoring tasks.
133
 
134
  ### Acknowledgements
135
  This work was supported by the AI Research Institutes program supported by the NSF and USDA-NIFA under [AI Institute: for Resilient Agriculture](https://aiira.iastate.edu/), Award No. 2021-67021-35329. This was also
 
6
  - zero-shot-image-classification
7
  - clip
8
  - biology
9
+ - biodiversity
10
+ - agronomy
11
  - CV
12
  - images
13
  - animals
 
19
  - multimodal
20
  - knowledge-guided
21
  datasets:
22
+ - Arboretum
23
  - imageomics/TreeOfLife-10M
24
  - iNat21
25
  - BIOSCAN-1M
 
49
  - **License:** MIT
50
  - **Fine-tuned from model:** [OpenAI CLIP](https://github.com/mlfoundations/open_clip), [MetaCLIP](https://github.com/facebookresearch/MetaCLIP), [BioCLIP](https://github.com/Imageomics/BioCLIP)
51
 
52
+ These models were developed for the benefit of the AI community as an open-source product. Thus, we request that any derivative products are also open-source.
53
 
54
 
55
  ### Model Description
 
57
  ArborCLIP is based on OpenAI's [CLIP](https://openai.com/research/clip) model.
58
  The models were trained on [ARBORETUM-40M](https://baskargroup.github.io/Arboretum/) for the following configurations:
59
 
60
+ - **ARBORCLIP-O:** Trained a ViT-B/16 backbone initialized from the [OpenCLIP's](https://github.com/mlfoundations/open_clip) checkpoint. The training was conducted for 40 epochs.
61
+ - **ARBORCLIP-B:** Trained a ViT-B/16 backbone initialized from the [BioCLIP's](https://github.com/Imageomics/BioCLIP) checkpoint. The training was conducted for 8 epochs.
62
+ - **ARBORCLIP-M:** Trained a ViT-L/14 backbone initialized from the [MetaCLIP's](https://github.com/facebookresearch/MetaCLIP) checkpoint. The training was conducted for 12 epochs.
63
 
64
 
65
+ To access the checkpoints of the above models, go to the `Files and versions` tab and download the weights. These weights can be directly used for zero-shot classification and finetuning. The filenames correspond to the specific model weights -
66
+ - **ARBORCLIP-O:** - `arborclip-vit-b-16-from-openai-epoch-40.pt`,
67
+ - **ARBORCLIP-B:** - `arborclip-vit-b-16-from-bioclip-epoch-8.pt`
68
+ - **ARBORCLIP-M** - `arborclip-vit-l-14-from-metaclip-epoch-12.pt`
69
 
70
  ### Model Training
71
  **See the [Model Training](https://github.com/baskargroup/Arboretum?tab=readme-ov-file#model-training) section on the [Github](https://github.com/baskargroup/Arboretum) for examples of how to use ArborCLIP models in zero-shot image classification tasks.**
 
134
  level (kingdom), while models begin to benefit from specialist datasets like [ARBORETUM-40M](https://baskargroup.github.io/Arboretum/) and
135
  [Tree-of-Life-10M](https://huggingface.co/datasets/imageomics/TreeOfLife-10M) at the lower taxonomic levels (order and species). From a practical standpoint, `ArborCLIP` is highly accurate at the species level, and higher-level taxa can be deterministically derived from lower ones.
136
 
137
+ Addressing these limitations will further enhance the applicability of models like `ArborCLIP` in real-world biodiversity monitoring tasks.
 
138
 
139
  ### Acknowledgements
140
  This work was supported by the AI Research Institutes program supported by the NSF and USDA-NIFA under [AI Institute: for Resilient Agriculture](https://aiira.iastate.edu/), Award No. 2021-67021-35329. This was also