xichen98cn commited on
Commit
3dac99f
1 Parent(s): 70fea71

Upload 135 files

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. GETTING_STARTED.md +67 -0
  2. INSTALL.md +24 -0
  3. README.md +133 -13
  4. app.py +202 -0
  5. configs/coco/Base-COCO-PanopticSegmentation.yaml +47 -0
  6. configs/coco/frozenseg/convnext_large_eval_a847.yaml +10 -0
  7. configs/coco/frozenseg/convnext_large_eval_ade20k.yaml +29 -0
  8. configs/coco/frozenseg/convnext_large_eval_bdd_panop.yaml +14 -0
  9. configs/coco/frozenseg/convnext_large_eval_bdd_sem.yaml +13 -0
  10. configs/coco/frozenseg/convnext_large_eval_cityscapes.yaml +8 -0
  11. configs/coco/frozenseg/convnext_large_eval_coco.yaml +3 -0
  12. configs/coco/frozenseg/convnext_large_eval_lvis.yaml +11 -0
  13. configs/coco/frozenseg/convnext_large_eval_mapillary_vistas.yaml +12 -0
  14. configs/coco/frozenseg/convnext_large_eval_pas21.yaml +10 -0
  15. configs/coco/frozenseg/convnext_large_eval_pc459.yaml +10 -0
  16. configs/coco/frozenseg/r50x64_eval_ade20k.yaml +13 -0
  17. configs/coco/maskformer2_R50_bs16_50ep.yaml +45 -0
  18. datasets/README.md +262 -0
  19. datasets/ade20k_instance_catid_mapping.txt +104 -0
  20. datasets/ade20k_instance_imgCatIds.json +0 -0
  21. datasets/prepare_ade20k_full_sem_seg.py +1004 -0
  22. datasets/prepare_ade20k_ins_seg.py +111 -0
  23. datasets/prepare_ade20k_pan_seg.py +499 -0
  24. datasets/prepare_ade20k_sem_seg.py +26 -0
  25. datasets/prepare_coco_semantic_annos_from_panoptic_annos.py +82 -0
  26. datasets/prepare_pascal_ctx_full_sem_seg.py +38 -0
  27. datasets/prepare_pascal_ctx_sem_seg.py +74 -0
  28. datasets/prepare_pascal_voc_sem_seg.py +55 -0
  29. demo/demo.py +189 -0
  30. demo/predictor.py +273 -0
  31. eval.sh +70 -0
  32. frozenseg/.DS_Store +0 -0
  33. frozenseg/__init__.py +26 -0
  34. frozenseg/config.py +132 -0
  35. frozenseg/data/.DS_Store +0 -0
  36. frozenseg/data/__init__.py +1 -0
  37. frozenseg/data/dataset_mappers/__init__.py +0 -0
  38. frozenseg/data/dataset_mappers/bdd_semseg_dataset_mapper.py +107 -0
  39. frozenseg/data/dataset_mappers/coco_instance_new_baseline_dataset_mapper.py +187 -0
  40. frozenseg/data/dataset_mappers/coco_panoptic_new_baseline_dataset_mapper.py +163 -0
  41. frozenseg/data/dataset_mappers/mask_former_instance_dataset_mapper.py +179 -0
  42. frozenseg/data/dataset_mappers/mask_former_panoptic_dataset_mapper.py +164 -0
  43. frozenseg/data/dataset_mappers/mask_former_semantic_dataset_mapper.py +183 -0
  44. frozenseg/data/datasets/__init__.py +18 -0
  45. frozenseg/data/datasets/ade20k_150_with_prompt_eng.txt +151 -0
  46. frozenseg/data/datasets/ade20k_847_with_prompt_eng.txt +848 -0
  47. frozenseg/data/datasets/cityscapes_with_prompt_eng.txt +19 -0
  48. frozenseg/data/datasets/coco_panoptic_with_prompt_eng.txt +201 -0
  49. frozenseg/data/datasets/coco_stuff_with_prompt_eng.txt +183 -0
  50. frozenseg/data/datasets/lvis_1203_with_prompt_eng.txt +1203 -0
GETTING_STARTED.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Getting Started with FrozenSeg
2
+
3
+ This document provides a brief intro of the usage of FrozenSeg.
4
+
5
+ Please see [Getting Started with Detectron2](https://github.com/facebookresearch/detectron2/blob/master/GETTING_STARTED.md) for full usage.
6
+
7
+
8
+ ### Inference Demo with Pre-trained Models
9
+
10
+ We provide `demo.py` that is able to demo builtin configs. Run it with:
11
+ ```
12
+ python demo.py \
13
+ --input input1.jpg input2.jpg \
14
+ [--other-options]
15
+ --opts MODEL.WEIGHTS /path/to/checkpoint_file
16
+ ```
17
+ The configs are made for training, therefore we need to specify `MODEL.WEIGHTS` to a model from model zoo for evaluation.
18
+ This command will run the inference and show visualizations in an OpenCV window.
19
+
20
+ For details of the command line arguments, see `demo.py -h` or look at its source code
21
+ to understand its behavior. Some common arguments are:
22
+ * To run on your __webcam__, replace --input files with --webcam
23
+ * To run on a __video__, replace --input files with --video-input video.mp4.
24
+ * To run on __cpu__, add `MODEL.DEVICE cpu` after `--opts`.
25
+ * To save outputs to a directory (for images) or a file (for webcam or video), use `--output`.
26
+
27
+
28
+ ### Training & Evaluation in Command Line
29
+
30
+ We provide a script `train_net.py`, that is made to train all the configs provided in FrozenSeg.
31
+
32
+ To train a model with "train_net.py", first setup the corresponding datasets following [datasets/README.md](./datasets/README.md), download [SAM checkpoints](https://github.com/facebookresearch/segment-anything?tab=readme-ov-file#model-checkpoints) and save it under `pretrained_checkpoint/`.
33
+ then run:
34
+ ```
35
+ python train_net.py --num-gpus 4\
36
+ --config-file configs/coco/frozenseg/convnext_large_eval_ade20k.yaml
37
+ ```
38
+
39
+ The configs are made for 4-GPU training.
40
+ Since we use ADAMW optimizer, it is not clear how to scale learning rate with batch size.
41
+ To train on 1 GPU, you need to figure out learning rate and batch size by yourself:
42
+ ```
43
+ python train_net.py \
44
+ --config-file configs/coco/frozenseg/convnext_large_eval_ade20k.yaml \
45
+ --num-gpus 1 SOLVER.IMS_PER_BATCH SET_TO_SOME_REASONABLE_VALUE SOLVER.BASE_LR SET_TO_SOME_REASONABLE_VALUE
46
+ ```
47
+
48
+ To evaluate a model's performance without `OpenSeg Ensemble`:
49
+ ```
50
+ python train_net.py \
51
+ --config-file configs/coco/frozenseg/convnext_large_eval_ade20k.yaml \
52
+ --eval-only MODEL.WEIGHTS /path/to/checkpoint_file \
53
+ TEST.USE_SAM_MASKS False
54
+ ```
55
+
56
+ For using `OpenSeg Ensemble`:
57
+ 1. generate SAM mask predictions (default saveing under `output/SAM_masks_pred`):
58
+ ```
59
+ python save_sam_masks.py --data_name pc_val --sam_model vit_h
60
+ ```
61
+ 2. run with:
62
+ ```
63
+ python train_net.py \
64
+ --config-file configs/coco/frozenseg/convnext_large_eval_ade20k.yaml \
65
+ --eval-only MODEL.WEIGHTS /path/to/checkpoint_file \
66
+ TEST.USE_SAM_MASKS True
67
+ ```
INSTALL.md ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Installation
2
+ The codebases are built on top of [Detectron2](https://detectron2.readthedocs.io/tutorials/install.html).
3
+
4
+
5
+ ### Dependencies and Installation
6
+ ```bash
7
+ conda create --name frozenseg python=3.10 -y
8
+ conda activate frozenseg
9
+ conda install pytorch==2.3.1 torchvision==0.18.1 -c pytorch -c nvidia
10
+
11
+ # under your working directory
12
+ python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
13
+ pip install git+https://github.com/cocodataset/panopticapi.git
14
+ pip install git+https://github.com/mcordts/cityscapesScripts.git
15
+
16
+ git clone https://github.com/chenxi52/FrozenSeg.git
17
+ cd FrozenSeg
18
+ pip install -r requirements.txt
19
+
20
+ # compile CUDA kernel for MSDeformAttn
21
+ cd frozenseg/modeling/pixel_decoder/ops
22
+ sh make.sh
23
+ cd ../../../..
24
+ ```
README.md CHANGED
@@ -1,13 +1,133 @@
1
- ---
2
- title: FrozenSeg
3
- emoji: 🚀
4
- colorFrom: green
5
- colorTo: red
6
- sdk: gradio
7
- sdk_version: 4.43.0
8
- app_file: app.py
9
- pinned: false
10
- license: apache-2.0
11
- ---
12
-
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # FrozenSeg: Harmonizing Frozen Foundation Models for Open-Vocabulary Segmentation
2
+
3
+ This repository is the official implementation of FrozenSeg introduced in the paper:
4
+ >[**FrozenSeg: Harmonizing Frozen Foundation Models for Open-Vocabulary Segmentation**](https://arxiv.org/abs/2409.03525)
5
+
6
+
7
+ ## Abstract
8
+
9
+ >Open-vocabulary segmentation is challenging, with the need of segmenting and recognizing objects for an open set of categories in unconstrained environments. Building on the success of powerful vision-language (ViL) foundation models like CLIP, recent efforts sought to harness their zero-short capabilities to recognize unseen categories. Despite demonstrating strong performances, they still face a fundamental challenge of generating precise mask proposals for unseen categories and scenarios, resulting in inferior segmentation performance eventually. To address this, we introduce a novel approach, FrozenSeg, designed to integrate spatial knowledge from a localization foundation model (e.g., SAM) and semantic knowledge extracted from a ViL model (e.g., CLIP), in a synergistic framework. Taking the ViL model's visual encoder as the feature backbone, we inject the space-aware feature into learnable query and CLIP feature in the transformer decoder. In addition, we devise a mask proposal ensemble strategy for further improving the recall rate and mask quality. To fully exploit pre-trained knowledge while minimizing training overhead, we freeze both foundation models, focusing optimization efforts solely on a light transformer decoder for mask proposal generation – the performance bottleneck. Extensive experiments show that FrozenSeg advances state-of-the-art results across various segmentation benchmarks, trained exclusively on COCO panoptic data and tested in a zero-shot manner.
10
+
11
+ ![FrozenSeg design](images/frozenseg.png)
12
+
13
+ ## Dependencies and Installation
14
+ See [installation instructions](INSTALL.md).
15
+
16
+ ## Getting Started
17
+ See [Preparing Datasets](datasets/README.md).
18
+
19
+ See [Getting Started](GETTING_STARTED.md).
20
+
21
+
22
+ ## Models
23
+ <table>
24
+ <thead>
25
+ <tr>
26
+ <th align="center"></th>
27
+ <th align="center" style="text-align:center" colspan="4"><a href="logs/testing/ade20k.log">ADE20K(A-150)</th>
28
+ <th align="center" style="text-align:center" colspan="3"><a href="logs/testing/cityscapes.log">Cityscapes</th>
29
+ <th align="center" style="text-align:center" colspan="2"><a href="logs/testing/mapillary_vistas.log">Mapillary Vistas</th>
30
+ <th align="center" style="text-align:center" colspan="2"><a href="logs/testing/bdd100k.log">BDD 100K</th>
31
+ <th align="center" style="text-align:center" colspan="2"><a href="logs/testing/a-847.log"> A-847 </th>
32
+ <th align="center" style="text-align:center" colspan="2"><a href="logs/testing/pc-459.log"> PC-459 </th>
33
+ <th align="center" style="text-align:center" colspan="2"><a href="logs/testing/pas-21.log">PAS-21 </th>
34
+ <th align="center" style="text-align:center" ><a href="logs/testing/lvis.log">Lvis </th>
35
+ <th align="center" style="text-align:center" colspan="3"><a href="logs/testing/coco.log">COCO <br> (training dataset)</th>
36
+ <th align="center" style="text-align:center">download </th>
37
+ </tr>
38
+ </thead>
39
+ <tbody>
40
+ <tr>
41
+ <td align="center"></td>
42
+ <td align="center">PQ</td>
43
+ <td align="center">mAP</td>
44
+ <td align="center">mIoU</td>
45
+ <td align="center">FWIoU</td>
46
+ <td align="center">PQ</td>
47
+ <td align="center">mAP</td>
48
+ <td align="center">mIoU</td>
49
+ <td align="center">PQ</td>
50
+ <td align="center">mIoU</td>
51
+ <td align="center">PQ</td>
52
+ <td align="center">mIoU</td>
53
+ <td align="center">mIoU</td>
54
+ <td align="center">FWIoU</td>
55
+ <td align="center">mIoU</td>
56
+ <td align="center">FWIoU</td>
57
+ <td align="center">mIoU</td>
58
+ <td align="center">FWIoU</td>
59
+ <td align="center">APr</td>
60
+ <td align="center">PQ</td>
61
+ <td align="center">mAP</td>
62
+ <td align="center">mIoU</td>
63
+ <td></td>
64
+ </tr>
65
+ <td align="center"><a href="configs/coco/frozenseg/r50x64_eval_ade20k.yaml"> FrozenSeg (ResNet50x64) </a></td>
66
+ <td align="center">23.1</td>
67
+ <td align="center">13.5</td>
68
+ <td align="center">30.7</td>
69
+ <td align="center">56.6</td>
70
+ <td align="center">45.2</td>
71
+ <td align="center">28.9</td>
72
+ <td align="center">56.0</td>
73
+ <td align="center">18.1</td>
74
+ <td align="center">27.7</td>
75
+ <td align="center">12.9</td>
76
+ <td align="center">46.2</td>
77
+ <td align="center">11.8</td>
78
+ <td align="center">52.8</td>
79
+ <td align="center">18.7</td>
80
+ <td align="center">60.1</td>
81
+ <td align="center">82.3</td>
82
+ <td align="center">92.1</td>
83
+ <td align="center">23.5</td>
84
+ <td align="center">55.7</td>
85
+ <td align="center">47.4</td>
86
+ <td align="center">65.4</td>
87
+ <td align="center"><a href=""> checkpoint </a></td>
88
+ </tr>
89
+ <tr>
90
+ <td align="center"><a href="configs/coco/frozenseg/convnext_large_eval_ade20k.yaml"> FrozenSeg (ConvNeXt-Large) </a></td>
91
+ <td align="center">25.9</td>
92
+ <td align="center">16.4</td>
93
+ <td align="center">34.4</td>
94
+ <td align="center">59.9</td>
95
+ <td align="center">45.8</td>
96
+ <td align="center">28.4</td>
97
+ <td align="center">56.8</td>
98
+ <td align="center">18.5</td>
99
+ <td align="center">27.3</td>
100
+ <td align="center">19.3</td>
101
+ <td align="center">52.3</td>
102
+ <td align="center">14.8</td>
103
+ <td align="center">51.4</td>
104
+ <td align="center">19.7</td>
105
+ <td align="center">60.2</td>
106
+ <td align="center">82.5</td>
107
+ <td align="center">92.1</td>
108
+ <td align="center">25.6</td>
109
+ <td align="center">56.2</td>
110
+ <td align="center">47.3</td>
111
+ <td align="center">65.5</td>
112
+ <td align="center"><a href="https://drive.google.com/file/d/1ThjVgY7nawm1AAP1LhrmGVlI3zr1EYMG/view?usp=drive_link"> checkpoint </a></td>
113
+ </tr>
114
+ </tbody>
115
+ </table>
116
+
117
+
118
+
119
+ ## Citing
120
+
121
+ If you use FrozenSeg in your research, please use the following BibTeX entry.
122
+
123
+ ```BibTeX
124
+ @misc{FrozenSeg,
125
+ title={FrozenSeg: Harmonizing Frozen Foundation Models for Open-Vocabulary Segmentation},
126
+ author={Xi Chen and Haosen Yang and Sheng Jin and Xiatian Zhu and Hongxun Yao},
127
+ publisher={arXiv:5835590},
128
+ year={2024}
129
+ }
130
+ ```
131
+
132
+ ## Acknowledgement
133
+ [Detectron2](https://github.com/facebookresearch/detectron2), [Mask2Former](https://github.com/facebookresearch/Mask2Former) and [OpenCLIP](https://github.com/mlfoundations/open_clip)
app.py ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import sys
3
+
4
+ # os.system("pip install gdown")
5
+ # os.system("pip install imutils")
6
+ # os.system("pip install gradio_client==0.2.7")
7
+ # os.system("python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'")
8
+ # os.system("pip install git+https://github.com/cocodataset/panopticapi.git")
9
+ # os.system("python frozenseg/modeling/pixel_decoder/ops/setup.py build install")
10
+
11
+ import gradio as gr
12
+ from detectron2.utils.logger import setup_logger
13
+ from contextlib import ExitStack
14
+ import numpy as np
15
+ import cv2
16
+ import torch
17
+ import itertools
18
+ from detectron2.config import get_cfg
19
+ from detectron2.utils.visualizer import ColorMode, random_color
20
+ from detectron2.data import MetadataCatalog
21
+
22
+ from frozenseg import add_maskformer2_config, add_frozenseg_config
23
+ from demo.predictor import DefaultPredictor, OpenVocabVisualizer
24
+ from PIL import Image
25
+ import json
26
+
27
+ setup_logger()
28
+ logger = setup_logger(name="frozenseg")
29
+ cfg = get_cfg()
30
+ cfg.MODEL.DEVICE='cuda'
31
+ add_maskformer2_config(cfg)
32
+ add_frozenseg_config(cfg)
33
+ cfg.merge_from_file("configs/coco/frozenseg/convnext_large_eval_ade20k.yaml")
34
+ # os.system("gdown 1-91PIns86vyNaL3CzMmDD39zKGnPMtvj")
35
+ cfg.MODEL.WEIGHTS = './modified_model.pth'
36
+ cfg.MODEL.MASK_FORMER.TEST.SEMANTIC_ON = False
37
+ cfg.MODEL.MASK_FORMER.TEST.INSTANCE_ON = False
38
+ cfg.MODEL.MASK_FORMER.TEST.PANOPTIC_ON = True
39
+ predictor = DefaultPredictor(cfg)
40
+
41
+
42
+ title = "FrozenSeg"
43
+ article = "<p style='text-align: center'><a href='' target='_blank'>FrozenSeg</a> | <a href='' target='_blank'>Github Repo</a></p>"
44
+
45
+ examples = [
46
+ [
47
+ "demo/examples/ADE_val_00000001.jpg",
48
+ "",
49
+ ["ADE (150 categories)"],
50
+ ],
51
+ [
52
+ "demo/examples/frankfurt_000000_005898_leftImg8bit.png",
53
+ "",
54
+ ["Cityscapes (19 categories)"],
55
+ ]
56
+ ]
57
+
58
+
59
+ coco_metadata = MetadataCatalog.get("openvocab_coco_2017_val_panoptic_with_sem_seg")
60
+ ade20k_metadata = MetadataCatalog.get("openvocab_ade20k_panoptic_val")
61
+ cityscapes_metadata = MetadataCatalog.get("openvocab_cityscapes_fine_panoptic_val")
62
+ lvis_classes = open("./frozenseg/data/datasets/lvis_1203_with_prompt_eng.txt", 'r').read().splitlines()
63
+ lvis_classes = [x[x.find(':')+1:] for x in lvis_classes]
64
+ lvis_colors = list(
65
+ itertools.islice(itertools.cycle(coco_metadata.stuff_colors), len(lvis_classes))
66
+ )
67
+ # rerrange to thing_classes, stuff_classes
68
+ coco_thing_classes = coco_metadata.thing_classes
69
+ coco_stuff_classes = [x for x in coco_metadata.stuff_classes if x not in coco_thing_classes]
70
+ coco_thing_colors = coco_metadata.thing_colors
71
+ coco_stuff_colors = [x for x in coco_metadata.stuff_colors if x not in coco_thing_colors]
72
+ ade20k_thing_classes = ade20k_metadata.thing_classes
73
+ ade20k_stuff_classes = [x for x in ade20k_metadata.stuff_classes if x not in ade20k_thing_classes]
74
+ ade20k_thing_colors = ade20k_metadata.thing_colors
75
+ ade20k_stuff_colors = [x for x in ade20k_metadata.stuff_colors if x not in ade20k_thing_colors]
76
+ cityscapes_stuff_classes = cityscapes_metadata.stuff_classes
77
+ cityscapes_stuff_color = cityscapes_metadata.stuff_colors
78
+ cityscapes_thing_classes = cityscapes_metadata.thing_classes
79
+ cityscapes_thing_color = cityscapes_metadata.thing_colors
80
+
81
+ def build_demo_classes_and_metadata(vocab, label_list):
82
+ extra_classes = []
83
+
84
+ if vocab:
85
+ for words in vocab.split(";"):
86
+ extra_classes.append(words)
87
+ extra_colors = [random_color(rgb=True, maximum=1) for _ in range(len(extra_classes))]
88
+ print("extra_classes:", extra_classes)
89
+ demo_thing_classes = extra_classes
90
+ demo_stuff_classes = []
91
+ demo_thing_colors = extra_colors
92
+ demo_stuff_colors = []
93
+
94
+ if any("COCO" in label for label in label_list):
95
+ demo_thing_classes += coco_thing_classes
96
+ demo_stuff_classes += coco_stuff_classes
97
+ demo_thing_colors += coco_thing_colors
98
+ demo_stuff_colors += coco_stuff_colors
99
+ if any("ADE" in label for label in label_list):
100
+ demo_thing_classes += ade20k_thing_classes
101
+ demo_stuff_classes += ade20k_stuff_classes
102
+ demo_thing_colors += ade20k_thing_colors
103
+ demo_stuff_colors += ade20k_stuff_colors
104
+ if any("LVIS" in label for label in label_list):
105
+ demo_thing_classes += lvis_classes
106
+ demo_thing_colors += lvis_colors
107
+ if any("Cityscapes" in label for label in label_list):
108
+ demo_thing_classes += cityscapes_thing_classes
109
+ demo_thing_colors += cityscapes_thing_color
110
+ demo_stuff_classes += cityscapes_stuff_classes
111
+ demo_stuff_colors += cityscapes_stuff_color
112
+
113
+
114
+ MetadataCatalog.pop("frozenseg_demo_metadata", None)
115
+ demo_metadata = MetadataCatalog.get("frozenseg_demo_metadata")
116
+ demo_metadata.thing_classes = demo_thing_classes
117
+ demo_metadata.stuff_classes = demo_thing_classes + demo_stuff_classes
118
+ demo_metadata.thing_colors = demo_thing_colors
119
+ demo_metadata.stuff_colors = demo_thing_colors + demo_stuff_colors
120
+ demo_metadata.stuff_dataset_id_to_contiguous_id = {
121
+ idx: idx for idx in range(len(demo_metadata.stuff_classes))
122
+ }
123
+ demo_metadata.thing_dataset_id_to_contiguous_id = {
124
+ idx: idx for idx in range(len(demo_metadata.thing_classes))
125
+ }
126
+ demo_classes = demo_thing_classes + demo_stuff_classes
127
+ return demo_classes, demo_metadata
128
+
129
+
130
+ def inference(image_path, vocab, label_list):
131
+ logger.info("building class names")
132
+ vocab = vocab.replace(", ", ",").replace("; ", ";")
133
+ demo_classes, demo_metadata = build_demo_classes_and_metadata(vocab, label_list)
134
+ predictor.set_metadata(demo_metadata)
135
+ im = cv2.imread(image_path)
136
+ outputs = predictor(im)
137
+ v = OpenVocabVisualizer(im[:, :, ::-1], demo_metadata, instance_mode=ColorMode.IMAGE)
138
+ panoptic_result = v.draw_panoptic_seg(outputs["panoptic_seg"][0].to("cpu"), outputs["panoptic_seg"][1]).get_image()
139
+ return Image.fromarray(np.uint8(panoptic_result)).convert('RGB')
140
+
141
+
142
+ with gr.Blocks(title=title,
143
+ css="""
144
+ #submit {background: #3498db; color: white; border: none; padding: 10px 20px; border-radius: 5px;width: 20%;margin: 0 auto; display: block;}
145
+ """
146
+ ) as demo:
147
+ gr.Markdown("<h1 style='text-align: center; margin-bottom: 1rem'>" + title + "</h1>")
148
+ input_components = []
149
+ output_components = []
150
+
151
+ with gr.Row():
152
+ output_image_gr = gr.Image(label="Panoptic Segmentation Output", type="pil")
153
+ output_components.append(output_image_gr)
154
+
155
+ with gr.Row().style(equal_height=True):
156
+ with gr.Column(scale=3, variant="panel") as input_component_column:
157
+ input_image_gr = gr.Image(type="filepath", label="Input Image")
158
+ extra_vocab_gr = gr.Textbox(label="Extra Vocabulary (separated by ;)", placeholder="house;sky")
159
+ category_list_gr = gr.CheckboxGroup(
160
+ choices=["COCO (133 categories)", "ADE (150 categories)", "LVIS (1203 categories)", "Cityscapes (19 categories)"],
161
+ label="Category to use",
162
+ )
163
+ input_components.extend([input_image_gr, extra_vocab_gr, category_list_gr])
164
+
165
+ with gr.Column(scale=2):
166
+ examples_handler = gr.Examples(
167
+ examples=examples,
168
+ inputs=[c for c in input_components if not isinstance(c, gr.State)],
169
+ outputs=[c for c in output_components if not isinstance(c, gr.State)],
170
+ fn=inference,
171
+ cache_examples=torch.cuda.is_available(),
172
+ examples_per_page=5,
173
+ )
174
+ with gr.Row():
175
+ clear_btn = gr.Button("Clear")
176
+ submit_btn = gr.Button("Submit", variant="primary")
177
+
178
+ gr.Markdown(article)
179
+
180
+ submit_btn.click(
181
+ inference,
182
+ input_components,
183
+ output_components,
184
+ api_name="predict",
185
+ scroll_to_output=True,
186
+ )
187
+
188
+ clear_btn.click(
189
+ None,
190
+ [],
191
+ (input_components + output_components + [input_component_column]),
192
+ _js=f"""() => {json.dumps(
193
+ [component.cleared_value if hasattr(component, "cleared_value") else None
194
+ for component in input_components + output_components] + (
195
+ [gr.Column.update(visible=True)]
196
+ )
197
+ + ([gr.Column.update(visible=False)])
198
+ )}
199
+ """,
200
+ )
201
+
202
+ demo.launch(server_port=7881)
configs/coco/Base-COCO-PanopticSegmentation.yaml ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MODEL:
2
+ BACKBONE:
3
+ FREEZE_AT: 0
4
+ NAME: "build_resnet_backbone"
5
+ WEIGHTS: "detectron2://ImageNetPretrained/torchvision/R-50.pkl"
6
+ PIXEL_MEAN: [123.675, 116.280, 103.530]
7
+ PIXEL_STD: [58.395, 57.120, 57.375]
8
+ RESNETS:
9
+ DEPTH: 50
10
+ # STEM_TYPE: "basic" # not used
11
+ STEM_OUT_CHANNELS: 64
12
+ STRIDE_IN_1X1: False
13
+ OUT_FEATURES: ["res2", "res3", "res4", "res5"]
14
+ # NORM: "SyncBN"
15
+ # RES5_MULTI_GRID: [1, 1, 1] # not used
16
+ DATASETS:
17
+ TRAIN: ("coco_2017_train_panoptic",)
18
+ TEST: ("coco_2017_val_panoptic_with_sem_seg",) # to evaluate instance and semantic performance as well
19
+ SOLVER:
20
+ IMS_PER_BATCH: 16
21
+ BASE_LR: 0.0001
22
+ STEPS: (327778, 355092)
23
+ MAX_ITER: 368750
24
+ WARMUP_FACTOR: 1.0
25
+ WARMUP_ITERS: 10
26
+ WEIGHT_DECAY: 0.05
27
+ OPTIMIZER: "ADAMW"
28
+ BACKBONE_MULTIPLIER: 0.1
29
+ CLIP_GRADIENTS:
30
+ ENABLED: True
31
+ CLIP_TYPE: "full_model"
32
+ CLIP_VALUE: 0.01
33
+ NORM_TYPE: 2.0
34
+ AMP:
35
+ ENABLED: True
36
+ INPUT:
37
+ IMAGE_SIZE: 1024
38
+ MIN_SCALE: 0.1
39
+ MAX_SCALE: 2.0
40
+ FORMAT: "RGB"
41
+ DATASET_MAPPER_NAME: "coco_panoptic_lsj"
42
+ TEST:
43
+ EVAL_PERIOD: 100000000
44
+ DATALOADER:
45
+ FILTER_EMPTY_ANNOTATIONS: True
46
+ NUM_WORKERS: 4
47
+ VERSION: 2
configs/coco/frozenseg/convnext_large_eval_a847.yaml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ MODEL:
4
+ MASK_FORMER:
5
+ TEST:
6
+ PANOPTIC_ON: False
7
+ INSTANCE_ON: False
8
+
9
+ DATASETS:
10
+ TEST: ("openvocab_ade20k_full_sem_seg_val",)
configs/coco/frozenseg/convnext_large_eval_ade20k.yaml ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ../maskformer2_R50_bs16_50ep.yaml
2
+ MODEL:
3
+ META_ARCHITECTURE: "FrozenSeg"
4
+ SEM_SEG_HEAD:
5
+ NAME: "FrozenSegHead"
6
+ # backbone part.
7
+ BACKBONE:
8
+ NAME: "CLIP"
9
+ WEIGHTS: ""
10
+ PIXEL_MEAN: [122.7709383, 116.7460125, 104.09373615]
11
+ PIXEL_STD: [68.5005327, 66.6321579, 70.32316305]
12
+ FROZEN_SEG:
13
+ CLIP_MODEL_NAME: "convnext_large_d_320"
14
+ # CLIP_PRETRAINED_WEIGHTS: "laion2b_s29b_b131k_ft_soup"
15
+ CLIP_PRETRAINED_WEIGHTS: "pretrained_checkpoint/models--laion--CLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soup/open_clip_pytorch_model.bin"
16
+ EMBED_DIM: 768
17
+ GEOMETRIC_ENSEMBLE_ALPHA: 0.4
18
+ GEOMETRIC_ENSEMBLE_BETA: 0.8
19
+ MASK_FORMER:
20
+ NUM_OBJECT_QUERIES: 250
21
+ TEST:
22
+ SEMANTIC_ON: True
23
+ INSTANCE_ON: True
24
+ PANOPTIC_ON: True
25
+ OBJECT_MASK_THRESHOLD: 0.0
26
+
27
+ DATASETS:
28
+ TRAIN: ("openvocab_coco_2017_train_panoptic_with_sem_seg",)
29
+ TEST: ("openvocab_ade20k_panoptic_val",)
configs/coco/frozenseg/convnext_large_eval_bdd_panop.yaml ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ MODEL:
4
+ MASK_FORMER:
5
+ TEST:
6
+ PANOPTIC_ON: True
7
+ INSTANCE_ON: False
8
+ SEMANTIC_ON: False
9
+ OBJECT_MASK_THRESHOLD: 0.4
10
+ INPUT:
11
+ MIN_SIZE_TEST: 800
12
+ MAX_SIZE_TEST: 1333
13
+ DATASETS:
14
+ TEST: ("bdd10k_40_panoptic_val",)
configs/coco/frozenseg/convnext_large_eval_bdd_sem.yaml ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ MODEL:
4
+ MASK_FORMER:
5
+ TEST:
6
+ PANOPTIC_ON: False
7
+ INSTANCE_ON: False
8
+ SEMANTIC_ON: True
9
+ INPUT:
10
+ MIN_SIZE_TEST: 800
11
+ MAX_SIZE_TEST: 1333
12
+ DATASETS:
13
+ TEST: ("bdd10k_val_sem_seg",)
configs/coco/frozenseg/convnext_large_eval_cityscapes.yaml ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ INPUT:
4
+ MIN_SIZE_TEST: 1024
5
+ MAX_SIZE_TEST: 2560
6
+
7
+ DATASETS:
8
+ TEST: ("openvocab_cityscapes_fine_panoptic_val",)
configs/coco/frozenseg/convnext_large_eval_coco.yaml ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+ DATASETS:
3
+ TEST: ("openvocab_coco_2017_val_panoptic_with_sem_seg",)
configs/coco/frozenseg/convnext_large_eval_lvis.yaml ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ MODEL:
4
+ MASK_FORMER:
5
+ TEST:
6
+ PANOPTIC_ON: False
7
+ INSTANCE_ON: True
8
+ SEMANTIC_ON: False
9
+
10
+ DATASETS:
11
+ TEST: ("openvocab_lvis_v1_val",)
configs/coco/frozenseg/convnext_large_eval_mapillary_vistas.yaml ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ MODEL:
4
+ MASK_FORMER:
5
+ TEST:
6
+ INSTANCE_ON: False
7
+ INPUT:
8
+ MIN_SIZE_TEST: 1024
9
+ MAX_SIZE_TEST: 2560
10
+
11
+ DATASETS:
12
+ TEST: ("openvocab_mapillary_vistas_panoptic_val",)
configs/coco/frozenseg/convnext_large_eval_pas21.yaml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ MODEL:
4
+ MASK_FORMER:
5
+ TEST:
6
+ PANOPTIC_ON: False
7
+ INSTANCE_ON: False
8
+
9
+ DATASETS:
10
+ TEST: ("openvocab_pascal21_sem_seg_val",)
configs/coco/frozenseg/convnext_large_eval_pc459.yaml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+
3
+ MODEL:
4
+ MASK_FORMER:
5
+ TEST:
6
+ PANOPTIC_ON: False
7
+ INSTANCE_ON: False
8
+
9
+ DATASETS:
10
+ TEST: ("openvocab_pascal_ctx459_sem_seg_val",)
configs/coco/frozenseg/r50x64_eval_ade20k.yaml ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: ./convnext_large_eval_ade20k.yaml
2
+ MODEL:
3
+ FROZEN_SEG:
4
+ CLIP_MODEL_NAME: "RN50x64"
5
+ CLIP_PRETRAINED_WEIGHTS: "openai"
6
+ EMBED_DIM: 1024
7
+ ENSEMBLE_ON_VALID_MASK: True
8
+ MASK_FORMER:
9
+ TEST:
10
+ PANOPTIC_ON: False
11
+ INSTANCE_ON: False
12
+ DATASETS:
13
+ TEST: ("openvocab_ade20k_full_sem_seg_val",)
configs/coco/maskformer2_R50_bs16_50ep.yaml ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ _BASE_: Base-COCO-PanopticSegmentation.yaml
2
+ MODEL:
3
+ META_ARCHITECTURE: "MaskFormer"
4
+ SEM_SEG_HEAD:
5
+ NAME: "MaskFormerHead"
6
+ IN_FEATURES: ["res2", "res3", "res4", "res5"]
7
+ IGNORE_VALUE: 255
8
+ NUM_CLASSES: 133
9
+ LOSS_WEIGHT: 1.0
10
+ CONVS_DIM: 256
11
+ MASK_DIM: 256
12
+ NORM: "GN"
13
+ # pixel decoder
14
+ PIXEL_DECODER_NAME: "MSDeformAttnPixelDecoder"
15
+ IN_FEATURES: ["res2", "res3", "res4", "res5"]
16
+ DEFORMABLE_TRANSFORMER_ENCODER_IN_FEATURES: ["res3", "res4", "res5"]
17
+ COMMON_STRIDE: 4
18
+ TRANSFORMER_ENC_LAYERS: 6
19
+ MASK_FORMER:
20
+ TRANSFORMER_DECODER_NAME: "MultiScaleMaskedTransformerDecoder"
21
+ TRANSFORMER_IN_FEATURE: "multi_scale_pixel_decoder"
22
+ DEEP_SUPERVISION: True
23
+ NO_OBJECT_WEIGHT: 0.1
24
+ CLASS_WEIGHT: 2.0
25
+ MASK_WEIGHT: 5.0
26
+ DICE_WEIGHT: 5.0
27
+ HIDDEN_DIM: 256
28
+ NUM_OBJECT_QUERIES: 100
29
+ NHEADS: 8
30
+ DROPOUT: 0.0
31
+ DIM_FEEDFORWARD: 2048
32
+ ENC_LAYERS: 0
33
+ PRE_NORM: False
34
+ ENFORCE_INPUT_PROJ: False
35
+ SIZE_DIVISIBILITY: 32
36
+ DEC_LAYERS: 10 # 9 decoder layers, add one for the loss on learnable query
37
+ TRAIN_NUM_POINTS: 12544
38
+ OVERSAMPLE_RATIO: 3.0
39
+ IMPORTANCE_SAMPLE_RATIO: 0.75
40
+ TEST:
41
+ SEMANTIC_ON: True
42
+ INSTANCE_ON: True
43
+ PANOPTIC_ON: True
44
+ OVERLAP_THRESHOLD: 0.8
45
+ OBJECT_MASK_THRESHOLD: 0.8
datasets/README.md ADDED
@@ -0,0 +1,262 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Prepare Datasets for FrozenSeg
2
+
3
+ A dataset can be used by accessing [DatasetCatalog](https://detectron2.readthedocs.io/modules/data.html#detectron2.data.DatasetCatalog)
4
+ for its data, or [MetadataCatalog](https://detectron2.readthedocs.io/modules/data.html#detectron2.data.MetadataCatalog) for its metadata (class names, etc).
5
+ This document explains how to setup the builtin datasets so they can be used by the above APIs.
6
+ [Use Custom Datasets](https://detectron2.readthedocs.io/tutorials/datasets.html) gives a deeper dive on how to use `DatasetCatalog` and `MetadataCatalog`,
7
+ and how to add new datasets to them.
8
+
9
+ FrozenSeg has builtin support for a few datasets.
10
+ The datasets are assumed to exist in a directory specified by the environment variable
11
+ `DETECTRON2_DATASETS`.
12
+ Under this directory, detectron2 will look for datasets in the structure described below, if needed.
13
+ ```
14
+ $DETECTRON2_DATASETS/
15
+ # panoptic datasets
16
+ ADEChallengeData2016/
17
+ coco/
18
+ cityscapes/
19
+ mapillary_vistas/
20
+ bdd100k/
21
+ # semantic datasets
22
+ VOCdevkit/
23
+ ADE20K_2021_17_01/
24
+ pascal_ctx_d2/
25
+ pascal_voc_d2/
26
+ ```
27
+
28
+ You can set the location for builtin datasets by `export DETECTRON2_DATASETS=/path/to/datasets`.
29
+ If left unset, the default is `./datasets` relative to your current working directory.
30
+
31
+
32
+ ## Expected dataset structure for [COCO](https://cocodataset.org/#download):
33
+
34
+ ```
35
+ coco/
36
+ annotations/
37
+ instances_{train,val}2017.json
38
+ panoptic_{train,val}2017.json
39
+ {train,val}2017/
40
+ # image files that are mentioned in the corresponding json
41
+ panoptic_{train,val}2017/ # png annotations
42
+ panoptic_semseg_{train,val}2017/ # generated by the script mentioned below
43
+ ```
44
+
45
+ Install panopticapi by:
46
+ ```
47
+ pip install git+https://github.com/cocodataset/panopticapi.git
48
+ ```
49
+ Then, run `python datasets/prepare_coco_semantic_annos_from_panoptic_annos.py`, to extract semantic annotations from panoptic annotations (only used for evaluation).
50
+
51
+
52
+ ## Expected dataset structure for [cityscapes](https://www.cityscapes-dataset.com/downloads/):
53
+ ```
54
+ cityscapes/
55
+ gtFine/
56
+ train/
57
+ aachen/
58
+ color.png, instanceIds.png, labelIds.png, polygons.json,
59
+ labelTrainIds.png
60
+ ...
61
+ val/
62
+ test/
63
+ # below are generated Cityscapes panoptic annotation
64
+ cityscapes_panoptic_train.json
65
+ cityscapes_panoptic_train/
66
+ cityscapes_panoptic_val.json
67
+ cityscapes_panoptic_val/
68
+ cityscapes_panoptic_test.json
69
+ cityscapes_panoptic_test/
70
+ leftImg8bit/
71
+ train/
72
+ val/
73
+ test/
74
+ ```
75
+ Install cityscapes scripts by:
76
+ ```
77
+ pip install git+https://github.com/mcordts/cityscapesScripts.git
78
+ ```
79
+
80
+ Note: to create labelTrainIds.png, first prepare the above structure, then run cityscapesescript with:
81
+ ```
82
+ CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createTrainIdLabelImgs.py
83
+ ```
84
+ These files are not needed for instance segmentation.
85
+
86
+ Note: to generate Cityscapes panoptic dataset, run cityscapesescript with:
87
+ ```
88
+ CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createPanopticImgs.py
89
+ ```
90
+ These files are not needed for semantic and instance segmentation.
91
+
92
+
93
+ ## Expected dataset structure for [ADE20k (A150)](http://sceneparsing.csail.mit.edu/):
94
+ ```
95
+ ADEChallengeData2016/
96
+ images/
97
+ annotations/
98
+ objectInfo150.txt
99
+ # download instance annotation
100
+ annotations_instance/
101
+ # generated by prepare_ade20k_sem_seg.py
102
+ annotations_detectron2/
103
+ # below are generated by prepare_ade20k_pan_seg.py
104
+ ade20k_panoptic_{train,val}.json
105
+ ade20k_panoptic_{train,val}/
106
+ # below are generated by prepare_ade20k_ins_seg.py
107
+ ade20k_instance_{train,val}.json
108
+ ```
109
+
110
+ The directory `annotations_detectron2` is generated by running `python datasets/prepare_ade20k_sem_seg.py`.
111
+
112
+ Install panopticapi by:
113
+ ```bash
114
+ pip install git+https://github.com/cocodataset/panopticapi.git
115
+ ```
116
+
117
+ Download the instance annotation from http://sceneparsing.csail.mit.edu/:
118
+ ```bash
119
+ wget http://sceneparsing.csail.mit.edu/data/ChallengeData2017/annotations_instance.tar
120
+ ```
121
+
122
+ Then, run `python datasets/prepare_ade20k_pan_seg.py`, to combine semantic and instance annotations for panoptic annotations.
123
+
124
+ And run `python datasets/prepare_ade20k_ins_seg.py`, to extract instance annotations in COCO format.
125
+
126
+
127
+ ## Expected dataset structure for [Mapillary Vistas](https://www.mapillary.com/dataset/vistas):
128
+ ```
129
+ mapillary_vistas/
130
+ training/
131
+ images/
132
+ instances/
133
+ labels/
134
+ panoptic/
135
+ validation/
136
+ images/
137
+ instances/
138
+ labels/
139
+ panoptic/
140
+ ```
141
+
142
+ No preprocessing is needed for Mapillary Vistas on semantic and panoptic segmentation.
143
+
144
+ ## Expected dataset structure for [BDD100K](https://doc.bdd100k.com/download.html#id1)
145
+ ```
146
+ bdd100k/
147
+ images/
148
+ 10k/
149
+ train/
150
+ val/
151
+ test/
152
+ json
153
+ labels/
154
+ pan_seg/
155
+ sem_seg/
156
+ ```
157
+
158
+ `coco-format` annotations is obtained by running:
159
+
160
+
161
+ ```
162
+ cd $DETECTRON2_DATASETS
163
+ wget https://github.com/chenxi52/FrozenSeg/releases/download/latest/bdd100k_json.zip
164
+ unzip bdd100k_json.zip
165
+ ```
166
+
167
+
168
+ ## Expected dataset structure for [ADE20k-Full (A-847)](https://groups.csail.mit.edu/vision/datasets/ADE20K/):
169
+ ```
170
+ ADE20K_2021_17_01/
171
+ images/
172
+ index_ade20k.pkl
173
+ objects.txt
174
+ # generated by prepare_ade20k_full_sem_seg.py
175
+ images_detectron2/
176
+ annotations_detectron2/
177
+ ```
178
+
179
+ Register and download the dataset from https://groups.csail.mit.edu/vision/datasets/ADE20K/:
180
+ ```bash
181
+ cd $DETECTRON2_DATASETS
182
+ wget your/personal/download/link/{username}_{hash}.zip
183
+ unzip {username}_{hash}.zip
184
+ ```
185
+
186
+ Generate the directories `ADE20K_2021_17_01/images_detectron2` and `ADE20K_2021_17_01/annotations_detectron2` by running:
187
+ ```bash
188
+ python datasets/prepare_ade20k_full_sem_seg.py
189
+ ```
190
+
191
+ ## Expected dataset structure for [PASCAL Context Full (PC-459)](https://www.cs.stanford.edu/~roozbeh/pascal-context/) and [PASCAL VOC (PAS-21)](http://host.robots.ox.ac.uk/pascal/VOC/):
192
+
193
+ ```bash
194
+ VOCdevkit/
195
+ VOC2012/
196
+ Annotations/
197
+ JPEGImages/
198
+ ImageSets/
199
+ Segmentation/
200
+ VOC2010/
201
+ JPEGImages/
202
+ trainval/
203
+ trainval_merged.json
204
+ # generated by prepare_pascal_voc_sem_seg.py
205
+ pascal_voc_d2/
206
+ images/
207
+ annotations_pascal21/
208
+ # pascal 20 excludes the background class
209
+ annotations_pascal20/
210
+ # generated by prepare_pascal_ctx_sem_seg.py
211
+ pascal_ctx_d2/
212
+ images/
213
+ annotations_ctx59/
214
+ # generated by prepare_pascal_ctx_full_sem_seg.py
215
+ annotations_ctx459/
216
+
217
+ ```
218
+ ### PASCAL VOC (PAS-21)
219
+
220
+ Download the dataset from http://host.robots.ox.ac.uk/pascal/VOC/:
221
+ ```bash
222
+ cd $DETECTRON2_DATASETS
223
+ wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
224
+ # generate folder VOCdevkit/VOC2012
225
+ tar -xvf VOCtrainval_11-May-2012.tar
226
+ ```
227
+
228
+ Generate directory `pascal_voc_d2` running:
229
+ ```bash
230
+ python datasets/prepare_pascal_voc_sem_seg.py
231
+ ```
232
+
233
+ ### PASCAL Context Full (PC-459)
234
+
235
+ Download the dataset from http://host.robots.ox.ac.uk/pascal/VOC/ and annotation from https://www.cs.stanford.edu/~roozbeh/pascal-context/:
236
+ ```bash
237
+ cd $DETECTRON2_DATASETS
238
+ wget http://host.robots.ox.ac.uk/pascal/VOC/voc2010/VOCtrainval_03-May-2010.tar
239
+ # generate folder VOCdevkit/VOC2010
240
+ tar -xvf VOCtrainval_03-May-2010.tar
241
+ wget https://www.cs.stanford.edu/~roozbeh/pascal-context/trainval.tar.gz
242
+ # generate folder VOCdevkit/VOC2010/trainval
243
+ tar -xvzf trainval.tar.gz -C VOCdevkit/VOC2010
244
+ wget https://codalabuser.blob.core.windows.net/public/trainval_merged.json -P VOCdevkit/VOC2010/
245
+ ```
246
+
247
+ Install [Detail API](https://github.com/zhanghang1989/detail-api) by:
248
+ ```bash
249
+ git clone https://github.com/zhanghang1989/detail-api.git
250
+ rm detail-api/PythonAPI/detail/_mask.c
251
+ pip install -e detail-api/PythonAPI/
252
+ ```
253
+
254
+ Generate directory `pascal_ctx_d2/images` running:
255
+ ```bash
256
+ python datasets/prepare_pascal_ctx_sem_seg.py
257
+ ```
258
+
259
+ Generate directory `pascal_ctx_d2/annotations_ctx459` running:
260
+ ```bash
261
+ python datasets/prepare_pascal_ctx_full_sem_seg.py
262
+ ```
datasets/ade20k_instance_catid_mapping.txt ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Instacne100 SceneParse150 FullADE20K
2
+ 1 8 165
3
+ 2 9 3055
4
+ 3 11 350
5
+ 4 13 1831
6
+ 5 15 774
7
+ 5 15 783
8
+ 6 16 2684
9
+ 7 19 687
10
+ 8 20 471
11
+ 9 21 401
12
+ 10 23 1735
13
+ 11 24 2473
14
+ 12 25 2329
15
+ 13 28 1564
16
+ 14 31 57
17
+ 15 32 2272
18
+ 16 33 907
19
+ 17 34 724
20
+ 18 36 2985
21
+ 18 36 533
22
+ 19 37 1395
23
+ 20 38 155
24
+ 21 39 2053
25
+ 22 40 689
26
+ 23 42 266
27
+ 24 43 581
28
+ 25 44 2380
29
+ 26 45 491
30
+ 27 46 627
31
+ 28 48 2388
32
+ 29 50 943
33
+ 30 51 2096
34
+ 31 54 2530
35
+ 32 56 420
36
+ 33 57 1948
37
+ 34 58 1869
38
+ 35 59 2251
39
+ 36 63 239
40
+ 37 65 571
41
+ 38 66 2793
42
+ 39 67 978
43
+ 40 68 236
44
+ 41 70 181
45
+ 42 71 629
46
+ 43 72 2598
47
+ 44 73 1744
48
+ 45 74 1374
49
+ 46 75 591
50
+ 47 76 2679
51
+ 48 77 223
52
+ 49 79 47
53
+ 50 81 327
54
+ 51 82 2821
55
+ 52 83 1451
56
+ 53 84 2880
57
+ 54 86 480
58
+ 55 87 77
59
+ 56 88 2616
60
+ 57 89 246
61
+ 57 89 247
62
+ 58 90 2733
63
+ 59 91 14
64
+ 60 93 38
65
+ 61 94 1936
66
+ 62 96 120
67
+ 63 98 1702
68
+ 64 99 249
69
+ 65 103 2928
70
+ 66 104 2337
71
+ 67 105 1023
72
+ 68 108 2989
73
+ 69 109 1930
74
+ 70 111 2586
75
+ 71 112 131
76
+ 72 113 146
77
+ 73 116 95
78
+ 74 117 1563
79
+ 75 119 1708
80
+ 76 120 103
81
+ 77 121 1002
82
+ 78 122 2569
83
+ 79 124 2833
84
+ 80 125 1551
85
+ 81 126 1981
86
+ 82 127 29
87
+ 83 128 187
88
+ 84 130 747
89
+ 85 131 2254
90
+ 86 133 2262
91
+ 87 134 1260
92
+ 88 135 2243
93
+ 89 136 2932
94
+ 90 137 2836
95
+ 91 138 2850
96
+ 92 139 64
97
+ 93 140 894
98
+ 94 143 1919
99
+ 95 144 1583
100
+ 96 145 318
101
+ 97 147 2046
102
+ 98 148 1098
103
+ 99 149 530
104
+ 100 150 954
datasets/ade20k_instance_imgCatIds.json ADDED
The diff for this file is too large to render. See raw diff
 
datasets/prepare_ade20k_full_sem_seg.py ADDED
@@ -0,0 +1,1004 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import pickle as pkl
3
+ from pathlib import Path
4
+
5
+ import cv2
6
+ import numpy as np
7
+ import tqdm
8
+ from PIL import Image
9
+
10
+ ADE20K_SEM_SEG_FULL_CATEGORIES = [
11
+ {"name": "wall", "id": 2978, "trainId": 0},
12
+ {"name": "building, edifice", "id": 312, "trainId": 1},
13
+ {"name": "sky", "id": 2420, "trainId": 2},
14
+ {"name": "tree", "id": 2855, "trainId": 3},
15
+ {"name": "road, route", "id": 2131, "trainId": 4},
16
+ {"name": "floor, flooring", "id": 976, "trainId": 5},
17
+ {"name": "ceiling", "id": 447, "trainId": 6},
18
+ {"name": "bed", "id": 165, "trainId": 7},
19
+ {"name": "sidewalk, pavement", "id": 2377, "trainId": 8},
20
+ {"name": "earth, ground", "id": 838, "trainId": 9},
21
+ {"name": "cabinet", "id": 350, "trainId": 10},
22
+ {"name": "person, individual, someone, somebody, mortal, soul", "id": 1831, "trainId": 11},
23
+ {"name": "grass", "id": 1125, "trainId": 12},
24
+ {"name": "windowpane, window", "id": 3055, "trainId": 13},
25
+ {"name": "car, auto, automobile, machine, motorcar", "id": 401, "trainId": 14},
26
+ {"name": "mountain, mount", "id": 1610, "trainId": 15},
27
+ {"name": "plant, flora, plant life", "id": 1910, "trainId": 16},
28
+ {"name": "table", "id": 2684, "trainId": 17},
29
+ {"name": "chair", "id": 471, "trainId": 18},
30
+ {"name": "curtain, drape, drapery, mantle, pall", "id": 687, "trainId": 19},
31
+ {"name": "door", "id": 774, "trainId": 20},
32
+ {"name": "sofa, couch, lounge", "id": 2473, "trainId": 21},
33
+ {"name": "sea", "id": 2264, "trainId": 22},
34
+ {"name": "painting, picture", "id": 1735, "trainId": 23},
35
+ {"name": "water", "id": 2994, "trainId": 24},
36
+ {"name": "mirror", "id": 1564, "trainId": 25},
37
+ {"name": "house", "id": 1276, "trainId": 26},
38
+ {"name": "rug, carpet, carpeting", "id": 2178, "trainId": 27},
39
+ {"name": "shelf", "id": 2329, "trainId": 28},
40
+ {"name": "armchair", "id": 57, "trainId": 29},
41
+ {"name": "fence, fencing", "id": 907, "trainId": 30},
42
+ {"name": "field", "id": 913, "trainId": 31},
43
+ {"name": "lamp", "id": 1395, "trainId": 32},
44
+ {"name": "rock, stone", "id": 2138, "trainId": 33},
45
+ {"name": "seat", "id": 2272, "trainId": 34},
46
+ {"name": "river", "id": 2128, "trainId": 35},
47
+ {"name": "desk", "id": 724, "trainId": 36},
48
+ {"name": "bathtub, bathing tub, bath, tub", "id": 155, "trainId": 37},
49
+ {"name": "railing, rail", "id": 2053, "trainId": 38},
50
+ {"name": "signboard, sign", "id": 2380, "trainId": 39},
51
+ {"name": "cushion", "id": 689, "trainId": 40},
52
+ {"name": "path", "id": 1788, "trainId": 41},
53
+ {"name": "work surface", "id": 3087, "trainId": 42},
54
+ {"name": "stairs, steps", "id": 2530, "trainId": 43},
55
+ {"name": "column, pillar", "id": 581, "trainId": 44},
56
+ {"name": "sink", "id": 2388, "trainId": 45},
57
+ {"name": "wardrobe, closet, press", "id": 2985, "trainId": 46},
58
+ {"name": "snow", "id": 2454, "trainId": 47},
59
+ {"name": "refrigerator, icebox", "id": 2096, "trainId": 48},
60
+ {"name": "base, pedestal, stand", "id": 137, "trainId": 49},
61
+ {"name": "bridge, span", "id": 294, "trainId": 50},
62
+ {"name": "blind, screen", "id": 212, "trainId": 51},
63
+ {"name": "runway", "id": 2185, "trainId": 52},
64
+ {"name": "cliff, drop, drop-off", "id": 524, "trainId": 53},
65
+ {"name": "sand", "id": 2212, "trainId": 54},
66
+ {"name": "fireplace, hearth, open fireplace", "id": 943, "trainId": 55},
67
+ {"name": "pillow", "id": 1869, "trainId": 56},
68
+ {"name": "screen door, screen", "id": 2251, "trainId": 57},
69
+ {"name": "toilet, can, commode, crapper, pot, potty, stool, throne", "id": 2793, "trainId": 58},
70
+ {"name": "skyscraper", "id": 2423, "trainId": 59},
71
+ {"name": "grandstand, covered stand", "id": 1121, "trainId": 60},
72
+ {"name": "box", "id": 266, "trainId": 61},
73
+ {"name": "pool table, billiard table, snooker table", "id": 1948, "trainId": 62},
74
+ {"name": "palm, palm tree", "id": 1744, "trainId": 63},
75
+ {"name": "double door", "id": 783, "trainId": 64},
76
+ {"name": "coffee table, cocktail table", "id": 571, "trainId": 65},
77
+ {"name": "counter", "id": 627, "trainId": 66},
78
+ {"name": "countertop", "id": 629, "trainId": 67},
79
+ {"name": "chest of drawers, chest, bureau, dresser", "id": 491, "trainId": 68},
80
+ {"name": "kitchen island", "id": 1374, "trainId": 69},
81
+ {"name": "boat", "id": 223, "trainId": 70},
82
+ {"name": "waterfall, falls", "id": 3016, "trainId": 71},
83
+ {
84
+ "name": "stove, kitchen stove, range, kitchen range, cooking stove",
85
+ "id": 2598,
86
+ "trainId": 72,
87
+ },
88
+ {"name": "flower", "id": 978, "trainId": 73},
89
+ {"name": "bookcase", "id": 239, "trainId": 74},
90
+ {"name": "controls", "id": 608, "trainId": 75},
91
+ {"name": "book", "id": 236, "trainId": 76},
92
+ {"name": "stairway, staircase", "id": 2531, "trainId": 77},
93
+ {"name": "streetlight, street lamp", "id": 2616, "trainId": 78},
94
+ {
95
+ "name": "computer, computing machine, computing device, data processor, electronic computer, information processing system",
96
+ "id": 591,
97
+ "trainId": 79,
98
+ },
99
+ {
100
+ "name": "bus, autobus, coach, charabanc, double-decker, jitney, motorbus, motorcoach, omnibus, passenger vehicle",
101
+ "id": 327,
102
+ "trainId": 80,
103
+ },
104
+ {"name": "swivel chair", "id": 2679, "trainId": 81},
105
+ {"name": "light, light source", "id": 1451, "trainId": 82},
106
+ {"name": "bench", "id": 181, "trainId": 83},
107
+ {"name": "case, display case, showcase, vitrine", "id": 420, "trainId": 84},
108
+ {"name": "towel", "id": 2821, "trainId": 85},
109
+ {"name": "fountain", "id": 1023, "trainId": 86},
110
+ {"name": "embankment", "id": 855, "trainId": 87},
111
+ {
112
+ "name": "television receiver, television, television set, tv, tv set, idiot box, boob tube, telly, goggle box",
113
+ "id": 2733,
114
+ "trainId": 88,
115
+ },
116
+ {"name": "van", "id": 2928, "trainId": 89},
117
+ {"name": "hill", "id": 1240, "trainId": 90},
118
+ {"name": "awning, sunshade, sunblind", "id": 77, "trainId": 91},
119
+ {"name": "poster, posting, placard, notice, bill, card", "id": 1969, "trainId": 92},
120
+ {"name": "truck, motortruck", "id": 2880, "trainId": 93},
121
+ {"name": "airplane, aeroplane, plane", "id": 14, "trainId": 94},
122
+ {"name": "pole", "id": 1936, "trainId": 95},
123
+ {"name": "tower", "id": 2828, "trainId": 96},
124
+ {"name": "court", "id": 631, "trainId": 97},
125
+ {"name": "ball", "id": 103, "trainId": 98},
126
+ {
127
+ "name": "aircraft carrier, carrier, flattop, attack aircraft carrier",
128
+ "id": 3144,
129
+ "trainId": 99,
130
+ },
131
+ {"name": "buffet, counter, sideboard", "id": 308, "trainId": 100},
132
+ {"name": "hovel, hut, hutch, shack, shanty", "id": 1282, "trainId": 101},
133
+ {"name": "apparel, wearing apparel, dress, clothes", "id": 38, "trainId": 102},
134
+ {"name": "minibike, motorbike", "id": 1563, "trainId": 103},
135
+ {"name": "animal, animate being, beast, brute, creature, fauna", "id": 29, "trainId": 104},
136
+ {"name": "chandelier, pendant, pendent", "id": 480, "trainId": 105},
137
+ {"name": "step, stair", "id": 2569, "trainId": 106},
138
+ {"name": "booth, cubicle, stall, kiosk", "id": 247, "trainId": 107},
139
+ {"name": "bicycle, bike, wheel, cycle", "id": 187, "trainId": 108},
140
+ {"name": "doorframe, doorcase", "id": 778, "trainId": 109},
141
+ {"name": "sconce", "id": 2243, "trainId": 110},
142
+ {"name": "pond", "id": 1941, "trainId": 111},
143
+ {"name": "trade name, brand name, brand, marque", "id": 2833, "trainId": 112},
144
+ {"name": "bannister, banister, balustrade, balusters, handrail", "id": 120, "trainId": 113},
145
+ {"name": "bag", "id": 95, "trainId": 114},
146
+ {"name": "traffic light, traffic signal, stoplight", "id": 2836, "trainId": 115},
147
+ {"name": "gazebo", "id": 1087, "trainId": 116},
148
+ {"name": "escalator, moving staircase, moving stairway", "id": 868, "trainId": 117},
149
+ {"name": "land, ground, soil", "id": 1401, "trainId": 118},
150
+ {"name": "board, plank", "id": 220, "trainId": 119},
151
+ {"name": "arcade machine", "id": 47, "trainId": 120},
152
+ {"name": "eiderdown, duvet, continental quilt", "id": 843, "trainId": 121},
153
+ {"name": "bar", "id": 123, "trainId": 122},
154
+ {"name": "stall, stand, sales booth", "id": 2537, "trainId": 123},
155
+ {"name": "playground", "id": 1927, "trainId": 124},
156
+ {"name": "ship", "id": 2337, "trainId": 125},
157
+ {"name": "ottoman, pouf, pouffe, puff, hassock", "id": 1702, "trainId": 126},
158
+ {
159
+ "name": "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin",
160
+ "id": 64,
161
+ "trainId": 127,
162
+ },
163
+ {"name": "bottle", "id": 249, "trainId": 128},
164
+ {"name": "cradle", "id": 642, "trainId": 129},
165
+ {"name": "pot, flowerpot", "id": 1981, "trainId": 130},
166
+ {
167
+ "name": "conveyer belt, conveyor belt, conveyer, conveyor, transporter",
168
+ "id": 609,
169
+ "trainId": 131,
170
+ },
171
+ {"name": "train, railroad train", "id": 2840, "trainId": 132},
172
+ {"name": "stool", "id": 2586, "trainId": 133},
173
+ {"name": "lake", "id": 1393, "trainId": 134},
174
+ {"name": "tank, storage tank", "id": 2704, "trainId": 135},
175
+ {"name": "ice, water ice", "id": 1304, "trainId": 136},
176
+ {"name": "basket, handbasket", "id": 146, "trainId": 137},
177
+ {"name": "manhole", "id": 1494, "trainId": 138},
178
+ {"name": "tent, collapsible shelter", "id": 2739, "trainId": 139},
179
+ {"name": "canopy", "id": 389, "trainId": 140},
180
+ {"name": "microwave, microwave oven", "id": 1551, "trainId": 141},
181
+ {"name": "barrel, cask", "id": 131, "trainId": 142},
182
+ {"name": "dirt track", "id": 738, "trainId": 143},
183
+ {"name": "beam", "id": 161, "trainId": 144},
184
+ {"name": "dishwasher, dish washer, dishwashing machine", "id": 747, "trainId": 145},
185
+ {"name": "plate", "id": 1919, "trainId": 146},
186
+ {"name": "screen, crt screen", "id": 3109, "trainId": 147},
187
+ {"name": "ruins", "id": 2179, "trainId": 148},
188
+ {"name": "washer, automatic washer, washing machine", "id": 2989, "trainId": 149},
189
+ {"name": "blanket, cover", "id": 206, "trainId": 150},
190
+ {"name": "plaything, toy", "id": 1930, "trainId": 151},
191
+ {"name": "food, solid food", "id": 1002, "trainId": 152},
192
+ {"name": "screen, silver screen, projection screen", "id": 2254, "trainId": 153},
193
+ {"name": "oven", "id": 1708, "trainId": 154},
194
+ {"name": "stage", "id": 2526, "trainId": 155},
195
+ {"name": "beacon, lighthouse, beacon light, pharos", "id": 160, "trainId": 156},
196
+ {"name": "umbrella", "id": 2901, "trainId": 157},
197
+ {"name": "sculpture", "id": 2262, "trainId": 158},
198
+ {"name": "aqueduct", "id": 44, "trainId": 159},
199
+ {"name": "container", "id": 597, "trainId": 160},
200
+ {"name": "scaffolding, staging", "id": 2235, "trainId": 161},
201
+ {"name": "hood, exhaust hood", "id": 1260, "trainId": 162},
202
+ {"name": "curb, curbing, kerb", "id": 682, "trainId": 163},
203
+ {"name": "roller coaster", "id": 2151, "trainId": 164},
204
+ {"name": "horse, equus caballus", "id": 3107, "trainId": 165},
205
+ {"name": "catwalk", "id": 432, "trainId": 166},
206
+ {"name": "glass, drinking glass", "id": 1098, "trainId": 167},
207
+ {"name": "vase", "id": 2932, "trainId": 168},
208
+ {"name": "central reservation", "id": 461, "trainId": 169},
209
+ {"name": "carousel", "id": 410, "trainId": 170},
210
+ {"name": "radiator", "id": 2046, "trainId": 171},
211
+ {"name": "closet", "id": 533, "trainId": 172},
212
+ {"name": "machine", "id": 1481, "trainId": 173},
213
+ {"name": "pier, wharf, wharfage, dock", "id": 1858, "trainId": 174},
214
+ {"name": "fan", "id": 894, "trainId": 175},
215
+ {"name": "inflatable bounce game", "id": 1322, "trainId": 176},
216
+ {"name": "pitch", "id": 1891, "trainId": 177},
217
+ {"name": "paper", "id": 1756, "trainId": 178},
218
+ {"name": "arcade, colonnade", "id": 49, "trainId": 179},
219
+ {"name": "hot tub", "id": 1272, "trainId": 180},
220
+ {"name": "helicopter", "id": 1229, "trainId": 181},
221
+ {"name": "tray", "id": 2850, "trainId": 182},
222
+ {"name": "partition, divider", "id": 1784, "trainId": 183},
223
+ {"name": "vineyard", "id": 2962, "trainId": 184},
224
+ {"name": "bowl", "id": 259, "trainId": 185},
225
+ {"name": "bullring", "id": 319, "trainId": 186},
226
+ {"name": "flag", "id": 954, "trainId": 187},
227
+ {"name": "pot", "id": 1974, "trainId": 188},
228
+ {"name": "footbridge, overcrossing, pedestrian bridge", "id": 1013, "trainId": 189},
229
+ {"name": "shower", "id": 2356, "trainId": 190},
230
+ {"name": "bag, traveling bag, travelling bag, grip, suitcase", "id": 97, "trainId": 191},
231
+ {"name": "bulletin board, notice board", "id": 318, "trainId": 192},
232
+ {"name": "confessional booth", "id": 592, "trainId": 193},
233
+ {"name": "trunk, tree trunk, bole", "id": 2885, "trainId": 194},
234
+ {"name": "forest", "id": 1017, "trainId": 195},
235
+ {"name": "elevator door", "id": 851, "trainId": 196},
236
+ {"name": "laptop, laptop computer", "id": 1407, "trainId": 197},
237
+ {"name": "instrument panel", "id": 1332, "trainId": 198},
238
+ {"name": "bucket, pail", "id": 303, "trainId": 199},
239
+ {"name": "tapestry, tapis", "id": 2714, "trainId": 200},
240
+ {"name": "platform", "id": 1924, "trainId": 201},
241
+ {"name": "jacket", "id": 1346, "trainId": 202},
242
+ {"name": "gate", "id": 1081, "trainId": 203},
243
+ {"name": "monitor, monitoring device", "id": 1583, "trainId": 204},
244
+ {
245
+ "name": "telephone booth, phone booth, call box, telephone box, telephone kiosk",
246
+ "id": 2727,
247
+ "trainId": 205,
248
+ },
249
+ {"name": "spotlight, spot", "id": 2509, "trainId": 206},
250
+ {"name": "ring", "id": 2123, "trainId": 207},
251
+ {"name": "control panel", "id": 602, "trainId": 208},
252
+ {"name": "blackboard, chalkboard", "id": 202, "trainId": 209},
253
+ {"name": "air conditioner, air conditioning", "id": 10, "trainId": 210},
254
+ {"name": "chest", "id": 490, "trainId": 211},
255
+ {"name": "clock", "id": 530, "trainId": 212},
256
+ {"name": "sand dune", "id": 2213, "trainId": 213},
257
+ {"name": "pipe, pipage, piping", "id": 1884, "trainId": 214},
258
+ {"name": "vault", "id": 2934, "trainId": 215},
259
+ {"name": "table football", "id": 2687, "trainId": 216},
260
+ {"name": "cannon", "id": 387, "trainId": 217},
261
+ {"name": "swimming pool, swimming bath, natatorium", "id": 2668, "trainId": 218},
262
+ {"name": "fluorescent, fluorescent fixture", "id": 982, "trainId": 219},
263
+ {"name": "statue", "id": 2547, "trainId": 220},
264
+ {
265
+ "name": "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system",
266
+ "id": 1474,
267
+ "trainId": 221,
268
+ },
269
+ {"name": "exhibitor", "id": 877, "trainId": 222},
270
+ {"name": "ladder", "id": 1391, "trainId": 223},
271
+ {"name": "carport", "id": 414, "trainId": 224},
272
+ {"name": "dam", "id": 698, "trainId": 225},
273
+ {"name": "pulpit", "id": 2019, "trainId": 226},
274
+ {"name": "skylight, fanlight", "id": 2422, "trainId": 227},
275
+ {"name": "water tower", "id": 3010, "trainId": 228},
276
+ {"name": "grill, grille, grillwork", "id": 1139, "trainId": 229},
277
+ {"name": "display board", "id": 753, "trainId": 230},
278
+ {"name": "pane, pane of glass, window glass", "id": 1747, "trainId": 231},
279
+ {"name": "rubbish, trash, scrap", "id": 2175, "trainId": 232},
280
+ {"name": "ice rink", "id": 1301, "trainId": 233},
281
+ {"name": "fruit", "id": 1033, "trainId": 234},
282
+ {"name": "patio", "id": 1789, "trainId": 235},
283
+ {"name": "vending machine", "id": 2939, "trainId": 236},
284
+ {"name": "telephone, phone, telephone set", "id": 2730, "trainId": 237},
285
+ {"name": "net", "id": 1652, "trainId": 238},
286
+ {
287
+ "name": "backpack, back pack, knapsack, packsack, rucksack, haversack",
288
+ "id": 90,
289
+ "trainId": 239,
290
+ },
291
+ {"name": "jar", "id": 1349, "trainId": 240},
292
+ {"name": "track", "id": 2830, "trainId": 241},
293
+ {"name": "magazine", "id": 1485, "trainId": 242},
294
+ {"name": "shutter", "id": 2370, "trainId": 243},
295
+ {"name": "roof", "id": 2155, "trainId": 244},
296
+ {"name": "banner, streamer", "id": 118, "trainId": 245},
297
+ {"name": "landfill", "id": 1402, "trainId": 246},
298
+ {"name": "post", "id": 1957, "trainId": 247},
299
+ {"name": "altarpiece, reredos", "id": 3130, "trainId": 248},
300
+ {"name": "hat, chapeau, lid", "id": 1197, "trainId": 249},
301
+ {"name": "arch, archway", "id": 52, "trainId": 250},
302
+ {"name": "table game", "id": 2688, "trainId": 251},
303
+ {"name": "bag, handbag, pocketbook, purse", "id": 96, "trainId": 252},
304
+ {"name": "document, written document, papers", "id": 762, "trainId": 253},
305
+ {"name": "dome", "id": 772, "trainId": 254},
306
+ {"name": "pier", "id": 1857, "trainId": 255},
307
+ {"name": "shanties", "id": 2315, "trainId": 256},
308
+ {"name": "forecourt", "id": 1016, "trainId": 257},
309
+ {"name": "crane", "id": 643, "trainId": 258},
310
+ {"name": "dog, domestic dog, canis familiaris", "id": 3105, "trainId": 259},
311
+ {"name": "piano, pianoforte, forte-piano", "id": 1849, "trainId": 260},
312
+ {"name": "drawing", "id": 791, "trainId": 261},
313
+ {"name": "cabin", "id": 349, "trainId": 262},
314
+ {
315
+ "name": "ad, advertisement, advertizement, advertising, advertizing, advert",
316
+ "id": 6,
317
+ "trainId": 263,
318
+ },
319
+ {"name": "amphitheater, amphitheatre, coliseum", "id": 3114, "trainId": 264},
320
+ {"name": "monument", "id": 1587, "trainId": 265},
321
+ {"name": "henhouse", "id": 1233, "trainId": 266},
322
+ {"name": "cockpit", "id": 559, "trainId": 267},
323
+ {"name": "heater, warmer", "id": 1223, "trainId": 268},
324
+ {"name": "windmill, aerogenerator, wind generator", "id": 3049, "trainId": 269},
325
+ {"name": "pool", "id": 1943, "trainId": 270},
326
+ {"name": "elevator, lift", "id": 853, "trainId": 271},
327
+ {"name": "decoration, ornament, ornamentation", "id": 709, "trainId": 272},
328
+ {"name": "labyrinth", "id": 1390, "trainId": 273},
329
+ {"name": "text, textual matter", "id": 2748, "trainId": 274},
330
+ {"name": "printer", "id": 2007, "trainId": 275},
331
+ {"name": "mezzanine, first balcony", "id": 1546, "trainId": 276},
332
+ {"name": "mattress", "id": 1513, "trainId": 277},
333
+ {"name": "straw", "id": 2600, "trainId": 278},
334
+ {"name": "stalls", "id": 2538, "trainId": 279},
335
+ {"name": "patio, terrace", "id": 1790, "trainId": 280},
336
+ {"name": "billboard, hoarding", "id": 194, "trainId": 281},
337
+ {"name": "bus stop", "id": 326, "trainId": 282},
338
+ {"name": "trouser, pant", "id": 2877, "trainId": 283},
339
+ {"name": "console table, console", "id": 594, "trainId": 284},
340
+ {"name": "rack", "id": 2036, "trainId": 285},
341
+ {"name": "notebook", "id": 1662, "trainId": 286},
342
+ {"name": "shrine", "id": 2366, "trainId": 287},
343
+ {"name": "pantry", "id": 1754, "trainId": 288},
344
+ {"name": "cart", "id": 418, "trainId": 289},
345
+ {"name": "steam shovel", "id": 2553, "trainId": 290},
346
+ {"name": "porch", "id": 1951, "trainId": 291},
347
+ {"name": "postbox, mailbox, letter box", "id": 1963, "trainId": 292},
348
+ {"name": "figurine, statuette", "id": 918, "trainId": 293},
349
+ {"name": "recycling bin", "id": 2086, "trainId": 294},
350
+ {"name": "folding screen", "id": 997, "trainId": 295},
351
+ {"name": "telescope", "id": 2731, "trainId": 296},
352
+ {"name": "deck chair, beach chair", "id": 704, "trainId": 297},
353
+ {"name": "kennel", "id": 1365, "trainId": 298},
354
+ {"name": "coffee maker", "id": 569, "trainId": 299},
355
+ {"name": "altar, communion table, lord's table", "id": 3108, "trainId": 300},
356
+ {"name": "fish", "id": 948, "trainId": 301},
357
+ {"name": "easel", "id": 839, "trainId": 302},
358
+ {"name": "artificial golf green", "id": 63, "trainId": 303},
359
+ {"name": "iceberg", "id": 1305, "trainId": 304},
360
+ {"name": "candlestick, candle holder", "id": 378, "trainId": 305},
361
+ {"name": "shower stall, shower bath", "id": 2362, "trainId": 306},
362
+ {"name": "television stand", "id": 2734, "trainId": 307},
363
+ {
364
+ "name": "wall socket, wall plug, electric outlet, electrical outlet, outlet, electric receptacle",
365
+ "id": 2982,
366
+ "trainId": 308,
367
+ },
368
+ {"name": "skeleton", "id": 2398, "trainId": 309},
369
+ {"name": "grand piano, grand", "id": 1119, "trainId": 310},
370
+ {"name": "candy, confect", "id": 382, "trainId": 311},
371
+ {"name": "grille door", "id": 1141, "trainId": 312},
372
+ {"name": "pedestal, plinth, footstall", "id": 1805, "trainId": 313},
373
+ {"name": "jersey, t-shirt, tee shirt", "id": 3102, "trainId": 314},
374
+ {"name": "shoe", "id": 2341, "trainId": 315},
375
+ {"name": "gravestone, headstone, tombstone", "id": 1131, "trainId": 316},
376
+ {"name": "shanty", "id": 2316, "trainId": 317},
377
+ {"name": "structure", "id": 2626, "trainId": 318},
378
+ {"name": "rocking chair, rocker", "id": 3104, "trainId": 319},
379
+ {"name": "bird", "id": 198, "trainId": 320},
380
+ {"name": "place mat", "id": 1896, "trainId": 321},
381
+ {"name": "tomb", "id": 2800, "trainId": 322},
382
+ {"name": "big top", "id": 190, "trainId": 323},
383
+ {"name": "gas pump, gasoline pump, petrol pump, island dispenser", "id": 3131, "trainId": 324},
384
+ {"name": "lockers", "id": 1463, "trainId": 325},
385
+ {"name": "cage", "id": 357, "trainId": 326},
386
+ {"name": "finger", "id": 929, "trainId": 327},
387
+ {"name": "bleachers", "id": 209, "trainId": 328},
388
+ {"name": "ferris wheel", "id": 912, "trainId": 329},
389
+ {"name": "hairdresser chair", "id": 1164, "trainId": 330},
390
+ {"name": "mat", "id": 1509, "trainId": 331},
391
+ {"name": "stands", "id": 2539, "trainId": 332},
392
+ {"name": "aquarium, fish tank, marine museum", "id": 3116, "trainId": 333},
393
+ {"name": "streetcar, tram, tramcar, trolley, trolley car", "id": 2615, "trainId": 334},
394
+ {"name": "napkin, table napkin, serviette", "id": 1644, "trainId": 335},
395
+ {"name": "dummy", "id": 818, "trainId": 336},
396
+ {"name": "booklet, brochure, folder, leaflet, pamphlet", "id": 242, "trainId": 337},
397
+ {"name": "sand trap", "id": 2217, "trainId": 338},
398
+ {"name": "shop, store", "id": 2347, "trainId": 339},
399
+ {"name": "table cloth", "id": 2686, "trainId": 340},
400
+ {"name": "service station", "id": 2300, "trainId": 341},
401
+ {"name": "coffin", "id": 572, "trainId": 342},
402
+ {"name": "drawer", "id": 789, "trainId": 343},
403
+ {"name": "cages", "id": 358, "trainId": 344},
404
+ {"name": "slot machine, coin machine", "id": 2443, "trainId": 345},
405
+ {"name": "balcony", "id": 101, "trainId": 346},
406
+ {"name": "volleyball court", "id": 2969, "trainId": 347},
407
+ {"name": "table tennis", "id": 2692, "trainId": 348},
408
+ {"name": "control table", "id": 606, "trainId": 349},
409
+ {"name": "shirt", "id": 2339, "trainId": 350},
410
+ {"name": "merchandise, ware, product", "id": 1533, "trainId": 351},
411
+ {"name": "railway", "id": 2060, "trainId": 352},
412
+ {"name": "parterre", "id": 1782, "trainId": 353},
413
+ {"name": "chimney", "id": 495, "trainId": 354},
414
+ {"name": "can, tin, tin can", "id": 371, "trainId": 355},
415
+ {"name": "tanks", "id": 2707, "trainId": 356},
416
+ {"name": "fabric, cloth, material, textile", "id": 889, "trainId": 357},
417
+ {"name": "alga, algae", "id": 3156, "trainId": 358},
418
+ {"name": "system", "id": 2683, "trainId": 359},
419
+ {"name": "map", "id": 1499, "trainId": 360},
420
+ {"name": "greenhouse", "id": 1135, "trainId": 361},
421
+ {"name": "mug", "id": 1619, "trainId": 362},
422
+ {"name": "barbecue", "id": 125, "trainId": 363},
423
+ {"name": "trailer", "id": 2838, "trainId": 364},
424
+ {"name": "toilet tissue, toilet paper, bathroom tissue", "id": 2792, "trainId": 365},
425
+ {"name": "organ", "id": 1695, "trainId": 366},
426
+ {"name": "dishrag, dishcloth", "id": 746, "trainId": 367},
427
+ {"name": "island", "id": 1343, "trainId": 368},
428
+ {"name": "keyboard", "id": 1370, "trainId": 369},
429
+ {"name": "trench", "id": 2858, "trainId": 370},
430
+ {"name": "basket, basketball hoop, hoop", "id": 145, "trainId": 371},
431
+ {"name": "steering wheel, wheel", "id": 2565, "trainId": 372},
432
+ {"name": "pitcher, ewer", "id": 1892, "trainId": 373},
433
+ {"name": "goal", "id": 1103, "trainId": 374},
434
+ {"name": "bread, breadstuff, staff of life", "id": 286, "trainId": 375},
435
+ {"name": "beds", "id": 170, "trainId": 376},
436
+ {"name": "wood", "id": 3073, "trainId": 377},
437
+ {"name": "file cabinet", "id": 922, "trainId": 378},
438
+ {"name": "newspaper, paper", "id": 1655, "trainId": 379},
439
+ {"name": "motorboat", "id": 1602, "trainId": 380},
440
+ {"name": "rope", "id": 2160, "trainId": 381},
441
+ {"name": "guitar", "id": 1151, "trainId": 382},
442
+ {"name": "rubble", "id": 2176, "trainId": 383},
443
+ {"name": "scarf", "id": 2239, "trainId": 384},
444
+ {"name": "barrels", "id": 132, "trainId": 385},
445
+ {"name": "cap", "id": 394, "trainId": 386},
446
+ {"name": "leaves", "id": 1424, "trainId": 387},
447
+ {"name": "control tower", "id": 607, "trainId": 388},
448
+ {"name": "dashboard", "id": 700, "trainId": 389},
449
+ {"name": "bandstand", "id": 116, "trainId": 390},
450
+ {"name": "lectern", "id": 1425, "trainId": 391},
451
+ {"name": "switch, electric switch, electrical switch", "id": 2676, "trainId": 392},
452
+ {"name": "baseboard, mopboard, skirting board", "id": 141, "trainId": 393},
453
+ {"name": "shower room", "id": 2360, "trainId": 394},
454
+ {"name": "smoke", "id": 2449, "trainId": 395},
455
+ {"name": "faucet, spigot", "id": 897, "trainId": 396},
456
+ {"name": "bulldozer", "id": 317, "trainId": 397},
457
+ {"name": "saucepan", "id": 2228, "trainId": 398},
458
+ {"name": "shops", "id": 2351, "trainId": 399},
459
+ {"name": "meter", "id": 1543, "trainId": 400},
460
+ {"name": "crevasse", "id": 656, "trainId": 401},
461
+ {"name": "gear", "id": 1088, "trainId": 402},
462
+ {"name": "candelabrum, candelabra", "id": 373, "trainId": 403},
463
+ {"name": "sofa bed", "id": 2472, "trainId": 404},
464
+ {"name": "tunnel", "id": 2892, "trainId": 405},
465
+ {"name": "pallet", "id": 1740, "trainId": 406},
466
+ {"name": "wire, conducting wire", "id": 3067, "trainId": 407},
467
+ {"name": "kettle, boiler", "id": 1367, "trainId": 408},
468
+ {"name": "bidet", "id": 188, "trainId": 409},
469
+ {
470
+ "name": "baby buggy, baby carriage, carriage, perambulator, pram, stroller, go-cart, pushchair, pusher",
471
+ "id": 79,
472
+ "trainId": 410,
473
+ },
474
+ {"name": "music stand", "id": 1633, "trainId": 411},
475
+ {"name": "pipe, tube", "id": 1885, "trainId": 412},
476
+ {"name": "cup", "id": 677, "trainId": 413},
477
+ {"name": "parking meter", "id": 1779, "trainId": 414},
478
+ {"name": "ice hockey rink", "id": 1297, "trainId": 415},
479
+ {"name": "shelter", "id": 2334, "trainId": 416},
480
+ {"name": "weeds", "id": 3027, "trainId": 417},
481
+ {"name": "temple", "id": 2735, "trainId": 418},
482
+ {"name": "patty, cake", "id": 1791, "trainId": 419},
483
+ {"name": "ski slope", "id": 2405, "trainId": 420},
484
+ {"name": "panel", "id": 1748, "trainId": 421},
485
+ {"name": "wallet", "id": 2983, "trainId": 422},
486
+ {"name": "wheel", "id": 3035, "trainId": 423},
487
+ {"name": "towel rack, towel horse", "id": 2824, "trainId": 424},
488
+ {"name": "roundabout", "id": 2168, "trainId": 425},
489
+ {"name": "canister, cannister, tin", "id": 385, "trainId": 426},
490
+ {"name": "rod", "id": 2148, "trainId": 427},
491
+ {"name": "soap dispenser", "id": 2465, "trainId": 428},
492
+ {"name": "bell", "id": 175, "trainId": 429},
493
+ {"name": "canvas", "id": 390, "trainId": 430},
494
+ {"name": "box office, ticket office, ticket booth", "id": 268, "trainId": 431},
495
+ {"name": "teacup", "id": 2722, "trainId": 432},
496
+ {"name": "trellis", "id": 2857, "trainId": 433},
497
+ {"name": "workbench", "id": 3088, "trainId": 434},
498
+ {"name": "valley, vale", "id": 2926, "trainId": 435},
499
+ {"name": "toaster", "id": 2782, "trainId": 436},
500
+ {"name": "knife", "id": 1378, "trainId": 437},
501
+ {"name": "podium", "id": 1934, "trainId": 438},
502
+ {"name": "ramp", "id": 2072, "trainId": 439},
503
+ {"name": "tumble dryer", "id": 2889, "trainId": 440},
504
+ {"name": "fireplug, fire hydrant, plug", "id": 944, "trainId": 441},
505
+ {"name": "gym shoe, sneaker, tennis shoe", "id": 1158, "trainId": 442},
506
+ {"name": "lab bench", "id": 1383, "trainId": 443},
507
+ {"name": "equipment", "id": 867, "trainId": 444},
508
+ {"name": "rocky formation", "id": 2145, "trainId": 445},
509
+ {"name": "plastic", "id": 1915, "trainId": 446},
510
+ {"name": "calendar", "id": 361, "trainId": 447},
511
+ {"name": "caravan", "id": 402, "trainId": 448},
512
+ {"name": "check-in-desk", "id": 482, "trainId": 449},
513
+ {"name": "ticket counter", "id": 2761, "trainId": 450},
514
+ {"name": "brush", "id": 300, "trainId": 451},
515
+ {"name": "mill", "id": 1554, "trainId": 452},
516
+ {"name": "covered bridge", "id": 636, "trainId": 453},
517
+ {"name": "bowling alley", "id": 260, "trainId": 454},
518
+ {"name": "hanger", "id": 1186, "trainId": 455},
519
+ {"name": "excavator", "id": 871, "trainId": 456},
520
+ {"name": "trestle", "id": 2859, "trainId": 457},
521
+ {"name": "revolving door", "id": 2103, "trainId": 458},
522
+ {"name": "blast furnace", "id": 208, "trainId": 459},
523
+ {"name": "scale, weighing machine", "id": 2236, "trainId": 460},
524
+ {"name": "projector", "id": 2012, "trainId": 461},
525
+ {"name": "soap", "id": 2462, "trainId": 462},
526
+ {"name": "locker", "id": 1462, "trainId": 463},
527
+ {"name": "tractor", "id": 2832, "trainId": 464},
528
+ {"name": "stretcher", "id": 2617, "trainId": 465},
529
+ {"name": "frame", "id": 1024, "trainId": 466},
530
+ {"name": "grating", "id": 1129, "trainId": 467},
531
+ {"name": "alembic", "id": 18, "trainId": 468},
532
+ {"name": "candle, taper, wax light", "id": 376, "trainId": 469},
533
+ {"name": "barrier", "id": 134, "trainId": 470},
534
+ {"name": "cardboard", "id": 407, "trainId": 471},
535
+ {"name": "cave", "id": 434, "trainId": 472},
536
+ {"name": "puddle", "id": 2017, "trainId": 473},
537
+ {"name": "tarp", "id": 2717, "trainId": 474},
538
+ {"name": "price tag", "id": 2005, "trainId": 475},
539
+ {"name": "watchtower", "id": 2993, "trainId": 476},
540
+ {"name": "meters", "id": 1545, "trainId": 477},
541
+ {
542
+ "name": "light bulb, lightbulb, bulb, incandescent lamp, electric light, electric-light bulb",
543
+ "id": 1445,
544
+ "trainId": 478,
545
+ },
546
+ {"name": "tracks", "id": 2831, "trainId": 479},
547
+ {"name": "hair dryer", "id": 1161, "trainId": 480},
548
+ {"name": "skirt", "id": 2411, "trainId": 481},
549
+ {"name": "viaduct", "id": 2949, "trainId": 482},
550
+ {"name": "paper towel", "id": 1769, "trainId": 483},
551
+ {"name": "coat", "id": 552, "trainId": 484},
552
+ {"name": "sheet", "id": 2327, "trainId": 485},
553
+ {"name": "fire extinguisher, extinguisher, asphyxiator", "id": 939, "trainId": 486},
554
+ {"name": "water wheel", "id": 3013, "trainId": 487},
555
+ {"name": "pottery, clayware", "id": 1986, "trainId": 488},
556
+ {"name": "magazine rack", "id": 1486, "trainId": 489},
557
+ {"name": "teapot", "id": 2723, "trainId": 490},
558
+ {"name": "microphone, mike", "id": 1549, "trainId": 491},
559
+ {"name": "support", "id": 2649, "trainId": 492},
560
+ {"name": "forklift", "id": 1020, "trainId": 493},
561
+ {"name": "canyon", "id": 392, "trainId": 494},
562
+ {"name": "cash register, register", "id": 422, "trainId": 495},
563
+ {"name": "leaf, leafage, foliage", "id": 1419, "trainId": 496},
564
+ {"name": "remote control, remote", "id": 2099, "trainId": 497},
565
+ {"name": "soap dish", "id": 2464, "trainId": 498},
566
+ {"name": "windshield, windscreen", "id": 3058, "trainId": 499},
567
+ {"name": "cat", "id": 430, "trainId": 500},
568
+ {"name": "cue, cue stick, pool cue, pool stick", "id": 675, "trainId": 501},
569
+ {"name": "vent, venthole, vent-hole, blowhole", "id": 2941, "trainId": 502},
570
+ {"name": "videos", "id": 2955, "trainId": 503},
571
+ {"name": "shovel", "id": 2355, "trainId": 504},
572
+ {"name": "eaves", "id": 840, "trainId": 505},
573
+ {"name": "antenna, aerial, transmitting aerial", "id": 32, "trainId": 506},
574
+ {"name": "shipyard", "id": 2338, "trainId": 507},
575
+ {"name": "hen, biddy", "id": 1232, "trainId": 508},
576
+ {"name": "traffic cone", "id": 2834, "trainId": 509},
577
+ {"name": "washing machines", "id": 2991, "trainId": 510},
578
+ {"name": "truck crane", "id": 2879, "trainId": 511},
579
+ {"name": "cds", "id": 444, "trainId": 512},
580
+ {"name": "niche", "id": 1657, "trainId": 513},
581
+ {"name": "scoreboard", "id": 2246, "trainId": 514},
582
+ {"name": "briefcase", "id": 296, "trainId": 515},
583
+ {"name": "boot", "id": 245, "trainId": 516},
584
+ {"name": "sweater, jumper", "id": 2661, "trainId": 517},
585
+ {"name": "hay", "id": 1202, "trainId": 518},
586
+ {"name": "pack", "id": 1714, "trainId": 519},
587
+ {"name": "bottle rack", "id": 251, "trainId": 520},
588
+ {"name": "glacier", "id": 1095, "trainId": 521},
589
+ {"name": "pergola", "id": 1828, "trainId": 522},
590
+ {"name": "building materials", "id": 311, "trainId": 523},
591
+ {"name": "television camera", "id": 2732, "trainId": 524},
592
+ {"name": "first floor", "id": 947, "trainId": 525},
593
+ {"name": "rifle", "id": 2115, "trainId": 526},
594
+ {"name": "tennis table", "id": 2738, "trainId": 527},
595
+ {"name": "stadium", "id": 2525, "trainId": 528},
596
+ {"name": "safety belt", "id": 2194, "trainId": 529},
597
+ {"name": "cover", "id": 634, "trainId": 530},
598
+ {"name": "dish rack", "id": 740, "trainId": 531},
599
+ {"name": "synthesizer", "id": 2682, "trainId": 532},
600
+ {"name": "pumpkin", "id": 2020, "trainId": 533},
601
+ {"name": "gutter", "id": 1156, "trainId": 534},
602
+ {"name": "fruit stand", "id": 1036, "trainId": 535},
603
+ {"name": "ice floe, floe", "id": 1295, "trainId": 536},
604
+ {"name": "handle, grip, handgrip, hold", "id": 1181, "trainId": 537},
605
+ {"name": "wheelchair", "id": 3037, "trainId": 538},
606
+ {"name": "mousepad, mouse mat", "id": 1614, "trainId": 539},
607
+ {"name": "diploma", "id": 736, "trainId": 540},
608
+ {"name": "fairground ride", "id": 893, "trainId": 541},
609
+ {"name": "radio", "id": 2047, "trainId": 542},
610
+ {"name": "hotplate", "id": 1274, "trainId": 543},
611
+ {"name": "junk", "id": 1361, "trainId": 544},
612
+ {"name": "wheelbarrow", "id": 3036, "trainId": 545},
613
+ {"name": "stream", "id": 2606, "trainId": 546},
614
+ {"name": "toll plaza", "id": 2797, "trainId": 547},
615
+ {"name": "punching bag", "id": 2022, "trainId": 548},
616
+ {"name": "trough", "id": 2876, "trainId": 549},
617
+ {"name": "throne", "id": 2758, "trainId": 550},
618
+ {"name": "chair desk", "id": 472, "trainId": 551},
619
+ {"name": "weighbridge", "id": 3028, "trainId": 552},
620
+ {"name": "extractor fan", "id": 882, "trainId": 553},
621
+ {"name": "hanging clothes", "id": 1189, "trainId": 554},
622
+ {"name": "dish, dish aerial, dish antenna, saucer", "id": 743, "trainId": 555},
623
+ {"name": "alarm clock, alarm", "id": 3122, "trainId": 556},
624
+ {"name": "ski lift", "id": 2401, "trainId": 557},
625
+ {"name": "chain", "id": 468, "trainId": 558},
626
+ {"name": "garage", "id": 1061, "trainId": 559},
627
+ {"name": "mechanical shovel", "id": 1523, "trainId": 560},
628
+ {"name": "wine rack", "id": 3059, "trainId": 561},
629
+ {"name": "tramway", "id": 2843, "trainId": 562},
630
+ {"name": "treadmill", "id": 2853, "trainId": 563},
631
+ {"name": "menu", "id": 1529, "trainId": 564},
632
+ {"name": "block", "id": 214, "trainId": 565},
633
+ {"name": "well", "id": 3032, "trainId": 566},
634
+ {"name": "witness stand", "id": 3071, "trainId": 567},
635
+ {"name": "branch", "id": 277, "trainId": 568},
636
+ {"name": "duck", "id": 813, "trainId": 569},
637
+ {"name": "casserole", "id": 426, "trainId": 570},
638
+ {"name": "frying pan", "id": 1039, "trainId": 571},
639
+ {"name": "desk organizer", "id": 727, "trainId": 572},
640
+ {"name": "mast", "id": 1508, "trainId": 573},
641
+ {"name": "spectacles, specs, eyeglasses, glasses", "id": 2490, "trainId": 574},
642
+ {"name": "service elevator", "id": 2299, "trainId": 575},
643
+ {"name": "dollhouse", "id": 768, "trainId": 576},
644
+ {"name": "hammock", "id": 1172, "trainId": 577},
645
+ {"name": "clothes hanging", "id": 537, "trainId": 578},
646
+ {"name": "photocopier", "id": 1847, "trainId": 579},
647
+ {"name": "notepad", "id": 1664, "trainId": 580},
648
+ {"name": "golf cart", "id": 1110, "trainId": 581},
649
+ {"name": "footpath", "id": 1014, "trainId": 582},
650
+ {"name": "cross", "id": 662, "trainId": 583},
651
+ {"name": "baptismal font", "id": 121, "trainId": 584},
652
+ {"name": "boiler", "id": 227, "trainId": 585},
653
+ {"name": "skip", "id": 2410, "trainId": 586},
654
+ {"name": "rotisserie", "id": 2165, "trainId": 587},
655
+ {"name": "tables", "id": 2696, "trainId": 588},
656
+ {"name": "water mill", "id": 3005, "trainId": 589},
657
+ {"name": "helmet", "id": 1231, "trainId": 590},
658
+ {"name": "cover curtain", "id": 635, "trainId": 591},
659
+ {"name": "brick", "id": 292, "trainId": 592},
660
+ {"name": "table runner", "id": 2690, "trainId": 593},
661
+ {"name": "ashtray", "id": 65, "trainId": 594},
662
+ {"name": "street box", "id": 2607, "trainId": 595},
663
+ {"name": "stick", "id": 2574, "trainId": 596},
664
+ {"name": "hangers", "id": 1188, "trainId": 597},
665
+ {"name": "cells", "id": 456, "trainId": 598},
666
+ {"name": "urinal", "id": 2913, "trainId": 599},
667
+ {"name": "centerpiece", "id": 459, "trainId": 600},
668
+ {"name": "portable fridge", "id": 1955, "trainId": 601},
669
+ {"name": "dvds", "id": 827, "trainId": 602},
670
+ {"name": "golf club", "id": 1111, "trainId": 603},
671
+ {"name": "skirting board", "id": 2412, "trainId": 604},
672
+ {"name": "water cooler", "id": 2997, "trainId": 605},
673
+ {"name": "clipboard", "id": 528, "trainId": 606},
674
+ {"name": "camera, photographic camera", "id": 366, "trainId": 607},
675
+ {"name": "pigeonhole", "id": 1863, "trainId": 608},
676
+ {"name": "chips", "id": 500, "trainId": 609},
677
+ {"name": "food processor", "id": 1001, "trainId": 610},
678
+ {"name": "post box", "id": 1958, "trainId": 611},
679
+ {"name": "lid", "id": 1441, "trainId": 612},
680
+ {"name": "drum", "id": 809, "trainId": 613},
681
+ {"name": "blender", "id": 210, "trainId": 614},
682
+ {"name": "cave entrance", "id": 435, "trainId": 615},
683
+ {"name": "dental chair", "id": 718, "trainId": 616},
684
+ {"name": "obelisk", "id": 1674, "trainId": 617},
685
+ {"name": "canoe", "id": 388, "trainId": 618},
686
+ {"name": "mobile", "id": 1572, "trainId": 619},
687
+ {"name": "monitors", "id": 1584, "trainId": 620},
688
+ {"name": "pool ball", "id": 1944, "trainId": 621},
689
+ {"name": "cue rack", "id": 674, "trainId": 622},
690
+ {"name": "baggage carts", "id": 99, "trainId": 623},
691
+ {"name": "shore", "id": 2352, "trainId": 624},
692
+ {"name": "fork", "id": 1019, "trainId": 625},
693
+ {"name": "paper filer", "id": 1763, "trainId": 626},
694
+ {"name": "bicycle rack", "id": 185, "trainId": 627},
695
+ {"name": "coat rack", "id": 554, "trainId": 628},
696
+ {"name": "garland", "id": 1066, "trainId": 629},
697
+ {"name": "sports bag", "id": 2508, "trainId": 630},
698
+ {"name": "fish tank", "id": 951, "trainId": 631},
699
+ {"name": "towel dispenser", "id": 2822, "trainId": 632},
700
+ {"name": "carriage", "id": 415, "trainId": 633},
701
+ {"name": "brochure", "id": 297, "trainId": 634},
702
+ {"name": "plaque", "id": 1914, "trainId": 635},
703
+ {"name": "stringer", "id": 2619, "trainId": 636},
704
+ {"name": "iron", "id": 1338, "trainId": 637},
705
+ {"name": "spoon", "id": 2505, "trainId": 638},
706
+ {"name": "flag pole", "id": 955, "trainId": 639},
707
+ {"name": "toilet brush", "id": 2786, "trainId": 640},
708
+ {"name": "book stand", "id": 238, "trainId": 641},
709
+ {"name": "water faucet, water tap, tap, hydrant", "id": 3000, "trainId": 642},
710
+ {"name": "ticket office", "id": 2763, "trainId": 643},
711
+ {"name": "broom", "id": 299, "trainId": 644},
712
+ {"name": "dvd", "id": 822, "trainId": 645},
713
+ {"name": "ice bucket", "id": 1288, "trainId": 646},
714
+ {"name": "carapace, shell, cuticle, shield", "id": 3101, "trainId": 647},
715
+ {"name": "tureen", "id": 2894, "trainId": 648},
716
+ {"name": "folders", "id": 992, "trainId": 649},
717
+ {"name": "chess", "id": 489, "trainId": 650},
718
+ {"name": "root", "id": 2157, "trainId": 651},
719
+ {"name": "sewing machine", "id": 2309, "trainId": 652},
720
+ {"name": "model", "id": 1576, "trainId": 653},
721
+ {"name": "pen", "id": 1810, "trainId": 654},
722
+ {"name": "violin", "id": 2964, "trainId": 655},
723
+ {"name": "sweatshirt", "id": 2662, "trainId": 656},
724
+ {"name": "recycling materials", "id": 2087, "trainId": 657},
725
+ {"name": "mitten", "id": 1569, "trainId": 658},
726
+ {"name": "chopping board, cutting board", "id": 503, "trainId": 659},
727
+ {"name": "mask", "id": 1505, "trainId": 660},
728
+ {"name": "log", "id": 1468, "trainId": 661},
729
+ {"name": "mouse, computer mouse", "id": 1613, "trainId": 662},
730
+ {"name": "grill", "id": 1138, "trainId": 663},
731
+ {"name": "hole", "id": 1256, "trainId": 664},
732
+ {"name": "target", "id": 2715, "trainId": 665},
733
+ {"name": "trash bag", "id": 2846, "trainId": 666},
734
+ {"name": "chalk", "id": 477, "trainId": 667},
735
+ {"name": "sticks", "id": 2576, "trainId": 668},
736
+ {"name": "balloon", "id": 108, "trainId": 669},
737
+ {"name": "score", "id": 2245, "trainId": 670},
738
+ {"name": "hair spray", "id": 1162, "trainId": 671},
739
+ {"name": "roll", "id": 2149, "trainId": 672},
740
+ {"name": "runner", "id": 2183, "trainId": 673},
741
+ {"name": "engine", "id": 858, "trainId": 674},
742
+ {"name": "inflatable glove", "id": 1324, "trainId": 675},
743
+ {"name": "games", "id": 1055, "trainId": 676},
744
+ {"name": "pallets", "id": 1741, "trainId": 677},
745
+ {"name": "baskets", "id": 149, "trainId": 678},
746
+ {"name": "coop", "id": 615, "trainId": 679},
747
+ {"name": "dvd player", "id": 825, "trainId": 680},
748
+ {"name": "rocking horse", "id": 2143, "trainId": 681},
749
+ {"name": "buckets", "id": 304, "trainId": 682},
750
+ {"name": "bread rolls", "id": 283, "trainId": 683},
751
+ {"name": "shawl", "id": 2322, "trainId": 684},
752
+ {"name": "watering can", "id": 3017, "trainId": 685},
753
+ {"name": "spotlights", "id": 2510, "trainId": 686},
754
+ {"name": "post-it", "id": 1960, "trainId": 687},
755
+ {"name": "bowls", "id": 265, "trainId": 688},
756
+ {"name": "security camera", "id": 2282, "trainId": 689},
757
+ {"name": "runner cloth", "id": 2184, "trainId": 690},
758
+ {"name": "lock", "id": 1461, "trainId": 691},
759
+ {"name": "alarm, warning device, alarm system", "id": 3113, "trainId": 692},
760
+ {"name": "side", "id": 2372, "trainId": 693},
761
+ {"name": "roulette", "id": 2166, "trainId": 694},
762
+ {"name": "bone", "id": 232, "trainId": 695},
763
+ {"name": "cutlery", "id": 693, "trainId": 696},
764
+ {"name": "pool balls", "id": 1945, "trainId": 697},
765
+ {"name": "wheels", "id": 3039, "trainId": 698},
766
+ {"name": "spice rack", "id": 2494, "trainId": 699},
767
+ {"name": "plant pots", "id": 1908, "trainId": 700},
768
+ {"name": "towel ring", "id": 2827, "trainId": 701},
769
+ {"name": "bread box", "id": 280, "trainId": 702},
770
+ {"name": "video", "id": 2950, "trainId": 703},
771
+ {"name": "funfair", "id": 1044, "trainId": 704},
772
+ {"name": "breads", "id": 288, "trainId": 705},
773
+ {"name": "tripod", "id": 2863, "trainId": 706},
774
+ {"name": "ironing board", "id": 1342, "trainId": 707},
775
+ {"name": "skimmer", "id": 2409, "trainId": 708},
776
+ {"name": "hollow", "id": 1258, "trainId": 709},
777
+ {"name": "scratching post", "id": 2249, "trainId": 710},
778
+ {"name": "tricycle", "id": 2862, "trainId": 711},
779
+ {"name": "file box", "id": 920, "trainId": 712},
780
+ {"name": "mountain pass", "id": 1607, "trainId": 713},
781
+ {"name": "tombstones", "id": 2802, "trainId": 714},
782
+ {"name": "cooker", "id": 610, "trainId": 715},
783
+ {"name": "card game, cards", "id": 3129, "trainId": 716},
784
+ {"name": "golf bag", "id": 1108, "trainId": 717},
785
+ {"name": "towel paper", "id": 2823, "trainId": 718},
786
+ {"name": "chaise lounge", "id": 476, "trainId": 719},
787
+ {"name": "sun", "id": 2641, "trainId": 720},
788
+ {"name": "toilet paper holder", "id": 2788, "trainId": 721},
789
+ {"name": "rake", "id": 2070, "trainId": 722},
790
+ {"name": "key", "id": 1368, "trainId": 723},
791
+ {"name": "umbrella stand", "id": 2903, "trainId": 724},
792
+ {"name": "dartboard", "id": 699, "trainId": 725},
793
+ {"name": "transformer", "id": 2844, "trainId": 726},
794
+ {"name": "fireplace utensils", "id": 942, "trainId": 727},
795
+ {"name": "sweatshirts", "id": 2663, "trainId": 728},
796
+ {
797
+ "name": "cellular telephone, cellular phone, cellphone, cell, mobile phone",
798
+ "id": 457,
799
+ "trainId": 729,
800
+ },
801
+ {"name": "tallboy", "id": 2701, "trainId": 730},
802
+ {"name": "stapler", "id": 2540, "trainId": 731},
803
+ {"name": "sauna", "id": 2231, "trainId": 732},
804
+ {"name": "test tube", "id": 2746, "trainId": 733},
805
+ {"name": "palette", "id": 1738, "trainId": 734},
806
+ {"name": "shopping carts", "id": 2350, "trainId": 735},
807
+ {"name": "tools", "id": 2808, "trainId": 736},
808
+ {"name": "push button, push, button", "id": 2025, "trainId": 737},
809
+ {"name": "star", "id": 2541, "trainId": 738},
810
+ {"name": "roof rack", "id": 2156, "trainId": 739},
811
+ {"name": "barbed wire", "id": 126, "trainId": 740},
812
+ {"name": "spray", "id": 2512, "trainId": 741},
813
+ {"name": "ear", "id": 831, "trainId": 742},
814
+ {"name": "sponge", "id": 2503, "trainId": 743},
815
+ {"name": "racket", "id": 2039, "trainId": 744},
816
+ {"name": "tins", "id": 2774, "trainId": 745},
817
+ {"name": "eyeglasses", "id": 886, "trainId": 746},
818
+ {"name": "file", "id": 919, "trainId": 747},
819
+ {"name": "scarfs", "id": 2240, "trainId": 748},
820
+ {"name": "sugar bowl", "id": 2636, "trainId": 749},
821
+ {"name": "flip flop", "id": 963, "trainId": 750},
822
+ {"name": "headstones", "id": 1218, "trainId": 751},
823
+ {"name": "laptop bag", "id": 1406, "trainId": 752},
824
+ {"name": "leash", "id": 1420, "trainId": 753},
825
+ {"name": "climbing frame", "id": 526, "trainId": 754},
826
+ {"name": "suit hanger", "id": 2639, "trainId": 755},
827
+ {"name": "floor spotlight", "id": 975, "trainId": 756},
828
+ {"name": "plate rack", "id": 1921, "trainId": 757},
829
+ {"name": "sewer", "id": 2305, "trainId": 758},
830
+ {"name": "hard drive", "id": 1193, "trainId": 759},
831
+ {"name": "sprinkler", "id": 2517, "trainId": 760},
832
+ {"name": "tools box", "id": 2809, "trainId": 761},
833
+ {"name": "necklace", "id": 1647, "trainId": 762},
834
+ {"name": "bulbs", "id": 314, "trainId": 763},
835
+ {"name": "steel industry", "id": 2560, "trainId": 764},
836
+ {"name": "club", "id": 545, "trainId": 765},
837
+ {"name": "jack", "id": 1345, "trainId": 766},
838
+ {"name": "door bars", "id": 775, "trainId": 767},
839
+ {
840
+ "name": "control panel, instrument panel, control board, board, panel",
841
+ "id": 603,
842
+ "trainId": 768,
843
+ },
844
+ {"name": "hairbrush", "id": 1163, "trainId": 769},
845
+ {"name": "napkin holder", "id": 1641, "trainId": 770},
846
+ {"name": "office", "id": 1678, "trainId": 771},
847
+ {"name": "smoke detector", "id": 2450, "trainId": 772},
848
+ {"name": "utensils", "id": 2915, "trainId": 773},
849
+ {"name": "apron", "id": 42, "trainId": 774},
850
+ {"name": "scissors", "id": 2242, "trainId": 775},
851
+ {"name": "terminal", "id": 2741, "trainId": 776},
852
+ {"name": "grinder", "id": 1143, "trainId": 777},
853
+ {"name": "entry phone", "id": 862, "trainId": 778},
854
+ {"name": "newspaper stand", "id": 1654, "trainId": 779},
855
+ {"name": "pepper shaker", "id": 1826, "trainId": 780},
856
+ {"name": "onions", "id": 1689, "trainId": 781},
857
+ {
858
+ "name": "central processing unit, cpu, c p u , central processor, processor, mainframe",
859
+ "id": 3124,
860
+ "trainId": 782,
861
+ },
862
+ {"name": "tape", "id": 2710, "trainId": 783},
863
+ {"name": "bat", "id": 152, "trainId": 784},
864
+ {"name": "coaster", "id": 549, "trainId": 785},
865
+ {"name": "calculator", "id": 360, "trainId": 786},
866
+ {"name": "potatoes", "id": 1982, "trainId": 787},
867
+ {"name": "luggage rack", "id": 1478, "trainId": 788},
868
+ {"name": "salt", "id": 2203, "trainId": 789},
869
+ {"name": "street number", "id": 2612, "trainId": 790},
870
+ {"name": "viewpoint", "id": 2956, "trainId": 791},
871
+ {"name": "sword", "id": 2681, "trainId": 792},
872
+ {"name": "cd", "id": 437, "trainId": 793},
873
+ {"name": "rowing machine", "id": 2171, "trainId": 794},
874
+ {"name": "plug", "id": 1933, "trainId": 795},
875
+ {"name": "andiron, firedog, dog, dog-iron", "id": 3110, "trainId": 796},
876
+ {"name": "pepper", "id": 1824, "trainId": 797},
877
+ {"name": "tongs", "id": 2803, "trainId": 798},
878
+ {"name": "bonfire", "id": 234, "trainId": 799},
879
+ {"name": "dog dish", "id": 764, "trainId": 800},
880
+ {"name": "belt", "id": 177, "trainId": 801},
881
+ {"name": "dumbbells", "id": 817, "trainId": 802},
882
+ {"name": "videocassette recorder, vcr", "id": 3145, "trainId": 803},
883
+ {"name": "hook", "id": 1262, "trainId": 804},
884
+ {"name": "envelopes", "id": 864, "trainId": 805},
885
+ {"name": "shower faucet", "id": 2359, "trainId": 806},
886
+ {"name": "watch", "id": 2992, "trainId": 807},
887
+ {"name": "padlock", "id": 1725, "trainId": 808},
888
+ {"name": "swimming pool ladder", "id": 2667, "trainId": 809},
889
+ {"name": "spanners", "id": 2484, "trainId": 810},
890
+ {"name": "gravy boat", "id": 1133, "trainId": 811},
891
+ {"name": "notice board", "id": 1667, "trainId": 812},
892
+ {"name": "trash bags", "id": 2847, "trainId": 813},
893
+ {"name": "fire alarm", "id": 932, "trainId": 814},
894
+ {"name": "ladle", "id": 1392, "trainId": 815},
895
+ {"name": "stethoscope", "id": 2573, "trainId": 816},
896
+ {"name": "rocket", "id": 2140, "trainId": 817},
897
+ {"name": "funnel", "id": 1046, "trainId": 818},
898
+ {"name": "bowling pins", "id": 264, "trainId": 819},
899
+ {"name": "valve", "id": 2927, "trainId": 820},
900
+ {"name": "thermometer", "id": 2752, "trainId": 821},
901
+ {"name": "cups", "id": 679, "trainId": 822},
902
+ {"name": "spice jar", "id": 2493, "trainId": 823},
903
+ {"name": "night light", "id": 1658, "trainId": 824},
904
+ {"name": "soaps", "id": 2466, "trainId": 825},
905
+ {"name": "games table", "id": 1057, "trainId": 826},
906
+ {"name": "slotted spoon", "id": 2444, "trainId": 827},
907
+ {"name": "reel", "id": 2093, "trainId": 828},
908
+ {"name": "scourer", "id": 2248, "trainId": 829},
909
+ {"name": "sleeping robe", "id": 2432, "trainId": 830},
910
+ {"name": "desk mat", "id": 726, "trainId": 831},
911
+ {"name": "dumbbell", "id": 816, "trainId": 832},
912
+ {"name": "hammer", "id": 1171, "trainId": 833},
913
+ {"name": "tie", "id": 2766, "trainId": 834},
914
+ {"name": "typewriter", "id": 2900, "trainId": 835},
915
+ {"name": "shaker", "id": 2313, "trainId": 836},
916
+ {"name": "cheese dish", "id": 488, "trainId": 837},
917
+ {"name": "sea star", "id": 2265, "trainId": 838},
918
+ {"name": "racquet", "id": 2043, "trainId": 839},
919
+ {"name": "butane gas cylinder", "id": 332, "trainId": 840},
920
+ {"name": "paper weight", "id": 1771, "trainId": 841},
921
+ {"name": "shaving brush", "id": 2320, "trainId": 842},
922
+ {"name": "sunglasses", "id": 2646, "trainId": 843},
923
+ {"name": "gear shift", "id": 1089, "trainId": 844},
924
+ {"name": "towel rail", "id": 2826, "trainId": 845},
925
+ {"name": "adding machine, totalizer, totaliser", "id": 3148, "trainId": 846},
926
+ ]
927
+
928
+
929
+ def loadAde20K(file):
930
+ fileseg = file.replace(".jpg", "_seg.png")
931
+ with Image.open(fileseg) as io:
932
+ seg = np.array(io)
933
+
934
+ R = seg[:, :, 0]
935
+ G = seg[:, :, 1]
936
+ ObjectClassMasks = (R / 10).astype(np.int32) * 256 + (G.astype(np.int32))
937
+
938
+ return {"img_name": file, "segm_name": fileseg, "class_mask": ObjectClassMasks}
939
+
940
+
941
+ if __name__ == "__main__":
942
+ dataset_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets"))
943
+ index_file = dataset_dir / "ade/ADE20K_2021_17_01" / "index_ade20k.pkl"
944
+ with open(index_file, "rb") as f:
945
+ index_ade20k = pkl.load(f)
946
+
947
+ id_map = {}
948
+ for cat in ADE20K_SEM_SEG_FULL_CATEGORIES:
949
+ id_map[cat["id"]] = cat["trainId"]
950
+
951
+ # make output dir
952
+ for name in ["training", "validation"]:
953
+ image_dir = dataset_dir / "ade/ADE20K_2021_17_01" / "images_detectron2" / name
954
+ image_dir.mkdir(parents=True, exist_ok=True)
955
+ annotation_dir = dataset_dir / "ade/ADE20K_2021_17_01" / "annotations_detectron2" / name
956
+ annotation_dir.mkdir(parents=True, exist_ok=True)
957
+
958
+ # process image and gt
959
+ for folder_name, file_name in tqdm.tqdm(
960
+ zip(index_ade20k["folder"], index_ade20k["filename"]),
961
+ total=len(index_ade20k["filename"]),
962
+ ):
963
+ split = "validation" if file_name.split("_")[1] == "val" else "training"
964
+ info = loadAde20K(str(dataset_dir / "ade" / folder_name / file_name))
965
+
966
+ # resize image and label
967
+ img = np.asarray(Image.open(info["img_name"]))
968
+ lab = np.asarray(info["class_mask"])
969
+
970
+ h, w = img.shape[0], img.shape[1]
971
+ max_size = 512
972
+ resize = True
973
+ if w >= h > max_size:
974
+ h_new, w_new = max_size, round(w / float(h) * max_size)
975
+ elif h >= w > max_size:
976
+ h_new, w_new = round(h / float(w) * max_size), max_size
977
+ else:
978
+ resize = False
979
+
980
+ if resize:
981
+ img = cv2.resize(img, (w_new, h_new), interpolation=cv2.INTER_LINEAR)
982
+ lab = cv2.resize(lab, (w_new, h_new), interpolation=cv2.INTER_NEAREST)
983
+
984
+ assert img.dtype == np.uint8
985
+ assert lab.dtype == np.int32
986
+
987
+ # apply label conversion and save into uint16 images
988
+ output = np.zeros_like(lab, dtype=np.uint16) + 65535
989
+ for obj_id in np.unique(lab):
990
+ if obj_id in id_map:
991
+ output[lab == obj_id] = id_map[obj_id]
992
+
993
+ output_img = dataset_dir / "ade/ADE20K_2021_17_01" / "images_detectron2" / split / file_name
994
+ output_lab = (
995
+ dataset_dir
996
+ / "ade/ADE20K_2021_17_01"
997
+ / "annotations_detectron2"
998
+ / split
999
+ / file_name.replace(".jpg", ".tif")
1000
+ )
1001
+ Image.fromarray(img).save(output_img)
1002
+
1003
+ assert output.dtype == np.uint16
1004
+ Image.fromarray(output).save(output_lab)
datasets/prepare_ade20k_ins_seg.py ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ # -*- coding: utf-8 -*-
3
+ import glob
4
+ import json
5
+ import os
6
+ from collections import Counter
7
+
8
+ import numpy as np
9
+ import tqdm
10
+ from panopticapi.utils import IdGenerator, save_json
11
+ from PIL import Image
12
+ import pycocotools.mask as mask_util
13
+
14
+
15
+ if __name__ == "__main__":
16
+ dataset_dir = os.getenv("DETECTRON2_DATASETS", "datasets")
17
+
18
+ for name, dirname in [("train", "training"), ("val", "validation")]:
19
+ image_dir = os.path.join(dataset_dir, f"ADEChallengeData2016/images/{dirname}/")
20
+ instance_dir = os.path.join(
21
+ dataset_dir, f"ADEChallengeData2016/annotations_instance/{dirname}/"
22
+ )
23
+
24
+ # img_id = 0
25
+ ann_id = 1
26
+
27
+ # json
28
+ out_file = os.path.join(dataset_dir, f"ADEChallengeData2016/ade20k_instance_{name}.json")
29
+
30
+ # json config
31
+ instance_config_file = "datasets/ade20k_instance_imgCatIds.json"
32
+ with open(instance_config_file) as f:
33
+ category_dict = json.load(f)["categories"]
34
+
35
+ # load catid mapping
36
+ # it is important to share category id for both instance and panoptic annotations
37
+ mapping_file = "datasets/ade20k_instance_catid_mapping.txt"
38
+ with open(mapping_file) as f:
39
+ map_id = {}
40
+ for i, line in enumerate(f.readlines()):
41
+ if i == 0:
42
+ continue
43
+ ins_id, sem_id, _ = line.strip().split()
44
+ # shift id by 1 because we want it to start from 0!
45
+ # ignore_label becomes 255
46
+ map_id[int(ins_id)] = int(sem_id) - 1
47
+
48
+ for cat in category_dict:
49
+ cat["id"] = map_id[cat["id"]]
50
+
51
+ filenames = sorted(glob.glob(os.path.join(image_dir, "*.jpg")))
52
+
53
+ ann_dict = {}
54
+ images = []
55
+ annotations = []
56
+
57
+ for idx, filename in enumerate(tqdm.tqdm(filenames)):
58
+ image = {}
59
+ image_id = os.path.basename(filename).split(".")[0]
60
+
61
+ image["id"] = image_id
62
+ image["file_name"] = os.path.basename(filename)
63
+
64
+ original_format = np.array(Image.open(filename))
65
+ image["width"] = original_format.shape[1]
66
+ image["height"] = original_format.shape[0]
67
+
68
+ images.append(image)
69
+
70
+ filename_instance = os.path.join(instance_dir, image_id + ".png")
71
+ ins_seg = np.asarray(Image.open(filename_instance))
72
+ assert ins_seg.dtype == np.uint8
73
+
74
+ instance_cat_ids = ins_seg[..., 0]
75
+ # instance id starts from 1!
76
+ # because 0 is reserved as VOID label
77
+ instance_ins_ids = ins_seg[..., 1]
78
+
79
+ # process things
80
+ for thing_id in np.unique(instance_ins_ids):
81
+ if thing_id == 0:
82
+ continue
83
+ mask = instance_ins_ids == thing_id
84
+ instance_cat_id = np.unique(instance_cat_ids[mask])
85
+ assert len(instance_cat_id) == 1
86
+
87
+ anno = {}
88
+ anno['id'] = ann_id
89
+ ann_id += 1
90
+ anno['image_id'] = image['id']
91
+ anno["iscrowd"] = int(0)
92
+ anno["category_id"] = int(map_id[instance_cat_id[0]])
93
+
94
+ inds = np.nonzero(mask)
95
+ ymin, ymax = inds[0].min(), inds[0].max()
96
+ xmin, xmax = inds[1].min(), inds[1].max()
97
+ anno["bbox"] = [int(xmin), int(ymin), int(xmax - xmin + 1), int(ymax - ymin + 1)]
98
+ # if xmax <= xmin or ymax <= ymin:
99
+ # continue
100
+ rle = mask_util.encode(np.array(mask[:, :, None], order="F", dtype="uint8"))[0]
101
+ rle["counts"] = rle["counts"].decode("utf-8")
102
+ anno["segmentation"] = rle
103
+ anno["area"] = int(mask_util.area(rle))
104
+ annotations.append(anno)
105
+
106
+ # save this
107
+ ann_dict['images'] = images
108
+ ann_dict['categories'] = category_dict
109
+ ann_dict['annotations'] = annotations
110
+
111
+ save_json(ann_dict, out_file)
datasets/prepare_ade20k_pan_seg.py ADDED
@@ -0,0 +1,499 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ # -*- coding: utf-8 -*-
3
+ import glob
4
+ import json
5
+ import os
6
+ from collections import Counter
7
+
8
+ import numpy as np
9
+ import tqdm
10
+ from panopticapi.utils import IdGenerator, save_json
11
+ from PIL import Image
12
+
13
+ ADE20K_SEM_SEG_CATEGORIES = [
14
+ "wall",
15
+ "building",
16
+ "sky",
17
+ "floor",
18
+ "tree",
19
+ "ceiling",
20
+ "road, route",
21
+ "bed",
22
+ "window ",
23
+ "grass",
24
+ "cabinet",
25
+ "sidewalk, pavement",
26
+ "person",
27
+ "earth, ground",
28
+ "door",
29
+ "table",
30
+ "mountain, mount",
31
+ "plant",
32
+ "curtain",
33
+ "chair",
34
+ "car",
35
+ "water",
36
+ "painting, picture",
37
+ "sofa",
38
+ "shelf",
39
+ "house",
40
+ "sea",
41
+ "mirror",
42
+ "rug",
43
+ "field",
44
+ "armchair",
45
+ "seat",
46
+ "fence",
47
+ "desk",
48
+ "rock, stone",
49
+ "wardrobe, closet, press",
50
+ "lamp",
51
+ "tub",
52
+ "rail",
53
+ "cushion",
54
+ "base, pedestal, stand",
55
+ "box",
56
+ "column, pillar",
57
+ "signboard, sign",
58
+ "chest of drawers, chest, bureau, dresser",
59
+ "counter",
60
+ "sand",
61
+ "sink",
62
+ "skyscraper",
63
+ "fireplace",
64
+ "refrigerator, icebox",
65
+ "grandstand, covered stand",
66
+ "path",
67
+ "stairs",
68
+ "runway",
69
+ "case, display case, showcase, vitrine",
70
+ "pool table, billiard table, snooker table",
71
+ "pillow",
72
+ "screen door, screen",
73
+ "stairway, staircase",
74
+ "river",
75
+ "bridge, span",
76
+ "bookcase",
77
+ "blind, screen",
78
+ "coffee table",
79
+ "toilet, can, commode, crapper, pot, potty, stool, throne",
80
+ "flower",
81
+ "book",
82
+ "hill",
83
+ "bench",
84
+ "countertop",
85
+ "stove",
86
+ "palm, palm tree",
87
+ "kitchen island",
88
+ "computer",
89
+ "swivel chair",
90
+ "boat",
91
+ "bar",
92
+ "arcade machine",
93
+ "hovel, hut, hutch, shack, shanty",
94
+ "bus",
95
+ "towel",
96
+ "light",
97
+ "truck",
98
+ "tower",
99
+ "chandelier",
100
+ "awning, sunshade, sunblind",
101
+ "street lamp",
102
+ "booth",
103
+ "tv",
104
+ "plane",
105
+ "dirt track",
106
+ "clothes",
107
+ "pole",
108
+ "land, ground, soil",
109
+ "bannister, banister, balustrade, balusters, handrail",
110
+ "escalator, moving staircase, moving stairway",
111
+ "ottoman, pouf, pouffe, puff, hassock",
112
+ "bottle",
113
+ "buffet, counter, sideboard",
114
+ "poster, posting, placard, notice, bill, card",
115
+ "stage",
116
+ "van",
117
+ "ship",
118
+ "fountain",
119
+ "conveyer belt, conveyor belt, conveyer, conveyor, transporter",
120
+ "canopy",
121
+ "washer, automatic washer, washing machine",
122
+ "plaything, toy",
123
+ "pool",
124
+ "stool",
125
+ "barrel, cask",
126
+ "basket, handbasket",
127
+ "falls",
128
+ "tent",
129
+ "bag",
130
+ "minibike, motorbike",
131
+ "cradle",
132
+ "oven",
133
+ "ball",
134
+ "food, solid food",
135
+ "step, stair",
136
+ "tank, storage tank",
137
+ "trade name",
138
+ "microwave",
139
+ "pot",
140
+ "animal",
141
+ "bicycle",
142
+ "lake",
143
+ "dishwasher",
144
+ "screen",
145
+ "blanket, cover",
146
+ "sculpture",
147
+ "hood, exhaust hood",
148
+ "sconce",
149
+ "vase",
150
+ "traffic light",
151
+ "tray",
152
+ "trash can",
153
+ "fan",
154
+ "pier",
155
+ "crt screen",
156
+ "plate",
157
+ "monitor",
158
+ "bulletin board",
159
+ "shower",
160
+ "radiator",
161
+ "glass, drinking glass",
162
+ "clock",
163
+ "flag", # noqa
164
+ ]
165
+
166
+ PALETTE = [
167
+ [120, 120, 120],
168
+ [180, 120, 120],
169
+ [6, 230, 230],
170
+ [80, 50, 50],
171
+ [4, 200, 3],
172
+ [120, 120, 80],
173
+ [140, 140, 140],
174
+ [204, 5, 255],
175
+ [230, 230, 230],
176
+ [4, 250, 7],
177
+ [224, 5, 255],
178
+ [235, 255, 7],
179
+ [150, 5, 61],
180
+ [120, 120, 70],
181
+ [8, 255, 51],
182
+ [255, 6, 82],
183
+ [143, 255, 140],
184
+ [204, 255, 4],
185
+ [255, 51, 7],
186
+ [204, 70, 3],
187
+ [0, 102, 200],
188
+ [61, 230, 250],
189
+ [255, 6, 51],
190
+ [11, 102, 255],
191
+ [255, 7, 71],
192
+ [255, 9, 224],
193
+ [9, 7, 230],
194
+ [220, 220, 220],
195
+ [255, 9, 92],
196
+ [112, 9, 255],
197
+ [8, 255, 214],
198
+ [7, 255, 224],
199
+ [255, 184, 6],
200
+ [10, 255, 71],
201
+ [255, 41, 10],
202
+ [7, 255, 255],
203
+ [224, 255, 8],
204
+ [102, 8, 255],
205
+ [255, 61, 6],
206
+ [255, 194, 7],
207
+ [255, 122, 8],
208
+ [0, 255, 20],
209
+ [255, 8, 41],
210
+ [255, 5, 153],
211
+ [6, 51, 255],
212
+ [235, 12, 255],
213
+ [160, 150, 20],
214
+ [0, 163, 255],
215
+ [140, 140, 200],
216
+ [250, 10, 15],
217
+ [20, 255, 0],
218
+ [31, 255, 0],
219
+ [255, 31, 0],
220
+ [255, 224, 0],
221
+ [153, 255, 0],
222
+ [0, 0, 255],
223
+ [255, 71, 0],
224
+ [0, 235, 255],
225
+ [0, 173, 255],
226
+ [31, 0, 255],
227
+ [11, 200, 200],
228
+ [255, 82, 0],
229
+ [0, 255, 245],
230
+ [0, 61, 255],
231
+ [0, 255, 112],
232
+ [0, 255, 133],
233
+ [255, 0, 0],
234
+ [255, 163, 0],
235
+ [255, 102, 0],
236
+ [194, 255, 0],
237
+ [0, 143, 255],
238
+ [51, 255, 0],
239
+ [0, 82, 255],
240
+ [0, 255, 41],
241
+ [0, 255, 173],
242
+ [10, 0, 255],
243
+ [173, 255, 0],
244
+ [0, 255, 153],
245
+ [255, 92, 0],
246
+ [255, 0, 255],
247
+ [255, 0, 245],
248
+ [255, 0, 102],
249
+ [255, 173, 0],
250
+ [255, 0, 20],
251
+ [255, 184, 184],
252
+ [0, 31, 255],
253
+ [0, 255, 61],
254
+ [0, 71, 255],
255
+ [255, 0, 204],
256
+ [0, 255, 194],
257
+ [0, 255, 82],
258
+ [0, 10, 255],
259
+ [0, 112, 255],
260
+ [51, 0, 255],
261
+ [0, 194, 255],
262
+ [0, 122, 255],
263
+ [0, 255, 163],
264
+ [255, 153, 0],
265
+ [0, 255, 10],
266
+ [255, 112, 0],
267
+ [143, 255, 0],
268
+ [82, 0, 255],
269
+ [163, 255, 0],
270
+ [255, 235, 0],
271
+ [8, 184, 170],
272
+ [133, 0, 255],
273
+ [0, 255, 92],
274
+ [184, 0, 255],
275
+ [255, 0, 31],
276
+ [0, 184, 255],
277
+ [0, 214, 255],
278
+ [255, 0, 112],
279
+ [92, 255, 0],
280
+ [0, 224, 255],
281
+ [112, 224, 255],
282
+ [70, 184, 160],
283
+ [163, 0, 255],
284
+ [153, 0, 255],
285
+ [71, 255, 0],
286
+ [255, 0, 163],
287
+ [255, 204, 0],
288
+ [255, 0, 143],
289
+ [0, 255, 235],
290
+ [133, 255, 0],
291
+ [255, 0, 235],
292
+ [245, 0, 255],
293
+ [255, 0, 122],
294
+ [255, 245, 0],
295
+ [10, 190, 212],
296
+ [214, 255, 0],
297
+ [0, 204, 255],
298
+ [20, 0, 255],
299
+ [255, 255, 0],
300
+ [0, 153, 255],
301
+ [0, 41, 255],
302
+ [0, 255, 204],
303
+ [41, 0, 255],
304
+ [41, 255, 0],
305
+ [173, 0, 255],
306
+ [0, 245, 255],
307
+ [71, 0, 255],
308
+ [122, 0, 255],
309
+ [0, 255, 184],
310
+ [0, 92, 255],
311
+ [184, 255, 0],
312
+ [0, 133, 255],
313
+ [255, 214, 0],
314
+ [25, 194, 194],
315
+ [102, 255, 0],
316
+ [92, 0, 255],
317
+ ]
318
+
319
+
320
+ if __name__ == "__main__":
321
+ dataset_dir = os.getenv("DETECTRON2_DATASETS", "datasets")
322
+
323
+ for name, dirname in [("train", "training"), ("val", "validation")]:
324
+ image_dir = os.path.join(dataset_dir, f"ADEChallengeData2016/images/{dirname}/")
325
+ semantic_dir = os.path.join(dataset_dir, f"ADEChallengeData2016/annotations/{dirname}/")
326
+ instance_dir = os.path.join(
327
+ dataset_dir, f"ADEChallengeData2016/annotations_instance/{dirname}/"
328
+ )
329
+
330
+ # folder to store panoptic PNGs
331
+ out_folder = os.path.join(dataset_dir, f"ADEChallengeData2016/ade20k_panoptic_{name}/")
332
+ # json with segmentations information
333
+ out_file = os.path.join(dataset_dir, f"ADEChallengeData2016/ade20k_panoptic_{name}.json")
334
+
335
+ if not os.path.isdir(out_folder):
336
+ print("Creating folder {} for panoptic segmentation PNGs".format(out_folder))
337
+ os.mkdir(out_folder)
338
+
339
+ # json config
340
+ config_file = "datasets/ade20k_instance_imgCatIds.json"
341
+ with open(config_file) as f:
342
+ config = json.load(f)
343
+
344
+ # load catid mapping
345
+ mapping_file = "datasets/ade20k_instance_catid_mapping.txt"
346
+ with open(mapping_file) as f:
347
+ map_id = {}
348
+ for i, line in enumerate(f.readlines()):
349
+ if i == 0:
350
+ continue
351
+ ins_id, sem_id, _ = line.strip().split()
352
+ # shift id by 1 because we want it to start from 0!
353
+ # ignore_label becomes 255
354
+ map_id[int(ins_id) - 1] = int(sem_id) - 1
355
+
356
+ ADE20K_150_CATEGORIES = []
357
+ for cat_id, cat_name in enumerate(ADE20K_SEM_SEG_CATEGORIES):
358
+ ADE20K_150_CATEGORIES.append(
359
+ {
360
+ "name": cat_name,
361
+ "id": cat_id,
362
+ "isthing": int(cat_id in map_id.values()),
363
+ "color": PALETTE[cat_id],
364
+ }
365
+ )
366
+ categories_dict = {cat["id"]: cat for cat in ADE20K_150_CATEGORIES}
367
+
368
+ panoptic_json_categories = ADE20K_150_CATEGORIES[:]
369
+ panoptic_json_images = []
370
+ panoptic_json_annotations = []
371
+
372
+ filenames = sorted(glob.glob(os.path.join(image_dir, "*.jpg")))
373
+ for idx, filename in enumerate(tqdm.tqdm(filenames)):
374
+ panoptic_json_image = {}
375
+ panoptic_json_annotation = {}
376
+
377
+ image_id = os.path.basename(filename).split(".")[0]
378
+
379
+ panoptic_json_image["id"] = image_id
380
+ panoptic_json_image["file_name"] = os.path.basename(filename)
381
+
382
+ original_format = np.array(Image.open(filename))
383
+ panoptic_json_image["width"] = original_format.shape[1]
384
+ panoptic_json_image["height"] = original_format.shape[0]
385
+
386
+ pan_seg = np.zeros(
387
+ (original_format.shape[0], original_format.shape[1], 3), dtype=np.uint8
388
+ )
389
+ id_generator = IdGenerator(categories_dict)
390
+
391
+ filename_semantic = os.path.join(semantic_dir, image_id + ".png")
392
+ filename_instance = os.path.join(instance_dir, image_id + ".png")
393
+
394
+ sem_seg = np.asarray(Image.open(filename_semantic))
395
+ ins_seg = np.asarray(Image.open(filename_instance))
396
+
397
+ assert sem_seg.dtype == np.uint8
398
+ assert ins_seg.dtype == np.uint8
399
+
400
+ semantic_cat_ids = sem_seg - 1
401
+ instance_cat_ids = ins_seg[..., 0] - 1
402
+ # instance id starts from 1!
403
+ # because 0 is reserved as VOID label
404
+ instance_ins_ids = ins_seg[..., 1]
405
+
406
+ segm_info = []
407
+
408
+ # NOTE: there is some overlap between semantic and instance annotation
409
+ # thus we paste stuffs first
410
+
411
+ # process stuffs
412
+ for semantic_cat_id in np.unique(semantic_cat_ids):
413
+ if semantic_cat_id == 255:
414
+ continue
415
+ if categories_dict[semantic_cat_id]["isthing"]:
416
+ continue
417
+ mask = semantic_cat_ids == semantic_cat_id
418
+ # should not have any overlap
419
+ assert pan_seg[mask].sum() == 0
420
+
421
+ segment_id, color = id_generator.get_id_and_color(semantic_cat_id)
422
+ pan_seg[mask] = color
423
+
424
+ area = np.sum(mask) # segment area computation
425
+ # bbox computation for a segment
426
+ hor = np.sum(mask, axis=0)
427
+ hor_idx = np.nonzero(hor)[0]
428
+ x = hor_idx[0]
429
+ width = hor_idx[-1] - x + 1
430
+ vert = np.sum(mask, axis=1)
431
+ vert_idx = np.nonzero(vert)[0]
432
+ y = vert_idx[0]
433
+ height = vert_idx[-1] - y + 1
434
+ bbox = [int(x), int(y), int(width), int(height)]
435
+
436
+ segm_info.append(
437
+ {
438
+ "id": int(segment_id),
439
+ "category_id": int(semantic_cat_id),
440
+ "area": int(area),
441
+ "bbox": bbox,
442
+ "iscrowd": 0,
443
+ }
444
+ )
445
+
446
+ # process things
447
+ for thing_id in np.unique(instance_ins_ids):
448
+ if thing_id == 0:
449
+ continue
450
+ mask = instance_ins_ids == thing_id
451
+ instance_cat_id = np.unique(instance_cat_ids[mask])
452
+ assert len(instance_cat_id) == 1
453
+
454
+ semantic_cat_id = map_id[instance_cat_id[0]]
455
+
456
+ segment_id, color = id_generator.get_id_and_color(semantic_cat_id)
457
+ pan_seg[mask] = color
458
+
459
+ area = np.sum(mask) # segment area computation
460
+ # bbox computation for a segment
461
+ hor = np.sum(mask, axis=0)
462
+ hor_idx = np.nonzero(hor)[0]
463
+ x = hor_idx[0]
464
+ width = hor_idx[-1] - x + 1
465
+ vert = np.sum(mask, axis=1)
466
+ vert_idx = np.nonzero(vert)[0]
467
+ y = vert_idx[0]
468
+ height = vert_idx[-1] - y + 1
469
+ bbox = [int(x), int(y), int(width), int(height)]
470
+
471
+ segm_info.append(
472
+ {
473
+ "id": int(segment_id),
474
+ "category_id": int(semantic_cat_id),
475
+ "area": int(area),
476
+ "bbox": bbox,
477
+ "iscrowd": 0,
478
+ }
479
+ )
480
+
481
+ panoptic_json_annotation = {
482
+ "image_id": image_id,
483
+ "file_name": image_id + ".png",
484
+ "segments_info": segm_info,
485
+ }
486
+
487
+ Image.fromarray(pan_seg).save(os.path.join(out_folder, image_id + ".png"))
488
+
489
+ panoptic_json_images.append(panoptic_json_image)
490
+ panoptic_json_annotations.append(panoptic_json_annotation)
491
+
492
+ # save this
493
+ d = {
494
+ "images": panoptic_json_images,
495
+ "annotations": panoptic_json_annotations,
496
+ "categories": panoptic_json_categories,
497
+ }
498
+
499
+ save_json(d, out_file)
datasets/prepare_ade20k_sem_seg.py ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ # -*- coding: utf-8 -*-
3
+ import os
4
+ from pathlib import Path
5
+
6
+ import numpy as np
7
+ import tqdm
8
+ from PIL import Image
9
+
10
+
11
+ def convert(input, output):
12
+ img = np.asarray(Image.open(input))
13
+ assert img.dtype == np.uint8
14
+ img = img - 1 # 0 (ignore) becomes 255. others are shifted by 1
15
+ Image.fromarray(img).save(output)
16
+
17
+
18
+ if __name__ == "__main__":
19
+ dataset_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets")) / "ADEChallengeData2016"
20
+ for name in ["training", "validation"]:
21
+ annotation_dir = dataset_dir / "annotations" / name
22
+ output_dir = dataset_dir / "annotations_detectron2" / name
23
+ output_dir.mkdir(parents=True, exist_ok=True)
24
+ for file in tqdm.tqdm(list(annotation_dir.iterdir())):
25
+ output_file = output_dir / file.name
26
+ convert(file, output_file)
datasets/prepare_coco_semantic_annos_from_panoptic_annos.py ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ # -*- coding: utf-8 -*-
3
+ import functools
4
+ import json
5
+ import multiprocessing as mp
6
+ import numpy as np
7
+ import os
8
+ import time
9
+ from fvcore.common.download import download
10
+ from panopticapi.utils import rgb2id
11
+ from PIL import Image
12
+
13
+ from detectron2.data.datasets.builtin_meta import COCO_CATEGORIES
14
+
15
+
16
+ def _process_panoptic_to_semantic(input_panoptic, output_semantic, segments, id_map):
17
+ panoptic = np.asarray(Image.open(input_panoptic), dtype=np.uint32)
18
+ panoptic = rgb2id(panoptic)
19
+ output = np.zeros_like(panoptic, dtype=np.uint8) + 255
20
+ for seg in segments:
21
+ cat_id = seg["category_id"]
22
+ new_cat_id = id_map[cat_id]
23
+ output[panoptic == seg["id"]] = new_cat_id
24
+ Image.fromarray(output).save(output_semantic)
25
+
26
+
27
+ def separate_coco_semantic_from_panoptic(panoptic_json, panoptic_root, sem_seg_root, categories):
28
+ """
29
+ Create semantic segmentation annotations from panoptic segmentation
30
+ annotations, to be used by PanopticFPN.
31
+ It maps all thing categories to class 0, and maps all unlabeled pixels to class 255.
32
+ It maps all stuff categories to contiguous ids starting from 1.
33
+ Args:
34
+ panoptic_json (str): path to the panoptic json file, in COCO's format.
35
+ panoptic_root (str): a directory with panoptic annotation files, in COCO's format.
36
+ sem_seg_root (str): a directory to output semantic annotation files
37
+ categories (list[dict]): category metadata. Each dict needs to have:
38
+ "id": corresponds to the "category_id" in the json annotations
39
+ "isthing": 0 or 1
40
+ """
41
+ os.makedirs(sem_seg_root, exist_ok=True)
42
+
43
+ id_map = {} # map from category id to id in the output semantic annotation
44
+ assert len(categories) <= 254
45
+ for i, k in enumerate(categories):
46
+ id_map[k["id"]] = i
47
+ # what is id = 0?
48
+ # id_map[0] = 255
49
+ print(id_map)
50
+
51
+ with open(panoptic_json) as f:
52
+ obj = json.load(f)
53
+
54
+ pool = mp.Pool(processes=max(mp.cpu_count() // 2, 4))
55
+
56
+ def iter_annotations():
57
+ for anno in obj["annotations"]:
58
+ file_name = anno["file_name"]
59
+ segments = anno["segments_info"]
60
+ input = os.path.join(panoptic_root, file_name)
61
+ output = os.path.join(sem_seg_root, file_name)
62
+ yield input, output, segments
63
+
64
+ print("Start writing to {} ...".format(sem_seg_root))
65
+ start = time.time()
66
+ pool.starmap(
67
+ functools.partial(_process_panoptic_to_semantic, id_map=id_map),
68
+ iter_annotations(),
69
+ chunksize=100,
70
+ )
71
+ print("Finished. time: {:.2f}s".format(time.time() - start))
72
+
73
+
74
+ if __name__ == "__main__":
75
+ dataset_dir = os.path.join(os.getenv("DETECTRON2_DATASETS", "datasets"), "coco")
76
+ for s in ["val2017", "train2017"]:
77
+ separate_coco_semantic_from_panoptic(
78
+ os.path.join(dataset_dir, "annotations/panoptic_{}.json".format(s)),
79
+ os.path.join(dataset_dir, "panoptic_{}".format(s)),
80
+ os.path.join(dataset_dir, "panoptic_semseg_{}".format(s)),
81
+ COCO_CATEGORIES,
82
+ )
datasets/prepare_pascal_ctx_full_sem_seg.py ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import numpy as np
3
+ from pathlib import Path
4
+ from PIL import Image
5
+ import scipy.io as sio
6
+
7
+ import tqdm
8
+
9
+
10
+ def generate_labels(mat_file, out_dir):
11
+
12
+ mat = sio.loadmat(mat_file)
13
+ label_map = mat["LabelMap"]
14
+ assert label_map.dtype == np.uint16
15
+ label_map[label_map == 0] = 65535
16
+ label_map = label_map - 1
17
+ label_map[label_map == 65534] = 65535
18
+
19
+ out_file = out_dir / Path(mat_file.name).with_suffix(".tif")
20
+ Image.fromarray(label_map).save(out_file)
21
+
22
+
23
+ if __name__ == "__main__":
24
+ dataset_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets")) / "pascal_ctx_d2"
25
+ voc_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets")) / "VOCdevkit/VOC2010"
26
+ mat_dir = voc_dir / "trainval"
27
+ for split in ["training", "validation"]:
28
+ file_names = list((dataset_dir / "images" / split).glob("*.jpg"))
29
+ output_img_dir = dataset_dir / "images" / split
30
+ output_ann_dir = dataset_dir / "annotations_ctx459" / split
31
+
32
+ output_img_dir.mkdir(parents=True, exist_ok=True)
33
+ output_ann_dir.mkdir(parents=True, exist_ok=True)
34
+
35
+ for file_name in tqdm.tqdm(file_names):
36
+ mat_file_path = mat_dir / f"{file_name.stem}.mat"
37
+
38
+ generate_labels(mat_file_path, output_ann_dir)
datasets/prepare_pascal_ctx_sem_seg.py ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ from pathlib import Path
3
+ import shutil
4
+
5
+ import numpy as np
6
+ import tqdm
7
+ from PIL import Image
8
+ import multiprocessing as mp
9
+ import functools
10
+ from detail import Detail
11
+
12
+ # fmt: off
13
+ _mapping = np.sort(
14
+ np.array([
15
+ 0, 2, 259, 260, 415, 324, 9, 258, 144, 18, 19, 22, 23, 397, 25, 284,
16
+ 158, 159, 416, 33, 162, 420, 454, 295, 296, 427, 44, 45, 46, 308, 59,
17
+ 440, 445, 31, 232, 65, 354, 424, 68, 326, 72, 458, 34, 207, 80, 355,
18
+ 85, 347, 220, 349, 360, 98, 187, 104, 105, 366, 189, 368, 113, 115
19
+ ]))
20
+ # fmt: on
21
+ _key = np.array(range(len(_mapping))).astype("uint8")
22
+
23
+
24
+ def generate_labels(img_info, detail_api, out_dir):
25
+ def _class_to_index(mask, _mapping, _key):
26
+ # assert the values
27
+ values = np.unique(mask)
28
+ for i in range(len(values)):
29
+ assert values[i] in _mapping
30
+ index = np.digitize(mask.ravel(), _mapping, right=True)
31
+ return _key[index].reshape(mask.shape)
32
+
33
+ sem_seg = _class_to_index(detail_api.getMask(img_info), _mapping=_mapping, _key=_key)
34
+ sem_seg = sem_seg - 1 # 0 (ignore) becomes 255. others are shifted by 1
35
+ filename = img_info["file_name"]
36
+
37
+ Image.fromarray(sem_seg).save(out_dir / filename.replace("jpg", "png"))
38
+
39
+
40
+ def copy_images(img_info, img_dir, out_dir):
41
+ filename = img_info["file_name"]
42
+ shutil.copy2(img_dir / filename, out_dir / filename)
43
+
44
+
45
+ if __name__ == "__main__":
46
+ dataset_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets")) / "pascal_ctx_d2"
47
+ voc_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets")) / "VOCdevkit/VOC2010"
48
+ for split in ["training", "validation"]:
49
+ img_dir = voc_dir / "JPEGImages"
50
+ if split == "training":
51
+ detail_api = Detail(voc_dir / "trainval_merged.json", img_dir, "train")
52
+ else:
53
+ detail_api = Detail(voc_dir / "trainval_merged.json", img_dir, "val")
54
+ img_infos = detail_api.getImgs()
55
+
56
+ output_img_dir = dataset_dir / "images" / split
57
+ output_ann_dir = dataset_dir / "annotations_ctx59" / split
58
+
59
+ output_img_dir.mkdir(parents=True, exist_ok=True)
60
+ output_ann_dir.mkdir(parents=True, exist_ok=True)
61
+
62
+ pool = mp.Pool(processes=max(mp.cpu_count() // 2, 4))
63
+
64
+ pool.map(
65
+ functools.partial(copy_images, img_dir=img_dir, out_dir=output_img_dir),
66
+ tqdm.tqdm(img_infos, desc=f"Writing {split} images to {output_img_dir} ..."),
67
+ chunksize=100,
68
+ )
69
+
70
+ pool.map(
71
+ functools.partial(generate_labels, detail_api=detail_api, out_dir=output_ann_dir),
72
+ tqdm.tqdm(img_infos, desc=f"Writing {split} images to {output_ann_dir} ..."),
73
+ chunksize=100,
74
+ )
datasets/prepare_pascal_voc_sem_seg.py ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ from pathlib import Path
3
+ import shutil
4
+
5
+ import numpy as np
6
+ import tqdm
7
+ from PIL import Image
8
+
9
+
10
+ def convert_pas21(input, output):
11
+ img = np.asarray(Image.open(input))
12
+ assert img.dtype == np.uint8
13
+ # do nothing
14
+ Image.fromarray(img).save(output)
15
+
16
+ def convert_pas20(input, output):
17
+ img = np.array(Image.open(input))
18
+ img[img == 0] = 255
19
+ img = img - 1
20
+ img[img == 254] = 255
21
+ assert img.dtype == np.uint8
22
+ # do nothing
23
+ Image.fromarray(img).save(output)
24
+
25
+
26
+ if __name__ == "__main__":
27
+ dataset_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets")) / "pascal_voc_d2"
28
+ voc_dir = Path(os.getenv("DETECTRON2_DATASETS", "datasets")) / "VOCdevkit/VOC2012"
29
+ for split in ["training", "validation"]:
30
+ if split == "training":
31
+ img_name_path = voc_dir / "ImageSets/Segmentation/train.txt"
32
+ else:
33
+ img_name_path = voc_dir / "ImageSets/Segmentation/val.txt"
34
+ img_dir = voc_dir / "JPEGImages"
35
+ ann_dir = voc_dir / "SegmentationClass"
36
+
37
+ output_img_dir = dataset_dir / "images" / split
38
+ output_ann_dir_21 = dataset_dir / "annotations_pascal21" / split
39
+ output_ann_dir_20 = dataset_dir / "annotations_pascal20" / split
40
+
41
+ output_img_dir.mkdir(parents=True, exist_ok=True)
42
+ output_ann_dir_21.mkdir(parents=True, exist_ok=True)
43
+ output_ann_dir_20.mkdir(parents=True, exist_ok=True)
44
+
45
+ with open(img_name_path) as f:
46
+ for line in tqdm.tqdm(f.readlines()):
47
+ img_name = line.strip()
48
+ img_path = img_dir / f"{img_name}.jpg"
49
+ ann_path = ann_dir / f"{img_name}.png"
50
+
51
+ # print(f'copy2 {output_img_dir}')
52
+ shutil.copy2(img_path, output_img_dir)
53
+ # print(f"convert {ann_dir} to {output_ann_dir / f'{img_name}.png'}")
54
+ convert_pas21(ann_path, output_ann_dir_21 / f"{img_name}.png")
55
+ convert_pas20(ann_path, output_ann_dir_20 / f"{img_name}.png")
demo/demo.py ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ import glob
3
+ import multiprocessing as mp
4
+ import os
5
+
6
+ # fmt: off
7
+ import sys
8
+ sys.path.insert(1, os.path.join(sys.path[0], '..'))
9
+ # fmt: on
10
+
11
+ import tempfile
12
+ import time
13
+ import warnings
14
+
15
+ import cv2
16
+ import numpy as np
17
+ import tqdm
18
+
19
+ from detectron2.config import get_cfg
20
+ from detectron2.data.detection_utils import read_image
21
+ from detectron2.utils.logger import setup_logger
22
+
23
+ from frozenseg import add_maskformer2_config, add_frozenseg_config
24
+ from predictor import VisualizationDemo
25
+
26
+ # constants
27
+ WINDOW_NAME = "frozenseg demo"
28
+
29
+
30
+ def setup_cfg(args):
31
+ # load config from file and command-line arguments
32
+ cfg = get_cfg()
33
+ add_maskformer2_config(cfg)
34
+ add_frozenseg_config(cfg)
35
+ cfg.merge_from_file(args.config_file)
36
+ cfg.merge_from_list(args.opts)
37
+ cfg.freeze()
38
+ return cfg
39
+
40
+
41
+ def get_parser():
42
+ parser = argparse.ArgumentParser(description="frozenseg demo for builtin configs")
43
+ parser.add_argument(
44
+ "--config-file",
45
+ default="configs/coco/frozenseg/convnext_large_eval_ade20k.yaml",
46
+ metavar="FILE",
47
+ help="path to config file",
48
+ )
49
+ parser.add_argument("--webcam", action="store_true", help="Take inputs from webcam.")
50
+ parser.add_argument("--video-input", help="Path to video file.")
51
+ parser.add_argument(
52
+ "--input",
53
+ nargs="+",
54
+ help="A list of space separated input images; "
55
+ "or a single glob pattern such as 'directory/*.jpg'",
56
+ )
57
+ parser.add_argument(
58
+ "--output",
59
+ help="A file or directory to save output visualizations. "
60
+ "If not given, will show output in an OpenCV window.",
61
+ )
62
+
63
+ parser.add_argument(
64
+ "--confidence-threshold",
65
+ type=float,
66
+ default=0.5,
67
+ help="Minimum score for instance predictions to be shown",
68
+ )
69
+ parser.add_argument(
70
+ "--opts",
71
+ help="Modify config options using the command-line 'KEY VALUE' pairs",
72
+ default=[],
73
+ nargs=argparse.REMAINDER,
74
+ )
75
+ return parser
76
+
77
+
78
+ def test_opencv_video_format(codec, file_ext):
79
+ with tempfile.TemporaryDirectory(prefix="video_format_test") as dir:
80
+ filename = os.path.join(dir, "test_file" + file_ext)
81
+ writer = cv2.VideoWriter(
82
+ filename=filename,
83
+ fourcc=cv2.VideoWriter_fourcc(*codec),
84
+ fps=float(30),
85
+ frameSize=(10, 10),
86
+ isColor=True,
87
+ )
88
+ [writer.write(np.zeros((10, 10, 3), np.uint8)) for _ in range(30)]
89
+ writer.release()
90
+ if os.path.isfile(filename):
91
+ return True
92
+ return False
93
+
94
+
95
+ if __name__ == "__main__":
96
+ mp.set_start_method("spawn", force=True)
97
+ args = get_parser().parse_args()
98
+ setup_logger(name="fvcore")
99
+ logger = setup_logger()
100
+ logger.info("Arguments: " + str(args))
101
+
102
+ cfg = setup_cfg(args)
103
+ demo = VisualizationDemo(cfg)
104
+
105
+ if args.input:
106
+ if len(args.input) == 1:
107
+ args.input = glob.glob(os.path.expanduser(args.input[0]))
108
+ assert args.input, "The input path(s) was not found"
109
+ for path in tqdm.tqdm(args.input, disable=not args.output):
110
+ # use PIL, to be consistent with evaluation
111
+ img = read_image(path, format="BGR")
112
+ start_time = time.time()
113
+ predictions, visualized_output = demo.run_on_image(img)
114
+ logger.info(
115
+ "{}: {} in {:.2f}s".format(
116
+ path,
117
+ "detected {} instances".format(len(predictions["instances"]))
118
+ if "instances" in predictions
119
+ else "finished",
120
+ time.time() - start_time,
121
+ )
122
+ )
123
+
124
+ if args.output:
125
+ if os.path.isdir(args.output):
126
+ assert os.path.isdir(args.output), args.output
127
+ out_filename = os.path.join(args.output, os.path.basename(path))
128
+ else:
129
+ assert len(args.input) == 1, "Please specify a directory with args.output"
130
+ out_filename = args.output
131
+ visualized_output.save(out_filename)
132
+ else:
133
+ cv2.namedWindow(WINDOW_NAME, cv2.WINDOW_NORMAL)
134
+ cv2.imshow(WINDOW_NAME, visualized_output.get_image()[:, :, ::-1])
135
+ if cv2.waitKey(0) == 27:
136
+ break # esc to quit
137
+ elif args.webcam:
138
+ assert args.input is None, "Cannot have both --input and --webcam!"
139
+ assert args.output is None, "output not yet supported with --webcam!"
140
+ cam = cv2.VideoCapture(0)
141
+ for vis in tqdm.tqdm(demo.run_on_video(cam)):
142
+ cv2.namedWindow(WINDOW_NAME, cv2.WINDOW_NORMAL)
143
+ cv2.imshow(WINDOW_NAME, vis)
144
+ if cv2.waitKey(1) == 27:
145
+ break # esc to quit
146
+ cam.release()
147
+ cv2.destroyAllWindows()
148
+ elif args.video_input:
149
+ video = cv2.VideoCapture(args.video_input)
150
+ width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
151
+ height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
152
+ frames_per_second = video.get(cv2.CAP_PROP_FPS)
153
+ num_frames = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
154
+ basename = os.path.basename(args.video_input)
155
+ codec, file_ext = (
156
+ ("x264", ".mkv") if test_opencv_video_format("x264", ".mkv") else ("mp4v", ".mp4")
157
+ )
158
+ if codec == ".mp4v":
159
+ warnings.warn("x264 codec not available, switching to mp4v")
160
+ if args.output:
161
+ if os.path.isdir(args.output):
162
+ output_fname = os.path.join(args.output, basename)
163
+ output_fname = os.path.splitext(output_fname)[0] + file_ext
164
+ else:
165
+ output_fname = args.output
166
+ assert not os.path.isfile(output_fname), output_fname
167
+ output_file = cv2.VideoWriter(
168
+ filename=output_fname,
169
+ # some installation of opencv may not support x264 (due to its license),
170
+ # you can try other format (e.g. MPEG)
171
+ fourcc=cv2.VideoWriter_fourcc(*codec),
172
+ fps=float(frames_per_second),
173
+ frameSize=(width, height),
174
+ isColor=True,
175
+ )
176
+ assert os.path.isfile(args.video_input)
177
+ for vis_frame in tqdm.tqdm(demo.run_on_video(video), total=num_frames):
178
+ if args.output:
179
+ output_file.write(vis_frame)
180
+ else:
181
+ cv2.namedWindow(basename, cv2.WINDOW_NORMAL)
182
+ cv2.imshow(basename, vis_frame)
183
+ if cv2.waitKey(1) == 27:
184
+ break # esc to quit
185
+ video.release()
186
+ if args.output:
187
+ output_file.release()
188
+ else:
189
+ cv2.destroyAllWindows()
demo/predictor.py ADDED
@@ -0,0 +1,273 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import atexit
2
+ import bisect
3
+ import multiprocessing as mp
4
+ from collections import deque
5
+
6
+ import cv2
7
+ import torch
8
+ import itertools
9
+
10
+
11
+ from detectron2.data import DatasetCatalog, MetadataCatalog
12
+ from detectron2.engine.defaults import DefaultPredictor as d2_defaultPredictor
13
+ from detectron2.utils.video_visualizer import VideoVisualizer
14
+ from detectron2.utils.visualizer import ColorMode, Visualizer, random_color
15
+ import detectron2.utils.visualizer as d2_visualizer
16
+
17
+
18
+ class DefaultPredictor(d2_defaultPredictor):
19
+
20
+ def set_metadata(self, metadata):
21
+ self.model.set_metadata(metadata)
22
+
23
+
24
+ class OpenVocabVisualizer(Visualizer):
25
+ def draw_panoptic_seg(self, panoptic_seg, segments_info, area_threshold=None, alpha=0.7):
26
+ """
27
+ Draw panoptic prediction annotations or results.
28
+
29
+ Args:
30
+ panoptic_seg (Tensor): of shape (height, width) where the values are ids for each
31
+ segment.
32
+ segments_info (list[dict] or None): Describe each segment in `panoptic_seg`.
33
+ If it is a ``list[dict]``, each dict contains keys "id", "category_id".
34
+ If None, category id of each pixel is computed by
35
+ ``pixel // metadata.label_divisor``.
36
+ area_threshold (int): stuff segments with less than `area_threshold` are not drawn.
37
+
38
+ Returns:
39
+ output (VisImage): image object with visualizations.
40
+ """
41
+ pred = d2_visualizer._PanopticPrediction(panoptic_seg, segments_info, self.metadata)
42
+
43
+ if self._instance_mode == ColorMode.IMAGE_BW:
44
+ self.output.reset_image(self._create_grayscale_image(pred.non_empty_mask()))
45
+ # draw mask for all semantic segments first i.e. "stuff"
46
+ for mask, sinfo in pred.semantic_masks():
47
+ category_idx = sinfo["category_id"]
48
+ try:
49
+ mask_color = [x / 255 for x in self.metadata.stuff_colors[category_idx]]
50
+ except AttributeError:
51
+ mask_color = None
52
+
53
+ text = self.metadata.stuff_classes[category_idx].split(',')[0]
54
+ self.draw_binary_mask(
55
+ mask,
56
+ color=mask_color,
57
+ edge_color=d2_visualizer._OFF_WHITE,
58
+ text=text,
59
+ alpha=alpha,
60
+ area_threshold=area_threshold,
61
+ )
62
+ # draw mask for all instances second
63
+ all_instances = list(pred.instance_masks())
64
+ if len(all_instances) == 0:
65
+ return self.output
66
+ masks, sinfo = list(zip(*all_instances))
67
+ category_ids = [x["category_id"] for x in sinfo]
68
+
69
+ try:
70
+ scores = [x["score"] for x in sinfo]
71
+ except KeyError:
72
+ scores = None
73
+ stuff_classes = self.metadata.stuff_classes
74
+ stuff_classes = [x.split(',')[0] for x in stuff_classes]
75
+ labels = d2_visualizer._create_text_labels(
76
+ category_ids, scores, stuff_classes, [x.get("iscrowd", 0) for x in sinfo]
77
+ )
78
+
79
+ try:
80
+ colors = [
81
+ self._jitter([x / 255 for x in self.metadata.stuff_colors[c]]) for c in category_ids
82
+ ]
83
+ except AttributeError:
84
+ colors = None
85
+ self.overlay_instances(masks=masks, labels=labels, assigned_colors=colors, alpha=alpha)
86
+
87
+ return self.output
88
+
89
+
90
+ class VisualizationDemo(object):
91
+ def __init__(self, cfg, instance_mode=ColorMode.IMAGE, parallel=False):
92
+ """
93
+ Args:
94
+ cfg (CfgNode):
95
+ instance_mode (ColorMode):
96
+ parallel (bool): whether to run the model in different processes from visualization.
97
+ Useful since the visualization logic can be slow.
98
+ """
99
+
100
+ coco_metadata = MetadataCatalog.get("openvocab_coco_2017_val_panoptic_with_sem_seg")
101
+ ade20k_metadata = MetadataCatalog.get("openvocab_ade20k_panoptic_val")
102
+ lvis_classes = open("./frozenseg/data/datasets/lvis_1203_with_prompt_eng.txt", 'r').read().splitlines()
103
+ lvis_classes = [x[x.find(':')+1:] for x in lvis_classes]
104
+ lvis_colors = list(
105
+ itertools.islice(itertools.cycle(coco_metadata.stuff_colors), len(lvis_classes))
106
+ )
107
+ # rerrange to thing_classes, stuff_classes
108
+ coco_thing_classes = coco_metadata.thing_classes
109
+ coco_stuff_classes = [x for x in coco_metadata.stuff_classes if x not in coco_thing_classes]
110
+ coco_thing_colors = coco_metadata.thing_colors
111
+ coco_stuff_colors = [x for x in coco_metadata.stuff_colors if x not in coco_thing_colors]
112
+ ade20k_thing_classes = ade20k_metadata.thing_classes
113
+ ade20k_stuff_classes = [x for x in ade20k_metadata.stuff_classes if x not in ade20k_thing_classes]
114
+ ade20k_thing_colors = ade20k_metadata.thing_colors
115
+ ade20k_stuff_colors = [x for x in ade20k_metadata.stuff_colors if x not in ade20k_thing_colors]
116
+
117
+ user_classes = []
118
+ user_colors = [random_color(rgb=True, maximum=1) for _ in range(len(user_classes))]
119
+
120
+ stuff_classes = coco_stuff_classes + ade20k_stuff_classes
121
+ stuff_colors = coco_stuff_colors + ade20k_stuff_colors
122
+ thing_classes = user_classes + coco_thing_classes + ade20k_thing_classes + lvis_classes
123
+ thing_colors = user_colors + coco_thing_colors + ade20k_thing_colors + lvis_colors
124
+
125
+ thing_dataset_id_to_contiguous_id = {x: x for x in range(len(thing_classes))}
126
+ DatasetCatalog.register(
127
+ "openvocab_dataset", lambda x: []
128
+ )
129
+ self.metadata = MetadataCatalog.get("openvocab_dataset").set(
130
+ stuff_classes=thing_classes+stuff_classes,
131
+ stuff_colors=thing_colors+stuff_colors,
132
+ thing_dataset_id_to_contiguous_id=thing_dataset_id_to_contiguous_id,
133
+ )
134
+ #print("self.metadata:", self.metadata)
135
+ self.cpu_device = torch.device("cpu")
136
+ self.instance_mode = instance_mode
137
+
138
+ self.parallel = parallel
139
+ if parallel:
140
+ num_gpu = torch.cuda.device_count()
141
+ self.predictor = AsyncPredictor(cfg, num_gpus=num_gpu)
142
+ else:
143
+ self.predictor = DefaultPredictor(cfg)
144
+ self.predictor.set_metadata(self.metadata)
145
+
146
+ def run_on_image(self, image):
147
+ """
148
+ Args:
149
+ image (np.ndarray): an image of shape (H, W, C) (in BGR order).
150
+ This is the format used by OpenCV.
151
+ Returns:
152
+ predictions (dict): the output of the model.
153
+ vis_output (VisImage): the visualized image output.
154
+ """
155
+ vis_output = None
156
+ predictions = self.predictor(image)
157
+ # Convert image from OpenCV BGR format to Matplotlib RGB format.
158
+ image = image[:, :, ::-1]
159
+ visualizer = OpenVocabVisualizer(image, self.metadata, instance_mode=self.instance_mode)
160
+ if "panoptic_seg" in predictions:
161
+ panoptic_seg, segments_info = predictions["panoptic_seg"]
162
+ vis_output = visualizer.draw_panoptic_seg(
163
+ panoptic_seg.to(self.cpu_device), segments_info
164
+ )
165
+ else:
166
+ if "sem_seg" in predictions:
167
+ vis_output = visualizer.draw_sem_seg(
168
+ predictions["sem_seg"].argmax(dim=0).to(self.cpu_device)
169
+ )
170
+ if "instances" in predictions:
171
+ instances = predictions["instances"].to(self.cpu_device)
172
+ vis_output = visualizer.draw_instance_predictions(predictions=instances)
173
+
174
+ return predictions, vis_output
175
+
176
+ def _frame_from_video(self, video):
177
+ while video.isOpened():
178
+ success, frame = video.read()
179
+ if success:
180
+ yield frame
181
+ else:
182
+ break
183
+
184
+
185
+ class AsyncPredictor:
186
+ """
187
+ A predictor that runs the model asynchronously, possibly on >1 GPUs.
188
+ Because rendering the visualization takes considerably amount of time,
189
+ this helps improve throughput a little bit when rendering videos.
190
+ """
191
+
192
+ class _StopToken:
193
+ pass
194
+
195
+ class _PredictWorker(mp.Process):
196
+ def __init__(self, cfg, task_queue, result_queue):
197
+ self.cfg = cfg
198
+ self.task_queue = task_queue
199
+ self.result_queue = result_queue
200
+ super().__init__()
201
+
202
+ def run(self):
203
+ predictor = DefaultPredictor(self.cfg)
204
+
205
+ while True:
206
+ task = self.task_queue.get()
207
+ if isinstance(task, AsyncPredictor._StopToken):
208
+ break
209
+ idx, data = task
210
+ result = predictor(data)
211
+ self.result_queue.put((idx, result))
212
+
213
+ def __init__(self, cfg, num_gpus: int = 1):
214
+ """
215
+ Args:
216
+ cfg (CfgNode):
217
+ num_gpus (int): if 0, will run on CPU
218
+ """
219
+ num_workers = max(num_gpus, 1)
220
+ self.task_queue = mp.Queue(maxsize=num_workers * 3)
221
+ self.result_queue = mp.Queue(maxsize=num_workers * 3)
222
+ self.procs = []
223
+ for gpuid in range(max(num_gpus, 1)):
224
+ cfg = cfg.clone()
225
+ cfg.defrost()
226
+ cfg.MODEL.DEVICE = "cuda:{}".format(gpuid) if num_gpus > 0 else "cpu"
227
+ self.procs.append(
228
+ AsyncPredictor._PredictWorker(cfg, self.task_queue, self.result_queue)
229
+ )
230
+
231
+ self.put_idx = 0
232
+ self.get_idx = 0
233
+ self.result_rank = []
234
+ self.result_data = []
235
+
236
+ for p in self.procs:
237
+ p.start()
238
+ atexit.register(self.shutdown)
239
+
240
+ def put(self, image):
241
+ self.put_idx += 1
242
+ self.task_queue.put((self.put_idx, image))
243
+
244
+ def get(self):
245
+ self.get_idx += 1 # the index needed for this request
246
+ if len(self.result_rank) and self.result_rank[0] == self.get_idx:
247
+ res = self.result_data[0]
248
+ del self.result_data[0], self.result_rank[0]
249
+ return res
250
+
251
+ while True:
252
+ # make sure the results are returned in the correct order
253
+ idx, res = self.result_queue.get()
254
+ if idx == self.get_idx:
255
+ return res
256
+ insert = bisect.bisect(self.result_rank, idx)
257
+ self.result_rank.insert(insert, idx)
258
+ self.result_data.insert(insert, res)
259
+
260
+ def __len__(self):
261
+ return self.put_idx - self.get_idx
262
+
263
+ def __call__(self, image):
264
+ self.put(image)
265
+ return self.get()
266
+
267
+ def shutdown(self):
268
+ for _ in self.procs:
269
+ self.task_queue.put(AsyncPredictor._StopToken())
270
+
271
+ @property
272
+ def default_buffer_size(self):
273
+ return len(self.procs) * 5
eval.sh ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ #SBATCH --job-name=frozenseg_eval
3
+ #SBATCH --output=output/slurm/%j.run.out
4
+ #SBATCH --error=output/slurm/%j.run.err
5
+ #SBATCH --partition=gpu-a100
6
+ #SBATCH --gres=gpu:1
7
+ #SBATCH --cpus-per-task=16
8
+ #SBATCH --comment=yhx_team
9
+ export MODULEPATH="/opt/app/spack/share/spack/modules/linux-centos7-haswell:/opt/app/spack/share/spack/modules/linux-centos7-cascadelake:/usr/share/Modules/modulefiles:/etc/modulefiles:/opt/app/modulefiles"
10
+ source /users/cx_xchen/.bashrc_12.1
11
+ export DETECTRON2_DATASETS=/users/cx_xchen/DATASETS/
12
+ export TORCH_DISTRIBUTED_DEBUG=DETAIL
13
+ export OMP_NUM_THREADS=1
14
+ export USE_SIMPLE_THREADED_LEVEL3=1
15
+ conda activate frozenseg
16
+
17
+ configs=(
18
+ # "configs/coco/frozenseg/convnext_large_eval_a847.yaml"
19
+ # "configs/coco/frozenseg/convnext_large_eval_ade20k.yaml"
20
+ # "configs/coco/frozenseg/convnext_large_eval_lvis.yaml"
21
+ # "configs/coco/frozenseg/convnext_large_eval_pas21.yaml"
22
+ "configs/coco/frozenseg/convnext_large_eval_pc459.yaml"
23
+ # "configs/coco/frozenseg/convnext_large_eval_cityscapes.yaml"
24
+ # "configs/coco/frozenseg/convnext_large_eval_coco.yaml"
25
+ # "configs/coco/frozenseg/convnext_large_eval_mapillary_vistas.yaml"
26
+ # configs/coco/frozenseg/convnext_large_eval_bdd_panop.yaml
27
+ # configs/coco/frozenseg/convnext_large_eval_bdd_sem.yaml
28
+ )
29
+ port=$((10000 + RANDOM % 50000))
30
+ sam=vit_b
31
+ path=output/ConvNext-L_${sam}_1x
32
+ for config in "${configs[@]}"; do
33
+ python train_net.py --eval-only --num-gpus 1 --dist-url tcp://127.0.0.1:$port \
34
+ --config-file $config \
35
+ OUTPUT_DIR $path/$(basename "$config" .yaml) \
36
+ MODEL.WEIGHTS modified_model.pth \
37
+ MODEL.SAM_NAME vit_b \
38
+ MODEL.FROZEN_SEG.CLIP_PRETRAINED_WEIGHTS pretrained_checkpoint/models--laion--CLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soup/open_clip_pytorch_model.bin \
39
+ TEST.USE_SAM_MASKS False \
40
+ MODEL.FROZEN_SEG.GEOMETRIC_ENSEMBLE_BETA 0.6
41
+ done
42
+
43
+ ########## with mask ensemble ########
44
+ # for config in "${configs[@]}"; do
45
+ # python train_net.py --eval-only --num-gpus 1 --dist-url tcp://127.0.0.1:$port \
46
+ # --config-file $config \
47
+ # OUTPUT_DIR $path/w_maskEnsemble/$(basename "$config" .yaml) \
48
+ # MODEL.WEIGHTS $path/model_final.pth \
49
+ # MODEL.MASK_FORMER.SAM_QUERY_FUSE_LAYER 2 \
50
+ # MODEL.MASK_FORMER.SAM_FEATURE_FUSE_LAYER 0 \
51
+ # MODEL.SAM_NAME vit_b \
52
+ # MODEL.FROZEN_SEG.CLIP_PRETRAINED_WEIGHTS pretrained_checkpoint/models--laion--CLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soup/open_clip_pytorch_model.bin \
53
+ # TEST.USE_SAM_MASKS True \
54
+ # TEST.PKL_SAM_MODEL_NAME vit_h
55
+ # done
56
+
57
+
58
+ ########### test recall ############
59
+ # path=output/Sam_query/ConvNext-L_vit_b_1x
60
+ # for config in "${configs[@]}"; do
61
+ # srun python train_net.py --eval-only --num-gpus 4 --dist-url tcp://127.0.0.1:$port \
62
+ # --config-file $config \
63
+ # OUTPUT_DIR "output/Ablation/recall_withEverything/$(basename "$config" .yaml)" \
64
+ # MODEL.WEIGHTS "$path/model_final.pth" \
65
+ # TEST.USE_SAM_MASKS True \
66
+ # MODEL.MASK_FORMER.TEST.RECALL_ON True \
67
+ # MODEL.MASK_FORMER.TEST.SEMANTIC_ON False \
68
+ # MODEL.MASK_FORMER.TEST.INSTANCE_ON False \
69
+ # MODEL.MASK_FORMER.TEST.PANOPTIC_ON False \
70
+ # done
frozenseg/.DS_Store ADDED
Binary file (6.15 kB). View file
 
frozenseg/__init__.py ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ from . import data # register all new datasets
3
+ from . import modeling
4
+
5
+ # config
6
+ from .config import add_maskformer2_config, add_frozenseg_config
7
+
8
+ # dataset loading
9
+ from .data.dataset_mappers.coco_instance_new_baseline_dataset_mapper import COCOInstanceNewBaselineDatasetMapper
10
+ from .data.dataset_mappers.coco_panoptic_new_baseline_dataset_mapper import COCOPanopticNewBaselineDatasetMapper
11
+ from .data.dataset_mappers.mask_former_instance_dataset_mapper import (
12
+ MaskFormerInstanceDatasetMapper,
13
+ )
14
+ from .data.dataset_mappers.mask_former_panoptic_dataset_mapper import (
15
+ MaskFormerPanopticDatasetMapper,
16
+ )
17
+ from .data.dataset_mappers.mask_former_semantic_dataset_mapper import (
18
+ MaskFormerSemanticDatasetMapper,
19
+ )
20
+
21
+ # models
22
+ from .frozenseg import FrozenSeg
23
+ from .test_time_augmentation import SemanticSegmentorWithTTA
24
+
25
+ # evaluation
26
+ from .evaluation.instance_evaluation import InstanceSegEvaluator
frozenseg/config.py ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # -*- coding: utf-8 -*-
2
+ from detectron2.config import CfgNode as CN
3
+
4
+ def add_maskformer2_config(cfg):
5
+ """
6
+ Add config for MASK_FORMER.
7
+ """
8
+ # NOTE: configs from original maskformer
9
+ # data config
10
+ # select the dataset mapper
11
+ cfg.INPUT.DATASET_MAPPER_NAME = "mask_former_semantic"
12
+ # Color augmentation
13
+ cfg.INPUT.COLOR_AUG_SSD = False
14
+ # We retry random cropping until no single category in semantic segmentation GT occupies more
15
+ # than `SINGLE_CATEGORY_MAX_AREA` part of the crop.
16
+ cfg.INPUT.CROP.SINGLE_CATEGORY_MAX_AREA = 1.0
17
+ # Pad image and segmentation GT in dataset mapper.
18
+ cfg.INPUT.SIZE_DIVISIBILITY = -1
19
+
20
+ # solver config
21
+ # weight decay on embedding
22
+ cfg.SOLVER.WEIGHT_DECAY_EMBED = 0.0
23
+ # optimizer
24
+ cfg.SOLVER.OPTIMIZER = "ADAMW"
25
+ cfg.SOLVER.BACKBONE_MULTIPLIER = 0.1
26
+
27
+ # mask_former model config
28
+ cfg.MODEL.MASK_FORMER = CN()
29
+
30
+ # loss
31
+ cfg.MODEL.MASK_FORMER.DEEP_SUPERVISION = True
32
+ cfg.MODEL.MASK_FORMER.NO_OBJECT_WEIGHT = 0.1
33
+ cfg.MODEL.MASK_FORMER.CLASS_WEIGHT = 1.0
34
+ cfg.MODEL.MASK_FORMER.DICE_WEIGHT = 1.0
35
+ cfg.MODEL.MASK_FORMER.MASK_WEIGHT = 20.0
36
+
37
+ # transformer config
38
+ cfg.MODEL.MASK_FORMER.NHEADS = 8
39
+ cfg.MODEL.MASK_FORMER.DROPOUT = 0.1
40
+ cfg.MODEL.MASK_FORMER.DIM_FEEDFORWARD = 2048
41
+ cfg.MODEL.MASK_FORMER.ENC_LAYERS = 0
42
+ cfg.MODEL.MASK_FORMER.DEC_LAYERS = 6
43
+ cfg.MODEL.MASK_FORMER.PRE_NORM = False
44
+
45
+ cfg.MODEL.MASK_FORMER.HIDDEN_DIM = 256
46
+ cfg.MODEL.MASK_FORMER.NUM_OBJECT_QUERIES = 100
47
+
48
+ cfg.MODEL.MASK_FORMER.TRANSFORMER_IN_FEATURE = "res5"
49
+ cfg.MODEL.MASK_FORMER.ENFORCE_INPUT_PROJ = False
50
+
51
+ # mask_former inference config
52
+ cfg.MODEL.MASK_FORMER.TEST = CN()
53
+ cfg.MODEL.MASK_FORMER.TEST.SEMANTIC_ON = True
54
+ cfg.MODEL.MASK_FORMER.TEST.INSTANCE_ON = False
55
+ cfg.MODEL.MASK_FORMER.TEST.PANOPTIC_ON = False
56
+ cfg.MODEL.MASK_FORMER.TEST.OBJECT_MASK_THRESHOLD = 0.0
57
+ cfg.MODEL.MASK_FORMER.TEST.OVERLAP_THRESHOLD = 0.0
58
+ cfg.MODEL.MASK_FORMER.TEST.SEM_SEG_POSTPROCESSING_BEFORE_INFERENCE = False
59
+
60
+ # Sometimes `backbone.size_divisibility` is set to 0 for some backbone (e.g. ResNet)
61
+ # you can use this config to override
62
+ cfg.MODEL.MASK_FORMER.SIZE_DIVISIBILITY = 32
63
+
64
+ # pixel decoder config
65
+ cfg.MODEL.SEM_SEG_HEAD.MASK_DIM = 256
66
+ # adding transformer in pixel decoder
67
+ cfg.MODEL.SEM_SEG_HEAD.TRANSFORMER_ENC_LAYERS = 0
68
+ # pixel decoder
69
+ cfg.MODEL.SEM_SEG_HEAD.PIXEL_DECODER_NAME = "BasePixelDecoder"
70
+
71
+ # swin transformer backbone
72
+ cfg.MODEL.SWIN = CN()
73
+ cfg.MODEL.SWIN.PRETRAIN_IMG_SIZE = 224
74
+ cfg.MODEL.SWIN.PATCH_SIZE = 4
75
+ cfg.MODEL.SWIN.EMBED_DIM = 96
76
+ cfg.MODEL.SWIN.DEPTHS = [2, 2, 6, 2]
77
+ cfg.MODEL.SWIN.NUM_HEADS = [3, 6, 12, 24]
78
+ cfg.MODEL.SWIN.WINDOW_SIZE = 7
79
+ cfg.MODEL.SWIN.MLP_RATIO = 4.0
80
+ cfg.MODEL.SWIN.QKV_BIAS = True
81
+ cfg.MODEL.SWIN.QK_SCALE = None
82
+ cfg.MODEL.SWIN.DROP_RATE = 0.0
83
+ cfg.MODEL.SWIN.ATTN_DROP_RATE = 0.0
84
+ cfg.MODEL.SWIN.DROP_PATH_RATE = 0.3
85
+ cfg.MODEL.SWIN.APE = False
86
+ cfg.MODEL.SWIN.PATCH_NORM = True
87
+ cfg.MODEL.SWIN.OUT_FEATURES = ["res2", "res3", "res4", "res5"]
88
+ cfg.MODEL.SWIN.USE_CHECKPOINT = False
89
+
90
+ # NOTE: maskformer2 extra configs
91
+ # transformer module
92
+ cfg.MODEL.MASK_FORMER.TRANSFORMER_DECODER_NAME = "MultiScaleMaskedTransformerDecoder"
93
+
94
+ # LSJ aug
95
+ cfg.INPUT.IMAGE_SIZE = 1024
96
+ cfg.INPUT.MIN_SCALE = 0.1
97
+ cfg.INPUT.MAX_SCALE = 2.0
98
+
99
+ # MSDeformAttn encoder configs
100
+ cfg.MODEL.SEM_SEG_HEAD.DEFORMABLE_TRANSFORMER_ENCODER_IN_FEATURES = ["res3", "res4", "res5"]
101
+ cfg.MODEL.SEM_SEG_HEAD.DEFORMABLE_TRANSFORMER_ENCODER_N_POINTS = 4
102
+ cfg.MODEL.SEM_SEG_HEAD.DEFORMABLE_TRANSFORMER_ENCODER_N_HEADS = 8
103
+
104
+ # point loss configs
105
+ # Number of points sampled during training for a mask point head.
106
+ cfg.MODEL.MASK_FORMER.TRAIN_NUM_POINTS = 112 * 112
107
+ # Oversampling parameter for PointRend point sampling during training. Parameter `k` in the
108
+ # original paper.
109
+ cfg.MODEL.MASK_FORMER.OVERSAMPLE_RATIO = 3.0
110
+ # Importance sampling parameter for PointRend point sampling during training. Parametr `beta` in
111
+ # the original paper.
112
+ cfg.MODEL.MASK_FORMER.IMPORTANCE_SAMPLE_RATIO = 0.75
113
+
114
+
115
+ def add_frozenseg_config(cfg):
116
+ cfg.MODEL.SAM_NAME = 'vit_b'
117
+ cfg.MODEL.MASK_FORMER.SAM_QUERY_FUSE_LAYER = 2
118
+ cfg.MODEL.MASK_FORMER.SAM_FEATURE_FUSE_LAYER = 0
119
+ cfg.MODEL.MASK_FORMER.TEST.RECALL_ON = False
120
+
121
+ cfg.TEST.SAM_MASK_PRED_ALPHA = 0.2
122
+ cfg.TEST.USE_SAM_MASKS = False
123
+ cfg.TEST.PKL_SAM_MODEL_NAME = 'vit_h'
124
+
125
+ cfg.MODEL.FROZEN_SEG = CN()
126
+ cfg.MODEL.FROZEN_SEG.CLIP_PRETRAINED_WEIGHTS = "laion2b_s29b_b131k_ft_soup"
127
+ cfg.MODEL.FROZEN_SEG.CLIP_MODEL_NAME = "convnext_large_d_320"
128
+ cfg.MODEL.FROZEN_SEG.EMBED_DIM = 768
129
+ cfg.MODEL.FROZEN_SEG.GEOMETRIC_ENSEMBLE_ALPHA = 0.4
130
+ cfg.MODEL.FROZEN_SEG.GEOMETRIC_ENSEMBLE_BETA = 0.8
131
+ cfg.MODEL.FROZEN_SEG.ENSEMBLE_ON_VALID_MASK = False
132
+
frozenseg/data/.DS_Store ADDED
Binary file (6.15 kB). View file
 
frozenseg/data/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ from . import datasets
frozenseg/data/dataset_mappers/__init__.py ADDED
File without changes
frozenseg/data/dataset_mappers/bdd_semseg_dataset_mapper.py ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # --------------------------------------------------------
2
+ # X-Decoder -- Generalized Decoding for Pixel, Image, and Language
3
+ # Copyright (c) 2022 Microsoft
4
+ # Licensed under The MIT License [see LICENSE for details]
5
+ # Modified by Xueyan Zou (xueyan@cs.wisc.edu)
6
+ # --------------------------------------------------------
7
+ # Copyright (c) Facebook, Inc. and its affiliates.
8
+ import copy
9
+
10
+ import scipy.io
11
+ import numpy as np
12
+ import torch
13
+ from PIL import Image
14
+
15
+ from torchvision import transforms
16
+ from detectron2.utils import configurable
17
+
18
+ __all__ = ["BDDSemDatasetMapper"]
19
+
20
+
21
+ # This is specifically designed for the COCO dataset.
22
+ class BDDSemDatasetMapper:
23
+ """
24
+ A callable which takes a dataset dict in Detectron2 Dataset format,
25
+ and map it into a format used by MaskFormer.
26
+
27
+ This dataset mapper applies the same transformation as DETR for COCO panoptic segmentation.
28
+
29
+ The callable currently does the following:
30
+
31
+ 1. Read the image from "file_name"
32
+ 2. Applies geometric transforms to the image and annotation
33
+ 3. Find and applies suitable cropping to the image and annotation
34
+ 4. Prepare image and annotation to Tensors
35
+ """
36
+
37
+ @configurable
38
+ def __init__(
39
+ self,
40
+ is_train=True,
41
+ min_size_test=None,
42
+ max_size_test=None,
43
+ mean=None,
44
+ std=None,
45
+ ):
46
+ """
47
+ NOTE: this interface is experimental.
48
+ Args:
49
+ is_train: for training or inference
50
+ augmentations: a list of augmentations or deterministic transforms to apply
51
+ tfm_gens: data augmentation
52
+ image_format: an image format supported by :func:`detection_utils.read_image`.
53
+ """
54
+ self.is_train = is_train
55
+ self.min_size_test = min_size_test
56
+ self.max_size_test = max_size_test
57
+ self.pixel_mean = torch.tensor(mean)[:,None,None]
58
+ self.pixel_std = torch.tensor(std)[:,None,None]
59
+
60
+ t = []
61
+ t.append(transforms.Resize(self.min_size_test, interpolation=Image.BICUBIC))
62
+ self.transform = transforms.Compose(t)
63
+
64
+ @classmethod
65
+ def from_config(cls, cfg, is_train=True):
66
+ ret = {
67
+ "is_train": is_train,
68
+ "min_size_test": cfg['INPUT']['MIN_SIZE_TEST'],
69
+ "max_size_test": cfg['INPUT']['MAX_SIZE_TEST'],
70
+ "mean": cfg['INPUT']['PIXEL_MEAN'],
71
+ "std": cfg['INPUT']['PIXEL_STD'],
72
+ }
73
+ return ret
74
+
75
+ def read_semseg(self, file_name):
76
+ if '.png' in file_name:
77
+ semseg = np.asarray(Image.open(file_name))
78
+ elif '.mat' in file_name:
79
+ semseg = scipy.io.loadmat(file_name)['LabelMap']
80
+ return semseg
81
+
82
+ def __call__(self, dataset_dict):
83
+ """
84
+ Args:
85
+ dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.
86
+
87
+ Returns:
88
+ dict: a format that builtin models in detectron2 accept
89
+ """
90
+ dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below
91
+ file_name = dataset_dict['file_name']
92
+ semseg_name = dataset_dict['sem_seg_file_name']
93
+ image = Image.open(file_name).convert('RGB')
94
+
95
+ dataset_dict['width'] = image.size[0]
96
+ dataset_dict['height'] = image.size[1]
97
+
98
+ if self.is_train == False:
99
+ image = self.transform(image)
100
+ image = torch.from_numpy(np.asarray(image).copy())
101
+ image = image.permute(2,0,1)
102
+
103
+ semseg = self.read_semseg(semseg_name)
104
+ semseg = torch.from_numpy(semseg.astype(np.int32))
105
+ dataset_dict['image'] = image
106
+ dataset_dict['semseg'] = semseg
107
+ return dataset_dict
frozenseg/data/dataset_mappers/coco_instance_new_baseline_dataset_mapper.py ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import copy
2
+ import logging
3
+
4
+ import numpy as np
5
+ import torch
6
+
7
+ from detectron2.config import configurable
8
+ from detectron2.data import detection_utils as utils
9
+ from detectron2.data import transforms as T
10
+ from detectron2.data.transforms import TransformGen
11
+ from detectron2.structures import BitMasks, Instances
12
+
13
+ from pycocotools import mask as coco_mask
14
+
15
+ __all__ = ["COCOInstanceNewBaselineDatasetMapper"]
16
+
17
+
18
+ def convert_coco_poly_to_mask(segmentations, height, width):
19
+ masks = []
20
+ for polygons in segmentations:
21
+ rles = coco_mask.frPyObjects(polygons, height, width)
22
+ mask = coco_mask.decode(rles)
23
+ if len(mask.shape) < 3:
24
+ mask = mask[..., None]
25
+ mask = torch.as_tensor(mask, dtype=torch.uint8)
26
+ mask = mask.any(dim=2)
27
+ masks.append(mask)
28
+ if masks:
29
+ masks = torch.stack(masks, dim=0)
30
+ else:
31
+ masks = torch.zeros((0, height, width), dtype=torch.uint8)
32
+ return masks
33
+
34
+
35
+ def build_transform_gen(cfg, is_train):
36
+ """
37
+ Create a list of default :class:`Augmentation` from config.
38
+ Now it includes resizing and flipping.
39
+ Returns:
40
+ list[Augmentation]
41
+ """
42
+ assert is_train, "Only support training augmentation"
43
+ image_size = cfg.INPUT.IMAGE_SIZE
44
+ min_scale = cfg.INPUT.MIN_SCALE
45
+ max_scale = cfg.INPUT.MAX_SCALE
46
+
47
+ augmentation = []
48
+
49
+ if cfg.INPUT.RANDOM_FLIP != "none":
50
+ augmentation.append(
51
+ T.RandomFlip(
52
+ horizontal=cfg.INPUT.RANDOM_FLIP == "horizontal",
53
+ vertical=cfg.INPUT.RANDOM_FLIP == "vertical",
54
+ )
55
+ )
56
+
57
+ augmentation.extend([
58
+ T.ResizeScale(
59
+ min_scale=min_scale, max_scale=max_scale, target_height=image_size, target_width=image_size
60
+ ),
61
+ T.FixedSizeCrop(crop_size=(image_size, image_size)),
62
+ ])
63
+
64
+ return augmentation
65
+
66
+
67
+ # This is specifically designed for the COCO dataset.
68
+ class COCOInstanceNewBaselineDatasetMapper:
69
+ """
70
+ A callable which takes a dataset dict in Detectron2 Dataset format,
71
+ and map it into a format used by MaskFormer.
72
+
73
+ This dataset mapper applies the same transformation as DETR for COCO panoptic segmentation.
74
+
75
+ The callable currently does the following:
76
+
77
+ 1. Read the image from "file_name"
78
+ 2. Applies geometric transforms to the image and annotation
79
+ 3. Find and applies suitable cropping to the image and annotation
80
+ 4. Prepare image and annotation to Tensors
81
+ """
82
+
83
+ @configurable
84
+ def __init__(
85
+ self,
86
+ is_train=True,
87
+ *,
88
+ tfm_gens,
89
+ image_format,
90
+ ):
91
+ """
92
+ NOTE: this interface is experimental.
93
+ Args:
94
+ is_train: for training or inference
95
+ augmentations: a list of augmentations or deterministic transforms to apply
96
+ tfm_gens: data augmentation
97
+ image_format: an image format supported by :func:`detection_utils.read_image`.
98
+ """
99
+ self.tfm_gens = tfm_gens
100
+ logging.getLogger(__name__).info(
101
+ "[COCOInstanceNewBaselineDatasetMapper] Full TransformGens used in training: {}".format(str(self.tfm_gens))
102
+ )
103
+
104
+ self.img_format = image_format
105
+ self.is_train = is_train
106
+
107
+ @classmethod
108
+ def from_config(cls, cfg, is_train=True):
109
+ # Build augmentation
110
+ tfm_gens = build_transform_gen(cfg, is_train)
111
+
112
+ ret = {
113
+ "is_train": is_train,
114
+ "tfm_gens": tfm_gens,
115
+ "image_format": cfg.INPUT.FORMAT,
116
+ }
117
+ return ret
118
+
119
+ def __call__(self, dataset_dict):
120
+ """
121
+ Args:
122
+ dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.
123
+
124
+ Returns:
125
+ dict: a format that builtin models in detectron2 accept
126
+ """
127
+ dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below
128
+ image = utils.read_image(dataset_dict["file_name"], format=self.img_format)
129
+ utils.check_image_size(dataset_dict, image)
130
+
131
+ # TODO: get padding mask
132
+ # by feeding a "segmentation mask" to the same transforms
133
+ padding_mask = np.ones(image.shape[:2])
134
+
135
+ image, transforms = T.apply_transform_gens(self.tfm_gens, image)
136
+ # the crop transformation has default padding value 0 for segmentation
137
+ padding_mask = transforms.apply_segmentation(padding_mask)
138
+ padding_mask = ~ padding_mask.astype(bool)
139
+
140
+ image_shape = image.shape[:2] # h, w
141
+
142
+ # Pytorch's dataloader is efficient on torch.Tensor due to shared-memory,
143
+ # but not efficient on large generic data structures due to the use of pickle & mp.Queue.
144
+ # Therefore it's important to use torch.Tensor.
145
+ dataset_dict["image"] = torch.as_tensor(np.ascontiguousarray(image.transpose(2, 0, 1)))
146
+ dataset_dict["padding_mask"] = torch.as_tensor(np.ascontiguousarray(padding_mask))
147
+
148
+ if not self.is_train:
149
+ # USER: Modify this if you want to keep them for some reason.
150
+ dataset_dict.pop("annotations", None)
151
+ return dataset_dict
152
+
153
+ if "annotations" in dataset_dict:
154
+ # USER: Modify this if you want to keep them for some reason.
155
+ for anno in dataset_dict["annotations"]:
156
+ # Let's always keep mask
157
+ # if not self.mask_on:
158
+ # anno.pop("segmentation", None)
159
+ anno.pop("keypoints", None)
160
+
161
+ # USER: Implement additional transformations if you have other types of data
162
+ annos = [
163
+ utils.transform_instance_annotations(obj, transforms, image_shape)
164
+ for obj in dataset_dict.pop("annotations")
165
+ if obj.get("iscrowd", 0) == 0
166
+ ]
167
+ # NOTE: does not support BitMask due to augmentation
168
+ # Current BitMask cannot handle empty objects
169
+ instances = utils.annotations_to_instances(annos, image_shape)
170
+ # After transforms such as cropping are applied, the bounding box may no longer
171
+ # tightly bound the object. As an example, imagine a triangle object
172
+ # [(0,0), (2,0), (0,2)] cropped by a box [(1,0),(2,2)] (XYXY format). The tight
173
+ # bounding box of the cropped triangle should be [(1,0),(2,1)], which is not equal to
174
+ # the intersection of original bounding box and the cropping box.
175
+ instances.gt_boxes = instances.gt_masks.get_bounding_boxes()
176
+ # Need to filter empty instances first (due to augmentation)
177
+ instances = utils.filter_empty_instances(instances)
178
+ # Generate masks from polygon
179
+ h, w = instances.image_size
180
+ # image_size_xyxy = torch.as_tensor([w, h, w, h], dtype=torch.float)
181
+ if hasattr(instances, 'gt_masks'):
182
+ gt_masks = instances.gt_masks
183
+ gt_masks = convert_coco_poly_to_mask(gt_masks.polygons, h, w)
184
+ instances.gt_masks = gt_masks
185
+ dataset_dict["instances"] = instances
186
+
187
+ return dataset_dict
frozenseg/data/dataset_mappers/coco_panoptic_new_baseline_dataset_mapper.py ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import copy
2
+ import logging
3
+
4
+ import numpy as np
5
+ import torch
6
+
7
+ from detectron2.config import configurable
8
+ from detectron2.data import detection_utils as utils
9
+ from detectron2.data import transforms as T
10
+ from detectron2.data.transforms import TransformGen
11
+ from detectron2.structures import BitMasks, Boxes, Instances
12
+
13
+ __all__ = ["COCOPanopticNewBaselineDatasetMapper"]
14
+
15
+
16
+ def build_transform_gen(cfg, is_train):
17
+ """
18
+ Create a list of default :class:`Augmentation` from config.
19
+ Now it includes resizing and flipping.
20
+ Returns:
21
+ list[Augmentation]
22
+ """
23
+ assert is_train, "Only support training augmentation"
24
+ image_size = cfg.INPUT.IMAGE_SIZE
25
+ min_scale = cfg.INPUT.MIN_SCALE
26
+ max_scale = cfg.INPUT.MAX_SCALE
27
+
28
+ augmentation = []
29
+
30
+ if cfg.INPUT.RANDOM_FLIP != "none":
31
+ augmentation.append(
32
+ T.RandomFlip(
33
+ horizontal=cfg.INPUT.RANDOM_FLIP == "horizontal",
34
+ vertical=cfg.INPUT.RANDOM_FLIP == "vertical",
35
+ )
36
+ )
37
+
38
+ augmentation.extend([
39
+ T.ResizeScale(
40
+ min_scale=min_scale, max_scale=max_scale, target_height=image_size, target_width=image_size
41
+ ),
42
+ T.FixedSizeCrop(crop_size=(image_size, image_size)),
43
+ ])
44
+
45
+ return augmentation
46
+
47
+
48
+ # This is specifically designed for the COCO dataset.
49
+ class COCOPanopticNewBaselineDatasetMapper:
50
+ """
51
+ A callable which takes a dataset dict in Detectron2 Dataset format,
52
+ and map it into a format used by MaskFormer.
53
+
54
+ This dataset mapper applies the same transformation as DETR for COCO panoptic segmentation.
55
+
56
+ The callable currently does the following:
57
+
58
+ 1. Read the image from "file_name"
59
+ 2. Applies geometric transforms to the image and annotation
60
+ 3. Find and applies suitable cropping to the image and annotation
61
+ 4. Prepare image and annotation to Tensors
62
+ """
63
+
64
+ @configurable
65
+ def __init__(
66
+ self,
67
+ is_train=True,
68
+ *,
69
+ tfm_gens,
70
+ image_format,
71
+ ):
72
+ """
73
+ NOTE: this interface is experimental.
74
+ Args:
75
+ is_train: for training or inference
76
+ augmentations: a list of augmentations or deterministic transforms to apply
77
+ crop_gen: crop augmentation
78
+ tfm_gens: data augmentation
79
+ image_format: an image format supported by :func:`detection_utils.read_image`.
80
+ """
81
+ self.tfm_gens = tfm_gens
82
+ logging.getLogger(__name__).info(
83
+ "[COCOPanopticNewBaselineDatasetMapper] Full TransformGens used in training: {}".format(
84
+ str(self.tfm_gens)
85
+ )
86
+ )
87
+
88
+ self.img_format = image_format
89
+ self.is_train = is_train
90
+
91
+ @classmethod
92
+ def from_config(cls, cfg, is_train=True):
93
+ # Build augmentation
94
+ tfm_gens = build_transform_gen(cfg, is_train)
95
+
96
+ ret = {
97
+ "is_train": is_train,
98
+ "tfm_gens": tfm_gens,
99
+ "image_format": cfg.INPUT.FORMAT,
100
+ }
101
+ return ret
102
+
103
+ def __call__(self, dataset_dict):
104
+ """
105
+ Args:
106
+ dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.
107
+
108
+ Returns:
109
+ dict: a format that builtin models in detectron2 accept
110
+ """
111
+ dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below
112
+ image = utils.read_image(dataset_dict["file_name"], format=self.img_format)
113
+ utils.check_image_size(dataset_dict, image)
114
+
115
+ image, transforms = T.apply_transform_gens(self.tfm_gens, image)
116
+ image_shape = image.shape[:2] # h, w
117
+
118
+ # Pytorch's dataloader is efficient on torch.Tensor due to shared-memory,
119
+ # but not efficient on large generic data structures due to the use of pickle & mp.Queue.
120
+ # Therefore it's important to use torch.Tensor.
121
+ dataset_dict["image"] = torch.as_tensor(np.ascontiguousarray(image.transpose(2, 0, 1)))
122
+
123
+ if not self.is_train:
124
+ # USER: Modify this if you want to keep them for some reason.
125
+ dataset_dict.pop("annotations", None)
126
+ return dataset_dict
127
+
128
+ if "pan_seg_file_name" in dataset_dict:
129
+ pan_seg_gt = utils.read_image(dataset_dict.pop("pan_seg_file_name"), "RGB")
130
+ segments_info = dataset_dict["segments_info"]
131
+
132
+ # apply the same transformation to panoptic segmentation
133
+ pan_seg_gt = transforms.apply_segmentation(pan_seg_gt)
134
+
135
+ from panopticapi.utils import rgb2id
136
+
137
+ pan_seg_gt = rgb2id(pan_seg_gt)
138
+
139
+ instances = Instances(image_shape)
140
+ classes = []
141
+ masks = []
142
+ for segment_info in segments_info:
143
+ class_id = segment_info["category_id"]
144
+ if not segment_info["iscrowd"]:
145
+ classes.append(class_id)
146
+ masks.append(pan_seg_gt == segment_info["id"])
147
+
148
+ classes = np.array(classes)
149
+ instances.gt_classes = torch.tensor(classes, dtype=torch.int64)
150
+ if len(masks) == 0:
151
+ # Some image does not have annotation (all ignored)
152
+ instances.gt_masks = torch.zeros((0, pan_seg_gt.shape[-2], pan_seg_gt.shape[-1]))
153
+ instances.gt_boxes = Boxes(torch.zeros((0, 4)))
154
+ else:
155
+ masks = BitMasks(
156
+ torch.stack([torch.from_numpy(np.ascontiguousarray(x.copy())) for x in masks])
157
+ )
158
+ instances.gt_masks = masks.tensor
159
+ instances.gt_boxes = masks.get_bounding_boxes()
160
+
161
+ dataset_dict["instances"] = instances
162
+
163
+ return dataset_dict
frozenseg/data/dataset_mappers/mask_former_instance_dataset_mapper.py ADDED
@@ -0,0 +1,179 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import copy
2
+ import logging
3
+
4
+ import numpy as np
5
+ import pycocotools.mask as mask_util
6
+ import torch
7
+ from torch.nn import functional as F
8
+
9
+ from detectron2.config import configurable
10
+ from detectron2.data import detection_utils as utils
11
+ from detectron2.data import transforms as T
12
+ from detectron2.projects.point_rend import ColorAugSSDTransform
13
+ from detectron2.structures import BitMasks, Instances, polygons_to_bitmask
14
+
15
+ __all__ = ["MaskFormerInstanceDatasetMapper"]
16
+
17
+
18
+ class MaskFormerInstanceDatasetMapper:
19
+ """
20
+ A callable which takes a dataset dict in Detectron2 Dataset format,
21
+ and map it into a format used by MaskFormer for instance segmentation.
22
+
23
+ The callable currently does the following:
24
+
25
+ 1. Read the image from "file_name"
26
+ 2. Applies geometric transforms to the image and annotation
27
+ 3. Find and applies suitable cropping to the image and annotation
28
+ 4. Prepare image and annotation to Tensors
29
+ """
30
+
31
+ @configurable
32
+ def __init__(
33
+ self,
34
+ is_train=True,
35
+ *,
36
+ augmentations,
37
+ image_format,
38
+ size_divisibility,
39
+ ):
40
+ """
41
+ NOTE: this interface is experimental.
42
+ Args:
43
+ is_train: for training or inference
44
+ augmentations: a list of augmentations or deterministic transforms to apply
45
+ image_format: an image format supported by :func:`detection_utils.read_image`.
46
+ size_divisibility: pad image size to be divisible by this value
47
+ """
48
+ self.is_train = is_train
49
+ self.tfm_gens = augmentations
50
+ self.img_format = image_format
51
+ self.size_divisibility = size_divisibility
52
+
53
+ logger = logging.getLogger(__name__)
54
+ mode = "training" if is_train else "inference"
55
+ logger.info(f"[{self.__class__.__name__}] Augmentations used in {mode}: {augmentations}")
56
+
57
+ @classmethod
58
+ def from_config(cls, cfg, is_train=True):
59
+ # Build augmentation
60
+ augs = [
61
+ T.ResizeShortestEdge(
62
+ cfg.INPUT.MIN_SIZE_TRAIN,
63
+ cfg.INPUT.MAX_SIZE_TRAIN,
64
+ cfg.INPUT.MIN_SIZE_TRAIN_SAMPLING,
65
+ )
66
+ ]
67
+ if cfg.INPUT.CROP.ENABLED:
68
+ augs.append(
69
+ T.RandomCrop(
70
+ cfg.INPUT.CROP.TYPE,
71
+ cfg.INPUT.CROP.SIZE,
72
+ )
73
+ )
74
+ if cfg.INPUT.COLOR_AUG_SSD:
75
+ augs.append(ColorAugSSDTransform(img_format=cfg.INPUT.FORMAT))
76
+ augs.append(T.RandomFlip())
77
+
78
+ ret = {
79
+ "is_train": is_train,
80
+ "augmentations": augs,
81
+ "image_format": cfg.INPUT.FORMAT,
82
+ "size_divisibility": cfg.INPUT.SIZE_DIVISIBILITY,
83
+ }
84
+ return ret
85
+
86
+ def __call__(self, dataset_dict):
87
+ """
88
+ Args:
89
+ dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.
90
+
91
+ Returns:
92
+ dict: a format that builtin models in detectron2 accept
93
+ """
94
+ assert self.is_train, "MaskFormerPanopticDatasetMapper should only be used for training!"
95
+
96
+ dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below
97
+ image = utils.read_image(dataset_dict["file_name"], format=self.img_format)
98
+ utils.check_image_size(dataset_dict, image)
99
+
100
+ aug_input = T.AugInput(image)
101
+ aug_input, transforms = T.apply_transform_gens(self.tfm_gens, aug_input)
102
+ image = aug_input.image
103
+
104
+ # transform instnace masks
105
+ assert "annotations" in dataset_dict
106
+ for anno in dataset_dict["annotations"]:
107
+ anno.pop("keypoints", None)
108
+
109
+ annos = [
110
+ utils.transform_instance_annotations(obj, transforms, image.shape[:2])
111
+ for obj in dataset_dict.pop("annotations")
112
+ if obj.get("iscrowd", 0) == 0
113
+ ]
114
+
115
+ if len(annos):
116
+ assert "segmentation" in annos[0]
117
+ segms = [obj["segmentation"] for obj in annos]
118
+ masks = []
119
+ for segm in segms:
120
+ if isinstance(segm, list):
121
+ # polygon
122
+ masks.append(polygons_to_bitmask(segm, *image.shape[:2]))
123
+ elif isinstance(segm, dict):
124
+ # COCO RLE
125
+ masks.append(mask_util.decode(segm))
126
+ elif isinstance(segm, np.ndarray):
127
+ assert segm.ndim == 2, "Expect segmentation of 2 dimensions, got {}.".format(
128
+ segm.ndim
129
+ )
130
+ # mask array
131
+ masks.append(segm)
132
+ else:
133
+ raise ValueError(
134
+ "Cannot convert segmentation of type '{}' to BitMasks!"
135
+ "Supported types are: polygons as list[list[float] or ndarray],"
136
+ " COCO-style RLE as a dict, or a binary segmentation mask "
137
+ " in a 2D numpy array of shape HxW.".format(type(segm))
138
+ )
139
+
140
+ # Pad image and segmentation label here!
141
+ image = torch.as_tensor(np.ascontiguousarray(image.transpose(2, 0, 1)))
142
+ masks = [torch.from_numpy(np.ascontiguousarray(x)) for x in masks]
143
+
144
+ classes = [int(obj["category_id"]) for obj in annos]
145
+ classes = torch.tensor(classes, dtype=torch.int64)
146
+
147
+ if self.size_divisibility > 0:
148
+ image_size = (image.shape[-2], image.shape[-1])
149
+ padding_size = [
150
+ 0,
151
+ self.size_divisibility - image_size[1],
152
+ 0,
153
+ self.size_divisibility - image_size[0],
154
+ ]
155
+ # pad image
156
+ image = F.pad(image, padding_size, value=128).contiguous()
157
+ # pad mask
158
+ masks = [F.pad(x, padding_size, value=0).contiguous() for x in masks]
159
+
160
+ image_shape = (image.shape[-2], image.shape[-1]) # h, w
161
+
162
+ # Pytorch's dataloader is efficient on torch.Tensor due to shared-memory,
163
+ # but not efficient on large generic data structures due to the use of pickle & mp.Queue.
164
+ # Therefore it's important to use torch.Tensor.
165
+ dataset_dict["image"] = image
166
+
167
+ # Prepare per-category binary masks
168
+ instances = Instances(image_shape)
169
+ instances.gt_classes = classes
170
+ if len(masks) == 0:
171
+ # Some image does not have annotation (all ignored)
172
+ instances.gt_masks = torch.zeros((0, image.shape[-2], image.shape[-1]))
173
+ else:
174
+ masks = BitMasks(torch.stack(masks))
175
+ instances.gt_masks = masks.tensor
176
+
177
+ dataset_dict["instances"] = instances
178
+
179
+ return dataset_dict
frozenseg/data/dataset_mappers/mask_former_panoptic_dataset_mapper.py ADDED
@@ -0,0 +1,164 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import copy
2
+ import logging
3
+
4
+ import numpy as np
5
+ import torch
6
+ from torch.nn import functional as F
7
+
8
+ from detectron2.config import configurable
9
+ from detectron2.data import detection_utils as utils
10
+ from detectron2.data import transforms as T
11
+ from detectron2.structures import BitMasks, Instances
12
+
13
+ from .mask_former_semantic_dataset_mapper import MaskFormerSemanticDatasetMapper
14
+
15
+ __all__ = ["MaskFormerPanopticDatasetMapper"]
16
+
17
+
18
+ class MaskFormerPanopticDatasetMapper(MaskFormerSemanticDatasetMapper):
19
+ """
20
+ A callable which takes a dataset dict in Detectron2 Dataset format,
21
+ and map it into a format used by MaskFormer for panoptic segmentation.
22
+
23
+ The callable currently does the following:
24
+
25
+ 1. Read the image from "file_name"
26
+ 2. Applies geometric transforms to the image and annotation
27
+ 3. Find and applies suitable cropping to the image and annotation
28
+ 4. Prepare image and annotation to Tensors
29
+ """
30
+
31
+ @configurable
32
+ def __init__(
33
+ self,
34
+ is_train=True,
35
+ *,
36
+ augmentations,
37
+ image_format,
38
+ ignore_label,
39
+ size_divisibility,
40
+ ):
41
+ """
42
+ NOTE: this interface is experimental.
43
+ Args:
44
+ is_train: for training or inference
45
+ augmentations: a list of augmentations or deterministic transforms to apply
46
+ image_format: an image format supported by :func:`detection_utils.read_image`.
47
+ ignore_label: the label that is ignored to evaluation
48
+ size_divisibility: pad image size to be divisible by this value
49
+ """
50
+ super().__init__(
51
+ is_train,
52
+ augmentations=augmentations,
53
+ image_format=image_format,
54
+ ignore_label=ignore_label,
55
+ size_divisibility=size_divisibility,
56
+ )
57
+
58
+ def __call__(self, dataset_dict):
59
+ """
60
+ Args:
61
+ dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.
62
+
63
+ Returns:
64
+ dict: a format that builtin models in detectron2 accept
65
+ """
66
+ assert self.is_train, "MaskFormerPanopticDatasetMapper should only be used for training!"
67
+
68
+ dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below
69
+ image = utils.read_image(dataset_dict["file_name"], format=self.img_format)
70
+ utils.check_image_size(dataset_dict, image)
71
+
72
+ # semantic segmentation
73
+ if "sem_seg_file_name" in dataset_dict:
74
+ # PyTorch transformation not implemented for uint16, so converting it to double first
75
+ sem_seg_gt = utils.read_image(dataset_dict.pop("sem_seg_file_name")).astype("double")
76
+ else:
77
+ sem_seg_gt = None
78
+
79
+ # panoptic segmentation
80
+ if "pan_seg_file_name" in dataset_dict:
81
+ pan_seg_gt = utils.read_image(dataset_dict.pop("pan_seg_file_name"), "RGB")
82
+ segments_info = dataset_dict["segments_info"]
83
+ else:
84
+ pan_seg_gt = None
85
+ segments_info = None
86
+
87
+ if pan_seg_gt is None:
88
+ raise ValueError(
89
+ "Cannot find 'pan_seg_file_name' for panoptic segmentation dataset {}.".format(
90
+ dataset_dict["file_name"]
91
+ )
92
+ )
93
+
94
+ aug_input = T.AugInput(image, sem_seg=sem_seg_gt)
95
+ aug_input, transforms = T.apply_transform_gens(self.tfm_gens, aug_input)
96
+ image = aug_input.image
97
+ if sem_seg_gt is not None:
98
+ sem_seg_gt = aug_input.sem_seg
99
+
100
+ # apply the same transformation to panoptic segmentation
101
+ pan_seg_gt = transforms.apply_segmentation(pan_seg_gt)
102
+
103
+ from panopticapi.utils import rgb2id
104
+
105
+ pan_seg_gt = rgb2id(pan_seg_gt)
106
+
107
+ # Pad image and segmentation label here!
108
+ image = torch.as_tensor(np.ascontiguousarray(image.transpose(2, 0, 1)))
109
+ if sem_seg_gt is not None:
110
+ sem_seg_gt = torch.as_tensor(sem_seg_gt.astype("long"))
111
+ pan_seg_gt = torch.as_tensor(pan_seg_gt.astype("long"))
112
+
113
+ if self.size_divisibility > 0:
114
+ image_size = (image.shape[-2], image.shape[-1])
115
+ padding_size = [
116
+ 0,
117
+ self.size_divisibility - image_size[1],
118
+ 0,
119
+ self.size_divisibility - image_size[0],
120
+ ]
121
+ image = F.pad(image, padding_size, value=128).contiguous()
122
+ if sem_seg_gt is not None:
123
+ sem_seg_gt = F.pad(sem_seg_gt, padding_size, value=self.ignore_label).contiguous()
124
+ pan_seg_gt = F.pad(
125
+ pan_seg_gt, padding_size, value=0
126
+ ).contiguous() # 0 is the VOID panoptic label
127
+
128
+ image_shape = (image.shape[-2], image.shape[-1]) # h, w
129
+
130
+ # Pytorch's dataloader is efficient on torch.Tensor due to shared-memory,
131
+ # but not efficient on large generic data structures due to the use of pickle & mp.Queue.
132
+ # Therefore it's important to use torch.Tensor.
133
+ dataset_dict["image"] = image
134
+ if sem_seg_gt is not None:
135
+ dataset_dict["sem_seg"] = sem_seg_gt.long()
136
+
137
+ if "annotations" in dataset_dict:
138
+ raise ValueError("Pemantic segmentation dataset should not have 'annotations'.")
139
+
140
+ # Prepare per-category binary masks
141
+ pan_seg_gt = pan_seg_gt.numpy()
142
+ instances = Instances(image_shape)
143
+ classes = []
144
+ masks = []
145
+ for segment_info in segments_info:
146
+ class_id = segment_info["category_id"]
147
+ if not segment_info["iscrowd"]:
148
+ classes.append(class_id)
149
+ masks.append(pan_seg_gt == segment_info["id"])
150
+
151
+ classes = np.array(classes)
152
+ instances.gt_classes = torch.tensor(classes, dtype=torch.int64)
153
+ if len(masks) == 0:
154
+ # Some image does not have annotation (all ignored)
155
+ instances.gt_masks = torch.zeros((0, pan_seg_gt.shape[-2], pan_seg_gt.shape[-1]))
156
+ else:
157
+ masks = BitMasks(
158
+ torch.stack([torch.from_numpy(np.ascontiguousarray(x.copy())) for x in masks])
159
+ )
160
+ instances.gt_masks = masks.tensor
161
+
162
+ dataset_dict["instances"] = instances
163
+
164
+ return dataset_dict
frozenseg/data/dataset_mappers/mask_former_semantic_dataset_mapper.py ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import copy
2
+ import logging
3
+
4
+ import numpy as np
5
+ import torch
6
+ from torch.nn import functional as F
7
+
8
+ from detectron2.config import configurable
9
+ from detectron2.data import MetadataCatalog
10
+ from detectron2.data import detection_utils as utils
11
+ from detectron2.data import transforms as T
12
+ from detectron2.projects.point_rend import ColorAugSSDTransform
13
+ from detectron2.structures import BitMasks, Instances
14
+
15
+ __all__ = ["MaskFormerSemanticDatasetMapper"]
16
+
17
+
18
+ class MaskFormerSemanticDatasetMapper:
19
+ """
20
+ A callable which takes a dataset dict in Detectron2 Dataset format,
21
+ and map it into a format used by MaskFormer for semantic segmentation.
22
+
23
+ The callable currently does the following:
24
+
25
+ 1. Read the image from "file_name"
26
+ 2. Applies geometric transforms to the image and annotation
27
+ 3. Find and applies suitable cropping to the image and annotation
28
+ 4. Prepare image and annotation to Tensors
29
+ """
30
+
31
+ @configurable
32
+ def __init__(
33
+ self,
34
+ is_train=True,
35
+ *,
36
+ augmentations,
37
+ image_format,
38
+ ignore_label,
39
+ size_divisibility,
40
+ ):
41
+ """
42
+ NOTE: this interface is experimental.
43
+ Args:
44
+ is_train: for training or inference
45
+ augmentations: a list of augmentations or deterministic transforms to apply
46
+ image_format: an image format supported by :func:`detection_utils.read_image`.
47
+ ignore_label: the label that is ignored to evaluation
48
+ size_divisibility: pad image size to be divisible by this value
49
+ """
50
+ self.is_train = is_train
51
+ self.tfm_gens = augmentations
52
+ self.img_format = image_format
53
+ self.ignore_label = ignore_label
54
+ self.size_divisibility = size_divisibility
55
+
56
+ logger = logging.getLogger(__name__)
57
+ mode = "training" if is_train else "inference"
58
+ logger.info(f"[{self.__class__.__name__}] Augmentations used in {mode}: {augmentations}")
59
+
60
+ @classmethod
61
+ def from_config(cls, cfg, is_train=True):
62
+ # Build augmentation
63
+ augs = [
64
+ T.ResizeShortestEdge(
65
+ cfg.INPUT.MIN_SIZE_TRAIN,
66
+ cfg.INPUT.MAX_SIZE_TRAIN,
67
+ cfg.INPUT.MIN_SIZE_TRAIN_SAMPLING,
68
+ )
69
+ ]
70
+ if cfg.INPUT.CROP.ENABLED:
71
+ augs.append(
72
+ T.RandomCrop_CategoryAreaConstraint(
73
+ cfg.INPUT.CROP.TYPE,
74
+ cfg.INPUT.CROP.SIZE,
75
+ cfg.INPUT.CROP.SINGLE_CATEGORY_MAX_AREA,
76
+ cfg.MODEL.SEM_SEG_HEAD.IGNORE_VALUE,
77
+ )
78
+ )
79
+ if cfg.INPUT.COLOR_AUG_SSD:
80
+ augs.append(ColorAugSSDTransform(img_format=cfg.INPUT.FORMAT))
81
+ augs.append(T.RandomFlip())
82
+
83
+ # Assume always applies to the training set.
84
+ dataset_names = cfg.DATASETS.TRAIN
85
+ meta = MetadataCatalog.get(dataset_names[0])
86
+ ignore_label = meta.ignore_label
87
+
88
+ ret = {
89
+ "is_train": is_train,
90
+ "augmentations": augs,
91
+ "image_format": cfg.INPUT.FORMAT,
92
+ "ignore_label": ignore_label,
93
+ "size_divisibility": cfg.INPUT.SIZE_DIVISIBILITY,
94
+ }
95
+ return ret
96
+
97
+ def __call__(self, dataset_dict):
98
+ """
99
+ Args:
100
+ dataset_dict (dict): Metadata of one image, in Detectron2 Dataset format.
101
+
102
+ Returns:
103
+ dict: a format that builtin models in detectron2 accept
104
+ """
105
+ assert self.is_train, "MaskFormerSemanticDatasetMapper should only be used for training!"
106
+
107
+ dataset_dict = copy.deepcopy(dataset_dict) # it will be modified by code below
108
+ image = utils.read_image(dataset_dict["file_name"], format=self.img_format)
109
+ utils.check_image_size(dataset_dict, image)
110
+
111
+ if "sem_seg_file_name" in dataset_dict:
112
+ # PyTorch transformation not implemented for uint16, so converting it to double first
113
+ sem_seg_gt = utils.read_image(dataset_dict.pop("sem_seg_file_name")).astype("double")
114
+ else:
115
+ sem_seg_gt = None
116
+
117
+ if sem_seg_gt is None:
118
+ raise ValueError(
119
+ "Cannot find 'sem_seg_file_name' for semantic segmentation dataset {}.".format(
120
+ dataset_dict["file_name"]
121
+ )
122
+ )
123
+
124
+ aug_input = T.AugInput(image, sem_seg=sem_seg_gt)
125
+ aug_input, transforms = T.apply_transform_gens(self.tfm_gens, aug_input)
126
+ image = aug_input.image
127
+ sem_seg_gt = aug_input.sem_seg
128
+
129
+ # Pad image and segmentation label here!
130
+ image = torch.as_tensor(np.ascontiguousarray(image.transpose(2, 0, 1)))
131
+ if sem_seg_gt is not None:
132
+ sem_seg_gt = torch.as_tensor(sem_seg_gt.astype("long"))
133
+
134
+ if self.size_divisibility > 0:
135
+ image_size = (image.shape[-2], image.shape[-1])
136
+ padding_size = [
137
+ 0,
138
+ self.size_divisibility - image_size[1],
139
+ 0,
140
+ self.size_divisibility - image_size[0],
141
+ ]
142
+ image = F.pad(image, padding_size, value=128).contiguous()
143
+ if sem_seg_gt is not None:
144
+ sem_seg_gt = F.pad(sem_seg_gt, padding_size, value=self.ignore_label).contiguous()
145
+
146
+ image_shape = (image.shape[-2], image.shape[-1]) # h, w
147
+
148
+ # Pytorch's dataloader is efficient on torch.Tensor due to shared-memory,
149
+ # but not efficient on large generic data structures due to the use of pickle & mp.Queue.
150
+ # Therefore it's important to use torch.Tensor.
151
+ dataset_dict["image"] = image
152
+
153
+ if sem_seg_gt is not None:
154
+ dataset_dict["sem_seg"] = sem_seg_gt.long()
155
+
156
+ if "annotations" in dataset_dict:
157
+ raise ValueError("Semantic segmentation dataset should not have 'annotations'.")
158
+
159
+ # Prepare per-category binary masks
160
+ if sem_seg_gt is not None:
161
+ sem_seg_gt = sem_seg_gt.numpy()
162
+ instances = Instances(image_shape)
163
+ classes = np.unique(sem_seg_gt)
164
+ # remove ignored region
165
+ classes = classes[classes != self.ignore_label]
166
+ instances.gt_classes = torch.tensor(classes, dtype=torch.int64)
167
+
168
+ masks = []
169
+ for class_id in classes:
170
+ masks.append(sem_seg_gt == class_id)
171
+
172
+ if len(masks) == 0:
173
+ # Some image does not have annotation (all ignored)
174
+ instances.gt_masks = torch.zeros((0, sem_seg_gt.shape[-2], sem_seg_gt.shape[-1]))
175
+ else:
176
+ masks = BitMasks(
177
+ torch.stack([torch.from_numpy(np.ascontiguousarray(x.copy())) for x in masks])
178
+ )
179
+ instances.gt_masks = masks.tensor
180
+
181
+ dataset_dict["instances"] = instances
182
+
183
+ return dataset_dict
frozenseg/data/datasets/__init__.py ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from . import (
2
+ register_lvis_instance,
3
+ register_coco_panoptic_annos_semseg,
4
+ register_ade20k_panoptic,
5
+ register_cityscapes_panoptic,
6
+ register_mapillary_vistas_panoptic,
7
+ register_ade20k_full,
8
+ register_pascal_voc_20_semantic,
9
+ register_pascal_voc_21_semantic,
10
+ register_pascal_ctx_59_sem_seg,
11
+ register_pascal_ctx_459_sem_seg,
12
+ register_coco_instance,
13
+ register_ade20k_instance,
14
+ register_coco_stuff_164k,
15
+ openseg_classes,
16
+ register_bdd100k_panoseg,
17
+ register_bdd100k_semseg,
18
+ )
frozenseg/data/datasets/ade20k_150_with_prompt_eng.txt ADDED
@@ -0,0 +1,151 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 0:invalid_class_id
2
+ 1:wall,walls,brick wall,stone wall,interior wall
3
+ 2:building,buildings,edifice,edifices
4
+ 3:sky,clouds
5
+ 4:floor,flooring
6
+ 5:tree,trees
7
+ 6:ceiling
8
+ 7:road,route,street,roads,streets,routes
9
+ 8:bed,beds
10
+ 9:windowpane,window,windows
11
+ 10:grass,grass field
12
+ 11:cabinet,cabinets,wall mounted cabine
13
+ 12:sidewalk,pavement
14
+ 13:person,child,girl,boy,woman,man,people,children,girls,boys,women,men
15
+ 14:earth,ground
16
+ 15:door,double door,doors
17
+ 16:table,tables,tablecloth
18
+ 17:mountain,mount,mountains
19
+ 18:plant,flora,plant life,plants,bushes
20
+ 19:curtain,drape,drapery,mantle,pall
21
+ 20:chair,chairs
22
+ 21:car,automobile,cars
23
+ 22:water
24
+ 23:painting,picture,paintings,pictures,wallart,framed canvas
25
+ 24:sofa,couch,sofas,couches
26
+ 25:shelf,shelves
27
+ 26:house exterior
28
+ 27:sea,ocean
29
+ 28:mirror,mirrors
30
+ 29:rug,carpet,carpeting
31
+ 30:field
32
+ 31:armchair,armchairs
33
+ 32:seat,seats
34
+ 33:fence,fencing
35
+ 34:desk,desks
36
+ 35:rock,stone,rocks,stones
37
+ 36:wardrobe,closet,press,wardrobes,closets
38
+ 37:lamp,lamps
39
+ 38:bathtub,bathing tub,bath,tub
40
+ 39:railing,rail
41
+ 40:cushion,cushions
42
+ 41:pedestal
43
+ 42:box,boxes
44
+ 43:column,pillar
45
+ 44:signboard,sign,signboards,signs
46
+ 45:chest of drawers,chest,bureau,dresser
47
+ 46:counter
48
+ 47:sand
49
+ 48:sink
50
+ 49:skyscraper,skyscrapers
51
+ 50:fireplace,hearth,open fireplace
52
+ 51:refrigerator,icebox
53
+ 52:grandstand,covered stand
54
+ 53:path
55
+ 54:stairs,steps
56
+ 55:runway
57
+ 56:case,display case,showcase,vitrine
58
+ 57:pool table,billiard table,snooker table
59
+ 58:pillow,pillows
60
+ 59:screen door,shower door
61
+ 60:stairway,staircase
62
+ 61:river
63
+ 62:bridge,span
64
+ 63:bookcase
65
+ 64:window screen,door screen
66
+ 65:coffee table,cocktail table
67
+ 66:toilet,commode,crapper,potty
68
+ 67:flower,flowers
69
+ 68:book,books
70
+ 69:hill
71
+ 70:bench,benches
72
+ 71:countertop,counter top,worktop
73
+ 72:stove,kitchen stove,kitchen range,kitchen range,cooking stove
74
+ 73:palm tree,palm trees
75
+ 74:kitchen island
76
+ 75:computer,computing machine,computing device,data processor,electronic computer,information processing system
77
+ 76:swivel chair
78
+ 77:boat
79
+ 78:bar
80
+ 79:arcade machine,arcade machines
81
+ 80:hovel,hut,hutch,shack,shanty
82
+ 81:bus,autobus,double-decker,jitney,motorbus,motorcoach,omnibus,passenger vehicle
83
+ 82:towel
84
+ 83:light bulb,lightbulb,bulb,incandescent lamp,electric light,electric-light bulb
85
+ 84:truck,motortruck
86
+ 85:tower,towers
87
+ 86:chandelier,pendant,pendent
88
+ 87:awning,sunshade,sunblind
89
+ 88:streetlight,street lamp
90
+ 89:booth,cubicle,stall,kiosk
91
+ 90:television receiver,television,television set,tv,tv set
92
+ 91:airplane,aeroplane,airplanes,aeroplanes
93
+ 92:dirt track
94
+ 93:apparel,wearing apparel,dress,clothes
95
+ 94:pole
96
+ 95:land,soil
97
+ 96:bannister,banister,balustrade,balusters,handrail
98
+ 97:escalator,moving staircase,moving stairway
99
+ 98:ottoman,pouf,pouffe,puff,hassock
100
+ 99:bottle,bottles,water bottle
101
+ 100:buffet,sideboard
102
+ 101:poster,posting,placard,notice,bill,card
103
+ 102:stage
104
+ 103:van
105
+ 104:ship
106
+ 105:fountain
107
+ 106:conveyer belt,conveyor belt,conveyer,conveyor,transporter
108
+ 107:canopy
109
+ 108:washer,automatic washer,washing machine
110
+ 109:plaything,toy,toys
111
+ 110:swimming pool,swimming bath
112
+ 111:stool,stools
113
+ 112:barrel,cask,barrels,casks
114
+ 113:basket,handbasket
115
+ 114:waterfall,falls
116
+ 115:tent,collapsible shelter
117
+ 116:bag,bags,gift bag,paper bag
118
+ 117:minibike,motorbike
119
+ 118:cradle
120
+ 119:oven
121
+ 120:ball,balls
122
+ 121:food,solid food
123
+ 122:step,stair
124
+ 123:tank,storage tank
125
+ 124:trade name,brand name,brand,marque
126
+ 125:microwave,microwave oven
127
+ 126:plant pots,plant pot,flower pot,flowerpot,planter
128
+ 127:animal,animate being,dog,cat,horse,cow,sheep,zebra,girraffe,bird
129
+ 128:bicycle,bike
130
+ 129:lake
131
+ 130:dishwasher,dish washer,dishwashing machine
132
+ 131:projection screen
133
+ 132:blanket,cover
134
+ 133:sculpture,sculptures
135
+ 134:exhaust hood
136
+ 135:sconce,sconce lamp,sconce light
137
+ 136:vase,vases
138
+ 137:traffic light,traffic signal,traffic lights
139
+ 138:tray,trays
140
+ 139:ashcan,trash can,garbage can,wastebin,ash bin,ash-bin,ashbin,dustbin,trash barrel,trash bin
141
+ 140:ceiling fan,floor fan
142
+ 141:pier,wharf,wharfage,dock
143
+ 142:crt screen
144
+ 143:plate,plates
145
+ 144:monitor,monitoring device,monitors
146
+ 145:bulletin board,notice board
147
+ 146:shower
148
+ 147:radiator
149
+ 148:cup,cups,drinking glass,drinking glasses
150
+ 149:clock
151
+ 150:flag,flags
frozenseg/data/datasets/ade20k_847_with_prompt_eng.txt ADDED
@@ -0,0 +1,848 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 0:invalid_class_id
2
+ 1:wall,walls,interior wall,brick wall,stone wall
3
+ 2:building,buildings,edifice,edifices
4
+ 3:sky,clouds
5
+ 4:tree,trees
6
+ 5:road,route,street,roads,streets,routes
7
+ 6:floor,flooring
8
+ 7:ceiling
9
+ 8:bed,beds
10
+ 9:sidewalk,pavement
11
+ 10:earth,ground
12
+ 11:cabinet,cabinets,wall mounted cabine
13
+ 12:person,child,girl,boy,woman,man,people,children,girls,boys,women,men
14
+ 13:grass,grass field
15
+ 14:windowpane,window,windows
16
+ 15:car,automobile,cars
17
+ 16:mountain,mount,mountains
18
+ 17:plant,flora,plant life,plants,bushes
19
+ 18:table,tables,tablecloth
20
+ 19:chair,chairs
21
+ 20:curtain,drape,drapery,mantle,pall
22
+ 21:door,double door,doors
23
+ 22:sofa,couch,sofas,couches
24
+ 23:sea,ocean
25
+ 24:painting,picture,paintings,pictures,wallart,framed canvas
26
+ 25:water
27
+ 26:mirror,mirrors
28
+ 27:house exterior
29
+ 28:rug,carpet,carpeting
30
+ 29:shelf,shelves
31
+ 30:armchair,armchairs
32
+ 31:fence,fencing
33
+ 32:field
34
+ 33:lamp,lamps
35
+ 34:rock,stone,rocks,stones
36
+ 35:seat,seats
37
+ 36:river
38
+ 37:desk,desks
39
+ 38:bathtub,bathing tub,bath,tub
40
+ 39:railing,rail
41
+ 40:signboard,sign,signboards,signs
42
+ 41:cushion,cushions
43
+ 42:path
44
+ 43:work surface
45
+ 44:stairs,steps
46
+ 45:column,pillar
47
+ 46:sink
48
+ 47:wardrobe,closet,press,wardrobes,closets
49
+ 48:snow
50
+ 49:refrigerator,icebox
51
+ 50:pedestal
52
+ 51:bridge,span
53
+ 52:blind
54
+ 53:runway
55
+ 54:cliff,drop,drop-off
56
+ 55:sand
57
+ 56:fireplace,hearth,open fireplace
58
+ 57:pillow,pillows
59
+ 58:screen door,shower door
60
+ 59:toilet,commode,crapper,potty
61
+ 60:skyscraper,skyscrapers
62
+ 61:grandstand,covered stand
63
+ 62:box,boxes
64
+ 63:pool table,billiard table,snooker table
65
+ 64:palm tree,palm trees
66
+ 65:double door
67
+ 66:coffee table,cocktail table
68
+ 67:counter
69
+ 68:countertop,counter top,worktop
70
+ 69:chest of drawers,chest,bureau,dresser
71
+ 70:kitchen island
72
+ 71:boat
73
+ 72:waterfall,falls
74
+ 73:stove,kitchen stove,kitchen range,kitchen range,cooking stove
75
+ 74:flower,flowers
76
+ 75:bookcase
77
+ 76:controls
78
+ 77:book,books
79
+ 78:stairway,staircase
80
+ 79:streetlight,street lamp
81
+ 80:computer,computing machine,computing device,data processor,electronic computer,information processing system
82
+ 81:bus,autobus,double-decker,jitney,motorbus,motorcoach,omnibus,passenger vehicle
83
+ 82:swivel chair
84
+ 83:light,light source
85
+ 84:bench,benches
86
+ 85:case,display case,showcase,vitrine
87
+ 86:towel
88
+ 87:fountain
89
+ 88:embankment
90
+ 89:television receiver,television,television set,tv,tv set
91
+ 90:van
92
+ 91:hill
93
+ 92:awning,sunshade,sunblind
94
+ 93:poster,posting,placard,notice,bill,card
95
+ 94:truck,motortruck
96
+ 95:airplane,aeroplane,airplanes,aeroplanes
97
+ 96:pole
98
+ 97:tower,towers
99
+ 98:court
100
+ 99:ball,balls
101
+ 100:aircraft carrier,carrier,flattop,attack aircraft carrier
102
+ 101:buffet,sideboard
103
+ 102:hovel,hut,hutch,shack,shanty
104
+ 103:apparel,wearing apparel,dress,clothes
105
+ 104:minibike,motorbike
106
+ 105:animal,animate being,dog,cat,horse,cow,sheep,zebra,giraffe,bird
107
+ 106:chandelier,pendant,pendent
108
+ 107:step,stair
109
+ 108:booth,cubicle,stall,kiosk
110
+ 109:bicycle,bike
111
+ 110:doorframe,doorcase
112
+ 111:sconce,sconce lamp,sconce light
113
+ 112:pond
114
+ 113:trade name,brand name
115
+ 114:bannister,banister,balustrade,balusters,handrail
116
+ 115:bag,bags,gift bag,paper bag
117
+ 116:traffic light,traffic signal,traffic lights
118
+ 117:gazebo
119
+ 118:escalator,moving staircase,moving stairway
120
+ 119:land,soil
121
+ 120:board,plank
122
+ 121:arcade machine,arcade machines
123
+ 122:eiderdown,duvet,continental quilt
124
+ 123:bar
125
+ 124:stall,stand,sales booth
126
+ 125:playground
127
+ 126:ship
128
+ 127:ottoman,pouf,pouffe,puff,hassock
129
+ 128:ashcan,trash can,garbage can,wastebin,ash bin,ash-bin,ashbin,dustbin,trash barrel,trash bin
130
+ 129:bottle,bottles,water bottle
131
+ 130:cradle
132
+ 131:pot,flowerpot
133
+ 132:conveyer belt,conveyor belt,conveyer,conveyor,transporter
134
+ 133:train,railroad train
135
+ 134:stool,stools
136
+ 135:lake
137
+ 136:tank,storage tank
138
+ 137:ice,water ice
139
+ 138:basket,handbasket
140
+ 139:manhole
141
+ 140:tent,collapsible shelter
142
+ 141:canopy
143
+ 142:microwave,microwave oven
144
+ 143:barrel,cask,barrels,casks
145
+ 144:dirt track
146
+ 145:beam
147
+ 146:dishwasher,dish washer,dishwashing machine
148
+ 147:plate,plates
149
+ 148:crt screen
150
+ 149:ruins
151
+ 150:washer,automatic washer,washing machine
152
+ 151:blanket,cover
153
+ 152:plaything,toy,toys
154
+ 153:food,solid food
155
+ 154:projection screen
156
+ 155:oven
157
+ 156:stage
158
+ 157:beacon,lighthouse,beacon light,pharos
159
+ 158:umbrella
160
+ 159:sculpture,sculptures
161
+ 160:aqueduct
162
+ 161:container
163
+ 162:scaffolding,staging
164
+ 163:exhaust hood
165
+ 164:curb,curbing,kerb
166
+ 165:roller coaster
167
+ 166:horse,equus caballus
168
+ 167:catwalk
169
+ 168:glass,drinking glass
170
+ 169:vase,vases
171
+ 170:central reservation
172
+ 171:carousel
173
+ 172:radiator
174
+ 173:closet
175
+ 174:machine
176
+ 175:pier,wharf,wharfage,dock
177
+ 176:ceiling fan,floor fan
178
+ 177:inflatable bounce game
179
+ 178:pitch
180
+ 179:paper
181
+ 180:arcade,colonnade
182
+ 181:hot tub
183
+ 182:helicopter
184
+ 183:tray,trays
185
+ 184:partition,divider
186
+ 185:vineyard
187
+ 186:bowl
188
+ 187:bullring
189
+ 188:flag,flags
190
+ 189:pot
191
+ 190:footbridge,overcrossing,pedestrian bridge
192
+ 191:shower
193
+ 192:bag,traveling bag,travelling bag,grip,suitcase
194
+ 193:bulletin board,notice board
195
+ 194:confessional booth
196
+ 195:trunk,tree trunk,bole
197
+ 196:forest
198
+ 197:elevator door
199
+ 198:laptop,laptop computer
200
+ 199:instrument panel
201
+ 200:bucket,pail
202
+ 201:tapestry,tapis
203
+ 202:platform
204
+ 203:jacket
205
+ 204:gate
206
+ 205:monitor,monitoring device,monitors
207
+ 206:telephone booth,phone booth,call box,telephone box,telephone kiosk
208
+ 207:spotlight,spot
209
+ 208:ring
210
+ 209:control panel
211
+ 210:blackboard,chalkboard
212
+ 211:air conditioner,air conditioning
213
+ 212:chest
214
+ 213:clock
215
+ 214:sand dune
216
+ 215:pipe,pipage,piping
217
+ 216:vault
218
+ 217:table football
219
+ 218:cannon
220
+ 219:swimming pool,swimming bath
221
+ 220:fluorescent,fluorescent fixture
222
+ 221:statue
223
+ 222:loudspeaker,speaker,speaker unit,loudspeaker system,speaker system
224
+ 223:exhibitor
225
+ 224:ladder
226
+ 225:carport
227
+ 226:dam
228
+ 227:pulpit
229
+ 228:skylight,fanlight
230
+ 229:water tower
231
+ 230:grill,grille,grillwork
232
+ 231:display board
233
+ 232:pane,pane of glass,window glass
234
+ 233:rubbish,trash,scrap
235
+ 234:ice rink
236
+ 235:fruit
237
+ 236:patio
238
+ 237:vending machine
239
+ 238:telephone,phone,telephone set
240
+ 239:net
241
+ 240:backpack,back pack,knapsack,packsack,rucksack,haversack
242
+ 241:jar
243
+ 242:track
244
+ 243:magazine
245
+ 244:shutter
246
+ 245:roof
247
+ 246:banner,streamer
248
+ 247:landfill
249
+ 248:post
250
+ 249:altarpiece,reredos
251
+ 250:hat,chapeau,lid
252
+ 251:arch,archway
253
+ 252:table game
254
+ 253:bag,handbag,pocketbook,purse
255
+ 254:document,written document,papers
256
+ 255:dome
257
+ 256:pier
258
+ 257:shanties
259
+ 258:forecourt
260
+ 259:crane
261
+ 260:dog,domestic dog,canis familiaris
262
+ 261:piano,pianoforte,forte-piano
263
+ 262:drawing
264
+ 263:cabin
265
+ 264:ad,advertisement,advertizement,advertising,advertizing,advert
266
+ 265:amphitheater,amphitheatre,coliseum
267
+ 266:monument
268
+ 267:henhouse
269
+ 268:cockpit
270
+ 269:heater,warmer
271
+ 270:windmill,aerogenerator,wind generator
272
+ 271:pool
273
+ 272:elevator,lift
274
+ 273:decoration,ornament,ornamentation
275
+ 274:labyrinth
276
+ 275:text,textual matter
277
+ 276:printer
278
+ 277:mezzanine,first balcony
279
+ 278:mattress
280
+ 279:straw
281
+ 280:stalls
282
+ 281:patio,terrace
283
+ 282:billboard,hoarding
284
+ 283:bus stop
285
+ 284:trouser,pant
286
+ 285:console table,console
287
+ 286:rack
288
+ 287:notebook
289
+ 288:shrine
290
+ 289:pantry
291
+ 290:cart
292
+ 291:steam shovel
293
+ 292:porch
294
+ 293:postbox,mailbox,letter box
295
+ 294:figurine,statuette
296
+ 295:recycling bin
297
+ 296:folding screen
298
+ 297:telescope
299
+ 298:deck chair,beach chair
300
+ 299:kennel
301
+ 300:coffee maker
302
+ 301:altar,communion table,lord's table
303
+ 302:fish
304
+ 303:easel
305
+ 304:artificial golf green
306
+ 305:iceberg
307
+ 306:candlestick,candle holder
308
+ 307:shower stall,shower bath
309
+ 308:television stand
310
+ 309:wall socket,wall plug,electric outlet,electrical outlet,outlet,electric receptacle
311
+ 310:skeleton
312
+ 311:grand piano,grand
313
+ 312:candy,confect
314
+ 313:grille door
315
+ 314:pedestal,plinth,footstall
316
+ 315:jersey,t-shirt,tee shirt
317
+ 316:shoe
318
+ 317:gravestone,headstone,tombstone
319
+ 318:shanty
320
+ 319:structure
321
+ 320:rocking chair,rocker
322
+ 321:bird
323
+ 322:place mat
324
+ 323:tomb
325
+ 324:big top
326
+ 325:gas pump,gasoline pump,petrol pump,island dispenser
327
+ 326:lockers
328
+ 327:cage
329
+ 328:finger
330
+ 329:bleachers
331
+ 330:ferris wheel
332
+ 331:hairdresser chair
333
+ 332:mat
334
+ 333:stands
335
+ 334:aquarium,fish tank,marine museum
336
+ 335:streetcar,tram,tramcar,trolley,trolley car
337
+ 336:napkin,table napkin,serviette
338
+ 337:dummy
339
+ 338:booklet,brochure,folder,leaflet,pamphlet
340
+ 339:sand trap
341
+ 340:shop,store
342
+ 341:table cloth
343
+ 342:service station
344
+ 343:coffin
345
+ 344:drawer
346
+ 345:cages
347
+ 346:slot machine,coin machine
348
+ 347:balcony
349
+ 348:volleyball court
350
+ 349:table tennis
351
+ 350:control table
352
+ 351:shirt
353
+ 352:merchandise,ware,product
354
+ 353:railway
355
+ 354:parterre
356
+ 355:chimney
357
+ 356:can,tin,tin can
358
+ 357:tanks
359
+ 358:fabric,cloth,material,textile
360
+ 359:alga,algae
361
+ 360:system
362
+ 361:map
363
+ 362:greenhouse
364
+ 363:mug
365
+ 364:barbecue
366
+ 365:trailer
367
+ 366:toilet tissue,toilet paper,bathroom tissue
368
+ 367:organ
369
+ 368:dishrag,dishcloth
370
+ 369:island
371
+ 370:keyboard
372
+ 371:trench
373
+ 372:basket,basketball hoop,hoop
374
+ 373:steering wheel,wheel
375
+ 374:pitcher,ewer
376
+ 375:goal
377
+ 376:bread,breadstuff,staff of life
378
+ 377:beds
379
+ 378:wood
380
+ 379:file cabinet
381
+ 380:newspaper,paper
382
+ 381:motorboat
383
+ 382:rope
384
+ 383:guitar
385
+ 384:rubble
386
+ 385:scarf
387
+ 386:barrels
388
+ 387:cap
389
+ 388:leaves
390
+ 389:control tower
391
+ 390:dashboard
392
+ 391:bandstand
393
+ 392:lectern
394
+ 393:switch,electric switch,electrical switch
395
+ 394:baseboard,mopboard,skirting board
396
+ 395:shower room
397
+ 396:smoke
398
+ 397:faucet,spigot
399
+ 398:bulldozer
400
+ 399:saucepan
401
+ 400:shops
402
+ 401:meter
403
+ 402:crevasse
404
+ 403:gear
405
+ 404:candelabrum,candelabra
406
+ 405:sofa bed
407
+ 406:tunnel
408
+ 407:pallet
409
+ 408:wire,conducting wire
410
+ 409:kettle,boiler
411
+ 410:bidet
412
+ 411:baby buggy,baby carriage,carriage,perambulator,pram,stroller,go-cart,pushchair,pusher
413
+ 412:music stand
414
+ 413:pipe,tube
415
+ 414:cup,cups,drinking glass,drinking glasses
416
+ 415:parking meter
417
+ 416:ice hockey rink
418
+ 417:shelter
419
+ 418:weeds
420
+ 419:temple
421
+ 420:patty,cake
422
+ 421:ski slope
423
+ 422:panel
424
+ 423:wallet
425
+ 424:wheel
426
+ 425:towel rack,towel horse
427
+ 426:roundabout
428
+ 427:canister,cannister,tin
429
+ 428:rod
430
+ 429:soap dispenser
431
+ 430:bell
432
+ 431:canvas
433
+ 432:box office,ticket office,ticket booth
434
+ 433:teacup
435
+ 434:trellis
436
+ 435:workbench
437
+ 436:valley,vale
438
+ 437:toaster
439
+ 438:knife
440
+ 439:podium
441
+ 440:ramp
442
+ 441:tumble dryer
443
+ 442:fireplug,fire hydrant,plug
444
+ 443:gym shoe,sneaker,tennis shoe
445
+ 444:lab bench
446
+ 445:equipment
447
+ 446:rocky formation
448
+ 447:plastic
449
+ 448:calendar
450
+ 449:caravan
451
+ 450:check-in-desk
452
+ 451:ticket counter
453
+ 452:brush
454
+ 453:mill
455
+ 454:covered bridge
456
+ 455:bowling alley
457
+ 456:hanger
458
+ 457:excavator
459
+ 458:trestle
460
+ 459:revolving door
461
+ 460:blast furnace
462
+ 461:scale,weighing machine
463
+ 462:projector
464
+ 463:soap
465
+ 464:locker
466
+ 465:tractor
467
+ 466:stretcher
468
+ 467:frame
469
+ 468:grating
470
+ 469:alembic
471
+ 470:candle,taper,wax light
472
+ 471:barrier
473
+ 472:cardboard
474
+ 473:cave
475
+ 474:puddle
476
+ 475:tarp
477
+ 476:price tag
478
+ 477:watchtower
479
+ 478:meters
480
+ 479:light bulb,bulb,bulbs
481
+ 480:tracks
482
+ 481:hair dryer
483
+ 482:skirt
484
+ 483:viaduct
485
+ 484:paper towel
486
+ 485:coat
487
+ 486:sheet
488
+ 487:fire extinguisher,extinguisher,asphyxiator
489
+ 488:water wheel
490
+ 489:pottery,clayware
491
+ 490:magazine rack
492
+ 491:teapot
493
+ 492:microphone,mike
494
+ 493:support
495
+ 494:forklift
496
+ 495:canyon
497
+ 496:cash register,register
498
+ 497:leaf,leafage,foliage
499
+ 498:remote control,remote
500
+ 499:soap dish
501
+ 500:windshield,windscreen
502
+ 501:cat
503
+ 502:cue,cue stick,pool cue,pool stick
504
+ 503:vent,venthole,vent-hole,blowhole
505
+ 504:videos
506
+ 505:shovel
507
+ 506:eaves
508
+ 507:antenna,aerial,transmitting aerial
509
+ 508:shipyard
510
+ 509:hen,biddy
511
+ 510:traffic cone
512
+ 511:washing machines
513
+ 512:truck crane
514
+ 513:cds
515
+ 514:niche
516
+ 515:scoreboard
517
+ 516:briefcase
518
+ 517:boot
519
+ 518:sweater,jumper
520
+ 519:hay
521
+ 520:pack
522
+ 521:bottle rack
523
+ 522:glacier
524
+ 523:pergola
525
+ 524:building materials
526
+ 525:television camera
527
+ 526:first floor
528
+ 527:rifle
529
+ 528:tennis table
530
+ 529:stadium
531
+ 530:safety belt
532
+ 531:cover
533
+ 532:dish rack
534
+ 533:synthesizer
535
+ 534:pumpkin
536
+ 535:gutter
537
+ 536:fruit stand
538
+ 537:ice floe,floe
539
+ 538:handle,grip,handgrip,hold
540
+ 539:wheelchair
541
+ 540:mousepad,mouse mat
542
+ 541:diploma
543
+ 542:fairground ride
544
+ 543:radio
545
+ 544:hotplate
546
+ 545:junk
547
+ 546:wheelbarrow
548
+ 547:stream
549
+ 548:toll plaza
550
+ 549:punching bag
551
+ 550:trough
552
+ 551:throne
553
+ 552:chair desk
554
+ 553:weighbridge
555
+ 554:extractor fan
556
+ 555:hanging clothes
557
+ 556:dish,dish aerial,dish antenna,saucer
558
+ 557:alarm clock,alarm
559
+ 558:ski lift
560
+ 559:chain
561
+ 560:garage
562
+ 561:mechanical shovel
563
+ 562:wine rack
564
+ 563:tramway
565
+ 564:treadmill
566
+ 565:menu
567
+ 566:block
568
+ 567:well
569
+ 568:witness stand
570
+ 569:branch
571
+ 570:duck
572
+ 571:casserole
573
+ 572:frying pan
574
+ 573:desk organizer
575
+ 574:mast
576
+ 575:spectacles,specs,eyeglasses,glasses
577
+ 576:service elevator
578
+ 577:dollhouse
579
+ 578:hammock
580
+ 579:clothes hanging
581
+ 580:photocopier
582
+ 581:notepad
583
+ 582:golf cart
584
+ 583:footpath
585
+ 584:cross
586
+ 585:baptismal font
587
+ 586:boiler
588
+ 587:skip
589
+ 588:rotisserie
590
+ 589:tables
591
+ 590:water mill
592
+ 591:helmet
593
+ 592:cover curtain
594
+ 593:brick
595
+ 594:table runner
596
+ 595:ashtray
597
+ 596:street box
598
+ 597:stick
599
+ 598:hangers
600
+ 599:cells
601
+ 600:urinal
602
+ 601:centerpiece
603
+ 602:portable fridge
604
+ 603:dvds
605
+ 604:golf club
606
+ 605:skirting board
607
+ 606:water cooler
608
+ 607:clipboard
609
+ 608:camera,photographic camera
610
+ 609:pigeonhole
611
+ 610:chips
612
+ 611:food processor
613
+ 612:post box
614
+ 613:lid
615
+ 614:drum
616
+ 615:blender
617
+ 616:cave entrance
618
+ 617:dental chair
619
+ 618:obelisk
620
+ 619:canoe
621
+ 620:mobile
622
+ 621:monitors
623
+ 622:pool ball
624
+ 623:cue rack
625
+ 624:baggage carts
626
+ 625:shore
627
+ 626:fork
628
+ 627:paper filer
629
+ 628:bicycle rack
630
+ 629:coat rack
631
+ 630:garland
632
+ 631:sports bag
633
+ 632:fish tank
634
+ 633:towel dispenser
635
+ 634:carriage
636
+ 635:brochure
637
+ 636:plaque
638
+ 637:stringer
639
+ 638:iron
640
+ 639:spoon
641
+ 640:flag pole
642
+ 641:toilet brush
643
+ 642:book stand
644
+ 643:water faucet,water tap,tap,hydrant
645
+ 644:ticket office
646
+ 645:broom
647
+ 646:dvd
648
+ 647:ice bucket
649
+ 648:carapace,shell,cuticle,shield
650
+ 649:tureen
651
+ 650:folders
652
+ 651:chess
653
+ 652:root
654
+ 653:sewing machine
655
+ 654:model
656
+ 655:pen
657
+ 656:violin
658
+ 657:sweatshirt
659
+ 658:recycling materials
660
+ 659:mitten
661
+ 660:chopping board,cutting board
662
+ 661:mask
663
+ 662:log
664
+ 663:mouse,computer mouse
665
+ 664:grill
666
+ 665:hole
667
+ 666:target
668
+ 667:trash bag
669
+ 668:chalk
670
+ 669:sticks
671
+ 670:balloon
672
+ 671:score
673
+ 672:hair spray
674
+ 673:roll
675
+ 674:runner
676
+ 675:engine
677
+ 676:inflatable glove
678
+ 677:games
679
+ 678:pallets
680
+ 679:baskets
681
+ 680:coop
682
+ 681:dvd player
683
+ 682:rocking horse
684
+ 683:buckets
685
+ 684:bread rolls
686
+ 685:shawl
687
+ 686:watering can
688
+ 687:spotlights
689
+ 688:post-it
690
+ 689:bowls
691
+ 690:security camera
692
+ 691:runner cloth
693
+ 692:lock
694
+ 693:alarm,warning device,alarm system
695
+ 694:side
696
+ 695:roulette
697
+ 696:bone
698
+ 697:cutlery
699
+ 698:pool balls
700
+ 699:wheels
701
+ 700:spice rack
702
+ 701:plant pots,plant pot,flower pot,flowerpot,planter
703
+ 702:towel ring
704
+ 703:bread box
705
+ 704:video
706
+ 705:funfair
707
+ 706:breads
708
+ 707:tripod
709
+ 708:ironing board
710
+ 709:skimmer
711
+ 710:hollow
712
+ 711:scratching post
713
+ 712:tricycle
714
+ 713:file box
715
+ 714:mountain pass
716
+ 715:tombstones
717
+ 716:cooker
718
+ 717:card game,cards
719
+ 718:golf bag
720
+ 719:towel paper
721
+ 720:chaise lounge
722
+ 721:sun
723
+ 722:toilet paper holder
724
+ 723:rake
725
+ 724:key
726
+ 725:umbrella stand
727
+ 726:dartboard
728
+ 727:transformer
729
+ 728:fireplace utensils
730
+ 729:sweatshirts
731
+ 730:cellular telephone,cellular phone,cellphone,cell,mobile phone
732
+ 731:tallboy
733
+ 732:stapler
734
+ 733:sauna
735
+ 734:test tube
736
+ 735:palette
737
+ 736:shopping carts
738
+ 737:tools
739
+ 738:push button,push,button
740
+ 739:star
741
+ 740:roof rack
742
+ 741:barbed wire
743
+ 742:spray
744
+ 743:ear
745
+ 744:sponge
746
+ 745:racket
747
+ 746:tins
748
+ 747:eyeglasses
749
+ 748:file
750
+ 749:scarfs
751
+ 750:sugar bowl
752
+ 751:flip flop
753
+ 752:headstones
754
+ 753:laptop bag
755
+ 754:leash
756
+ 755:climbing frame
757
+ 756:suit hanger
758
+ 757:floor spotlight
759
+ 758:plate rack
760
+ 759:sewer
761
+ 760:hard drive
762
+ 761:sprinkler
763
+ 762:tools box
764
+ 763:necklace
765
+ 764:bulbs
766
+ 765:steel industry
767
+ 766:club
768
+ 767:jack
769
+ 768:door bars
770
+ 769:control panel,instrument panel,control board,board,panel
771
+ 770:hairbrush
772
+ 771:napkin holder
773
+ 772:office
774
+ 773:smoke detector
775
+ 774:utensils
776
+ 775:apron
777
+ 776:scissors
778
+ 777:terminal
779
+ 778:grinder
780
+ 779:entry phone
781
+ 780:newspaper stand
782
+ 781:pepper shaker
783
+ 782:onions
784
+ 783:central processing unit,cpu,central processor,processor,mainframe
785
+ 784:tape
786
+ 785:bat
787
+ 786:coaster
788
+ 787:calculator
789
+ 788:potatoes
790
+ 789:luggage rack
791
+ 790:salt
792
+ 791:street number
793
+ 792:viewpoint
794
+ 793:sword
795
+ 794:cd
796
+ 795:rowing machine
797
+ 796:plug
798
+ 797:andiron,firedog,dog,dog-iron
799
+ 798:pepper
800
+ 799:tongs
801
+ 800:bonfire
802
+ 801:dog dish
803
+ 802:belt
804
+ 803:dumbbells
805
+ 804:videocassette recorder,vcr
806
+ 805:hook
807
+ 806:envelopes
808
+ 807:shower faucet
809
+ 808:watch
810
+ 809:padlock
811
+ 810:swimming pool ladder
812
+ 811:spanners
813
+ 812:gravy boat
814
+ 813:notice board
815
+ 814:trash bags
816
+ 815:fire alarm
817
+ 816:ladle
818
+ 817:stethoscope
819
+ 818:rocket
820
+ 819:funnel
821
+ 820:bowling pins
822
+ 821:valve
823
+ 822:thermometer
824
+ 823:cups
825
+ 824:spice jar
826
+ 825:night light
827
+ 826:soaps
828
+ 827:games table
829
+ 828:slotted spoon
830
+ 829:reel
831
+ 830:scourer
832
+ 831:sleeping robe
833
+ 832:desk mat
834
+ 833:dumbbell
835
+ 834:hammer
836
+ 835:tie
837
+ 836:typewriter
838
+ 837:shaker
839
+ 838:cheese dish
840
+ 839:sea star
841
+ 840:racquet
842
+ 841:butane gas cylinder
843
+ 842:paper weight
844
+ 843:shaving brush
845
+ 844:sunglasses
846
+ 845:gear shift
847
+ 846:towel rail
848
+ 847:adding machine,totalizer,totaliser
frozenseg/data/datasets/cityscapes_with_prompt_eng.txt ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 0:road,railroad
2
+ 1:sidewalk,pavement
3
+ 2:building,buildings,edifice,edifices,house,ceiling
4
+ 3:wall,walls,brick wall,stone wall,tile wall,wood wall
5
+ 4:fence,fences
6
+ 5:pole,poles
7
+ 6:traffic light,traffic lights
8
+ 7:traffic sign,stop sign
9
+ 8:vegetation,tree,trees,palm tree,bushes
10
+ 9:terrain,river,sand,sea,snow,water,mountain,grass,dirt,rock
11
+ 10:sky,clouds
12
+ 11:person
13
+ 12:rider
14
+ 13:car,cars
15
+ 14:truck,trucks
16
+ 15:bus,buses
17
+ 16:train,trains,locomotive,locomotives,freight train
18
+ 17:motorcycle,motorcycles
19
+ 18:bicycle,bicycles,bike,bikes
frozenseg/data/datasets/coco_panoptic_with_prompt_eng.txt ADDED
@@ -0,0 +1,201 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 0:invalid_class_id
2
+ 1:person,child,girl,boy,woman,man,people,children,girls,boys,women,men,lady,guy,ladies,guys,clothes
3
+ 2:bicycle,bicycles,bike,bikes
4
+ 3:car,cars
5
+ 4:motorcycle,motorcycles
6
+ 5:airplane,airplanes
7
+ 6:bus,buses
8
+ 7:train,trains,locomotive,locomotives,freight train
9
+ 8:truck,trucks
10
+ 9:boat,boats
11
+ 10:traffic light
12
+ 11:fire hydrant
13
+ 12:invalid_class_id
14
+ 13:stop sign
15
+ 14:parking meter
16
+ 15:bench,benches
17
+ 16:bird,birds
18
+ 17:cat,cats,kitties,kitty
19
+ 18:dog,dogs,puppy,puppies
20
+ 19:horse,horses,foal
21
+ 20:sheep
22
+ 21:cow,cows,calf
23
+ 22:elephant,elephants
24
+ 23:bear,bears
25
+ 24:zebra,zebras
26
+ 25:giraffe,giraffes
27
+ 26:invalid_class_id
28
+ 27:backpack,backpacks
29
+ 28:umbrella,umbrellas
30
+ 29:invalid_class_id
31
+ 30:invalid_class_id
32
+ 31:handbag,handbags
33
+ 32:tie
34
+ 33:suitcase,suitcases
35
+ 34:frisbee
36
+ 35:skis
37
+ 36:snowboard
38
+ 37:sports ball
39
+ 38:kite,kites
40
+ 39:baseball bat
41
+ 40:baseball glove
42
+ 41:skateboard
43
+ 42:surfboard
44
+ 43:tennis racket
45
+ 44:bottle,bottles,water bottle
46
+ 45:invalid_class_id
47
+ 46:wine glass,wine glasses,wineglass
48
+ 47:cup,cups,water cup,water glass
49
+ 48:fork,forks
50
+ 49:knife,knives
51
+ 50:spoon,spoons
52
+ 51:bowl,bowls
53
+ 52:banana,bananas
54
+ 53:apple,apples,apple fruit
55
+ 54:sandwich,sandwiches
56
+ 55:orange fruit
57
+ 56:broccoli
58
+ 57:carrot,carrots
59
+ 58:hot dog
60
+ 59:pizza
61
+ 60:donut,donuts
62
+ 61:cake,cakes
63
+ 62:chair,chairs
64
+ 63:couch,sofa,sofas
65
+ 64:potted plant,potted plants,pottedplant,pottedplants,planter,planters
66
+ 65:bed,beds
67
+ 66:invalid_class_id
68
+ 67:dining table,dining tables,diningtable,diningtables,plate,plates,diningtable tablecloth
69
+ 68:invalid_class_id
70
+ 69:invalid_class_id
71
+ 70:toilet
72
+ 71:invalid_class_id
73
+ 72:tv
74
+ 73:laptop
75
+ 74:mouse
76
+ 75:tv remote,remote control
77
+ 76:keyboard
78
+ 77:cell phone,mobile
79
+ 78:microwave
80
+ 79:oven,ovens
81
+ 80:toaster
82
+ 81:sink,sinks
83
+ 82:refrigerator,fridge
84
+ 83:invalid_class_id
85
+ 84:book,books
86
+ 85:clock
87
+ 86:vase,vases
88
+ 87:scissor,scissors
89
+ 88:teddy bear,teddy bears
90
+ 89:hair drier
91
+ 90:toothbrush,toothbrushes
92
+ 91:invalid_class_id
93
+ 92:banner,banners
94
+ 93:blanket,blankets
95
+ 94:invalid_class_id
96
+ 95:bridge
97
+ 96:invalid_class_id
98
+ 97:invalid_class_id
99
+ 98:invalid_class_id
100
+ 99:invalid_class_id
101
+ 100:cardboard
102
+ 101:invalid_class_id
103
+ 102:invalid_class_id
104
+ 103:invalid_class_id
105
+ 104:invalid_class_id
106
+ 105:invalid_class_id
107
+ 106:invalid_class_id
108
+ 107:counter
109
+ 108:invalid_class_id
110
+ 109:curtain,curtains
111
+ 110:invalid_class_id
112
+ 111:invalid_class_id
113
+ 112:door,doors
114
+ 113:invalid_class_id
115
+ 114:invalid_class_id
116
+ 115:invalid_class_id
117
+ 116:invalid_class_id
118
+ 117:invalid_class_id
119
+ 118:wood floor
120
+ 119:flower,flowers
121
+ 120:invalid_class_id
122
+ 121:invalid_class_id
123
+ 122:fruit,fruits
124
+ 123:invalid_class_id
125
+ 124:invalid_class_id
126
+ 125:gravel
127
+ 126:invalid_class_id
128
+ 127:invalid_class_id
129
+ 128:house
130
+ 129:invalid_class_id
131
+ 130:lamp,bulb,lamps,bulbs
132
+ 131:invalid_class_id
133
+ 132:invalid_class_id
134
+ 133:mirror
135
+ 134:invalid_class_id
136
+ 135:invalid_class_id
137
+ 136:invalid_class_id
138
+ 137:invalid_class_id
139
+ 138:tennis net
140
+ 139:invalid_class_id
141
+ 140:invalid_class_id
142
+ 141:pillow,pillows
143
+ 142:invalid_class_id
144
+ 143:invalid_class_id
145
+ 144:platform
146
+ 145:playingfield,tennis court,baseball field,soccer field,tennis field
147
+ 146:invalid_class_id
148
+ 147:railroad
149
+ 148:river
150
+ 149:road
151
+ 150:invalid_class_id
152
+ 151:roof
153
+ 152:invalid_class_id
154
+ 153:invalid_class_id
155
+ 154:sand
156
+ 155:sea,sea wave,wave,waves
157
+ 156:shelf
158
+ 157:invalid_class_id
159
+ 158:invalid_class_id
160
+ 159:snow
161
+ 160:invalid_class_id
162
+ 161:stairs
163
+ 162:invalid_class_id
164
+ 163:invalid_class_id
165
+ 164:invalid_class_id
166
+ 165:invalid_class_id
167
+ 166:tent
168
+ 167:invalid_class_id
169
+ 168:towel
170
+ 169:invalid_class_id
171
+ 170:invalid_class_id
172
+ 171:brick wall
173
+ 172:invalid_class_id
174
+ 173:invalid_class_id
175
+ 174:invalid_class_id
176
+ 175:stone wall
177
+ 176:tile wall
178
+ 177:wood wall
179
+ 178:water
180
+ 179:invalid_class_id
181
+ 180:window blind
182
+ 181:window
183
+ 182:invalid_class_id
184
+ 183:invalid_class_id
185
+ 184:tree,trees,palm tree,bushes
186
+ 185:fence,fences
187
+ 186:ceiling
188
+ 187:sky,clouds
189
+ 188:cabinet,cabinets
190
+ 189:table
191
+ 190:floor,flooring,tile floor
192
+ 191:pavement
193
+ 192:mountain,mountains
194
+ 193:grass
195
+ 194:dirt
196
+ 195:paper
197
+ 196:food
198
+ 197:building,buildings
199
+ 198:rock
200
+ 199:wall,walls
201
+ 200:rug
frozenseg/data/datasets/coco_stuff_with_prompt_eng.txt ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 0:invalid_class_id
2
+ 1:person,child,girl,boy,woman,man,people,children,girls,boys,women,men,lady,guy,ladies,guys
3
+ 2:bicycle,bicycles,bike,bikes
4
+ 3:car,cars
5
+ 4:motorcycle,motorcycles
6
+ 5:airplane,airplanes
7
+ 6:bus,buses
8
+ 7:train,trains,locomotive,locomotives,freight train
9
+ 8:truck,trucks
10
+ 9:boat,boats
11
+ 10:traffic light
12
+ 11:fire hydrant
13
+ 12:invalid_class_id
14
+ 13:stop sign
15
+ 14:parking meter
16
+ 15:bench,benches
17
+ 16:bird,birds
18
+ 17:cat,cats,kitties,kitty
19
+ 18:dog,dogs,puppy,puppies
20
+ 19:horse,horses,foal
21
+ 20:sheep
22
+ 21:cow,cows,calf
23
+ 22:elephant,elephants
24
+ 23:bear,bears
25
+ 24:zebra,zebras
26
+ 25:giraffe,giraffes
27
+ 26:invalid_class_id
28
+ 27:backpack,backpacks
29
+ 28:umbrella,umbrellas
30
+ 29:invalid_class_id
31
+ 30:invalid_class_id
32
+ 31:handbag,handbags
33
+ 32:tie
34
+ 33:suitcase,suitcases
35
+ 34:frisbee
36
+ 35:skis
37
+ 36:snowboard
38
+ 37:sports ball
39
+ 38:kite,kites
40
+ 39:baseball bat
41
+ 40:baseball glove
42
+ 41:skateboard
43
+ 42:surfboard
44
+ 43:tennis racket
45
+ 44:bottle,bottles,water bottle
46
+ 45:invalid_class_id
47
+ 46:wine glass,wine glasses,wineglass
48
+ 47:cup,cups,water cup,water glass
49
+ 48:fork,forks
50
+ 49:knife,knives
51
+ 50:spoon,spoons
52
+ 51:bowl,bowls
53
+ 52:banana,bananas
54
+ 53:apple,apples,apple fruit
55
+ 54:sandwich,sandwiches
56
+ 55:orange,oranges,orange fruit
57
+ 56:broccoli
58
+ 57:carrot,carrots
59
+ 58:hot dog
60
+ 59:pizza
61
+ 60:donut,donuts
62
+ 61:cake,cakes
63
+ 62:chair,chairs
64
+ 63:couch,sofa,sofas
65
+ 64:potted plant,potted plants,pottedplant,pottedplants,planter,planters
66
+ 65:bed,beds
67
+ 66:invalid_class_id
68
+ 67:dining table,dining tables,diningtable,diningtables,plate,plates,diningtable tablecloth
69
+ 68:invalid_class_id
70
+ 69:invalid_class_id
71
+ 70:toilet
72
+ 71:invalid_class_id
73
+ 72:tv
74
+ 73:laptop
75
+ 74:mouse
76
+ 75:remote,tv remote,remote control
77
+ 76:keyboard
78
+ 77:cell phone,mobile
79
+ 78:microwave
80
+ 79:oven,ovens
81
+ 80:toaster
82
+ 81:sink,sinks
83
+ 82:refrigerator,fridge
84
+ 83:invalid_class_id
85
+ 84:book,books
86
+ 85:clock
87
+ 86:vase,vases
88
+ 87:scissors,scissor
89
+ 88:teddy bear,teddy bears
90
+ 89:hair drier
91
+ 90:toothbrush,toothbrushes
92
+ 91:invalid_class_id
93
+ 92:banner,banners
94
+ 93:blanket,blankets
95
+ 94:branch
96
+ 95:bridge
97
+ 96:building,buildings
98
+ 97:bush,bushes
99
+ 98:cabinet,cabinets
100
+ 99:cage,cages
101
+ 100:cardboard
102
+ 101:carpet,carpets
103
+ 102:ceiling-other,ceiling
104
+ 103:ceiling-tile,ceiling tile
105
+ 104:cloth
106
+ 105:clothes
107
+ 106:clouds
108
+ 107:counter
109
+ 108:cupboard,cupboards
110
+ 109:curtain,curtains
111
+ 110:desk-stuff,desk,desks
112
+ 111:dirt
113
+ 112:door-stuff,door,doors
114
+ 113:fence,fences
115
+ 114:floor-marble,marble floor,floor marble
116
+ 115:floor-other,floor
117
+ 116:floor-stone,stone floor,floor stone
118
+ 117:floor-tile,tile floor,floor tile
119
+ 118:floor-wood,wood floor,floor wood
120
+ 119:flower,flowers
121
+ 120:fog
122
+ 121:food-other,food
123
+ 122:fruit,fruits
124
+ 123:furniture-other,furniture
125
+ 124:grass
126
+ 125:gravel
127
+ 126:ground-other,ground
128
+ 127:hill
129
+ 128:house
130
+ 129:leaves
131
+ 130:light
132
+ 131:mat
133
+ 132:metal
134
+ 133:mirror-stuff,mirror
135
+ 134:moss
136
+ 135:mountain,mountains
137
+ 136:mud
138
+ 137:napkin
139
+ 138:net
140
+ 139:paper
141
+ 140:pavement
142
+ 141:pillow,pillows
143
+ 142:plant-other
144
+ 143:plastic
145
+ 144:platform
146
+ 145:playingfield,tennis court,baseball field,soccer field,tennis field
147
+ 146:railing
148
+ 147:railroad
149
+ 148:river
150
+ 149:road
151
+ 150:rock
152
+ 151:roof
153
+ 152:rug
154
+ 153:salad
155
+ 154:sand
156
+ 155:sea,sea wave,wave,waves
157
+ 156:shelf
158
+ 157:sky-other,sky
159
+ 158:skyscraper
160
+ 159:snow
161
+ 160:solid-other,solid
162
+ 161:stairs
163
+ 162:stone
164
+ 163:straw
165
+ 164:structural-other,structural
166
+ 165:table
167
+ 166:tent
168
+ 167:textile-other,textile
169
+ 168:towel
170
+ 169:tree,trees,palm tree
171
+ 170:vegetable
172
+ 171:wall-brick,brick wall,wall brick
173
+ 172:wall-concrete,concrete wall,wall concrete
174
+ 173:wall-other,wall
175
+ 174:wall-panel,wall panel,panel wall
176
+ 175:wall-stone,stone wall,wall stone
177
+ 176:wall-tile,wall tile,tile wall
178
+ 177:wall-wood,wood wall, wall wood
179
+ 178:water-other,water
180
+ 179:waterdrops
181
+ 180:window-blind,window blind
182
+ 181:window-other,window
183
+ 182:wood
frozenseg/data/datasets/lvis_1203_with_prompt_eng.txt ADDED
@@ -0,0 +1,1203 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 1:aerosol can,spray can
2
+ 2:air conditioner
3
+ 3:airplane,aeroplane
4
+ 4:alarm clock
5
+ 5:alcohol,alcoholic beverage
6
+ 6:alligator,gator
7
+ 7:almond
8
+ 8:ambulance
9
+ 9:amplifier
10
+ 10:anklet,ankle bracelet
11
+ 11:antenna,aerial,transmitting aerial
12
+ 12:apple
13
+ 13:applesauce
14
+ 14:apricot
15
+ 15:apron
16
+ 16:aquarium,fish tank
17
+ 17:arctic (type of shoe),galosh,golosh,rubber (type of shoe),gumshoe
18
+ 18:armband
19
+ 19:armchair
20
+ 20:armoire
21
+ 21:armor,armour
22
+ 22:artichoke
23
+ 23:trash can,garbage can,wastebin,dustbin,trash barrel,trash bin
24
+ 24:ashtray
25
+ 25:asparagus
26
+ 26:atomizer,atomiser,spray,sprayer,nebulizer,nebuliser
27
+ 27:avocado
28
+ 28:award,accolade
29
+ 29:awning
30
+ 30:ax,axe
31
+ 31:baboon
32
+ 32:baby buggy,baby carriage,perambulator,pram,stroller
33
+ 33:basketball backboard
34
+ 34:backpack,knapsack,packsack,rucksack,haversack
35
+ 35:handbag,purse,pocketbook
36
+ 36:suitcase,baggage,luggage
37
+ 37:bagel,beigel
38
+ 38:bagpipe
39
+ 39:baguet,baguette
40
+ 40:bait,lure
41
+ 41:ball
42
+ 42:ballet skirt,tutu
43
+ 43:balloon
44
+ 44:bamboo
45
+ 45:banana
46
+ 46:Band Aid
47
+ 47:bandage
48
+ 48:bandanna,bandana
49
+ 49:banjo
50
+ 50:banner,streamer
51
+ 51:barbell
52
+ 52:barge
53
+ 53:barrel,cask
54
+ 54:barrette
55
+ 55:barrow,garden cart,lawn cart,wheelbarrow
56
+ 56:baseball base
57
+ 57:baseball
58
+ 58:baseball bat
59
+ 59:baseball cap,jockey cap,golf cap
60
+ 60:baseball glove,baseball mitt
61
+ 61:basket,handbasket
62
+ 62:basketball
63
+ 63:bass horn,sousaphone,tuba
64
+ 64:bat (animal)
65
+ 65:bath mat
66
+ 66:bath towel
67
+ 67:bathrobe
68
+ 68:bathtub,bathing tub
69
+ 69:batter (food)
70
+ 70:battery
71
+ 71:beachball
72
+ 72:bead
73
+ 73:bean curd,tofu
74
+ 74:beanbag
75
+ 75:beanie,beany
76
+ 76:bear
77
+ 77:bed
78
+ 78:bedpan
79
+ 79:bedspread,bedcover,bed covering,counterpane,spread
80
+ 80:cow
81
+ 81:beef (food),boeuf (food)
82
+ 82:beeper,pager
83
+ 83:beer bottle
84
+ 84:beer can
85
+ 85:beetle
86
+ 86:bell
87
+ 87:bell pepper,capsicum
88
+ 88:belt
89
+ 89:belt buckle
90
+ 90:bench
91
+ 91:beret
92
+ 92:bib
93
+ 93:Bible
94
+ 94:bicycle,bike (bicycle)
95
+ 95:visor,vizor
96
+ 96:billboard
97
+ 97:binder,ring-binder
98
+ 98:binoculars,field glasses,opera glasses
99
+ 99:bird
100
+ 100:birdfeeder
101
+ 101:birdbath
102
+ 102:birdcage
103
+ 103:birdhouse
104
+ 104:birthday cake
105
+ 105:birthday card
106
+ 106:pirate flag
107
+ 107:black sheep
108
+ 108:blackberry
109
+ 109:blackboard,chalkboard
110
+ 110:blanket
111
+ 111:blazer,sport jacket,sport coat,sports jacket,sports coat
112
+ 112:blender,liquidizer,liquidiser
113
+ 113:blimp
114
+ 114:blinker,flasher
115
+ 115:blouse
116
+ 116:blueberry
117
+ 117:gameboard
118
+ 118:boat,ship (boat)
119
+ 119:bob,bobber,bobfloat
120
+ 120:bobbin,spool,reel
121
+ 121:bobby pin,hairgrip
122
+ 122:boiled egg,coddled egg
123
+ 123:bolo tie,bolo,bola tie,bola
124
+ 124:deadbolt
125
+ 125:bolt
126
+ 126:bonnet
127
+ 127:book
128
+ 128:bookcase
129
+ 129:booklet,brochure,leaflet,pamphlet
130
+ 130:bookmark,bookmarker
131
+ 131:boom microphone,microphone boom
132
+ 132:boot
133
+ 133:bottle
134
+ 134:bottle opener
135
+ 135:bouquet
136
+ 136:bow (weapon)
137
+ 137:bow (decorative ribbons)
138
+ 138:bow-tie,bowtie
139
+ 139:bowl
140
+ 140:pipe bowl
141
+ 141:bowler hat,bowler,derby hat,derby,plug hat
142
+ 142:bowling ball
143
+ 143:box
144
+ 144:boxing glove
145
+ 145:suspenders
146
+ 146:bracelet,bangle
147
+ 147:brass plaque
148
+ 148:brassiere,bra,bandeau
149
+ 149:bread-bin,breadbox
150
+ 150:bread
151
+ 151:breechcloth,breechclout,loincloth
152
+ 152:bridal gown,wedding gown,wedding dress
153
+ 153:briefcase
154
+ 154:broccoli
155
+ 155:broach
156
+ 156:broom
157
+ 157:brownie
158
+ 158:brussels sprouts
159
+ 159:bubble gum
160
+ 160:bucket,pail
161
+ 161:horse buggy
162
+ 162:horned cow
163
+ 163:bulldog
164
+ 164:bulldozer,dozer
165
+ 165:bullet train
166
+ 166:bulletin board,notice board
167
+ 167:bulletproof vest
168
+ 168:bullhorn,megaphone
169
+ 169:bun,roll
170
+ 170:bunk bed
171
+ 171:buoy
172
+ 172:burrito
173
+ 173:bus (vehicle),autobus,charabanc,double-decker,motorbus,motorcoach
174
+ 174:business card
175
+ 175:butter
176
+ 176:butterfly
177
+ 177:button
178
+ 178:cab (taxi),taxi,taxicab
179
+ 179:cabana
180
+ 180:cabin car,caboose
181
+ 181:cabinet
182
+ 182:locker,storage locker
183
+ 183:cake
184
+ 184:calculator
185
+ 185:calendar
186
+ 186:calf
187
+ 187:camcorder
188
+ 188:camel
189
+ 189:camera
190
+ 190:camera lens
191
+ 191:camper (vehicle),camping bus,motor home
192
+ 192:can,tin can
193
+ 193:can opener,tin opener
194
+ 194:candle,candlestick
195
+ 195:candle holder
196
+ 196:candy bar
197
+ 197:candy cane
198
+ 198:walking cane
199
+ 199:canister,cannister
200
+ 200:canoe
201
+ 201:cantaloup,cantaloupe
202
+ 202:canteen
203
+ 203:cap (headwear)
204
+ 204:bottle cap,cap (container lid)
205
+ 205:cape
206
+ 206:cappuccino,coffee cappuccino
207
+ 207:car (automobile),auto (automobile),automobile
208
+ 208:railcar (part of a train),railway car (part of a train),railroad car (part of a train)
209
+ 209:elevator car
210
+ 210:car battery,automobile battery
211
+ 211:identity card
212
+ 212:card
213
+ 213:cardigan
214
+ 214:cargo ship,cargo vessel
215
+ 215:carnation
216
+ 216:horse carriage
217
+ 217:carrot
218
+ 218:tote bag
219
+ 219:cart
220
+ 220:carton
221
+ 221:cash register,register (for cash transactions)
222
+ 222:casserole
223
+ 223:cassette
224
+ 224:cast,plaster cast,plaster bandage
225
+ 225:cat
226
+ 226:cauliflower
227
+ 227:cayenne (spice),cayenne pepper (spice),red pepper (spice)
228
+ 228:CD player
229
+ 229:celery
230
+ 230:cellular telephone,cellular phone,cellphone,mobile phone,smart phone
231
+ 231:chain mail,ring mail,chain armor,chain armour,ring armor,ring armour
232
+ 232:chair
233
+ 233:chaise longue,chaise,daybed
234
+ 234:chalice
235
+ 235:chandelier
236
+ 236:chap
237
+ 237:checkbook,chequebook
238
+ 238:checkerboard
239
+ 239:cherry
240
+ 240:chessboard
241
+ 241:chicken (animal)
242
+ 242:chickpea,garbanzo
243
+ 243:chili (vegetable),chili pepper (vegetable),chilli (vegetable),chilly (vegetable),chile (vegetable)
244
+ 244:chime,gong
245
+ 245:chinaware
246
+ 246:crisp (potato chip),potato chip
247
+ 247:poker chip
248
+ 248:chocolate bar
249
+ 249:chocolate cake
250
+ 250:chocolate milk
251
+ 251:chocolate mousse
252
+ 252:choker,collar,neckband
253
+ 253:chopping board,cutting board,chopping block
254
+ 254:chopstick
255
+ 255:Christmas tree
256
+ 256:slide
257
+ 257:cider,cyder
258
+ 258:cigar box
259
+ 259:cigarette
260
+ 260:cigarette case,cigarette pack
261
+ 261:cistern,water tank
262
+ 262:clarinet
263
+ 263:clasp
264
+ 264:cleansing agent,cleanser,cleaner
265
+ 265:cleat (for securing rope)
266
+ 266:clementine
267
+ 267:clip
268
+ 268:clipboard
269
+ 269:clippers (for plants)
270
+ 270:cloak
271
+ 271:clock,timepiece,timekeeper
272
+ 272:clock tower
273
+ 273:clothes hamper,laundry basket,clothes basket
274
+ 274:clothespin,clothes peg
275
+ 275:clutch bag
276
+ 276:coaster
277
+ 277:coat
278
+ 278:coat hanger,clothes hanger,dress hanger
279
+ 279:coatrack,hatrack
280
+ 280:cock,rooster
281
+ 281:cockroach
282
+ 282:cocoa (beverage),hot chocolate (beverage),drinking chocolate
283
+ 283:coconut,cocoanut
284
+ 284:coffee maker,coffee machine
285
+ 285:coffee table,cocktail table
286
+ 286:coffeepot
287
+ 287:coil
288
+ 288:coin
289
+ 289:colander,cullender
290
+ 290:coleslaw,slaw
291
+ 291:coloring material,colouring material
292
+ 292:combination lock
293
+ 293:pacifier,teething ring
294
+ 294:comic book
295
+ 295:compass
296
+ 296:computer keyboard,keyboard (computer)
297
+ 297:condiment
298
+ 298:cone,traffic cone
299
+ 299:control,controller
300
+ 300:convertible (automobile)
301
+ 301:sofa bed
302
+ 302:cooker
303
+ 303:cookie,cooky,biscuit (cookie)
304
+ 304:cooking utensil
305
+ 305:cooler (for food),ice chest
306
+ 306:cork (bottle plug),bottle cork
307
+ 307:corkboard
308
+ 308:corkscrew,bottle screw
309
+ 309:edible corn,corn,maize
310
+ 310:cornbread
311
+ 311:cornet,horn,trumpet
312
+ 312:cornice,valance,valance board,pelmet
313
+ 313:cornmeal
314
+ 314:corset,girdle
315
+ 315:costume
316
+ 316:cougar,puma,catamount,mountain lion,panther
317
+ 317:coverall
318
+ 318:cowbell
319
+ 319:cowboy hat,ten-gallon hat
320
+ 320:crab (animal)
321
+ 321:crabmeat
322
+ 322:cracker
323
+ 323:crape,crepe,French pancake
324
+ 324:crate
325
+ 325:crayon,wax crayon
326
+ 326:cream pitcher
327
+ 327:crescent roll,croissant
328
+ 328:crib,cot
329
+ 329:crock pot,earthenware jar
330
+ 330:crossbar
331
+ 331:crouton
332
+ 332:crow
333
+ 333:crowbar,wrecking bar,pry bar
334
+ 334:crown
335
+ 335:crucifix
336
+ 336:cruise ship,cruise liner
337
+ 337:police cruiser,patrol car,police car,squad car
338
+ 338:crumb
339
+ 339:crutch
340
+ 340:cub (animal)
341
+ 341:cube,square block
342
+ 342:cucumber,cuke
343
+ 343:cufflink
344
+ 344:cup
345
+ 345:trophy cup
346
+ 346:cupboard,closet
347
+ 347:cupcake
348
+ 348:hair curler,hair roller,hair crimper
349
+ 349:curling iron
350
+ 350:curtain,drapery
351
+ 351:cushion
352
+ 352:cylinder
353
+ 353:cymbal
354
+ 354:dagger
355
+ 355:dalmatian
356
+ 356:dartboard
357
+ 357:date (fruit)
358
+ 358:deck chair,beach chair
359
+ 359:deer,cervid
360
+ 360:dental floss,floss
361
+ 361:desk
362
+ 362:detergent
363
+ 363:diaper
364
+ 364:diary,journal
365
+ 365:die,dice
366
+ 366:dinghy,dory,rowboat
367
+ 367:dining table
368
+ 368:tux,tuxedo
369
+ 369:dish
370
+ 370:dish antenna
371
+ 371:dishrag,dishcloth
372
+ 372:dishtowel,tea towel
373
+ 373:dishwasher,dishwashing machine
374
+ 374:dishwasher detergent,dishwashing detergent,dishwashing liquid,dishsoap
375
+ 375:dispenser
376
+ 376:diving board
377
+ 377:Dixie cup,paper cup
378
+ 378:dog
379
+ 379:dog collar
380
+ 380:doll
381
+ 381:dollar,dollar bill,one dollar bill
382
+ 382:dollhouse,doll's house
383
+ 383:dolphin
384
+ 384:domestic ass,donkey
385
+ 385:doorknob,doorhandle
386
+ 386:doormat,welcome mat
387
+ 387:doughnut,donut
388
+ 388:dove
389
+ 389:dragonfly
390
+ 390:drawer
391
+ 391:underdrawers,boxers,boxershorts
392
+ 392:dress,frock
393
+ 393:dress hat,high hat,opera hat,silk hat,top hat
394
+ 394:dress suit
395
+ 395:dresser
396
+ 396:drill
397
+ 397:drone
398
+ 398:dropper,eye dropper
399
+ 399:drum (musical instrument)
400
+ 400:drumstick
401
+ 401:duck
402
+ 402:duckling
403
+ 403:duct tape
404
+ 404:duffel bag,duffle bag,duffel,duffle
405
+ 405:dumbbell
406
+ 406:dumpster
407
+ 407:dustpan
408
+ 408:eagle
409
+ 409:earphone,earpiece,headphone
410
+ 410:earplug
411
+ 411:earring
412
+ 412:easel
413
+ 413:eclair
414
+ 414:eel
415
+ 415:egg,eggs
416
+ 416:egg roll,spring roll
417
+ 417:egg yolk,yolk (egg)
418
+ 418:eggbeater,eggwhisk
419
+ 419:eggplant,aubergine
420
+ 420:electric chair
421
+ 421:refrigerator
422
+ 422:elephant
423
+ 423:elk,moose
424
+ 424:envelope
425
+ 425:eraser
426
+ 426:escargot
427
+ 427:eyepatch
428
+ 428:falcon
429
+ 429:fan
430
+ 430:faucet,spigot,tap
431
+ 431:fedora
432
+ 432:ferret
433
+ 433:Ferris wheel
434
+ 434:ferry,ferryboat
435
+ 435:fig (fruit)
436
+ 436:fighter jet,fighter aircraft,attack aircraft
437
+ 437:figurine
438
+ 438:file cabinet,filing cabinet
439
+ 439:file (tool)
440
+ 440:fire alarm,smoke alarm
441
+ 441:fire engine,fire truck
442
+ 442:fire extinguisher,extinguisher
443
+ 443:fire hose
444
+ 444:fireplace
445
+ 445:fireplug,fire hydrant,hydrant
446
+ 446:first-aid kit
447
+ 447:fish
448
+ 448:fish (food)
449
+ 449:fishbowl,goldfish bowl
450
+ 450:fishing rod,fishing pole
451
+ 451:flag
452
+ 452:flagpole,flagstaff
453
+ 453:flamingo
454
+ 454:flannel
455
+ 455:flap
456
+ 456:flash,flashbulb
457
+ 457:flashlight,torch
458
+ 458:fleece
459
+ 459:flip-flop (sandal)
460
+ 460:flipper (footwear),fin (footwear)
461
+ 461:flower arrangement,floral arrangement
462
+ 462:flute glass,champagne flute
463
+ 463:foal
464
+ 464:folding chair
465
+ 465:food processor
466
+ 466:football (American)
467
+ 467:football helmet
468
+ 468:footstool,footrest
469
+ 469:fork
470
+ 470:forklift
471
+ 471:freight car
472
+ 472:French toast
473
+ 473:freshener,air freshener
474
+ 474:frisbee
475
+ 475:frog,toad,toad frog
476
+ 476:fruit juice
477
+ 477:frying pan,frypan,skillet
478
+ 478:fudge
479
+ 479:funnel
480
+ 480:futon
481
+ 481:gag,muzzle
482
+ 482:garbage
483
+ 483:garbage truck
484
+ 484:garden hose
485
+ 485:gargle,mouthwash
486
+ 486:gargoyle
487
+ 487:garlic,ail
488
+ 488:gasmask,respirator,gas helmet
489
+ 489:gazelle
490
+ 490:gelatin,jelly
491
+ 491:gemstone
492
+ 492:generator
493
+ 493:giant panda,panda,panda bear
494
+ 494:gift wrap
495
+ 495:ginger,gingerroot
496
+ 496:giraffe
497
+ 497:cincture,sash,waistband,waistcloth
498
+ 498:glass (drink container),drinking glass
499
+ 499:globe
500
+ 500:glove
501
+ 501:goat
502
+ 502:goggles
503
+ 503:goldfish
504
+ 504:golf club,golf-club
505
+ 505:golfcart
506
+ 506:gondola (boat)
507
+ 507:goose
508
+ 508:gorilla
509
+ 509:gourd
510
+ 510:grape
511
+ 511:grater
512
+ 512:gravestone,headstone,tombstone
513
+ 513:gravy boat,gravy holder
514
+ 514:green bean
515
+ 515:green onion,spring onion,scallion
516
+ 516:griddle
517
+ 517:grill,grille,grillwork,radiator grille
518
+ 518:grits,hominy grits
519
+ 519:grizzly,grizzly bear
520
+ 520:grocery bag
521
+ 521:guitar
522
+ 522:gull,seagull
523
+ 523:gun
524
+ 524:hairbrush
525
+ 525:hairnet
526
+ 526:hairpin
527
+ 527:halter top
528
+ 528:ham,jambon,gammon
529
+ 529:hamburger,beefburger,burger
530
+ 530:hammer
531
+ 531:hammock
532
+ 532:hamper
533
+ 533:hamster
534
+ 534:hair dryer
535
+ 535:hand glass,hand mirror
536
+ 536:hand towel,face towel
537
+ 537:handcart,pushcart,hand truck
538
+ 538:handcuff
539
+ 539:handkerchief
540
+ 540:handle,grip,handgrip
541
+ 541:handsaw,carpenter's saw
542
+ 542:hardback book,hardcover book
543
+ 543:harmonium,organ (musical instrument),reed organ (musical instrument)
544
+ 544:hat
545
+ 545:hatbox
546
+ 546:veil
547
+ 547:headband
548
+ 548:headboard
549
+ 549:headlight,headlamp
550
+ 550:headscarf
551
+ 551:headset
552
+ 552:headstall (for horses),headpiece (for horses)
553
+ 553:heart
554
+ 554:heater,warmer
555
+ 555:helicopter
556
+ 556:helmet
557
+ 557:heron
558
+ 558:highchair,feeding chair
559
+ 559:hinge
560
+ 560:hippopotamus
561
+ 561:hockey stick
562
+ 562:hog,pig
563
+ 563:home plate (baseball),home base (baseball)
564
+ 564:honey
565
+ 565:fume hood,exhaust hood
566
+ 566:hook
567
+ 567:hookah,narghile,nargileh,sheesha,shisha,water pipe
568
+ 568:hornet
569
+ 569:horse
570
+ 570:hose,hosepipe
571
+ 571:hot-air balloon
572
+ 572:hotplate
573
+ 573:hot sauce
574
+ 574:hourglass
575
+ 575:houseboat
576
+ 576:hummingbird
577
+ 577:hummus,humus,hommos,hoummos,humous
578
+ 578:polar bear
579
+ 579:icecream
580
+ 580:popsicle
581
+ 581:ice maker
582
+ 582:ice pack,ice bag
583
+ 583:ice skate
584
+ 584:igniter,ignitor,lighter
585
+ 585:inhaler,inhalator
586
+ 586:iPod
587
+ 587:iron (for clothing),smoothing iron (for clothing)
588
+ 588:ironing board
589
+ 589:jacket
590
+ 590:jam
591
+ 591:jar
592
+ 592:jean,blue jean,denim
593
+ 593:jeep,landrover
594
+ 594:jelly bean,jelly egg
595
+ 595:jersey,T-shirt,tee shirt
596
+ 596:jet plane,jet-propelled plane
597
+ 597:jewel,gem,precious stone
598
+ 598:jewelry,jewellery
599
+ 599:joystick
600
+ 600:jumpsuit
601
+ 601:kayak
602
+ 602:keg
603
+ 603:kennel,doghouse
604
+ 604:kettle,boiler
605
+ 605:key
606
+ 606:keycard
607
+ 607:kilt
608
+ 608:kimono
609
+ 609:kitchen sink
610
+ 610:kitchen table
611
+ 611:kite
612
+ 612:kitten,kitty
613
+ 613:kiwi fruit
614
+ 614:knee pad
615
+ 615:knife
616
+ 616:knitting needle
617
+ 617:knob
618
+ 618:knocker (on a door),doorknocker
619
+ 619:koala,koala bear
620
+ 620:lab coat,laboratory coat
621
+ 621:ladder
622
+ 622:ladle
623
+ 623:ladybug,ladybeetle,ladybird beetle
624
+ 624:lamb (animal)
625
+ 625:lamb-chop,lambchop
626
+ 626:lamp
627
+ 627:lamppost
628
+ 628:lampshade
629
+ 629:lantern
630
+ 630:lanyard,laniard
631
+ 631:laptop computer,notebook computer
632
+ 632:lasagna,lasagne
633
+ 633:latch
634
+ 634:lawn mower
635
+ 635:leather
636
+ 636:legging (clothing),leging (clothing),leg covering
637
+ 637:Lego,Lego set
638
+ 638:legume
639
+ 639:lemon
640
+ 640:lemonade
641
+ 641:lettuce
642
+ 642:license plate,numberplate
643
+ 643:life buoy,lifesaver,life belt,life ring
644
+ 644:life jacket,life vest
645
+ 645:lightbulb
646
+ 646:lightning rod,lightning conductor
647
+ 647:lime
648
+ 648:limousine
649
+ 649:lion
650
+ 650:lip balm
651
+ 651:liquor,spirits,hard liquor,liqueur,cordial
652
+ 652:lizard
653
+ 653:log
654
+ 654:lollipop
655
+ 655:speaker (stero equipment)
656
+ 656:loveseat
657
+ 657:machine gun
658
+ 658:magazine
659
+ 659:magnet
660
+ 660:mail slot
661
+ 661:mailbox (at home),letter box (at home)
662
+ 662:mallard
663
+ 663:mallet
664
+ 664:mammoth
665
+ 665:manatee
666
+ 666:mandarin orange
667
+ 667:manger,trough
668
+ 668:manhole
669
+ 669:map
670
+ 670:marker
671
+ 671:martini
672
+ 672:mascot
673
+ 673:mashed potato
674
+ 674:masher
675
+ 675:mask,facemask
676
+ 676:mast
677
+ 677:mat (gym equipment),gym mat
678
+ 678:matchbox
679
+ 679:mattress
680
+ 680:measuring cup
681
+ 681:measuring stick,ruler (measuring stick),measuring rod
682
+ 682:meatball
683
+ 683:medicine
684
+ 684:melon
685
+ 685:microphone
686
+ 686:microscope
687
+ 687:microwave oven
688
+ 688:milestone,milepost
689
+ 689:milk
690
+ 690:milk can
691
+ 691:milkshake
692
+ 692:minivan
693
+ 693:mint candy
694
+ 694:mirror
695
+ 695:mitten
696
+ 696:mixer (kitchen tool),stand mixer
697
+ 697:money
698
+ 698:monitor (computer equipment) computer monitor
699
+ 699:monkey
700
+ 700:motor
701
+ 701:motor scooter,scooter
702
+ 702:motor vehicle,automotive vehicle
703
+ 703:motorcycle
704
+ 704:mound (baseball),pitcher's mound
705
+ 705:mouse (computer equipment),computer mouse
706
+ 706:mousepad
707
+ 707:muffin
708
+ 708:mug
709
+ 709:mushroom
710
+ 710:music stool,piano stool
711
+ 711:musical instrument,instrument (musical)
712
+ 712:nailfile
713
+ 713:napkin,table napkin,serviette
714
+ 714:neckerchief
715
+ 715:necklace
716
+ 716:necktie,tie (necktie)
717
+ 717:needle
718
+ 718:nest
719
+ 719:newspaper,paper (newspaper)
720
+ 720:newsstand
721
+ 721:nightshirt,nightwear,sleepwear,nightclothes
722
+ 722:nosebag (for animals),feedbag
723
+ 723:noseband (for animals),nosepiece (for animals)
724
+ 724:notebook
725
+ 725:notepad
726
+ 726:nut
727
+ 727:nutcracker
728
+ 728:oar
729
+ 729:octopus (food)
730
+ 730:octopus (animal)
731
+ 731:oil lamp,kerosene lamp,kerosine lamp
732
+ 732:olive oil
733
+ 733:omelet,omelette
734
+ 734:onion
735
+ 735:orange (fruit)
736
+ 736:orange juice
737
+ 737:ostrich
738
+ 738:ottoman,pouf,pouffe,hassock
739
+ 739:oven
740
+ 740:overalls (clothing)
741
+ 741:owl
742
+ 742:packet
743
+ 743:inkpad,inking pad,stamp pad
744
+ 744:pad
745
+ 745:paddle,boat paddle
746
+ 746:padlock
747
+ 747:paintbrush
748
+ 748:painting
749
+ 749:pajamas,pyjamas
750
+ 750:palette,pallet
751
+ 751:pan (for cooking),cooking pan
752
+ 752:pan (metal container)
753
+ 753:pancake
754
+ 754:pantyhose
755
+ 755:papaya
756
+ 756:paper plate
757
+ 757:paper towel
758
+ 758:paperback book,paper-back book,softback book,soft-cover book
759
+ 759:paperweight
760
+ 760:parachute
761
+ 761:parakeet,parrakeet,parroket,paraquet,paroquet,parroquet
762
+ 762:parasail (sports)
763
+ 763:parasol,sunshade
764
+ 764:parchment
765
+ 765:parka,anorak
766
+ 766:parking meter
767
+ 767:parrot
768
+ 768:passenger car (part of a train),coach (part of a train)
769
+ 769:passenger ship
770
+ 770:passport
771
+ 771:pastry
772
+ 772:patty (food)
773
+ 773:pea (food)
774
+ 774:peach
775
+ 775:peanut butter
776
+ 776:pear
777
+ 777:peeler (tool for fruit and vegetables)
778
+ 778:wooden leg,pegleg
779
+ 779:pegboard
780
+ 780:pelican
781
+ 781:pen
782
+ 782:pencil
783
+ 783:pencil box,pencil case
784
+ 784:pencil sharpener
785
+ 785:pendulum
786
+ 786:penguin
787
+ 787:pennant
788
+ 788:penny (coin)
789
+ 789:pepper,peppercorn
790
+ 790:pepper mill,pepper grinder
791
+ 791:perfume
792
+ 792:persimmon
793
+ 793:person,baby,child,boy,girl,man,woman,human
794
+ 794:pet
795
+ 795:pew (church bench),church bench
796
+ 796:phonebook,telephone book,telephone directory
797
+ 797:phonograph record,phonograph recording,record (phonograph recording)
798
+ 798:piano
799
+ 799:pickle
800
+ 800:pickup truck
801
+ 801:pie
802
+ 802:pigeon
803
+ 803:piggy bank,penny bank
804
+ 804:pillow
805
+ 805:pin (non jewelry)
806
+ 806:pineapple
807
+ 807:pinecone
808
+ 808:ping-pong ball
809
+ 809:pinwheel
810
+ 810:tobacco pipe
811
+ 811:pipe,piping
812
+ 812:pistol,handgun
813
+ 813:pita (bread),pocket bread
814
+ 814:pitcher (vessel for liquid),ewer
815
+ 815:pitchfork
816
+ 816:pizza
817
+ 817:place mat
818
+ 818:plate
819
+ 819:platter
820
+ 820:playpen
821
+ 821:pliers,plyers
822
+ 822:plow (farm equipment),plough (farm equipment)
823
+ 823:plume
824
+ 824:pocket watch
825
+ 825:pocketknife
826
+ 826:poker (fire stirring tool),stove poker,fire hook
827
+ 827:pole,post
828
+ 828:polo shirt,sport shirt
829
+ 829:poncho
830
+ 830:pony
831
+ 831:pool table,billiard table,snooker table
832
+ 832:pop (soda),soda (pop),tonic,soft drink
833
+ 833:postbox (public),mailbox (public)
834
+ 834:postcard,postal card,mailing-card
835
+ 835:poster,placard
836
+ 836:pot
837
+ 837:flowerpot
838
+ 838:potato
839
+ 839:potholder
840
+ 840:pottery,clayware
841
+ 841:pouch
842
+ 842:power shovel,excavator,digger
843
+ 843:prawn,shrimp
844
+ 844:pretzel
845
+ 845:printer,printing machine
846
+ 846:projectile (weapon),missile
847
+ 847:projector
848
+ 848:propeller,propellor
849
+ 849:prune
850
+ 850:pudding
851
+ 851:puffer (fish),pufferfish,blowfish,globefish
852
+ 852:puffin
853
+ 853:pug-dog
854
+ 854:pumpkin
855
+ 855:puncher
856
+ 856:puppet,marionette
857
+ 857:puppy
858
+ 858:quesadilla
859
+ 859:quiche
860
+ 860:quilt,comforter
861
+ 861:rabbit
862
+ 862:race car,racing car
863
+ 863:racket,racquet
864
+ 864:radar
865
+ 865:radiator
866
+ 866:radio receiver,radio set,radio,tuner (radio)
867
+ 867:radish,daikon
868
+ 868:raft
869
+ 869:rag doll
870
+ 870:raincoat,waterproof jacket
871
+ 871:ram (animal)
872
+ 872:raspberry
873
+ 873:rat
874
+ 874:razorblade
875
+ 875:reamer (juicer),juicer,juice reamer
876
+ 876:rearview mirror
877
+ 877:receipt
878
+ 878:recliner,reclining chair,lounger (chair)
879
+ 879:record player,phonograph (record player),turntable
880
+ 880:reflector
881
+ 881:remote control
882
+ 882:rhinoceros
883
+ 883:rib (food)
884
+ 884:rifle
885
+ 885:ring
886
+ 886:river boat
887
+ 887:road map
888
+ 888:robe
889
+ 889:rocking chair
890
+ 890:rodent
891
+ 891:roller skate
892
+ 892:Rollerblade
893
+ 893:rolling pin
894
+ 894:root beer
895
+ 895:router (computer equipment)
896
+ 896:rubber band,elastic band
897
+ 897:runner (carpet)
898
+ 898:plastic bag,paper bag
899
+ 899:saddle (on an animal)
900
+ 900:saddle blanket,saddlecloth,horse blanket
901
+ 901:saddlebag
902
+ 902:safety pin
903
+ 903:sail
904
+ 904:salad
905
+ 905:salad plate,salad bowl
906
+ 906:salami
907
+ 907:salmon (fish)
908
+ 908:salmon (food)
909
+ 909:salsa
910
+ 910:saltshaker
911
+ 911:sandal (type of shoe)
912
+ 912:sandwich
913
+ 913:satchel
914
+ 914:saucepan
915
+ 915:saucer
916
+ 916:sausage
917
+ 917:sawhorse,sawbuck
918
+ 918:saxophone
919
+ 919:scale (measuring instrument)
920
+ 920:scarecrow,strawman
921
+ 921:scarf
922
+ 922:school bus
923
+ 923:scissors
924
+ 924:scoreboard
925
+ 925:scraper
926
+ 926:screwdriver
927
+ 927:scrubbing brush
928
+ 928:sculpture
929
+ 929:seabird,seafowl
930
+ 930:seahorse
931
+ 931:seaplane,hydroplane
932
+ 932:seashell
933
+ 933:sewing machine
934
+ 934:shaker
935
+ 935:shampoo
936
+ 936:shark
937
+ 937:sharpener
938
+ 938:Sharpie
939
+ 939:shaver (electric),electric shaver,electric razor
940
+ 940:shaving cream,shaving soap
941
+ 941:shawl
942
+ 942:shears
943
+ 943:sheep
944
+ 944:shepherd dog,sheepdog
945
+ 945:sherbert,sherbet
946
+ 946:shield
947
+ 947:shirt
948
+ 948:shoe,sneaker (type of shoe),tennis shoe
949
+ 949:shopping bag
950
+ 950:shopping cart
951
+ 951:short pants,shorts (clothing),trunks (clothing)
952
+ 952:shot glass
953
+ 953:shoulder bag
954
+ 954:shovel
955
+ 955:shower head
956
+ 956:shower cap
957
+ 957:shower curtain
958
+ 958:shredder (for paper)
959
+ 959:signboard
960
+ 960:silo
961
+ 961:sink
962
+ 962:skateboard
963
+ 963:skewer
964
+ 964:ski
965
+ 965:ski boot
966
+ 966:ski parka,ski jacket
967
+ 967:ski pole
968
+ 968:skirt
969
+ 969:skullcap
970
+ 970:sled,sledge,sleigh
971
+ 971:sleeping bag
972
+ 972:sling (bandage),triangular bandage
973
+ 973:slipper (footwear),carpet slipper (footwear)
974
+ 974:smoothie
975
+ 975:snake,serpent
976
+ 976:snowboard
977
+ 977:snowman
978
+ 978:snowmobile
979
+ 979:soap
980
+ 980:soccer ball
981
+ 981:sock
982
+ 982:sofa,couch,lounge
983
+ 983:softball
984
+ 984:solar array,solar battery,solar panel
985
+ 985:sombrero
986
+ 986:soup
987
+ 987:soup bowl
988
+ 988:soupspoon
989
+ 989:sour cream,soured cream
990
+ 990:soya milk,soybean milk,soymilk
991
+ 991:space shuttle
992
+ 992:sparkler (fireworks)
993
+ 993:spatula
994
+ 994:spear,lance
995
+ 995:spectacles,specs,eyeglasses,glasses
996
+ 996:spice rack
997
+ 997:spider
998
+ 998:crawfish,crayfish
999
+ 999:sponge
1000
+ 1000:spoon
1001
+ 1001:sportswear,athletic wear,activewear
1002
+ 1002:spotlight
1003
+ 1003:squid (food),calamari,calamary
1004
+ 1004:squirrel
1005
+ 1005:stagecoach
1006
+ 1006:stapler (stapling machine)
1007
+ 1007:starfish,sea star
1008
+ 1008:statue (sculpture)
1009
+ 1009:steak (food)
1010
+ 1010:steak knife
1011
+ 1011:steering wheel
1012
+ 1012:stepladder
1013
+ 1013:step stool
1014
+ 1014:stereo (sound system)
1015
+ 1015:stew
1016
+ 1016:stirrer
1017
+ 1017:stirrup
1018
+ 1018:stool
1019
+ 1019:stop sign
1020
+ 1020:brake light
1021
+ 1021:stove,kitchen stove,range (kitchen appliance),kitchen range,cooking stove
1022
+ 1022:strainer
1023
+ 1023:strap
1024
+ 1024:straw (for drinking),drinking straw
1025
+ 1025:strawberry
1026
+ 1026:street sign
1027
+ 1027:streetlight,street lamp
1028
+ 1028:string cheese
1029
+ 1029:stylus
1030
+ 1030:subwoofer
1031
+ 1031:sugar bowl
1032
+ 1032:sugarcane (plant)
1033
+ 1033:suit (clothing)
1034
+ 1034:sunflower
1035
+ 1035:sunglasses
1036
+ 1036:sunhat
1037
+ 1037:surfboard
1038
+ 1038:sushi
1039
+ 1039:mop
1040
+ 1040:sweat pants
1041
+ 1041:sweatband
1042
+ 1042:sweater
1043
+ 1043:sweatshirt
1044
+ 1044:sweet potato
1045
+ 1045:swimsuit,swimwear,bathing suit,swimming costume,bathing costume,swimming trunks,bathing trunks
1046
+ 1046:sword
1047
+ 1047:syringe
1048
+ 1048:Tabasco sauce
1049
+ 1049:table-tennis table,ping-pong table
1050
+ 1050:table
1051
+ 1051:table lamp
1052
+ 1052:tablecloth
1053
+ 1053:tachometer
1054
+ 1054:taco
1055
+ 1055:tag
1056
+ 1056:taillight,rear light
1057
+ 1057:tambourine
1058
+ 1058:army tank,armored combat vehicle,armoured combat vehicle
1059
+ 1059:tank (storage vessel),storage tank
1060
+ 1060:tank top (clothing)
1061
+ 1061:tape (sticky cloth or paper)
1062
+ 1062:tape measure,measuring tape
1063
+ 1063:tapestry
1064
+ 1064:tarp
1065
+ 1065:tartan,plaid
1066
+ 1066:tassel
1067
+ 1067:tea bag
1068
+ 1068:teacup
1069
+ 1069:teakettle
1070
+ 1070:teapot
1071
+ 1071:teddy bear
1072
+ 1072:telephone,phone,telephone set
1073
+ 1073:telephone booth,phone booth,call box,telephone box,telephone kiosk
1074
+ 1074:telephone pole,telegraph pole,telegraph post
1075
+ 1075:telephoto lens,zoom lens
1076
+ 1076:television camera,tv camera
1077
+ 1077:television set,tv,tv set
1078
+ 1078:tennis ball
1079
+ 1079:tennis racket
1080
+ 1080:tequila
1081
+ 1081:thermometer
1082
+ 1082:thermos bottle
1083
+ 1083:thermostat
1084
+ 1084:thimble
1085
+ 1085:thread,yarn
1086
+ 1086:thumbtack,drawing pin,pushpin
1087
+ 1087:tiara
1088
+ 1088:tiger
1089
+ 1089:tights (clothing),leotards
1090
+ 1090:timer,stopwatch
1091
+ 1091:tinfoil
1092
+ 1092:tinsel
1093
+ 1093:tissue paper
1094
+ 1094:toast (food)
1095
+ 1095:toaster
1096
+ 1096:toaster oven
1097
+ 1097:toilet
1098
+ 1098:toilet tissue,toilet paper,bathroom tissue
1099
+ 1099:tomato
1100
+ 1100:tongs
1101
+ 1101:toolbox
1102
+ 1102:toothbrush
1103
+ 1103:toothpaste
1104
+ 1104:toothpick
1105
+ 1105:cover
1106
+ 1106:tortilla
1107
+ 1107:tow truck
1108
+ 1108:towel
1109
+ 1109:towel rack,towel rail,towel bar
1110
+ 1110:toy
1111
+ 1111:tractor (farm equipment)
1112
+ 1112:traffic light
1113
+ 1113:dirt bike
1114
+ 1114:trailer truck,tractor trailer,trucking rig,articulated lorry,semi truck
1115
+ 1115:train (railroad vehicle),railroad train
1116
+ 1116:trampoline
1117
+ 1117:tray
1118
+ 1118:trench coat
1119
+ 1119:triangle (musical instrument)
1120
+ 1120:tricycle
1121
+ 1121:tripod
1122
+ 1122:trousers,pants (clothing)
1123
+ 1123:truck
1124
+ 1124:truffle (chocolate),chocolate truffle
1125
+ 1125:trunk
1126
+ 1126:vat
1127
+ 1127:turban
1128
+ 1128:turkey (food)
1129
+ 1129:turnip
1130
+ 1130:turtle
1131
+ 1131:turtleneck (clothing),polo-neck
1132
+ 1132:typewriter
1133
+ 1133:umbrella
1134
+ 1134:underwear,underclothes,underclothing,underpants
1135
+ 1135:unicycle
1136
+ 1136:urinal
1137
+ 1137:urn
1138
+ 1138:vacuum cleaner
1139
+ 1139:vase
1140
+ 1140:vending machine
1141
+ 1141:vent,blowhole,air vent
1142
+ 1142:vest,waistcoat
1143
+ 1143:videotape
1144
+ 1144:vinegar
1145
+ 1145:violin,fiddle
1146
+ 1146:vodka
1147
+ 1147:volleyball
1148
+ 1148:vulture
1149
+ 1149:waffle
1150
+ 1150:waffle iron
1151
+ 1151:wagon
1152
+ 1152:wagon wheel
1153
+ 1153:walking stick
1154
+ 1154:wall clock
1155
+ 1155:wall socket,wall plug,electric outlet,electrical outlet,outlet,electric receptacle
1156
+ 1156:wallet,billfold
1157
+ 1157:walrus
1158
+ 1158:wardrobe
1159
+ 1159:washbasin,basin (for washing),washbowl,washstand,handbasin
1160
+ 1160:automatic washer,washing machine
1161
+ 1161:watch,wristwatch
1162
+ 1162:water bottle
1163
+ 1163:water cooler
1164
+ 1164:water faucet,water tap,tap (water faucet)
1165
+ 1165:water heater,hot-water heater
1166
+ 1166:water jug
1167
+ 1167:water gun,squirt gun
1168
+ 1168:water scooter,sea scooter,jet ski
1169
+ 1169:water ski
1170
+ 1170:water tower
1171
+ 1171:watering can
1172
+ 1172:watermelon
1173
+ 1173:weathervane,vane (weathervane),wind vane
1174
+ 1174:webcam
1175
+ 1175:wedding cake,bridecake
1176
+ 1176:wedding ring,wedding band
1177
+ 1177:wet suit
1178
+ 1178:wheel
1179
+ 1179:wheelchair
1180
+ 1180:whipped cream
1181
+ 1181:whistle
1182
+ 1182:wig
1183
+ 1183:wind chime
1184
+ 1184:windmill
1185
+ 1185:window box (for plants)
1186
+ 1186:windshield wiper,windscreen wiper,wiper (for windshield/screen)
1187
+ 1187:windsock,air sock,air-sleeve,wind sleeve,wind cone
1188
+ 1188:wine bottle
1189
+ 1189:wine bucket,wine cooler
1190
+ 1190:wineglass
1191
+ 1191:blinder (for horses)
1192
+ 1192:wok
1193
+ 1193:wolf
1194
+ 1194:wooden spoon
1195
+ 1195:wreath
1196
+ 1196:wrench,spanner
1197
+ 1197:wristband
1198
+ 1198:wristlet,wrist band
1199
+ 1199:yacht
1200
+ 1200:yogurt,yoghurt,yoghourt
1201
+ 1201:yoke (animal equipment)
1202
+ 1202:zebra
1203
+ 1203:zucchini,courgette