Hugging Face's logo

Spaces:
Awiny
/
Image2Paragraph
Runtime error

App Files Community
3
Awiny commited on
Commit
e175cc1
•
1 Parent(s): 9fed418

update requirements.txt

Browse files
Files changed (37) hide show
  1. examples/3.jpg +0 -0
  2. main.py +20 -0
  3. models/__pycache__/blip2_model.cpython-38.pyc +0 -0
  4. models/__pycache__/controlnet_model.cpython-38.pyc +0 -0
  5. models/__pycache__/gpt_model.cpython-38.pyc +0 -0
  6. models/__pycache__/grit_model.cpython-38.pyc +0 -0
  7. models/__pycache__/image_text_transformation.cpython-38.pyc +0 -0
  8. models/__pycache__/region_semantic.cpython-38.pyc +0 -0
  9. models/grit_src/__pycache__/image_dense_captions.cpython-38.pyc +0 -0
  10. models/grit_src/grit/__pycache__/__init__.cpython-38.pyc +0 -0
  11. models/grit_src/grit/__pycache__/config.cpython-38.pyc +0 -0
  12. models/grit_src/grit/__pycache__/predictor.cpython-38.pyc +0 -0
  13. models/grit_src/grit/data/__pycache__/custom_build_augmentation.cpython-38.pyc +0 -0
  14. models/grit_src/grit/data/__pycache__/custom_dataset_mapper.cpython-38.pyc +0 -0
  15. models/grit_src/grit/data/datasets/__pycache__/grit_coco.cpython-38.pyc +0 -0
  16. models/grit_src/grit/data/datasets/__pycache__/object365.cpython-38.pyc +0 -0
  17. models/grit_src/grit/data/datasets/__pycache__/vg.cpython-38.pyc +0 -0
  18. models/grit_src/grit/data/transforms/__pycache__/custom_augmentation_impl.cpython-38.pyc +0 -0
  19. models/grit_src/grit/data/transforms/__pycache__/custom_transform.cpython-38.pyc +0 -0
  20. models/grit_src/grit/modeling/__pycache__/soft_nms.cpython-38.pyc +0 -0
  21. models/grit_src/grit/modeling/backbone/__pycache__/utils.cpython-38.pyc +0 -0
  22. models/grit_src/grit/modeling/backbone/__pycache__/vit.cpython-38.pyc +0 -0
  23. models/grit_src/grit/modeling/meta_arch/__pycache__/grit.cpython-38.pyc +0 -0
  24. models/grit_src/grit/modeling/roi_heads/__pycache__/grit_fast_rcnn.cpython-38.pyc +0 -0
  25. models/grit_src/grit/modeling/roi_heads/__pycache__/grit_roi_heads.cpython-38.pyc +0 -0
  26. models/grit_src/grit/modeling/text/__pycache__/file_utils.cpython-38.pyc +0 -0
  27. models/grit_src/grit/modeling/text/__pycache__/load_text_token.cpython-38.pyc +0 -0
  28. models/grit_src/grit/modeling/text/__pycache__/modeling_bert.cpython-38.pyc +0 -0
  29. models/grit_src/grit/modeling/text/__pycache__/text_decoder.cpython-38.pyc +0 -0
  30. models/segment_models/__pycache__/semantic_segment_anything_model.cpython-38.pyc +0 -0
  31. models/segment_models/__pycache__/semgent_anything_model.cpython-38.pyc +0 -0
  32. models/segment_models/configs/__pycache__/__init__.cpython-38.pyc +0 -0
  33. models/segment_models/configs/__pycache__/ade20k_id2label.cpython-38.pyc +0 -0
  34. models/segment_models/configs/__pycache__/coco_id2label.cpython-38.pyc +0 -0
  35. output/3_result_3.jpg +0 -0
  36. requirements.txt +11 -5
  37. utils/__pycache__/util.cpython-38.pyc +0 -0
examples/3.jpg ADDED
main.py ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ from models.image_text_transformation import ImageTextTransformation
3
+ from utils.util import display_images_and_text
4
+
5
+ if __name__ == '__main__':
6
+ parser = argparse.ArgumentParser()
7
+ parser.add_argument('--image_src', default='examples/1.jpg')
8
+ parser.add_argument('--out_image_name', default='output/1_result.jpg')
9
+ args = parser.parse_args()
10
+
11
+ processor = ImageTextTransformation()
12
+ generated_text = processor.image_to_text(args.image_src)
13
+ generated_image = processor.text_to_image(generated_text)
14
+ ## then text to image
15
+ print("*" * 50)
16
+ print("Generated Text:")
17
+ print(generated_text)
18
+ print("*" * 50)
19
+
20
+ results = display_images_and_text(args.image_src, generated_image, generated_text, args.out_image_name)
models/__pycache__/blip2_model.cpython-38.pyc CHANGED
Binary files a/models/__pycache__/blip2_model.cpython-38.pyc and b/models/__pycache__/blip2_model.cpython-38.pyc differ
 
models/__pycache__/controlnet_model.cpython-38.pyc CHANGED
Binary files a/models/__pycache__/controlnet_model.cpython-38.pyc and b/models/__pycache__/controlnet_model.cpython-38.pyc differ
 
models/__pycache__/gpt_model.cpython-38.pyc CHANGED
Binary files a/models/__pycache__/gpt_model.cpython-38.pyc and b/models/__pycache__/gpt_model.cpython-38.pyc differ
 
models/__pycache__/grit_model.cpython-38.pyc CHANGED
Binary files a/models/__pycache__/grit_model.cpython-38.pyc and b/models/__pycache__/grit_model.cpython-38.pyc differ
 
models/__pycache__/image_text_transformation.cpython-38.pyc CHANGED
Binary files a/models/__pycache__/image_text_transformation.cpython-38.pyc and b/models/__pycache__/image_text_transformation.cpython-38.pyc differ
 
models/__pycache__/region_semantic.cpython-38.pyc CHANGED
Binary files a/models/__pycache__/region_semantic.cpython-38.pyc and b/models/__pycache__/region_semantic.cpython-38.pyc differ
 
models/grit_src/__pycache__/image_dense_captions.cpython-38.pyc CHANGED
Binary files a/models/grit_src/__pycache__/image_dense_captions.cpython-38.pyc and b/models/grit_src/__pycache__/image_dense_captions.cpython-38.pyc differ
 
models/grit_src/grit/__pycache__/__init__.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/__pycache__/__init__.cpython-38.pyc and b/models/grit_src/grit/__pycache__/__init__.cpython-38.pyc differ
 
models/grit_src/grit/__pycache__/config.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/__pycache__/config.cpython-38.pyc and b/models/grit_src/grit/__pycache__/config.cpython-38.pyc differ
 
models/grit_src/grit/__pycache__/predictor.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/__pycache__/predictor.cpython-38.pyc and b/models/grit_src/grit/__pycache__/predictor.cpython-38.pyc differ
 
models/grit_src/grit/data/__pycache__/custom_build_augmentation.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/data/__pycache__/custom_build_augmentation.cpython-38.pyc and b/models/grit_src/grit/data/__pycache__/custom_build_augmentation.cpython-38.pyc differ
 
models/grit_src/grit/data/__pycache__/custom_dataset_mapper.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/data/__pycache__/custom_dataset_mapper.cpython-38.pyc and b/models/grit_src/grit/data/__pycache__/custom_dataset_mapper.cpython-38.pyc differ
 
models/grit_src/grit/data/datasets/__pycache__/grit_coco.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/data/datasets/__pycache__/grit_coco.cpython-38.pyc and b/models/grit_src/grit/data/datasets/__pycache__/grit_coco.cpython-38.pyc differ
 
models/grit_src/grit/data/datasets/__pycache__/object365.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/data/datasets/__pycache__/object365.cpython-38.pyc and b/models/grit_src/grit/data/datasets/__pycache__/object365.cpython-38.pyc differ
 
models/grit_src/grit/data/datasets/__pycache__/vg.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/data/datasets/__pycache__/vg.cpython-38.pyc and b/models/grit_src/grit/data/datasets/__pycache__/vg.cpython-38.pyc differ
 
models/grit_src/grit/data/transforms/__pycache__/custom_augmentation_impl.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/data/transforms/__pycache__/custom_augmentation_impl.cpython-38.pyc and b/models/grit_src/grit/data/transforms/__pycache__/custom_augmentation_impl.cpython-38.pyc differ
 
models/grit_src/grit/data/transforms/__pycache__/custom_transform.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/data/transforms/__pycache__/custom_transform.cpython-38.pyc and b/models/grit_src/grit/data/transforms/__pycache__/custom_transform.cpython-38.pyc differ
 
models/grit_src/grit/modeling/__pycache__/soft_nms.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/__pycache__/soft_nms.cpython-38.pyc and b/models/grit_src/grit/modeling/__pycache__/soft_nms.cpython-38.pyc differ
 
models/grit_src/grit/modeling/backbone/__pycache__/utils.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/backbone/__pycache__/utils.cpython-38.pyc and b/models/grit_src/grit/modeling/backbone/__pycache__/utils.cpython-38.pyc differ
 
models/grit_src/grit/modeling/backbone/__pycache__/vit.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/backbone/__pycache__/vit.cpython-38.pyc and b/models/grit_src/grit/modeling/backbone/__pycache__/vit.cpython-38.pyc differ
 
models/grit_src/grit/modeling/meta_arch/__pycache__/grit.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/meta_arch/__pycache__/grit.cpython-38.pyc and b/models/grit_src/grit/modeling/meta_arch/__pycache__/grit.cpython-38.pyc differ
 
models/grit_src/grit/modeling/roi_heads/__pycache__/grit_fast_rcnn.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/roi_heads/__pycache__/grit_fast_rcnn.cpython-38.pyc and b/models/grit_src/grit/modeling/roi_heads/__pycache__/grit_fast_rcnn.cpython-38.pyc differ
 
models/grit_src/grit/modeling/roi_heads/__pycache__/grit_roi_heads.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/roi_heads/__pycache__/grit_roi_heads.cpython-38.pyc and b/models/grit_src/grit/modeling/roi_heads/__pycache__/grit_roi_heads.cpython-38.pyc differ
 
models/grit_src/grit/modeling/text/__pycache__/file_utils.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/text/__pycache__/file_utils.cpython-38.pyc and b/models/grit_src/grit/modeling/text/__pycache__/file_utils.cpython-38.pyc differ
 
models/grit_src/grit/modeling/text/__pycache__/load_text_token.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/text/__pycache__/load_text_token.cpython-38.pyc and b/models/grit_src/grit/modeling/text/__pycache__/load_text_token.cpython-38.pyc differ
 
models/grit_src/grit/modeling/text/__pycache__/modeling_bert.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/text/__pycache__/modeling_bert.cpython-38.pyc and b/models/grit_src/grit/modeling/text/__pycache__/modeling_bert.cpython-38.pyc differ
 
models/grit_src/grit/modeling/text/__pycache__/text_decoder.cpython-38.pyc CHANGED
Binary files a/models/grit_src/grit/modeling/text/__pycache__/text_decoder.cpython-38.pyc and b/models/grit_src/grit/modeling/text/__pycache__/text_decoder.cpython-38.pyc differ
 
models/segment_models/__pycache__/semantic_segment_anything_model.cpython-38.pyc CHANGED
Binary files a/models/segment_models/__pycache__/semantic_segment_anything_model.cpython-38.pyc and b/models/segment_models/__pycache__/semantic_segment_anything_model.cpython-38.pyc differ
 
models/segment_models/__pycache__/semgent_anything_model.cpython-38.pyc CHANGED
Binary files a/models/segment_models/__pycache__/semgent_anything_model.cpython-38.pyc and b/models/segment_models/__pycache__/semgent_anything_model.cpython-38.pyc differ
 
models/segment_models/configs/__pycache__/__init__.cpython-38.pyc CHANGED
Binary files a/models/segment_models/configs/__pycache__/__init__.cpython-38.pyc and b/models/segment_models/configs/__pycache__/__init__.cpython-38.pyc differ
 
models/segment_models/configs/__pycache__/ade20k_id2label.cpython-38.pyc CHANGED
Binary files a/models/segment_models/configs/__pycache__/ade20k_id2label.cpython-38.pyc and b/models/segment_models/configs/__pycache__/ade20k_id2label.cpython-38.pyc differ
 
models/segment_models/configs/__pycache__/coco_id2label.cpython-38.pyc CHANGED
Binary files a/models/segment_models/configs/__pycache__/coco_id2label.cpython-38.pyc and b/models/segment_models/configs/__pycache__/coco_id2label.cpython-38.pyc differ
 
output/3_result_3.jpg ADDED
requirements.txt CHANGED
@@ -1,8 +1,11 @@
1
- torch
2
- torchvision
3
- torchaudio
4
- detectron2
 
 
5
  transformers
 
6
  openai
7
  diffusers[torch]
8
  setuptools==59.5.0
@@ -10,4 +13,7 @@ git+https://github.com/facebookresearch/segment-anything.git
10
  openmim
11
  mmcv
12
  spacy
13
- en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.0.0/en_core_web_sm-3.0.0.tar.gz
 
 
 
 
1
+ --extra-index-url https://download.pytorch.org/whl
2
+ torch==1.9.0+cu111
3
+ torchvision==0.10.0+cu111
4
+ torchaudio==0.9.0
5
+ -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html
6
+ detectron2==0.6
7
  transformers
8
+ timm==0.4.12
9
  openai
10
  diffusers[torch]
11
  setuptools==59.5.0
 
13
  openmim
14
  mmcv
15
  spacy
16
+ en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.0.0/en_core_web_sm-3.0.0.tar.gz
17
+ lvis
18
+ scipy
19
+ boto3
utils/__pycache__/util.cpython-38.pyc CHANGED
Binary files a/utils/__pycache__/util.cpython-38.pyc and b/utils/__pycache__/util.cpython-38.pyc differ