complete the model package

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+models/model.ts filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,85 @@
+---
+tags:
+- monai
+- medical
+library_name: monai
+license: unknown
+---
+# Description
+A pre-trained model for volumetric (3D) detection of the lung lesion from CT image.
+
+# Model Overview
+This model is trained on LUNA16 dataset (https://luna16.grand-challenge.org/Home/), using the RetinaNet (Lin, Tsung-Yi, et al. "Focal loss for dense object detection." ICCV 2017. https://arxiv.org/abs/1708.02002).
+
+LUNA16 is a public dataset of CT lung nodule detection. Using raw CT scans, the goal is to identify locations of possible nodules, and to assign a probability for being a nodule to each location.
+
+Disclaimer: We are not the host of the data. Please make sure to read the requirements and usage policies of the data and give credit to the authors of the dataset!
+
+## Data
+The dataset we are experimenting in this example is LUNA16 (https://luna16.grand-challenge.org/Home/).
+LUNA16 is a public dataset of CT lung nodule detection. Using raw CT scans, the goal is to identify locations of possible nodules, and to assign a probability for being a nodule to each location.
+
+Disclaimer: We are not the host of the data. Please make sure to read the requirements and usage policies of the data and give credit to the authors of the dataset!
+
+We follow the official 10-fold data splitting from LUNA16 challenge and generate data split json files using the script from [nnDetection](https://github.com/MIC-DKFZ/nnDetection/blob/main/projects/Task016_Luna/scripts/prepare.py).
+The resulted json files can be downloaded from https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/LUNA16_datasplit-20220615T233840Z-001.zip.
+In these files, the values of "box" are the ground truth boxes in world coordinate.
+
+The raw CT images in LUNA16 have various of voxel sizes. The first step is to resample them to the same voxel size.
+In this model, we resampled them into 0.703125 x 0.703125 x 1.25 mm. The code can be found in Section 3.1 of https://github.com/Project-MONAI/tutorials/tree/main/detection
+
+## Training configuration
+The training was performed with at least 12GB-memory GPUs.
+
+Actual Model Input: 192 x 192 x 80
+
+## Input and output formats
+Input: list of 1 channel 3D CT patches
+
+Output: dictionary of classification and box regression loss in training mode;
+list of dictionary of predicted box, classification label, and classification score in evaluation mode.
+
+## Scores
+The script to compute FROC sensitivity value on inference results can be found in https://github.com/Project-MONAI/tutorials/tree/main/detection
+
+This model achieves the following FROC sensitivity value on the validation data (our own split from the training dataset):
+
+| Methods             | 1/8   | 1/4   | 1/2   | 1     | 2     | 4     | 8     |
+| :---:               | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| [Liu et al. (2019)](https://arxiv.org/pdf/1906.03467.pdf)   | **0.848** | 0.876 | 0.905 | 0.933 | 0.943 | 0.957 | 0.970 |
+| [nnDetection (2021)](https://arxiv.org/pdf/2106.00817.pdf)  | 0.812 | **0.885** | 0.927 | 0.950 | 0.969 | 0.979 | 0.985 |
+| MONAI detection     | 0.835 | **0.885** | **0.931** | **0.957** | **0.974** | **0.983** | **0.988** |
+
+**Table 1**. The FROC sensitivity values at the predefined false positive per scan thresholds of the LUNA16 challenge.
+
+## commands example
+Execute training:
+
+```
+python -m monai.bundle run training --meta_file configs/metadata.json --config_file configs/train.json --logging_file configs/logging.conf
+```
+
+Override the `train` config to execute evaluation with the trained model:
+
+```
+python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file "['configs/train.json','configs/evaluate.json']" --logging_file configs/logging.conf
+```
+
+Execute inference:
+
+```
+python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file configs/inference.json --logging_file configs/logging.conf
+```
+
+Note that in inference.json, the transform "AffineBoxToWorldCoordinated" in "postprocessing" has `"affine_lps_to_ras": true`.
+This depends on the input images. It is possible that your inference dataset should set "affine_lps_to_ras": false.
+Please set it as `true` only when the original images were read by itkreader with affine_lps_to_ras=True.
+
+
+# Disclaimer
+This is an example, not to be used for diagnostic purposes.
+
+# References
+[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." ICCV 2017. https://arxiv.org/abs/1708.02002)
+
+[2] Baumgartner and Jaeger et al. "nnDetection: A self-configuring method for medical object detection." MICCAI 2021. https://arxiv.org/pdf/2106.00817.pdf

configs/evaluate.json

@@ -0,0 +1,37 @@
+{
+    "test_datalist": "$monai.data.load_decathlon_datalist(@data_list_file_path, is_segmentation=True, data_list_key='validation', base_dir=@data_file_base_dir)",
+    "validate#dataset": {
+        "_target_": "Dataset",
+        "data": "$@test_datalist",
+        "transform": "@validate#preprocessing"
+    },
+    "validate#handlers": [
+        {
+            "_target_": "CheckpointLoader",
+            "load_path": "$@ckpt_dir + '/model.pt'",
+            "load_dict": {
+                "model": "@network"
+            }
+        },
+        {
+            "_target_": "StatsHandler",
+            "iteration_log": false
+        },
+        {
+            "_target_": "MetricsSaver",
+            "save_dir": "@output_dir",
+            "metrics": [
+                "val_coco"
+            ],
+            "metric_details": [
+                "val_coco"
+            ],
+            "batch_transform": "$monai.handlers.from_engine(['image_meta_dict'])",
+            "summary_ops": "*"
+        }
+    ],
+    "evaluating": [
+        "$setattr(torch.backends.cudnn, 'benchmark', True)",
+        "$@validate#evaluator.run()"
+    ]
+}

configs/inference.json

@@ -0,0 +1,209 @@
+{
+    "imports": [
+        "$import glob",
+        "$import os"
+    ],
+    "bundle_root": "./",
+    "ckpt_dir": "$@bundle_root + '/models'",
+    "output_dir": "$@bundle_root + '/eval'",
+    "data_list_file_path": "$@bundle_root + '/annotation/dataset_fold0.json'",
+    "data_file_base_dir": "/home/canz/Projects/datasets/LUNA16/93176/Images_resample",
+    "test_datalist": "$monai.data.load_decathlon_datalist(@data_list_file_path, is_segmentation=True, data_list_key='validation', base_dir=@data_file_base_dir)",
+    "device": "$torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')",
+    "amp": true,
+    "val_patch_size": [
+        512,
+        512,
+        208
+    ],
+    "anchor_generator": {
+        "_target_": "monai.apps.detection.utils.anchor_utils.AnchorGeneratorWithAnchorShape",
+        "feature_map_scales": [
+            1,
+            2,
+            4
+        ],
+        "base_anchor_shapes": [
+            [
+                6,
+                8,
+                4
+            ],
+            [
+                8,
+                6,
+                5
+            ],
+            [
+                10,
+                10,
+                6
+            ]
+        ]
+    },
+    "backbone": "$monai.networks.nets.resnet.resnet50(spatial_dims=3,n_input_channels=1,conv1_t_stride=[2,2,1],conv1_t_size=[7,7,7])",
+    "feature_extractor": "$monai.apps.detection.networks.retinanet_network.resnet_fpn_feature_extractor(@backbone,3,False,[1,2],None)",
+    "network_def": {
+        "_target_": "RetinaNet",
+        "spatial_dims": 3,
+        "num_classes": 1,
+        "num_anchors": 3,
+        "feature_extractor": "@feature_extractor",
+        "size_divisible": [
+            16,
+            16,
+            8
+        ]
+    },
+    "network": "$@network_def.to(@device)",
+    "detector": {
+        "_target_": "RetinaNetDetector",
+        "network": "@network",
+        "anchor_generator": "@anchor_generator",
+        "debug": false
+    },
+    "detector_ops": [
+        "$@detector.set_target_keys(box_key='box', label_key='label')",
+        "$@detector.set_box_selector_parameters(score_thresh=0.02,topk_candidates_per_level=1000,nms_thresh=0.22,detections_per_img=300)",
+        "$@detector.set_sliding_window_inferer(roi_size=@val_patch_size,overlap=0.25,sw_batch_size=1,mode='constant',device='cpu')"
+    ],
+    "preprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "DeleteItemsd",
+                "keys": [
+                    "box",
+                    "label"
+                ]
+            },
+            {
+                "_target_": "LoadImaged",
+                "keys": "image",
+                "meta_key_postfix": "meta_dict"
+            },
+            {
+                "_target_": "EnsureChannelFirstd",
+                "keys": "image",
+                "meta_key_postfix": "meta_dict"
+            },
+            {
+                "_target_": "Orientationd",
+                "keys": "image",
+                "axcodes": "RAS"
+            },
+            {
+                "_target_": "Spacingd",
+                "keys": "image",
+                "pixdim": [
+                    0.703125,
+                    0.703125,
+                    1.25
+                ]
+            },
+            {
+                "_target_": "ScaleIntensityRanged",
+                "keys": "image",
+                "a_min": -1024.0,
+                "a_max": 300.0,
+                "b_min": 0.0,
+                "b_max": 1.0,
+                "clip": true
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": "image"
+            }
+        ]
+    },
+    "dataset": {
+        "_target_": "Dataset",
+        "data": "$@test_datalist",
+        "transform": "@preprocessing"
+    },
+    "dataloader": {
+        "_target_": "DataLoader",
+        "dataset": "@dataset",
+        "batch_size": 1,
+        "shuffle": false,
+        "num_workers": 4,
+        "collate_fn": "$monai.data.utils.no_collation"
+    },
+    "inferer": {
+        "_target_": "SlidingWindowInferer",
+        "roi_size": [
+            240,
+            240,
+            160
+        ],
+        "sw_batch_size": 1,
+        "overlap": 0.5
+    },
+    "postprocessing": {
+        "_target_": "Compose",
+        "transforms": [
+            {
+                "_target_": "ClipBoxToImaged",
+                "box_keys": "box",
+                "label_keys": "label",
+                "box_ref_image_keys": "image",
+                "remove_empty": true
+            },
+            {
+                "_target_": "AffineBoxToWorldCoordinated",
+                "box_keys": "box",
+                "box_ref_image_keys": "image",
+                "image_meta_key_postfix": "meta_dict",
+                "affine_lps_to_ras": true
+            },
+            {
+                "_target_": "ConvertBoxModed",
+                "box_keys": "box",
+                "src_mode": "xyzxyz",
+                "dst_mode": "cccwhd"
+            },
+            {
+                "_target_": "DeleteItemsd",
+                "keys": [
+                    "image"
+                ]
+            }
+        ]
+    },
+    "handlers": [
+        {
+            "_target_": "CheckpointLoader",
+            "load_path": "$@bundle_root + '/models/model.pt'",
+            "load_dict": {
+                "model": "@network"
+            }
+        },
+        {
+            "_target_": "StatsHandler",
+            "iteration_log": false
+        },
+        {
+            "_target_": "scripts.detection_saver.DetectionSaver",
+            "output_dir": "@output_dir",
+            "filename": "result_luna16_fold0.json",
+            "batch_transform": "$lambda x: [xx['image_meta_dict'] for xx in x]",
+            "output_transform": "$lambda x: [@postprocessing({**xx['pred'],'image':xx['image']}) for xx in x]",
+            "pred_box_key": "box",
+            "pred_label_key": "label",
+            "pred_score_key": "label_scores"
+        }
+    ],
+    "evaluator": {
+        "_target_": "scripts.evaluator.DetectionEvaluator",
+        "_requires_": "@detector_ops",
+        "device": "@device",
+        "val_data_loader": "@dataloader",
+        "detector": "@detector",
+        "val_handlers": "@handlers",
+        "amp": "@amp"
+    },
+    "evaluating": [
+        "$setattr(torch.backends.cudnn, 'benchmark', True)",
+        "$@evaluator.run()"
+    ]
+}

configs/logging.conf

@@ -0,0 +1,21 @@
+[loggers]
+keys=root
+
+[handlers]
+keys=consoleHandler
+
+[formatters]
+keys=fullFormatter
+
+[logger_root]
+level=INFO
+handlers=consoleHandler
+
+[handler_consoleHandler]
+class=StreamHandler
+level=INFO
+formatter=fullFormatter
+args=(sys.stdout,)
+
+[formatter_fullFormatter]
+format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

configs/metadata.json

@@ -0,0 +1,72 @@
+{
+    "schema": "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json",
+    "version": "0.1.0",
+    "changelog": {
+        "0.1.0": "complete the model package"
+    },
+    "monai_version": "0.9.1",
+    "pytorch_version": "1.12.0",
+    "numpy_version": "1.22.4",
+    "optional_packages_version": {
+        "nibabel": "4.0.1",
+        "pytorch-ignite": "0.4.9"
+    },
+    "task": "CT lung nodule detection",
+    "description": "A pre-trained model for volumetric (3D) detection of the lung lesion from CT image on LUNA16 dataset",
+    "authors": "MONAI team",
+    "copyright": "Copyright (c) MONAI Consortium",
+    "data_source": "https://luna16.grand-challenge.org/Home/",
+    "data_type": "nibabel",
+    "image_classes": "1 channel data, CT at 0.703125 x 0.703125 x 1.25 mm",
+    "label_classes": "dict data, containing Nx6 box and Nx1 classification labels.",
+    "pred_classes": "dict data, containing Nx6 box, Nx1 classification labels, Nx1 classification scores.",
+    "eval_metrics": {
+        "val_coco": 0,
+        "froc": 0
+    },
+    "intended_use": "This is an example, not to be used for diagnostic purposes",
+    "references": [
+        "Lin, Tsung-Yi, et al. 'Focal loss for dense object detection. ICCV 2017"
+    ],
+    "network_data_format": {
+        "inputs": {
+            "image": {
+                "type": "image",
+                "format": "magnitude",
+                "modality": "CT",
+                "num_channels": 1,
+                "spatial_shape": [
+                    "16*n",
+                    "16*n",
+                    "8*n"
+                ],
+                "dtype": "float16",
+                "value_range": [
+                    0,
+                    1
+                ],
+                "is_patch_data": true,
+                "channel_def": {
+                    "0": "image"
+                }
+            }
+        },
+        "outputs": {
+            "pred": {
+                "type": "object",
+                "format": "dict",
+                "dtype": "float16",
+                "num_channels": 1,
+                "spatial_shape": [
+                    "n",
+                    "n",
+                    "n"
+                ],
+                "value_range": [
+                    -10000,
+                    10000
+                ]
+            }
+        }
+    }
+}

configs/train.json

@@ -0,0 +1,450 @@
+{
+    "imports": [
+        "$import glob",
+        "$import os"
+    ],
+    "bundle_root": "./",
+    "ckpt_dir": "$@bundle_root + '/models'",
+    "output_dir": "$@bundle_root + '/eval'",
+    "data_list_file_path": "$@bundle_root + '/annotation/dataset_fold0.json'",
+    "data_file_base_dir": "/home/canz/Projects/datasets/LUNA16/93176/Images_resample",
+    "train_datalist": "$monai.data.load_decathlon_datalist(@data_list_file_path, is_segmentation=True, data_list_key='training', base_dir=@data_file_base_dir)",
+    "device": "$torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')",
+    "epochs": 300,
+    "num_interval_per_valid": 10,
+    "learning_rate": 0.01,
+    "amp": true,
+    "batch_size": 3,
+    "patch_size": [
+        192,
+        192,
+        80
+    ],
+    "val_patch_size": [
+        512,
+        512,
+        208
+    ],
+    "anchor_generator": {
+        "_target_": "monai.apps.detection.utils.anchor_utils.AnchorGeneratorWithAnchorShape",
+        "feature_map_scales": [
+            1,
+            2,
+            4
+        ],
+        "base_anchor_shapes": [
+            [
+                6,
+                8,
+                4
+            ],
+            [
+                8,
+                6,
+                5
+            ],
+            [
+                10,
+                10,
+                6
+            ]
+        ]
+    },
+    "backbone": "$monai.networks.nets.resnet.resnet50(spatial_dims=3,n_input_channels=1,conv1_t_stride=[2,2,1],conv1_t_size=[7,7,7])",
+    "feature_extractor": "$monai.apps.detection.networks.retinanet_network.resnet_fpn_feature_extractor(@backbone,3,False,[1,2],None)",
+    "network_def": {
+        "_target_": "RetinaNet",
+        "spatial_dims": 3,
+        "num_classes": 1,
+        "num_anchors": 3,
+        "feature_extractor": "@feature_extractor",
+        "size_divisible": [
+            16,
+            16,
+            8
+        ]
+    },
+    "network": "$@network_def.to(@device)",
+    "detector": {
+        "_target_": "RetinaNetDetector",
+        "network": "@network",
+        "anchor_generator": "@anchor_generator",
+        "debug": false
+    },
+    "detector_ops": [
+        "$@detector.set_atss_matcher(num_candidates=4, center_in_gt=False)",
+        "$@detector.set_hard_negative_sampler(batch_size_per_image=64,positive_fraction=0.3,pool_size=20,min_neg=16)",
+        "$@detector.set_target_keys(box_key='box', label_key='label')",
+        "$@detector.set_box_selector_parameters(score_thresh=0.02,topk_candidates_per_level=1000,nms_thresh=0.22,detections_per_img=300)",
+        "$@detector.set_sliding_window_inferer(roi_size=@val_patch_size,overlap=0.25,sw_batch_size=1,mode='constant',device='cpu')"
+    ],
+    "optimizer": {
+        "_target_": "torch.optim.SGD",
+        "params": "$@detector.network.parameters()",
+        "lr": "@learning_rate",
+        "momentum": 0.9,
+        "weight_decay": 3e-05,
+        "nesterov": true
+    },
+    "after_scheduler": {
+        "_target_": "torch.optim.lr_scheduler.StepLR",
+        "optimizer": "@optimizer",
+        "step_size": 150,
+        "gamma": 0.1
+    },
+    "lr_scheduler": {
+        "_target_": "scripts.warmup_scheduler.GradualWarmupScheduler",
+        "optimizer": "@optimizer",
+        "multiplier": 1,
+        "total_epoch": 10,
+        "after_scheduler": "@after_scheduler"
+    },
+    "train": {
+        "preprocessing_transforms": [
+            {
+                "_target_": "LoadImaged",
+                "keys": "image",
+                "meta_key_postfix": "meta_dict"
+            },
+            {
+                "_target_": "EnsureChannelFirstd",
+                "keys": "image",
+                "meta_key_postfix": "meta_dict"
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": [
+                    "image",
+                    "box"
+                ]
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": "label",
+                "dtype": "$torch.long"
+            },
+            {
+                "_target_": "Orientationd",
+                "keys": "image",
+                "axcodes": "RAS"
+            },
+            {
+                "_target_": "ScaleIntensityRanged",
+                "keys": "image",
+                "a_min": -1024.0,
+                "a_max": 300.0,
+                "b_min": 0.0,
+                "b_max": 1.0,
+                "clip": true
+            },
+            {
+                "_target_": "ConvertBoxToStandardModed",
+                "box_keys": "box",
+                "mode": "cccwhd"
+            },
+            {
+                "_target_": "AffineBoxToImageCoordinated",
+                "box_keys": "box",
+                "box_ref_image_keys": "image",
+                "image_meta_key_postfix": "meta_dict",
+                "affine_lps_to_ras": true
+            }
+        ],
+        "random_transforms": [
+            {
+                "_target_": "RandCropBoxByPosNegLabeld",
+                "image_keys": "image",
+                "box_keys": "box",
+                "label_keys": "label",
+                "spatial_size": "@patch_size",
+                "whole_box": true,
+                "num_samples": "@batch_size",
+                "pos": 1,
+                "neg": 1
+            },
+            {
+                "_target_": "RandZoomBoxd",
+                "image_keys": "image",
+                "box_keys": "box",
+                "label_keys": "label",
+                "box_ref_image_keys": "image",
+                "prob": 0.2,
+                "min_zoom": 0.7,
+                "max_zoom": 1.4,
+                "padding_mode": "constant",
+                "keep_size": true
+            },
+            {
+                "_target_": "ClipBoxToImaged",
+                "box_keys": "box",
+                "label_keys": "label",
+                "box_ref_image_keys": "image",
+                "remove_empty": true
+            },
+            {
+                "_target_": "RandFlipBoxd",
+                "image_keys": "image",
+                "box_keys": "box",
+                "box_ref_image_keys": "image",
+                "prob": 0.5,
+                "spatial_axis": 0
+            },
+            {
+                "_target_": "RandFlipBoxd",
+                "image_keys": "image",
+                "box_keys": "box",
+                "box_ref_image_keys": "image",
+                "prob": 0.5,
+                "spatial_axis": 1
+            },
+            {
+                "_target_": "RandFlipBoxd",
+                "image_keys": "image",
+                "box_keys": "box",
+                "box_ref_image_keys": "image",
+                "prob": 0.5,
+                "spatial_axis": 2
+            },
+            {
+                "_target_": "RandRotateBox90d",
+                "image_keys": "image",
+                "box_keys": "box",
+                "box_ref_image_keys": "image",
+                "prob": 0.75,
+                "max_k": 3,
+                "spatial_axes": [
+                    0,
+                    1
+                ]
+            },
+            {
+                "_target_": "BoxToMaskd",
+                "box_keys": "box",
+                "label_keys": "label",
+                "box_mask_keys": "box_mask",
+                "box_ref_image_keys": "image",
+                "min_fg_label": 0,
+                "ellipse_mask": true
+            },
+            {
+                "_target_": "RandRotated",
+                "keys": [
+                    "image",
+                    "box_mask"
+                ],
+                "mode": [
+                    "nearest",
+                    "nearest"
+                ],
+                "prob": 0.2,
+                "range_x": 0.5236,
+                "range_y": 0.5236,
+                "range_z": 0.5236,
+                "keep_size": true,
+                "padding_mode": "zeros"
+            },
+            {
+                "_target_": "MaskToBoxd",
+                "box_keys": [
+                    "box"
+                ],
+                "label_keys": [
+                    "label"
+                ],
+                "box_mask_keys": [
+                    "box_mask"
+                ],
+                "min_fg_label": 0
+            },
+            {
+                "_target_": "DeleteItemsd",
+                "keys": "box_mask"
+            },
+            {
+                "_target_": "RandGaussianNoised",
+                "keys": "image",
+                "prob": 0.1,
+                "mean": 0.0,
+                "std": 0.1
+            },
+            {
+                "_target_": "RandGaussianSmoothd",
+                "keys": "image",
+                "prob": 0.1,
+                "sigma_x": [
+                    0.5,
+                    1.0
+                ],
+                "sigma_y": [
+                    0.5,
+                    1.0
+                ],
+                "sigma_z": [
+                    0.5,
+                    1.0
+                ]
+            },
+            {
+                "_target_": "RandScaleIntensityd",
+                "keys": "image",
+                "factors": 0.25,
+                "prob": 0.15
+            },
+            {
+                "_target_": "RandShiftIntensityd",
+                "keys": "image",
+                "offsets": 0.1,
+                "prob": 0.15
+            },
+            {
+                "_target_": "RandAdjustContrastd",
+                "keys": "image",
+                "prob": 0.3,
+                "gamma": [
+                    0.7,
+                    1.5
+                ]
+            }
+        ],
+        "final_transforms": [
+            {
+                "_target_": "EnsureTyped",
+                "keys": [
+                    "image",
+                    "box"
+                ]
+            },
+            {
+                "_target_": "EnsureTyped",
+                "keys": "label",
+                "dtype": "$torch.long"
+            },
+            {
+                "_target_": "ToTensord",
+                "keys": [
+                    "image",
+                    "box",
+                    "label"
+                ]
+            }
+        ],
+        "preprocessing": {
+            "_target_": "Compose",
+            "transforms": "$@train#preprocessing_transforms + @train#random_transforms + @train#final_transforms"
+        },
+        "dataset": {
+            "_target_": "Dataset",
+            "data": "$@train_datalist[: int(0.95 * len(@train_datalist))]",
+            "transform": "@train#preprocessing"
+        },
+        "dataloader": {
+            "_target_": "DataLoader",
+            "dataset": "@train#dataset",
+            "batch_size": 1,
+            "shuffle": true,
+            "num_workers": 4,
+            "collate_fn": "$monai.data.utils.no_collation"
+        },
+        "handlers": [
+            {
+                "_target_": "LrScheduleHandler",
+                "lr_scheduler": "@lr_scheduler",
+                "print_lr": true
+            },
+            {
+                "_target_": "ValidationHandler",
+                "validator": "@validate#evaluator",
+                "epoch_level": true,
+                "interval": "@num_interval_per_valid"
+            },
+            {
+                "_target_": "StatsHandler",
+                "tag_name": "train_loss",
+                "output_transform": "$lambda x: monai.handlers.from_engine(['loss'], first=True)(x)[0]"
+            },
+            {
+                "_target_": "TensorBoardStatsHandler",
+                "log_dir": "@output_dir",
+                "tag_name": "train_loss",
+                "output_transform": "$lambda x: monai.handlers.from_engine(['loss'], first=True)(x)[0]"
+            }
+        ],
+        "trainer": {
+            "_target_": "scripts.trainer.DetectionTrainer",
+            "_requires_": "@detector_ops",
+            "max_epochs": "@epochs",
+            "device": "@device",
+            "train_data_loader": "@train#dataloader",
+            "detector": "@detector",
+            "optimizer": "@optimizer",
+            "train_handlers": "@train#handlers",
+            "amp": "@amp"
+        }
+    },
+    "validate": {
+        "preprocessing": {
+            "_target_": "Compose",
+            "transforms": "$@train#preprocessing_transforms + @train#final_transforms"
+        },
+        "dataset": {
+            "_target_": "Dataset",
+            "data": "$@train_datalist[int(0.95 * len(@train_datalist)): ]",
+            "transform": "@validate#preprocessing"
+        },
+        "dataloader": {
+            "_target_": "DataLoader",
+            "dataset": "@validate#dataset",
+            "batch_size": 1,
+            "shuffle": false,
+            "num_workers": 2,
+            "collate_fn": "$monai.data.utils.no_collation"
+        },
+        "handlers": [
+            {
+                "_target_": "StatsHandler",
+                "iteration_log": false
+            },
+            {
+                "_target_": "TensorBoardStatsHandler",
+                "log_dir": "@output_dir",
+                "iteration_log": false
+            },
+            {
+                "_target_": "CheckpointSaver",
+                "save_dir": "@ckpt_dir",
+                "save_dict": {
+                    "model": "@network"
+                },
+                "save_key_metric": true,
+                "key_metric_filename": "model.pt"
+            }
+        ],
+        "key_metric": {
+            "val_coco": {
+                "_target_": "scripts.cocometric_ignite.IgniteCocoMetric",
+                "coco_metric_monai": "$monai.apps.detection.metrics.coco.COCOMetric(classes=['nodule'], iou_list=[0.1], max_detection=[100])",
+                "output_transform": "$monai.handlers.from_engine(['pred', 'label'])",
+                "box_key": "box",
+                "label_key": "label",
+                "pred_score_key": "label_scores",
+                "reduce_scalar": true
+            }
+        },
+        "evaluator": {
+            "_target_": "scripts.evaluator.DetectionEvaluator",
+            "_requires_": "@detector_ops",
+            "device": "@device",
+            "val_data_loader": "@validate#dataloader",
+            "detector": "@detector",
+            "key_val_metric": "@validate#key_metric",
+            "val_handlers": "@validate#handlers",
+            "amp": "@amp"
+        }
+    },
+    "training": [
+        "os.environ['CUDA_LAUNCH_BLOCKING']=1",
+        "$monai.utils.set_determinism(seed=123)",
+        "$setattr(torch.backends.cudnn, 'benchmark', True)",
+        "$@train#trainer.run()"
+    ]
+}

docs/README.md

@@ -0,0 +1,78 @@
+# Description
+A pre-trained model for volumetric (3D) detection of the lung lesion from CT image.
+
+# Model Overview
+This model is trained on LUNA16 dataset (https://luna16.grand-challenge.org/Home/), using the RetinaNet (Lin, Tsung-Yi, et al. "Focal loss for dense object detection." ICCV 2017. https://arxiv.org/abs/1708.02002).
+
+LUNA16 is a public dataset of CT lung nodule detection. Using raw CT scans, the goal is to identify locations of possible nodules, and to assign a probability for being a nodule to each location.
+
+Disclaimer: We are not the host of the data. Please make sure to read the requirements and usage policies of the data and give credit to the authors of the dataset!
+
+## Data
+The dataset we are experimenting in this example is LUNA16 (https://luna16.grand-challenge.org/Home/).
+LUNA16 is a public dataset of CT lung nodule detection. Using raw CT scans, the goal is to identify locations of possible nodules, and to assign a probability for being a nodule to each location.
+
+Disclaimer: We are not the host of the data. Please make sure to read the requirements and usage policies of the data and give credit to the authors of the dataset!
+
+We follow the official 10-fold data splitting from LUNA16 challenge and generate data split json files using the script from [nnDetection](https://github.com/MIC-DKFZ/nnDetection/blob/main/projects/Task016_Luna/scripts/prepare.py).
+The resulted json files can be downloaded from https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/LUNA16_datasplit-20220615T233840Z-001.zip.
+In these files, the values of "box" are the ground truth boxes in world coordinate.
+
+The raw CT images in LUNA16 have various of voxel sizes. The first step is to resample them to the same voxel size.
+In this model, we resampled them into 0.703125 x 0.703125 x 1.25 mm. The code can be found in Section 3.1 of https://github.com/Project-MONAI/tutorials/tree/main/detection
+
+## Training configuration
+The training was performed with at least 12GB-memory GPUs.
+
+Actual Model Input: 192 x 192 x 80
+
+## Input and output formats
+Input: list of 1 channel 3D CT patches
+
+Output: dictionary of classification and box regression loss in training mode;
+list of dictionary of predicted box, classification label, and classification score in evaluation mode.
+
+## Scores
+The script to compute FROC sensitivity value on inference results can be found in https://github.com/Project-MONAI/tutorials/tree/main/detection
+
+This model achieves the following FROC sensitivity value on the validation data (our own split from the training dataset):
+
+| Methods             | 1/8   | 1/4   | 1/2   | 1     | 2     | 4     | 8     |
+| :---:               | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| [Liu et al. (2019)](https://arxiv.org/pdf/1906.03467.pdf)   | **0.848** | 0.876 | 0.905 | 0.933 | 0.943 | 0.957 | 0.970 |
+| [nnDetection (2021)](https://arxiv.org/pdf/2106.00817.pdf)  | 0.812 | **0.885** | 0.927 | 0.950 | 0.969 | 0.979 | 0.985 |
+| MONAI detection     | 0.835 | **0.885** | **0.931** | **0.957** | **0.974** | **0.983** | **0.988** |
+
+**Table 1**. The FROC sensitivity values at the predefined false positive per scan thresholds of the LUNA16 challenge.
+
+## commands example
+Execute training:
+
+```
+python -m monai.bundle run training --meta_file configs/metadata.json --config_file configs/train.json --logging_file configs/logging.conf
+```
+
+Override the `train` config to execute evaluation with the trained model:
+
+```
+python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file "['configs/train.json','configs/evaluate.json']" --logging_file configs/logging.conf
+```
+
+Execute inference:
+
+```
+python -m monai.bundle run evaluating --meta_file configs/metadata.json --config_file configs/inference.json --logging_file configs/logging.conf
+```
+
+Note that in inference.json, the transform "AffineBoxToWorldCoordinated" in "postprocessing" has `"affine_lps_to_ras": true`.
+This depends on the input images. It is possible that your inference dataset should set "affine_lps_to_ras": false.
+Please set it as `true` only when the original images were read by itkreader with affine_lps_to_ras=True.
+
+
+# Disclaimer
+This is an example, not to be used for diagnostic purposes.
+
+# References
+[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." ICCV 2017. https://arxiv.org/abs/1708.02002)
+
+[2] Baumgartner and Jaeger et al. "nnDetection: A self-configuring method for medical object detection." MICCAI 2021. https://arxiv.org/pdf/2106.00817.pdf

docs/license.txt

@@ -0,0 +1,6 @@
+Third Party Licenses
+-----------------------------------------------------------------------
+
+/*********************************************************************/
+i. LUng Nodule Analysis 2016
+   https://luna16.grand-challenge.org/Home/

models/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0caff53e6cc00e7f40e0ed10944f3462b45d42b152bc811ddae839ffcb13c0df
+size 83719685

models/model.ts

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97d30237b8f328ff99fc3f7b3d5c560b5081b5c074253975eb28ebadd8e69dcc
+size 83796462

scripts/__init__.py

@@ -0,0 +1,14 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# from .evaluator import EnsembleEvaluator, Evaluator, SupervisedEvaluator
+# from .multi_gpu_supervised_trainer import create_multigpu_supervised_evaluator, create_multigpu_supervised_trainer
+from .trainer import DetectionTrainer

scripts/cocometric_ignite.py

@@ -0,0 +1,112 @@
+from typing import Callable, Dict, Sequence, Union
+
+import torch
+from ignite.metrics.metric import Metric, reinit__is_reduced, sync_all_reduce
+from monai.apps.detection.metrics.coco import COCOMetric
+from monai.apps.detection.metrics.matching import matching_batch
+from monai.data import box_utils
+
+from .utils import detach_to_numpy
+
+
+class IgniteCocoMetric(Metric):
+    def __init__(
+        self,
+        coco_metric_monai: Union[None, COCOMetric] = None,
+        box_key="box",
+        label_key="label",
+        pred_score_key="label_scores",
+        output_transform: Callable = lambda x: x,
+        device: Union[str, torch.device, None] = None,
+        reduce_scalar: bool = True,
+    ):
+        r"""
+        Computes coco detection metric in Ignite.
+
+        Args:
+            coco_metric_monai: the coco metric in monai.
+                If not given, will asume COCOMetric(classes=[0], iou_list=[0.1], max_detection=[100])
+            box_key: box key in the ground truth target dict and prediction dict.
+            label_key: classification label key in the ground truth target dict and prediction dict.
+            pred_score_key: classification score key in the prediction dict.
+            output_transform: A callable that is used to transform the Engine’s
+                process_function’s output into the form expected by the metric.
+            device: specifies which device updates are accumulated on.
+                Setting the metric’s device to be the same as your update arguments ensures
+                the update method is non-blocking. By default, CPU.
+            reduce_scalar: if True, will return the average value of coc metric values;
+                if False, will return an dictionary of coc metric.
+
+        Examples:
+            To use with ``Engine`` and ``process_function``,
+             simply attach the metric instance to the engine.
+            The output of the engine's ``process_function`` needs to be in format of
+            ``(y_pred, y)`` or ``{'y_pred': y_pred, 'y': y, ...}``.
+            For more information on how metric works with :class:`~ignite.engine.engine.Engine`,
+             visit :ref:`attach-engine`.
+            .. include:: defaults.rst
+                :start-after: :orphan:
+            .. testcode::
+                coco = IgniteCocoMetric()
+                coco.attach(default_evaluator, 'coco')
+                preds = [
+                    {
+                        'box': torch.Tensor([[1,1,1,2,2,2]]),
+                        'label':torch.Tensor([0]),
+                        'label_scores':torch.Tensor([0.8])
+                    }
+                ]
+                target = [{'box': torch.Tensor([[1,1,1,2,2,2]]), 'label':torch.Tensor([0])}]
+                state = default_evaluator.run([[preds, target]])
+                print(state.metrics['coco'])
+            .. testoutput::
+                1.0...
+        .. versionadded:: 0.4.3
+        """
+        self.box_key = box_key
+        self.label_key = label_key
+        self.pred_score_key = pred_score_key
+        if coco_metric_monai is None:
+            self.coco_metric = COCOMetric(classes=[0], iou_list=[0.1], max_detection=[100])
+        else:
+            self.coco_metric = coco_metric_monai
+        self.reduce_scalar = reduce_scalar
+
+        if device is None:
+            device = torch.device("cpu")
+        super(IgniteCocoMetric, self).__init__(output_transform=output_transform, device=device)
+
+    @reinit__is_reduced
+    def reset(self) -> None:
+        self.val_targets_all = []
+        self.val_outputs_all = []
+
+    @reinit__is_reduced
+    def update(self, output: Sequence[Dict]) -> None:
+        y_pred, y = output[0], output[1]
+        self.val_outputs_all += y_pred
+        self.val_targets_all += y
+
+    @sync_all_reduce("val_targets_all", "val_outputs_all")
+    def compute(self) -> float:
+
+        self.val_outputs_all = detach_to_numpy(self.val_outputs_all)
+        self.val_targets_all = detach_to_numpy(self.val_targets_all)
+
+        results_metric = matching_batch(
+            iou_fn=box_utils.box_iou,
+            iou_thresholds=self.coco_metric.iou_thresholds,
+            pred_boxes=[val_data_i[self.box_key] for val_data_i in self.val_outputs_all],
+            pred_classes=[val_data_i[self.label_key] for val_data_i in self.val_outputs_all],
+            pred_scores=[val_data_i[self.pred_score_key] for val_data_i in self.val_outputs_all],
+            gt_boxes=[val_data_i[self.box_key] for val_data_i in self.val_targets_all],
+            gt_classes=[val_data_i[self.label_key] for val_data_i in self.val_targets_all],
+        )
+        val_epoch_metric_dict = self.coco_metric(results_metric)[0]
+
+        if self.reduce_scalar:
+            val_epoch_metric = val_epoch_metric_dict.values()
+            val_epoch_metric = sum(val_epoch_metric) / len(val_epoch_metric)
+            return val_epoch_metric
+        else:
+            return val_epoch_metric_dict

scripts/detection_saver.py

@@ -0,0 +1,127 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import os
+import warnings
+from typing import TYPE_CHECKING, Callable, Optional
+
+from monai.config import IgniteInfo
+from monai.handlers.classification_saver import ClassificationSaver
+from monai.utils import evenly_divisible_all_gather, min_version, optional_import, string_list_all_gather
+
+from .utils import detach_to_numpy
+
+idist, _ = optional_import("ignite", IgniteInfo.OPT_IMPORT_VERSION, min_version, "distributed")
+Events, _ = optional_import("ignite.engine", IgniteInfo.OPT_IMPORT_VERSION, min_version, "Events")
+if TYPE_CHECKING:
+    from ignite.engine import Engine
+else:
+    Engine, _ = optional_import("ignite.engine", IgniteInfo.OPT_IMPORT_VERSION, min_version, "Engine")
+
+
+class DetectionSaver(ClassificationSaver):
+    """
+    Event handler triggered on completing every iteration to save the classification predictions as json file.
+    If running in distributed data parallel, only saves json file in the specified rank.
+
+    """
+
+    def __init__(
+        self,
+        output_dir: str = "./",
+        filename: str = "predictions.json",
+        overwrite: bool = True,
+        batch_transform: Callable = lambda x: x,
+        output_transform: Callable = lambda x: x,
+        name: Optional[str] = None,
+        save_rank: int = 0,
+        pred_box_key: str = "box",
+        pred_label_key: str = "label",
+        pred_score_key: str = "label_scores",
+    ) -> None:
+        """
+        Args:
+            output_dir: if `saver=None`, output json file directory.
+            filename: if `saver=None`, name of the saved json file name.
+            overwrite: if `saver=None`, whether to overwriting existing file content, if True,
+                will clear the file before saving. otherwise, will append new content to the file.
+            batch_transform: a callable that is used to extract the `meta_data` dictionary of
+                the input images from `ignite.engine.state.batch`. the purpose is to get the input
+                filenames from the `meta_data` and store with classification results together.
+                `engine.state` and `batch_transform` inherit from the ignite concept:
+                https://pytorch.org/ignite/concepts.html#state, explanation and usage example are in the tutorial:
+                https://github.com/Project-MONAI/tutorials/blob/master/modules/batch_output_transform.ipynb.
+            output_transform: a callable that is used to extract the model prediction data from
+                `ignite.engine.state.output`. the first dimension of its output will be treated as
+                the batch dimension. each item in the batch will be saved individually.
+                `engine.state` and `output_transform` inherit from the ignite concept:
+                https://pytorch.org/ignite/concepts.html#state, explanation and usage example are in the tutorial:
+                https://github.com/Project-MONAI/tutorials/blob/master/modules/batch_output_transform.ipynb.
+            name: identifier of logging.logger to use, defaulting to `engine.logger`.
+            save_rank: only the handler on specified rank will save to json file in multi-gpus validation,
+                default to 0.
+            pred_box_key: box key in the prediction dict.
+            pred_label_key: classification label key in the prediction dict.
+            pred_score_key: classification score key in the prediction dict.
+
+        """
+        super().__init__(
+            output_dir=output_dir,
+            filename=filename,
+            overwrite=overwrite,
+            batch_transform=batch_transform,
+            output_transform=output_transform,
+            name=name,
+            save_rank=save_rank,
+            saver=None,
+        )
+        self.pred_box_key = pred_box_key
+        self.pred_label_key = pred_label_key
+        self.pred_score_key = pred_score_key
+
+    def _finalize(self, _engine: Engine) -> None:
+        """
+        All gather classification results from ranks and save to json file.
+
+        Args:
+            _engine: Ignite Engine, unused argument.
+        """
+        ws = idist.get_world_size()
+        if self.save_rank >= ws:
+            raise ValueError("target save rank is greater than the distributed group size.")
+
+        # self._outputs is supposed to be a list of dict
+        # self._outputs[i] should be have at least three keys: pred_box_key, pred_label_key, pred_score_key
+        # self._filenames is supposed to be a list of str
+        outputs = self._outputs
+        filenames = self._filenames
+        if ws > 1:
+            outputs = evenly_divisible_all_gather(outputs, concat=False)
+            filenames = string_list_all_gather(filenames)
+
+        if len(filenames) != len(outputs):
+            warnings.warn(f"filenames length: {len(filenames)} doesn't match outputs length: {len(outputs)}.")
+
+        # save to json file only in the expected rank
+        if idist.get_rank() == self.save_rank:
+            results = [
+                {
+                    self.pred_box_key: detach_to_numpy(o[self.pred_box_key]).tolist(),
+                    self.pred_label_key: detach_to_numpy(o[self.pred_label_key]).tolist(),
+                    self.pred_score_key: detach_to_numpy(o[self.pred_score_key]).tolist(),
+                    "image": f,
+                }
+                for o, f in zip(outputs, filenames)
+            ]
+
+            with open(os.path.join(self.output_dir, self.filename), "w") as outfile:
+                json.dump(results, outfile, indent=4)

scripts/evaluator.py

@@ -0,0 +1,228 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, Callable, Dict, Iterable, List, Optional, Sequence, Tuple, Union
+
+import numpy as np
+import torch
+from monai.config import IgniteInfo
+from monai.engines.evaluator import Evaluator
+from monai.engines.utils import IterationEvents, default_metric_cmp_fn
+from monai.inferers import Inferer
+from monai.networks.utils import eval_mode, train_mode
+from monai.transforms import Transform
+from monai.utils import ForwardMode, min_version, optional_import
+from monai.utils.enums import CommonKeys as Keys
+from monai.utils.module import look_up_option
+from torch.utils.data import DataLoader
+
+if TYPE_CHECKING:
+    from ignite.engine import Engine, EventEnum
+    from ignite.metrics import Metric
+else:
+    Engine, _ = optional_import("ignite.engine", IgniteInfo.OPT_IMPORT_VERSION, min_version, "Engine")
+    Metric, _ = optional_import("ignite.metrics", IgniteInfo.OPT_IMPORT_VERSION, min_version, "Metric")
+    EventEnum, _ = optional_import("ignite.engine", IgniteInfo.OPT_IMPORT_VERSION, min_version, "EventEnum")
+
+__all__ = ["DetectionEvaluator"]
+
+
+def detection_prepare_val_batch(
+    batchdata: List[Dict[str, torch.Tensor]],
+    device: Optional[Union[str, torch.device]] = None,
+    non_blocking: bool = False,
+    **kwargs,
+) -> Union[Tuple[torch.Tensor, Optional[torch.Tensor]], torch.Tensor]:
+    """
+    Default function to prepare the data for current iteration.
+    Args `batchdata`, `device`, `non_blocking` refer to the ignite API:
+    https://pytorch.org/ignite/v0.4.8/generated/ignite.engine.create_supervised_trainer.html.
+    `kwargs` supports other args for `Tensor.to()` API.
+    Returns:
+        image, label(optional).
+    """
+    inputs = [
+        batch_data_i["image"].to(device=device, non_blocking=non_blocking, **kwargs) for batch_data_i in batchdata
+    ]
+
+    if isinstance(batchdata[0].get(Keys.LABEL), torch.Tensor):
+        targets = [
+            dict(
+                label=batch_data_i["label"].to(device=device, non_blocking=non_blocking, **kwargs),
+                box=batch_data_i["box"].to(device=device, non_blocking=non_blocking, **kwargs),
+            )
+            for batch_data_i in batchdata
+        ]
+        return (inputs, targets)
+    return inputs, None
+
+
+class DetectionEvaluator(Evaluator):
+    """
+    Supervised detection evaluation method with image and label, inherits from ``Evaluator`` and ``Workflow``.
+    Args:
+        device: an object representing the device on which to run.
+        val_data_loader: Ignite engine use data_loader to run, must be Iterable or torch.DataLoader.
+        detector: detector to train in the trainer, should be regular PyTorch `torch.nn.Module`.
+        epoch_length: number of iterations for one epoch, default to `len(val_data_loader)`.
+        non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
+            with respect to the host. For other cases, this argument has no effect.
+        prepare_batch: function to parse expected data (usually `image`,`box`, `label` and other detector args)
+            from `engine.state.batch` for every iteration, for more details please refer to:
+            https://pytorch.org/ignite/generated/ignite.engine.create_supervised_trainer.html.
+        iteration_update: the callable function for every iteration, expect to accept `engine`
+            and `engine.state.batch` as inputs, return data will be stored in `engine.state.output`.
+            if not provided, use `self._iteration()` instead. for more details please refer to:
+            https://pytorch.org/ignite/generated/ignite.engine.engine.Engine.html.
+        inferer: inference method that execute model forward on input data, like: SlidingWindow, etc.
+        postprocessing: execute additional transformation for the model output data.
+            Typically, several Tensor based transforms composed by `Compose`.
+        key_val_metric: compute metric when every iteration completed, and save average value to
+            engine.state.metrics when epoch completed. key_val_metric is the main metric to compare and save the
+            checkpoint into files.
+        additional_metrics: more Ignite metrics that also attach to Ignite Engine.
+        metric_cmp_fn: function to compare current key metric with previous best key metric value,
+            it must accept 2 args (current_metric, previous_best) and return a bool result: if `True`, will update
+            `best_metric` and `best_metric_epoch` with current metric and epoch, default to `greater than`.
+        val_handlers: every handler is a set of Ignite Event-Handlers, must have `attach` function, like:
+            CheckpointHandler, StatsHandler, etc.
+        amp: whether to enable auto-mixed-precision evaluation, default is False.
+        mode: model forward mode during evaluation, should be 'eval' or 'train',
+            which maps to `model.eval()` or `model.train()`, default to 'eval'.
+        event_names: additional custom ignite events that will register to the engine.
+            new events can be a list of str or `ignite.engine.events.EventEnum`.
+        event_to_attr: a dictionary to map an event to a state attribute, then add to `engine.state`.
+            for more details, check: https://pytorch.org/ignite/generated/ignite.engine.engine.Engine.html
+            #ignite.engine.engine.Engine.register_events.
+        decollate: whether to decollate the batch-first data to a list of data after model computation,
+            recommend `decollate=True` when `postprocessing` uses components from `monai.transforms`.
+            default to `True`.
+        to_kwargs: dict of other args for `prepare_batch` API when converting the input data, except for
+            `device`, `non_blocking`.
+        amp_kwargs: dict of the args for `torch.cuda.amp.autocast()` API, for more details:
+            https://pytorch.org/docs/stable/amp.html#torch.cuda.amp.autocast.
+    """
+
+    def __init__(
+        self,
+        device: torch.device,
+        val_data_loader: Iterable | DataLoader,
+        detector: torch.nn.Module,
+        epoch_length: int | None = None,
+        non_blocking: bool = False,
+        prepare_batch: Callable = detection_prepare_val_batch,
+        iteration_update: Callable[[Engine, Any], Any] | None = None,
+        inferer: Inferer | None = None,
+        postprocessing: Transform | None = None,
+        key_val_metric: dict[str, Metric] | None = None,
+        additional_metrics: dict[str, Metric] | None = None,
+        metric_cmp_fn: Callable = default_metric_cmp_fn,
+        val_handlers: Sequence | None = None,
+        amp: bool = False,
+        mode: ForwardMode | str = ForwardMode.EVAL,
+        event_names: list[str | EventEnum] | None = None,
+        event_to_attr: dict | None = None,
+        decollate: bool = True,
+        to_kwargs: dict | None = None,
+        amp_kwargs: dict | None = None,
+    ) -> None:
+        super().__init__(
+            device=device,
+            val_data_loader=val_data_loader,
+            epoch_length=epoch_length,
+            non_blocking=non_blocking,
+            prepare_batch=prepare_batch,
+            iteration_update=iteration_update,
+            postprocessing=postprocessing,
+            key_val_metric=key_val_metric,
+            additional_metrics=additional_metrics,
+            metric_cmp_fn=metric_cmp_fn,
+            val_handlers=val_handlers,
+            amp=amp,
+            mode=mode,
+            event_names=event_names,
+            event_to_attr=event_to_attr,
+            decollate=decollate,
+            to_kwargs=to_kwargs,
+            amp_kwargs=amp_kwargs,
+        )
+
+        self.detector = detector
+
+        mode = look_up_option(mode, ForwardMode)
+        if mode == ForwardMode.EVAL:
+            self.mode = eval_mode
+        elif mode == ForwardMode.TRAIN:
+            self.mode = train_mode
+        else:
+            raise ValueError(f"unsupported mode: {mode}, should be 'eval' or 'train'.")
+
+    def _register_decollate(self):
+        """
+        Register the decollate operation for batch data, will execute after model forward and loss forward.
+        """
+
+        @self.on(IterationEvents.MODEL_COMPLETED)
+        def _decollate_data(engine: Engine) -> None:
+            output_list = []
+            for i in range(len(engine.state.output[Keys.IMAGE])):
+                output_list.append({})
+                for k in engine.state.output.keys():
+                    if engine.state.output[k] is not None:
+                        output_list[i][k] = engine.state.output[k][i]
+            engine.state.output = output_list
+
+    def _iteration(self, engine, batchdata: dict[str, torch.Tensor]):
+        """
+        callback function for the Supervised Evaluation processing logic of 1 iteration in Ignite Engine.
+        Return below items in a dictionary:
+            - IMAGE: image Tensor data for model input, already moved to device.
+            - LABEL: label Tensor data corresponding to the image, already moved to device.
+            - PRED: prediction result of model.
+        Args:
+            engine: `SupervisedEvaluator` to execute operation for an iteration.
+            batchdata: input data for this iteration, usually can be dictionary or tuple of Tensor data.
+        Raises:
+            ValueError: When ``batchdata`` is None.
+        """
+
+        if batchdata is None:
+            raise ValueError("Must provide batch data for current iteration.")
+
+        batch = engine.prepare_batch(batchdata, engine.state.device, engine.non_blocking, **engine.to_kwargs)
+        if len(batch) == 2:
+            inputs, targets = batch
+            args: tuple = ()
+            kwargs: dict = {}
+        else:
+            inputs, targets, args, kwargs = batch
+        # put iteration outputs into engine.state
+        engine.state.output = {Keys.IMAGE: inputs, Keys.LABEL: targets}
+
+        # execute forward computation
+        sliding_window_size = np.prod(engine.detector.inferer.roi_size)
+
+        with engine.mode(engine.detector):
+
+            use_inferer = not all([val_data_i[0, ...].numel() < sliding_window_size for val_data_i in inputs])
+
+            if engine.amp:
+                with torch.cuda.amp.autocast(**engine.amp_kwargs):
+                    engine.state.output[Keys.PRED] = engine.detector(inputs, use_inferer=use_inferer)
+            else:
+                engine.state.output[Keys.PRED] = engine.detector(inputs, use_inferer=use_inferer)
+
+        engine.fire_event(IterationEvents.FORWARD_COMPLETED)
+        engine.fire_event(IterationEvents.MODEL_COMPLETED)
+
+        return engine.state.output

scripts/trainer.py

@@ -0,0 +1,229 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, Callable, Dict, Iterable, List, Optional, Sequence, Tuple, Union
+
+import torch
+from monai.config import IgniteInfo
+from monai.engines.trainer import Trainer
+from monai.engines.utils import IterationEvents, default_metric_cmp_fn
+from monai.inferers import Inferer
+from monai.transforms import Transform
+from monai.utils import min_version, optional_import
+from monai.utils.enums import CommonKeys as Keys
+from torch.optim.optimizer import Optimizer
+from torch.utils.data import DataLoader
+
+if TYPE_CHECKING:
+    from ignite.engine import Engine, EventEnum
+    from ignite.metrics import Metric
+else:
+    Engine, _ = optional_import("ignite.engine", IgniteInfo.OPT_IMPORT_VERSION, min_version, "Engine")
+    Metric, _ = optional_import("ignite.metrics", IgniteInfo.OPT_IMPORT_VERSION, min_version, "Metric")
+    EventEnum, _ = optional_import("ignite.engine", IgniteInfo.OPT_IMPORT_VERSION, min_version, "EventEnum")
+
+__all__ = ["DetectionTrainer"]
+
+
+def detection_prepare_batch(
+    batchdata: List[Dict[str, torch.Tensor]],
+    device: Optional[Union[str, torch.device]] = None,
+    non_blocking: bool = False,
+    **kwargs,
+) -> Union[Tuple[torch.Tensor, Optional[torch.Tensor]], torch.Tensor]:
+    """
+    Default function to prepare the data for current iteration.
+    Args `batchdata`, `device`, `non_blocking` refer to the ignite API:
+    https://pytorch.org/ignite/v0.4.8/generated/ignite.engine.create_supervised_trainer.html.
+    `kwargs` supports other args for `Tensor.to()` API.
+    Returns:
+        image, label(optional).
+    """
+    inputs = [
+        batch_data_ii["image"].to(device=device, non_blocking=non_blocking, **kwargs)
+        for batch_data_i in batchdata
+        for batch_data_ii in batch_data_i
+    ]
+
+    if isinstance(batchdata[0][0].get(Keys.LABEL), torch.Tensor):
+        targets = [
+            dict(
+                label=batch_data_ii["label"].to(device=device, non_blocking=non_blocking, **kwargs),
+                box=batch_data_ii["box"].to(device=device, non_blocking=non_blocking, **kwargs),
+            )
+            for batch_data_i in batchdata
+            for batch_data_ii in batch_data_i
+        ]
+        return (inputs, targets)
+    return inputs, None
+
+
+class DetectionTrainer(Trainer):
+    """
+    Supervised detection training method with image and label, inherits from ``Trainer`` and ``Workflow``.
+    Args:
+        device: an object representing the device on which to run.
+        max_epochs: the total epoch number for trainer to run.
+        train_data_loader: Ignite engine use data_loader to run, must be Iterable or torch.DataLoader.
+        detector: detector to train in the trainer, should be regular PyTorch `torch.nn.Module`.
+        optimizer: the optimizer associated to the detector, should be regular PyTorch optimizer from `torch.optim`
+            or its subclass.
+        epoch_length: number of iterations for one epoch, default to `len(train_data_loader)`.
+        non_blocking: if True and this copy is between CPU and GPU, the copy may occur asynchronously
+            with respect to the host. For other cases, this argument has no effect.
+        prepare_batch: function to parse expected data (usually `image`,`box`, `label` and other detector args)
+            from `engine.state.batch` for every iteration, for more details please refer to:
+            https://pytorch.org/ignite/generated/ignite.engine.create_supervised_trainer.html.
+        iteration_update: the callable function for every iteration, expect to accept `engine`
+            and `engine.state.batch` as inputs, return data will be stored in `engine.state.output`.
+            if not provided, use `self._iteration()` instead. for more details please refer to:
+            https://pytorch.org/ignite/generated/ignite.engine.engine.Engine.html.
+        inferer: inference method that execute model forward on input data, like: SlidingWindow, etc.
+        postprocessing: execute additional transformation for the model output data.
+            Typically, several Tensor based transforms composed by `Compose`.
+        key_train_metric: compute metric when every iteration completed, and save average value to
+            engine.state.metrics when epoch completed. key_train_metric is the main metric to compare and save the
+            checkpoint into files.
+        additional_metrics: more Ignite metrics that also attach to Ignite Engine.
+        metric_cmp_fn: function to compare current key metric with previous best key metric value,
+            it must accept 2 args (current_metric, previous_best) and return a bool result: if `True`, will update
+            `best_metric` and `best_metric_epoch` with current metric and epoch, default to `greater than`.
+        train_handlers: every handler is a set of Ignite Event-Handlers, must have `attach` function, like:
+            CheckpointHandler, StatsHandler, etc.
+        amp: whether to enable auto-mixed-precision training, default is False.
+        event_names: additional custom ignite events that will register to the engine.
+            new events can be a list of str or `ignite.engine.events.EventEnum`.
+        event_to_attr: a dictionary to map an event to a state attribute, then add to `engine.state`.
+            for more details, check: https://pytorch.org/ignite/generated/ignite.engine.engine.Engine.html
+            #ignite.engine.engine.Engine.register_events.
+        decollate: whether to decollate the batch-first data to a list of data after model computation,
+            recommend `decollate=True` when `postprocessing` uses components from `monai.transforms`.
+            default to `True`.
+        optim_set_to_none: when calling `optimizer.zero_grad()`, instead of setting to zero, set the grads to None.
+            more details: https://pytorch.org/docs/stable/generated/torch.optim.Optimizer.zero_grad.html.
+        to_kwargs: dict of other args for `prepare_batch` API when converting the input data, except for
+            `device`, `non_blocking`.
+        amp_kwargs: dict of the args for `torch.cuda.amp.autocast()` API, for more details:
+            https://pytorch.org/docs/stable/amp.html#torch.cuda.amp.autocast.
+    """
+
+    def __init__(
+        self,
+        device: torch.device,
+        max_epochs: int,
+        train_data_loader: Iterable | DataLoader,
+        detector: torch.nn.Module,
+        optimizer: Optimizer,
+        epoch_length: int | None = None,
+        non_blocking: bool = False,
+        prepare_batch: Callable = detection_prepare_batch,
+        iteration_update: Callable[[Engine, Any], Any] | None = None,
+        inferer: Inferer | None = None,
+        postprocessing: Transform | None = None,
+        key_train_metric: dict[str, Metric] | None = None,
+        additional_metrics: dict[str, Metric] | None = None,
+        metric_cmp_fn: Callable = default_metric_cmp_fn,
+        train_handlers: Sequence | None = None,
+        amp: bool = False,
+        event_names: list[str | EventEnum] | None = None,
+        event_to_attr: dict | None = None,
+        decollate: bool = True,
+        optim_set_to_none: bool = False,
+        to_kwargs: dict | None = None,
+        amp_kwargs: dict | None = None,
+    ) -> None:
+        super().__init__(
+            device=device,
+            max_epochs=max_epochs,
+            data_loader=train_data_loader,
+            epoch_length=epoch_length,
+            non_blocking=non_blocking,
+            prepare_batch=prepare_batch,
+            iteration_update=iteration_update,
+            postprocessing=postprocessing,
+            key_metric=key_train_metric,
+            additional_metrics=additional_metrics,
+            metric_cmp_fn=metric_cmp_fn,
+            handlers=train_handlers,
+            amp=amp,
+            event_names=event_names,
+            event_to_attr=event_to_attr,
+            decollate=decollate,
+            to_kwargs=to_kwargs,
+            amp_kwargs=amp_kwargs,
+        )
+
+        self.detector = detector
+        self.optimizer = optimizer
+        self.optim_set_to_none = optim_set_to_none
+
+    def _iteration(self, engine, batchdata: dict[str, torch.Tensor]):
+        """
+        Callback function for the Supervised Training processing logic of 1 iteration in Ignite Engine.
+        Return below items in a dictionary:
+            - IMAGE: image Tensor data for model input, already moved to device.
+            - BOX: box regression loss corresponding to the image, already moved to device.
+            - LABEL: classification loss corresponding to the image, already moved to device.
+            - LOSS: weighted sum of loss values computed by loss function.
+        Args:
+            engine: `DetectionTrainer` to execute operation for an iteration.
+            batchdata: input data for this iteration, usually can be dictionary or tuple of Tensor data.
+        Raises:
+            ValueError: When ``batchdata`` is None.
+        """
+
+        if batchdata is None:
+            raise ValueError("Must provide batch data for current iteration.")
+
+        batch = engine.prepare_batch(batchdata, engine.state.device, engine.non_blocking, **engine.to_kwargs)
+        if len(batch) == 2:
+            inputs, targets = batch
+            args: tuple = ()
+            kwargs: dict = {}
+        else:
+            inputs, targets, args, kwargs = batch
+        # put iteration outputs into engine.state
+        engine.state.output = {Keys.IMAGE: inputs, Keys.LABEL: targets}
+
+        def _compute_pred_loss(w_cls: float = 1.0, w_box_reg: float = 1.0):
+            """
+            Args:
+                w_cls: weight of classification loss
+                w_box_reg: weight of box regression loss
+            """
+            outputs = engine.detector(inputs, targets)
+            engine.state.output[engine.detector.cls_key] = outputs[engine.detector.cls_key]
+            engine.state.output[engine.detector.box_reg_key] = outputs[engine.detector.box_reg_key]
+            engine.state.output[Keys.LOSS] = (
+                w_cls * outputs[engine.detector.cls_key] + w_box_reg * outputs[engine.detector.box_reg_key]
+            )
+            engine.fire_event(IterationEvents.LOSS_COMPLETED)
+
+        engine.detector.train()
+        engine.optimizer.zero_grad(set_to_none=engine.optim_set_to_none)
+
+        if engine.amp and engine.scaler is not None:
+            with torch.cuda.amp.autocast(**engine.amp_kwargs):
+                inputs = [img.to(torch.float16) for img in inputs]
+                _compute_pred_loss()
+            engine.scaler.scale(engine.state.output[Keys.LOSS]).backward()
+            engine.fire_event(IterationEvents.BACKWARD_COMPLETED)
+            engine.scaler.step(engine.optimizer)
+            engine.scaler.update()
+        else:
+            _compute_pred_loss()
+            engine.state.output[Keys.LOSS].backward()
+            engine.fire_event(IterationEvents.BACKWARD_COMPLETED)
+            engine.optimizer.step()
+
+        return engine.state.output

scripts/utils.py

@@ -0,0 +1,26 @@
+from typing import Dict, List, Union
+
+import numpy as np
+import torch
+
+
+def detach_to_numpy(data: Union[List, Dict, torch.Tensor]) -> Union[List, Dict, torch.Tensor]:
+    """
+    Recursively detach elements in data
+    """
+    if isinstance(data, torch.Tensor):
+        return data.cpu().detach().numpy()  # pytype: disable=attribute-error
+
+    elif isinstance(data, np.ndarray):
+        return data
+
+    elif isinstance(data, list):
+        return [detach_to_numpy(d) for d in data]
+
+    elif isinstance(data, dict):
+        for k in data.keys():
+            data[k] = detach_to_numpy(data[k])
+        return data
+
+    else:
+        raise ValueError("data should be tensor, numpy array, dict, or list.")

scripts/warmup_scheduler.py

@@ -0,0 +1,88 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+This script is adapted from
+https://github.com/ildoonet/pytorch-gradual-warmup-lr/blob/master/warmup_scheduler/scheduler.py
+"""
+
+from torch.optim.lr_scheduler import ReduceLROnPlateau, _LRScheduler
+
+
+class GradualWarmupScheduler(_LRScheduler):
+    """Gradually warm-up(increasing) learning rate in optimizer.
+    Proposed in 'Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour'.
+
+    Args:
+        optimizer (Optimizer): Wrapped optimizer.
+        multiplier: target learning rate = base lr * multiplier if multiplier > 1.0.
+            if multiplier = 1.0, lr starts from 0 and ends up with the base_lr.
+        total_epoch: target learning rate is reached at total_epoch, gradually
+        after_scheduler: after target_epoch, use this scheduler(eg. ReduceLROnPlateau)
+    """
+
+    def __init__(self, optimizer, multiplier, total_epoch, after_scheduler=None):
+        self.multiplier = multiplier
+        if self.multiplier < 1.0:
+            raise ValueError("multiplier should be greater thant or equal to 1.")
+        self.total_epoch = total_epoch
+        self.after_scheduler = after_scheduler
+        self.finished = False
+        super(GradualWarmupScheduler, self).__init__(optimizer)
+
+    def get_lr(self):
+        if self.last_epoch > self.total_epoch:
+            if self.after_scheduler:
+                if not self.finished:
+                    self.after_scheduler.base_lrs = [base_lr * self.multiplier for base_lr in self.base_lrs]
+                    self.finished = True
+                return self.after_scheduler.get_last_lr()
+            return [base_lr * self.multiplier for base_lr in self.base_lrs]
+
+        if self.multiplier == 1.0:
+            return [base_lr * (float(self.last_epoch) / self.total_epoch) for base_lr in self.base_lrs]
+        else:
+            return [
+                base_lr * ((self.multiplier - 1.0) * self.last_epoch / self.total_epoch + 1.0)
+                for base_lr in self.base_lrs
+            ]
+
+    def step_reduce_lr_on_plateau(self, metrics, epoch=None):
+        if epoch is None:
+            epoch = self.last_epoch + 1
+        self.last_epoch = (
+            epoch if epoch != 0 else 1
+        )  # ReduceLROnPlateau is called at the end of epoch, whereas others are called at beginning
+        if self.last_epoch <= self.total_epoch:
+            warmup_lr = [
+                base_lr * ((self.multiplier - 1.0) * self.last_epoch / self.total_epoch + 1.0)
+                for base_lr in self.base_lrs
+            ]
+            for param_group, lr in zip(self.optimizer.param_groups, warmup_lr):
+                param_group["lr"] = lr
+        else:
+            if epoch is None:
+                self.after_scheduler.step(metrics, None)
+            else:
+                self.after_scheduler.step(metrics, epoch - self.total_epoch)
+
+    def step(self, epoch=None, metrics=None):
+        if type(self.after_scheduler) != ReduceLROnPlateau:
+            if self.finished and self.after_scheduler:
+                if epoch is None:
+                    self.after_scheduler.step(None)
+                else:
+                    self.after_scheduler.step(epoch - self.total_epoch)
+                self._last_lr = self.after_scheduler.get_last_lr()
+            else:
+                return super(GradualWarmupScheduler, self).step(epoch)
+        else:
+            self.step_reduce_lr_on_plateau(metrics, epoch)