LambdaSuperRes initial commit

.gitignore

@@ -0,0 +1,10 @@
+*.swp
+*.swo
+
+__pycache__
+*.pyc
+
+sr_interactive_tmp
+sr_interactive_tmp_output
+
+gradio_cached_examples

KAIR/LICENSE

@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) 2019 Kai Zhang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

KAIR/README.md

@@ -0,0 +1,343 @@
+## Training and testing codes for USRNet, DnCNN, FFDNet, SRMD, DPSR, MSRResNet, ESRGAN, BSRGAN, SwinIR, VRT
+[![download](https://img.shields.io/github/downloads/cszn/KAIR/total.svg)](https://github.com/cszn/KAIR/releases) ![visitors](https://visitor-badge.glitch.me/badge?page_id=cszn/KAIR) 
+
+[Kai Zhang](https://cszn.github.io/)
+
+*[Computer Vision Lab](https://vision.ee.ethz.ch/the-institute.html), ETH Zurich, Switzerland*
+
+_______
+- **_News (2022-02-15)_**: We release [the training codes](https://github.com/cszn/KAIR/blob/master/docs/README_VRT.md) of [VRT ![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/VRT?style=social)](https://github.com/JingyunLiang/VRT) for video SR, deblurring and denoising.
+<p align="center">
+  <a href="https://github.com/JingyunLiang/VRT">
+    <img width=30% src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/teaser_vsr.gif"/>
+    <img width=30% src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/teaser_vdb.gif"/>
+    <img width=30% src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/teaser_vdn.gif"/>
+  </a>
+</p>
+
+- **_News (2021-12-23)_**: Our techniques are adopted in [https://www.amemori.ai/](https://www.amemori.ai/).
+- **_News (2021-12-23)_**: Our new work for practical image denoising.
+
+- <img src="figs/palace.png" height="320px"/> <img src="figs/palace_HSCU.png" height="320px"/> 
+- [<img src="https://github.com/cszn/KAIR/raw/master/figs/denoising_02.png" height="256px"/>](https://imgsli.com/ODczMTc) 
+[<img src="https://github.com/cszn/KAIR/raw/master/figs/denoising_01.png" height="256px"/>](https://imgsli.com/ODczMTY) 
+- **_News (2021-09-09)_**: Add [main_download_pretrained_models.py](https://github.com/cszn/KAIR/blob/master/main_download_pretrained_models.py) to download pre-trained models.
+- **_News (2021-09-08)_**: Add [matlab code](https://github.com/cszn/KAIR/tree/master/matlab) to zoom local part of an image for the purpose of comparison between different results.
+- **_News (2021-09-07)_**: We upload [the training code](https://github.com/cszn/KAIR/blob/master/docs/README_SwinIR.md) of [SwinIR ![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/SwinIR?style=social)](https://github.com/JingyunLiang/SwinIR) and provide an [interactive online Colob demo for real-world image SR](https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb). Try to super-resolve your own images on Colab! <a href="https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>
+
+|Real-World Image (x4)|[BSRGAN, ICCV2021](https://github.com/cszn/BSRGAN)|[Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN)|SwinIR (ours)|
+|      :---      |     :---:        |        :-----:         |        :-----:         | 
+|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_LR.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_BSRGAN.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_realESRGAN.jpg">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_SwinIR.png">
+|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_LR.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_BSRGAN.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_realESRGAN.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_SwinIR.png">|
+
+- **_News (2021-08-31)_**: We upload the [training code of BSRGAN](https://github.com/cszn/BSRGAN#training).
+- **_News (2021-08-24)_**: We upload the BSRGAN degradation model.
+- **_News (2021-08-22)_**: Support multi-feature-layer VGG perceptual loss and UNet discriminator. 
+- **_News (2021-08-18)_**: We upload the extended BSRGAN degradation model. It is slightly different from our published version. 
+
+- **_News (2021-06-03)_**: Add testing codes of [GPEN (CVPR21)](https://github.com/yangxy/GPEN) for face image enhancement: [main_test_face_enhancement.py](https://github.com/cszn/KAIR/blob/master/main_test_face_enhancement.py)
+
+<img src="figs/face_04_comparison.png" width="730px"/> 
+<img src="figs/face_13_comparison.png" width="730px"/> 
+<img src="figs/face_08_comparison.png" width="730px"/> 
+<img src="figs/face_01_comparison.png" width="730px"/> 
+<img src="figs/face_12_comparison.png" width="730px"/> 
+<img src="figs/face_10_comparison.png" width="730px"/> 
+
+
+- **_News (2021-05-13)_**: Add [PatchGAN discriminator](https://github.com/cszn/KAIR/blob/master/models/network_discriminator.py).
+
+- **_News (2021-05-12)_**: Support distributed training, see also [https://github.com/xinntao/BasicSR/blob/master/docs/TrainTest.md](https://github.com/xinntao/BasicSR/blob/master/docs/TrainTest.md).
+
+- **_News (2021-01)_**: [BSRGAN](https://github.com/cszn/BSRGAN) for blind real image super-resolution will be added.
+
+- **_Pull requests are welcome!_**
+
+- **Correction (2020-10)**: If you use multiple GPUs for GAN training, remove or comment [Line 105](https://github.com/cszn/KAIR/blob/e52a6944c6a40ba81b88430ffe38fd6517e0449e/models/model_gan.py#L105) to enable `DataParallel` for fast training
+
+- **News (2020-10)**: Add [utils_receptivefield.py](https://github.com/cszn/KAIR/blob/master/utils/utils_receptivefield.py) to calculate receptive field.
+
+- **News (2020-8)**: A `deep plug-and-play image restoration toolbox` is released at [cszn/DPIR](https://github.com/cszn/DPIR).
+
+- **Tips (2020-8)**: Use [this](https://github.com/cszn/KAIR/blob/9fd17abff001ab82a22070f7e442bb5246d2d844/main_challenge_sr.py#L147) to avoid `out of memory` issue.
+
+- **News (2020-7)**: Add [main_challenge_sr.py](https://github.com/cszn/KAIR/blob/23b0d0f717980e48fad02513ba14045d57264fe1/main_challenge_sr.py#L90) to get `FLOPs`, `#Params`, `Runtime`, `#Activations`, `#Conv`, and `Max Memory Allocated`.
+```python
+from utils.utils_modelsummary import get_model_activation, get_model_flops
+input_dim = (3, 256, 256)  # set the input dimension
+activations, num_conv2d = get_model_activation(model, input_dim)
+logger.info('{:>16s} : {:<.4f} [M]'.format('#Activations', activations/10**6))
+logger.info('{:>16s} : {:<d}'.format('#Conv2d', num_conv2d))
+flops = get_model_flops(model, input_dim, False)
+logger.info('{:>16s} : {:<.4f} [G]'.format('FLOPs', flops/10**9))
+num_parameters = sum(map(lambda x: x.numel(), model.parameters()))
+logger.info('{:>16s} : {:<.4f} [M]'.format('#Params', num_parameters/10**6))
+```
+
+- **News (2020-6)**: Add [USRNet (CVPR 2020)](https://github.com/cszn/USRNet) for training and testing.
+  - [Network Architecture](https://github.com/cszn/KAIR/blob/3357aa0e54b81b1e26ceb1cee990f39add235e17/models/network_usrnet.py#L309)
+  - [Dataset](https://github.com/cszn/KAIR/blob/6c852636d3715bb281637863822a42c72739122a/data/dataset_usrnet.py#L16)
+
+
+Clone repo
+----------
+```
+git clone https://github.com/cszn/KAIR.git
+```
+```
+pip install -r requirement.txt
+```
+
+
+
+Training
+----------
+
+You should modify the json file from [options](https://github.com/cszn/KAIR/tree/master/options) first, for example,
+setting ["gpu_ids": [0,1,2,3]](https://github.com/cszn/KAIR/blob/ff80d265f64de67dfb3ffa9beff8949773c81a3d/options/train_msrresnet_psnr.json#L4) if 4 GPUs are used,
+setting ["dataroot_H": "trainsets/trainH"](https://github.com/cszn/KAIR/blob/ff80d265f64de67dfb3ffa9beff8949773c81a3d/options/train_msrresnet_psnr.json#L24) if path of the high quality dataset is `trainsets/trainH`.
+
+- Training with `DataParallel` - PSNR
+
+
+```python
+python main_train_psnr.py --opt options/train_msrresnet_psnr.json
+```
+
+- Training with `DataParallel` - GAN
+
+```python
+python main_train_gan.py --opt options/train_msrresnet_gan.json
+```
+
+- Training with `DistributedDataParallel` - PSNR - 4 GPUs
+
+```python
+python -m torch.distributed.launch --nproc_per_node=4 --master_port=1234 main_train_psnr.py --opt options/train_msrresnet_psnr.json  --dist True
+```
+
+- Training with `DistributedDataParallel` - PSNR - 8 GPUs
+
+```python
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/train_msrresnet_psnr.json  --dist True
+```
+
+- Training with `DistributedDataParallel` - GAN - 4 GPUs
+
+```python
+python -m torch.distributed.launch --nproc_per_node=4 --master_port=1234 main_train_gan.py --opt options/train_msrresnet_gan.json  --dist True
+```
+
+- Training with `DistributedDataParallel` - GAN - 8 GPUs
+
+```python
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_gan.py --opt options/train_msrresnet_gan.json  --dist True
+```
+
+- Kill distributed training processes of `main_train_gan.py`
+
+```python
+kill $(ps aux | grep main_train_gan.py | grep -v grep | awk '{print $2}')
+```
+
+----------
+| Method | Original Link |
+|---|---|
+| DnCNN |[https://github.com/cszn/DnCNN](https://github.com/cszn/DnCNN)|
+| FDnCNN |[https://github.com/cszn/DnCNN](https://github.com/cszn/DnCNN)|
+| FFDNet | [https://github.com/cszn/FFDNet](https://github.com/cszn/FFDNet)|
+| SRMD | [https://github.com/cszn/SRMD](https://github.com/cszn/SRMD)|
+| DPSR-SRResNet | [https://github.com/cszn/DPSR](https://github.com/cszn/DPSR)|
+| SRResNet | [https://github.com/xinntao/BasicSR](https://github.com/xinntao/BasicSR)|
+| ESRGAN | [https://github.com/xinntao/ESRGAN](https://github.com/xinntao/ESRGAN)|
+| RRDB | [https://github.com/xinntao/ESRGAN](https://github.com/xinntao/ESRGAN)|
+| IMDB | [https://github.com/Zheng222/IMDN](https://github.com/Zheng222/IMDN)|
+| USRNet | [https://github.com/cszn/USRNet](https://github.com/cszn/USRNet)|
+| DRUNet | [https://github.com/cszn/DPIR](https://github.com/cszn/DPIR)|
+| DPIR | [https://github.com/cszn/DPIR](https://github.com/cszn/DPIR)|
+| BSRGAN | [https://github.com/cszn/BSRGAN](https://github.com/cszn/BSRGAN)|
+| SwinIR | [https://github.com/JingyunLiang/SwinIR](https://github.com/JingyunLiang/SwinIR)|
+| VRT | [https://github.com/JingyunLiang/VRT](https://github.com/JingyunLiang/VRT)       |
+
+Network architectures
+----------
+* [USRNet](https://github.com/cszn/USRNet)
+
+  <img src="https://github.com/cszn/USRNet/blob/master/figs/architecture.png" width="600px"/> 
+
+* DnCNN
+
+  <img src="https://github.com/cszn/DnCNN/blob/master/figs/dncnn.png" width="600px"/> 
+ 
+* IRCNN denoiser
+
+ <img src="https://github.com/lipengFu/IRCNN/raw/master/Image/image_2.png" width="680px"/> 
+
+* FFDNet
+
+  <img src="https://github.com/cszn/FFDNet/blob/master/figs/ffdnet.png" width="600px"/> 
+
+* SRMD
+
+  <img src="https://github.com/cszn/SRMD/blob/master/figs/architecture.png" width="605px"/> 
+
+* SRResNet, SRGAN, RRDB, ESRGAN
+
+  <img src="https://github.com/xinntao/ESRGAN/blob/master/figures/architecture.jpg" width="595px"/> 
+  
+* IMDN
+
+  <img src="figs/imdn.png" width="460px"/>  ----- <img src="figs/imdn_block.png" width="100px"/> 
+
+
+
+Testing
+----------
+|Method | [model_zoo](model_zoo)|
+|---|---|
+| [main_test_dncnn.py](main_test_dncnn.py) |```dncnn_15.pth, dncnn_25.pth, dncnn_50.pth, dncnn_gray_blind.pth, dncnn_color_blind.pth, dncnn3.pth```|
+| [main_test_ircnn_denoiser.py](main_test_ircnn_denoiser.py) | ```ircnn_gray.pth, ircnn_color.pth```| 
+| [main_test_fdncnn.py](main_test_fdncnn.py) | ```fdncnn_gray.pth, fdncnn_color.pth, fdncnn_gray_clip.pth, fdncnn_color_clip.pth```|
+| [main_test_ffdnet.py](main_test_ffdnet.py) | ```ffdnet_gray.pth, ffdnet_color.pth, ffdnet_gray_clip.pth, ffdnet_color_clip.pth```|
+| [main_test_srmd.py](main_test_srmd.py) | ```srmdnf_x2.pth, srmdnf_x3.pth, srmdnf_x4.pth, srmd_x2.pth, srmd_x3.pth, srmd_x4.pth```| 
+|  | **The above models are converted from MatConvNet.** |
+| [main_test_dpsr.py](main_test_dpsr.py) | ```dpsr_x2.pth, dpsr_x3.pth, dpsr_x4.pth, dpsr_x4_gan.pth```|
+| [main_test_msrresnet.py](main_test_msrresnet.py) | ```msrresnet_x4_psnr.pth, msrresnet_x4_gan.pth```|
+| [main_test_rrdb.py](main_test_rrdb.py) | ```rrdb_x4_psnr.pth, rrdb_x4_esrgan.pth```|
+| [main_test_imdn.py](main_test_imdn.py) | ```imdn_x4.pth```|
+
+[model_zoo](model_zoo)
+--------
+- download link [https://drive.google.com/drive/folders/13kfr3qny7S2xwG9h7v95F5mkWs0OmU0D](https://drive.google.com/drive/folders/13kfr3qny7S2xwG9h7v95F5mkWs0OmU0D)
+
+[trainsets](trainsets)
+----------
+- [https://github.com/xinntao/BasicSR/blob/master/docs/DatasetPreparation.md](https://github.com/xinntao/BasicSR/blob/master/docs/DatasetPreparation.md)
+- [train400](https://github.com/cszn/DnCNN/tree/master/TrainingCodes/DnCNN_TrainingCodes_v1.0/data)
+- [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/)
+- [Flickr2K](https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar)
+- optional: use [split_imageset(original_dataroot, taget_dataroot, n_channels=3, p_size=512, p_overlap=96, p_max=800)](https://github.com/cszn/KAIR/blob/3ee0bf3e07b90ec0b7302d97ee2adb780617e637/utils/utils_image.py#L123) to get ```trainsets/trainH``` with small images for fast data loading
+
+[testsets](testsets)
+-----------
+- [https://github.com/xinntao/BasicSR/blob/master/docs/DatasetPreparation.md](https://github.com/xinntao/BasicSR/blob/master/docs/DatasetPreparation.md)
+- [set12](https://github.com/cszn/FFDNet/tree/master/testsets)
+- [bsd68](https://github.com/cszn/FFDNet/tree/master/testsets)
+- [cbsd68](https://github.com/cszn/FFDNet/tree/master/testsets)
+- [kodak24](https://github.com/cszn/FFDNet/tree/master/testsets)
+- [srbsd68](https://github.com/cszn/DPSR/tree/master/testsets/BSD68/GT)
+- set5
+- set14
+- cbsd100
+- urban100
+- manga109
+
+
+References
+----------
+```BibTex
+@article{liang2022vrt,
+title={VRT: A Video Restoration Transformer},
+author={Liang, Jingyun and Cao, Jiezhang and Fan, Yuchen and Zhang, Kai and Ranjan, Rakesh and Li, Yawei and Timofte, Radu and Van Gool, Luc},
+journal={arXiv preprint arXiv:2022.00000},
+year={2022}
+}
+@inproceedings{liang2021swinir,
+title={SwinIR: Image Restoration Using Swin Transformer},
+author={Liang, Jingyun and Cao, Jiezhang and Sun, Guolei and Zhang, Kai and Van Gool, Luc and Timofte, Radu},
+booktitle={IEEE International Conference on Computer Vision Workshops},
+pages={1833--1844},
+year={2021}
+}
+@inproceedings{zhang2021designing,
+title={Designing a Practical Degradation Model for Deep Blind Image Super-Resolution},
+author={Zhang, Kai and Liang, Jingyun and Van Gool, Luc and Timofte, Radu},
+booktitle={IEEE International Conference on Computer Vision},
+pages={4791--4800},
+year={2021}
+}
+@article{zhang2021plug, % DPIR & DRUNet & IRCNN
+  title={Plug-and-Play Image Restoration with Deep Denoiser Prior},
+  author={Zhang, Kai and Li, Yawei and Zuo, Wangmeng and Zhang, Lei and Van Gool, Luc and Timofte, Radu},
+  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
+  year={2021}
+}
+@inproceedings{zhang2020aim, % efficientSR_challenge
+  title={AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results},
+  author={Kai Zhang and Martin Danelljan and Yawei Li and Radu Timofte and others},
+  booktitle={European Conference on Computer Vision Workshops},
+  year={2020}
+}
+@inproceedings{zhang2020deep, % USRNet
+  title={Deep unfolding network for image super-resolution},
+  author={Zhang, Kai and Van Gool, Luc and Timofte, Radu},
+  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
+  pages={3217--3226},
+  year={2020}
+}
+@article{zhang2017beyond, % DnCNN
+  title={Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising},
+  author={Zhang, Kai and Zuo, Wangmeng and Chen, Yunjin and Meng, Deyu and Zhang, Lei},
+  journal={IEEE Transactions on Image Processing},
+  volume={26},
+  number={7},
+  pages={3142--3155},
+  year={2017}
+}
+@inproceedings{zhang2017learning, % IRCNN
+title={Learning deep CNN denoiser prior for image restoration},
+author={Zhang, Kai and Zuo, Wangmeng and Gu, Shuhang and Zhang, Lei},
+booktitle={IEEE conference on computer vision and pattern recognition},
+pages={3929--3938},
+year={2017}
+}
+@article{zhang2018ffdnet, % FFDNet, FDnCNN
+  title={FFDNet: Toward a fast and flexible solution for CNN-based image denoising},
+  author={Zhang, Kai and Zuo, Wangmeng and Zhang, Lei},
+  journal={IEEE Transactions on Image Processing},
+  volume={27},
+  number={9},
+  pages={4608--4622},
+  year={2018}
+}
+@inproceedings{zhang2018learning, % SRMD
+  title={Learning a single convolutional super-resolution network for multiple degradations},
+  author={Zhang, Kai and Zuo, Wangmeng and Zhang, Lei},
+  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
+  pages={3262--3271},
+  year={2018}
+}
+@inproceedings{zhang2019deep, % DPSR
+  title={Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernels},
+  author={Zhang, Kai and Zuo, Wangmeng and Zhang, Lei},
+  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
+  pages={1671--1681},
+  year={2019}
+}
+@InProceedings{wang2018esrgan, % ESRGAN, MSRResNet
+    author = {Wang, Xintao and Yu, Ke and Wu, Shixiang and Gu, Jinjin and Liu, Yihao and Dong, Chao and Qiao, Yu and Loy, Chen Change},
+    title = {ESRGAN: Enhanced super-resolution generative adversarial networks},
+    booktitle = {The European Conference on Computer Vision Workshops (ECCVW)},
+    month = {September},
+    year = {2018}
+}
+@inproceedings{hui2019lightweight, % IMDN
+  title={Lightweight Image Super-Resolution with Information Multi-distillation Network},
+  author={Hui, Zheng and Gao, Xinbo and Yang, Yunchu and Wang, Xiumei},
+  booktitle={Proceedings of the 27th ACM International Conference on Multimedia (ACM MM)},
+  pages={2024--2032},
+  year={2019}
+}
+@inproceedings{zhang2019aim, % IMDN
+  title={AIM 2019 Challenge on Constrained Super-Resolution: Methods and Results},
+  author={Kai Zhang and Shuhang Gu and Radu Timofte and others},
+  booktitle={IEEE International Conference on Computer Vision Workshops},
+  year={2019}
+}
+@inproceedings{yang2021gan,
+    title={GAN Prior Embedded Network for Blind Face Restoration in the Wild},
+    author={Tao Yang, Peiran Ren, Xuansong Xie, and Lei Zhang},
+    booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
+    year={2021}
+}
+```

KAIR/data/__init__.py

@@ -0,0 +1 @@
+

KAIR/data/dataset_blindsr.py

@@ -0,0 +1,92 @@
+import random
+import numpy as np
+import torch.utils.data as data
+import utils.utils_image as util
+import os
+from utils import utils_blindsr as blindsr
+
+
+class DatasetBlindSR(data.Dataset):
+    '''
+    # -----------------------------------------
+    # dataset for BSRGAN
+    # -----------------------------------------
+    '''
+    def __init__(self, opt):
+        super(DatasetBlindSR, self).__init__()
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.sf = opt['scale'] if opt['scale'] else 4
+        self.shuffle_prob = opt['shuffle_prob'] if opt['shuffle_prob'] else 0.1
+        self.use_sharp = opt['use_sharp'] if opt['use_sharp'] else False
+        self.degradation_type = opt['degradation_type'] if opt['degradation_type'] else 'bsrgan'
+        self.lq_patchsize = self.opt['lq_patchsize'] if self.opt['lq_patchsize'] else 64
+        self.patch_size = self.opt['H_size'] if self.opt['H_size'] else self.lq_patchsize*self.sf
+
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+        print(len(self.paths_H))
+
+#        for n, v in enumerate(self.paths_H):
+#            if 'face' in v:
+#                del self.paths_H[n]
+#        time.sleep(1)
+        assert self.paths_H, 'Error: H path is empty.'
+
+    def __getitem__(self, index):
+
+        L_path = None
+
+        # ------------------------------------
+        # get H image
+        # ------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+        img_name, ext = os.path.splitext(os.path.basename(H_path))
+        H, W, C = img_H.shape
+
+        if H < self.patch_size or W < self.patch_size:
+            img_H = np.tile(np.random.randint(0, 256, size=[1, 1, self.n_channels], dtype=np.uint8), (self.patch_size, self.patch_size, 1))
+
+        # ------------------------------------
+        # if train, get L/H patch pair
+        # ------------------------------------
+        if self.opt['phase'] == 'train':
+
+            H, W, C = img_H.shape
+
+            rnd_h_H = random.randint(0, max(0, H - self.patch_size))
+            rnd_w_H = random.randint(0, max(0, W - self.patch_size))
+            img_H = img_H[rnd_h_H:rnd_h_H + self.patch_size, rnd_w_H:rnd_w_H + self.patch_size, :]
+
+            if 'face' in img_name:
+                mode = random.choice([0, 4])
+                img_H = util.augment_img(img_H, mode=mode)
+            else:
+                mode = random.randint(0, 7)
+                img_H = util.augment_img(img_H, mode=mode)
+
+            img_H = util.uint2single(img_H)
+            if self.degradation_type == 'bsrgan':
+                img_L, img_H = blindsr.degradation_bsrgan(img_H, self.sf, lq_patchsize=self.lq_patchsize, isp_model=None)
+            elif self.degradation_type == 'bsrgan_plus':
+                img_L, img_H = blindsr.degradation_bsrgan_plus(img_H, self.sf, shuffle_prob=self.shuffle_prob, use_sharp=self.use_sharp, lq_patchsize=self.lq_patchsize)
+
+        else:
+            img_H = util.uint2single(img_H)
+            if self.degradation_type == 'bsrgan':
+                img_L, img_H = blindsr.degradation_bsrgan(img_H, self.sf, lq_patchsize=self.lq_patchsize, isp_model=None)
+            elif self.degradation_type == 'bsrgan_plus':
+                img_L, img_H = blindsr.degradation_bsrgan_plus(img_H, self.sf, shuffle_prob=self.shuffle_prob, use_sharp=self.use_sharp, lq_patchsize=self.lq_patchsize)
+
+        # ------------------------------------
+        # L/H pairs, HWC to CHW, numpy to tensor
+        # ------------------------------------
+        img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+
+        if L_path is None:
+            L_path = H_path
+
+        return {'L': img_L, 'H': img_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_dncnn.py

@@ -0,0 +1,101 @@
+import os.path
+import random
+import numpy as np
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+class DatasetDnCNN(data.Dataset):
+    """
+    # -----------------------------------------
+    # Get L/H for denosing on AWGN with fixed sigma.
+    # Only dataroot_H is needed.
+    # -----------------------------------------
+    # e.g., DnCNN
+    # -----------------------------------------
+    """
+
+    def __init__(self, opt):
+        super(DatasetDnCNN, self).__init__()
+        print('Dataset: Denosing on AWGN with fixed sigma. Only dataroot_H is needed.')
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = opt['H_size'] if opt['H_size'] else 64
+        self.sigma = opt['sigma'] if opt['sigma'] else 25
+        self.sigma_test = opt['sigma_test'] if opt['sigma_test'] else self.sigma
+
+        # ------------------------------------
+        # get path of H
+        # return None if input is None
+        # ------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+
+    def __getitem__(self, index):
+
+        # ------------------------------------
+        # get H image
+        # ------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+
+        L_path = H_path
+
+        if self.opt['phase'] == 'train':
+            """
+            # --------------------------------
+            # get L/H patch pairs
+            # --------------------------------
+            """
+            H, W, _ = img_H.shape
+
+            # --------------------------------
+            # randomly crop the patch
+            # --------------------------------
+            rnd_h = random.randint(0, max(0, H - self.patch_size))
+            rnd_w = random.randint(0, max(0, W - self.patch_size))
+            patch_H = img_H[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size, :]
+
+            # --------------------------------
+            # augmentation - flip, rotate
+            # --------------------------------
+            mode = random.randint(0, 7)
+            patch_H = util.augment_img(patch_H, mode=mode)
+
+            # --------------------------------
+            # HWC to CHW, numpy(uint) to tensor
+            # --------------------------------
+            img_H = util.uint2tensor3(patch_H)
+            img_L = img_H.clone()
+
+            # --------------------------------
+            # add noise
+            # --------------------------------
+            noise = torch.randn(img_L.size()).mul_(self.sigma/255.0)
+            img_L.add_(noise)
+
+        else:
+            """
+            # --------------------------------
+            # get L/H image pairs
+            # --------------------------------
+            """
+            img_H = util.uint2single(img_H)
+            img_L = np.copy(img_H)
+
+            # --------------------------------
+            # add noise
+            # --------------------------------
+            np.random.seed(seed=0)
+            img_L += np.random.normal(0, self.sigma_test/255.0, img_L.shape)
+
+            # --------------------------------
+            # HWC to CHW, numpy to tensor
+            # --------------------------------
+            img_L = util.single2tensor3(img_L)
+            img_H = util.single2tensor3(img_H)
+
+        return {'L': img_L, 'H': img_H, 'H_path': H_path, 'L_path': L_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_dnpatch.py

@@ -0,0 +1,133 @@
+import random
+import numpy as np
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+class DatasetDnPatch(data.Dataset):
+    """
+    # -----------------------------------------
+    # Get L/H for denosing on AWGN with fixed sigma.
+    # ****Get all H patches first****
+    # Only dataroot_H is needed.
+    # -----------------------------------------
+    # e.g., DnCNN with BSD400
+    # -----------------------------------------
+    """
+
+    def __init__(self, opt):
+        super(DatasetDnPatch, self).__init__()
+        print('Get L/H for denosing on AWGN with fixed sigma. Only dataroot_H is needed.')
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = opt['H_size'] if opt['H_size'] else 64
+
+        self.sigma = opt['sigma'] if opt['sigma'] else 25
+        self.sigma_test = opt['sigma_test'] if opt['sigma_test'] else self.sigma
+
+        self.num_patches_per_image = opt['num_patches_per_image'] if opt['num_patches_per_image'] else 40
+        self.num_sampled = opt['num_sampled'] if opt['num_sampled'] else 3000
+
+        # ------------------------------------
+        # get paths of H
+        # ------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+        assert self.paths_H, 'Error: H path is empty.'
+
+        # ------------------------------------
+        # number of sampled H images
+        # ------------------------------------
+        self.num_sampled = min(self.num_sampled, len(self.paths_H))
+
+        # ------------------------------------
+        # reserve space with zeros
+        # ------------------------------------
+        self.total_patches = self.num_sampled * self.num_patches_per_image
+        self.H_data = np.zeros([self.total_patches, self.patch_size, self.patch_size, self.n_channels], dtype=np.uint8)
+
+        # ------------------------------------
+        # update H patches
+        # ------------------------------------
+        self.update_data()
+
+    def update_data(self):
+        """
+        # ------------------------------------
+        # update whole H patches
+        # ------------------------------------
+        """
+        self.index_sampled = random.sample(range(0, len(self.paths_H), 1), self.num_sampled)
+        n_count = 0
+
+        for i in range(len(self.index_sampled)):
+            H_patches = self.get_patches(self.index_sampled[i])
+            for H_patch in H_patches:
+                self.H_data[n_count,:,:,:] = H_patch
+                n_count += 1
+
+        print('Training data updated! Total number of patches is:  %5.2f X %5.2f = %5.2f\n' % (len(self.H_data)//128, 128, len(self.H_data)))
+
+    def get_patches(self, index):
+        """
+        # ------------------------------------
+        # get H patches from an H image
+        # ------------------------------------
+        """
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)  # uint format
+
+        H, W = img_H.shape[:2]
+
+        H_patches = []
+
+        num = self.num_patches_per_image
+        for _ in range(num):
+            rnd_h = random.randint(0, max(0, H - self.patch_size))
+            rnd_w = random.randint(0, max(0, W - self.patch_size))
+            H_patch = img_H[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size, :]
+            H_patches.append(H_patch)
+
+        return H_patches
+
+    def __getitem__(self, index):
+
+        H_path = 'toy.png'
+        if self.opt['phase'] == 'train':
+
+            patch_H = self.H_data[index]
+
+            # --------------------------------
+            # augmentation - flip and/or rotate
+            # --------------------------------
+            mode = random.randint(0, 7)
+            patch_H = util.augment_img(patch_H, mode=mode)
+
+            patch_H = util.uint2tensor3(patch_H)
+            patch_L = patch_H.clone()
+
+            # ------------------------------------
+            # add noise
+            # ------------------------------------
+            noise = torch.randn(patch_L.size()).mul_(self.sigma/255.0)
+            patch_L.add_(noise)
+
+        else:
+
+            H_path = self.paths_H[index]
+            img_H = util.imread_uint(H_path, self.n_channels)
+            img_H = util.uint2single(img_H)
+            img_L = np.copy(img_H)
+
+            # ------------------------------------
+            # add noise
+            # ------------------------------------
+            np.random.seed(seed=0)
+            img_L += np.random.normal(0, self.sigma_test/255.0, img_L.shape)
+            patch_L, patch_H = util.single2tensor3(img_L), util.single2tensor3(img_H)
+
+        L_path = H_path
+        return {'L': patch_L, 'H': patch_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.H_data)

KAIR/data/dataset_dpsr.py

@@ -0,0 +1,131 @@
+import random
+import numpy as np
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+class DatasetDPSR(data.Dataset):
+    '''
+    # -----------------------------------------
+    # Get L/H/M for noisy image SR.
+    # Only "paths_H" is needed, sythesize bicubicly downsampled L on-the-fly.
+    # -----------------------------------------
+    # e.g., SRResNet super-resolver prior for DPSR
+    # -----------------------------------------
+    '''
+
+    def __init__(self, opt):
+        super(DatasetDPSR, self).__init__()
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.sf = opt['scale'] if opt['scale'] else 4
+        self.patch_size = self.opt['H_size'] if self.opt['H_size'] else 96
+        self.L_size = self.patch_size // self.sf
+        self.sigma = opt['sigma'] if opt['sigma'] else [0, 50]
+        self.sigma_min, self.sigma_max = self.sigma[0], self.sigma[1]
+        self.sigma_test = opt['sigma_test'] if opt['sigma_test'] else 0
+
+        # ------------------------------------
+        # get paths of L/H
+        # ------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+        self.paths_L = util.get_image_paths(opt['dataroot_L'])
+
+        assert self.paths_H, 'Error: H path is empty.'
+
+    def __getitem__(self, index):
+
+        # ------------------------------------
+        # get H image
+        # ------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+        img_H = util.uint2single(img_H)
+
+        # ------------------------------------
+        # modcrop for SR
+        # ------------------------------------
+        img_H = util.modcrop(img_H, self.sf)
+
+        # ------------------------------------
+        # sythesize L image via matlab's bicubic
+        # ------------------------------------
+        H, W, _ = img_H.shape
+        img_L = util.imresize_np(img_H, 1 / self.sf, True)
+
+        if self.opt['phase'] == 'train':
+            """
+            # --------------------------------
+            # get L/H patch pairs
+            # --------------------------------
+            """
+            H, W, C = img_L.shape
+
+            # --------------------------------
+            # randomly crop L patch
+            # --------------------------------
+            rnd_h = random.randint(0, max(0, H - self.L_size))
+            rnd_w = random.randint(0, max(0, W - self.L_size))
+            img_L = img_L[rnd_h:rnd_h + self.L_size, rnd_w:rnd_w + self.L_size, :]
+
+            # --------------------------------
+            # crop corresponding H patch
+            # --------------------------------
+            rnd_h_H, rnd_w_H = int(rnd_h * self.sf), int(rnd_w * self.sf)
+            img_H = img_H[rnd_h_H:rnd_h_H + self.patch_size, rnd_w_H:rnd_w_H + self.patch_size, :]
+
+            # --------------------------------
+            # augmentation - flip and/or rotate
+            # --------------------------------
+            mode = random.randint(0, 7)
+            img_L, img_H = util.augment_img(img_L, mode=mode), util.augment_img(img_H, mode=mode)
+
+            # --------------------------------
+            # get patch pairs
+            # --------------------------------
+            img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+
+            # --------------------------------
+            # select noise level and get Gaussian noise
+            # --------------------------------
+            if random.random() < 0.1:
+                noise_level = torch.zeros(1).float()
+            else:
+                noise_level = torch.FloatTensor([np.random.uniform(self.sigma_min, self.sigma_max)])/255.0
+                # noise_level = torch.rand(1)*50/255.0
+                # noise_level = torch.min(torch.from_numpy(np.float32([7*np.random.chisquare(2.5)/255.0])),torch.Tensor([50./255.]))
+    
+        else:
+
+            img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+
+            noise_level = torch.FloatTensor([self.sigma_test])
+
+        # ------------------------------------
+        # add noise
+        # ------------------------------------
+        noise = torch.randn(img_L.size()).mul_(noise_level).float()
+        img_L.add_(noise)
+
+        # ------------------------------------
+        # get noise level map M
+        # ------------------------------------
+        M_vector = noise_level.unsqueeze(1).unsqueeze(1)
+        M = M_vector.repeat(1, img_L.size()[-2], img_L.size()[-1])
+
+
+        """
+        # -------------------------------------
+        # concat L and noise level map M
+        # -------------------------------------
+        """
+        img_L = torch.cat((img_L, M), 0)
+
+
+        L_path = H_path
+
+        return {'L': img_L, 'H': img_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_fdncnn.py

@@ -0,0 +1,109 @@
+import random
+import numpy as np
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+class DatasetFDnCNN(data.Dataset):
+    """
+    # -----------------------------------------
+    # Get L/H/M for denosing on AWGN with a range of sigma.
+    # Only dataroot_H is needed.
+    # -----------------------------------------
+    # e.g., FDnCNN, H = f(cat(L, M)), M is noise level map
+    # -----------------------------------------
+    """
+
+    def __init__(self, opt):
+        super(DatasetFDnCNN, self).__init__()
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = self.opt['H_size'] if opt['H_size'] else 64
+        self.sigma = opt['sigma'] if opt['sigma'] else [0, 75]
+        self.sigma_min, self.sigma_max = self.sigma[0], self.sigma[1]
+        self.sigma_test = opt['sigma_test'] if opt['sigma_test'] else 25
+
+        # -------------------------------------
+        # get the path of H, return None if input is None
+        # -------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+
+    def __getitem__(self, index):
+        # -------------------------------------
+        # get H image
+        # -------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+
+        L_path = H_path
+
+        if self.opt['phase'] == 'train':
+            """
+            # --------------------------------
+            # get L/H/M patch pairs
+            # --------------------------------
+            """
+            H, W = img_H.shape[:2]
+
+            # ---------------------------------
+            # randomly crop the patch
+            # ---------------------------------
+            rnd_h = random.randint(0, max(0, H - self.patch_size))
+            rnd_w = random.randint(0, max(0, W - self.patch_size))
+            patch_H = img_H[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size, :]
+
+            # ---------------------------------
+            # augmentation - flip, rotate
+            # ---------------------------------
+            mode = random.randint(0, 7)
+            patch_H = util.augment_img(patch_H, mode=mode)
+
+            # ---------------------------------
+            # HWC to CHW, numpy(uint) to tensor
+            # ---------------------------------
+            img_H = util.uint2tensor3(patch_H)
+            img_L = img_H.clone()
+
+            # ---------------------------------
+            # get noise level
+            # ---------------------------------
+            # noise_level = torch.FloatTensor([np.random.randint(self.sigma_min, self.sigma_max)])/255.0
+            noise_level = torch.FloatTensor([np.random.uniform(self.sigma_min, self.sigma_max)])/255.0
+
+            noise_level_map = torch.ones((1, img_L.size(1), img_L.size(2))).mul_(noise_level).float()  # torch.full((1, img_L.size(1), img_L.size(2)), noise_level)
+
+            # ---------------------------------
+            # add noise
+            # ---------------------------------
+            noise = torch.randn(img_L.size()).mul_(noise_level).float()
+            img_L.add_(noise)
+
+        else:
+            """
+            # --------------------------------
+            # get L/H/M image pairs
+            # --------------------------------
+            """
+            img_H = util.uint2single(img_H)
+            img_L = np.copy(img_H)
+            np.random.seed(seed=0)
+            img_L += np.random.normal(0, self.sigma_test/255.0, img_L.shape)
+            noise_level_map = torch.ones((1, img_L.shape[0], img_L.shape[1])).mul_(self.sigma_test/255.0).float()  # torch.full((1, img_L.size(1), img_L.size(2)), noise_level)
+
+            # ---------------------------------
+            # L/H image pairs
+            # ---------------------------------
+            img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+
+        """
+        # -------------------------------------
+        # concat L and noise level map M
+        # -------------------------------------
+        """
+        img_L = torch.cat((img_L, noise_level_map), 0)
+
+        return {'L': img_L, 'H': img_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_ffdnet.py

@@ -0,0 +1,103 @@
+import random
+import numpy as np
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+class DatasetFFDNet(data.Dataset):
+    """
+    # -----------------------------------------
+    # Get L/H/M for denosing on AWGN with a range of sigma.
+    # Only dataroot_H is needed.
+    # -----------------------------------------
+    # e.g., FFDNet, H = f(L, sigma), sigma is noise level
+    # -----------------------------------------
+    """
+
+    def __init__(self, opt):
+        super(DatasetFFDNet, self).__init__()
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = self.opt['H_size'] if opt['H_size'] else 64
+        self.sigma = opt['sigma'] if opt['sigma'] else [0, 75]
+        self.sigma_min, self.sigma_max = self.sigma[0], self.sigma[1]
+        self.sigma_test = opt['sigma_test'] if opt['sigma_test'] else 25
+
+        # -------------------------------------
+        # get the path of H, return None if input is None
+        # -------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+
+    def __getitem__(self, index):
+        # -------------------------------------
+        # get H image
+        # -------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+
+        L_path = H_path
+
+        if self.opt['phase'] == 'train':
+            """
+            # --------------------------------
+            # get L/H/M patch pairs
+            # --------------------------------
+            """
+            H, W = img_H.shape[:2]
+
+            # ---------------------------------
+            # randomly crop the patch
+            # ---------------------------------
+            rnd_h = random.randint(0, max(0, H - self.patch_size))
+            rnd_w = random.randint(0, max(0, W - self.patch_size))
+            patch_H = img_H[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size, :]
+
+            # ---------------------------------
+            # augmentation - flip, rotate
+            # ---------------------------------
+            mode = random.randint(0, 7)
+            patch_H = util.augment_img(patch_H, mode=mode)
+
+            # ---------------------------------
+            # HWC to CHW, numpy(uint) to tensor
+            # ---------------------------------
+            img_H = util.uint2tensor3(patch_H)
+            img_L = img_H.clone()
+
+            # ---------------------------------
+            # get noise level
+            # ---------------------------------
+            # noise_level = torch.FloatTensor([np.random.randint(self.sigma_min, self.sigma_max)])/255.0
+            noise_level = torch.FloatTensor([np.random.uniform(self.sigma_min, self.sigma_max)])/255.0
+
+            # ---------------------------------
+            # add noise
+            # ---------------------------------
+            noise = torch.randn(img_L.size()).mul_(noise_level).float()
+            img_L.add_(noise)
+
+        else:
+            """
+            # --------------------------------
+            # get L/H/sigma image pairs
+            # --------------------------------
+            """
+            img_H = util.uint2single(img_H)
+            img_L = np.copy(img_H)
+            np.random.seed(seed=0)
+            img_L += np.random.normal(0, self.sigma_test/255.0, img_L.shape)
+            noise_level = torch.FloatTensor([self.sigma_test/255.0])
+
+            # ---------------------------------
+            # L/H image pairs
+            # ---------------------------------
+            img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+
+        noise_level = noise_level.unsqueeze(1).unsqueeze(1)
+
+
+        return {'L': img_L, 'H': img_H, 'C': noise_level, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_jpeg.py

@@ -0,0 +1,118 @@
+import random
+import torch.utils.data as data
+import utils.utils_image as util
+import cv2
+
+
+class DatasetJPEG(data.Dataset):
+    def __init__(self, opt):
+        super(DatasetJPEG, self).__init__()
+        print('Dataset: JPEG compression artifact reduction (deblocking) with quality factor. Only dataroot_H is needed.')
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = self.opt['H_size'] if opt['H_size'] else 128
+
+        self.quality_factor = opt['quality_factor'] if opt['quality_factor'] else 40
+        self.quality_factor_test = opt['quality_factor_test'] if opt['quality_factor_test'] else 40
+        self.is_color = opt['is_color'] if opt['is_color'] else False
+
+        # -------------------------------------
+        # get the path of H, return None if input is None
+        # -------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+
+    def __getitem__(self, index):
+
+        if self.opt['phase'] == 'train':
+            # -------------------------------------
+            # get H image
+            # -------------------------------------
+            H_path = self.paths_H[index]
+            img_H = util.imread_uint(H_path, 3)
+            L_path = H_path
+
+            H, W = img_H.shape[:2]
+            self.patch_size_plus = self.patch_size + 8
+
+            # ---------------------------------
+            # randomly crop a large patch
+            # ---------------------------------
+            rnd_h = random.randint(0, max(0, H - self.patch_size_plus))
+            rnd_w = random.randint(0, max(0, W - self.patch_size_plus))
+            patch_H = img_H[rnd_h:rnd_h + self.patch_size_plus, rnd_w:rnd_w + self.patch_size_plus, ...]
+
+            # ---------------------------------
+            # augmentation - flip, rotate
+            # ---------------------------------
+            mode = random.randint(0, 7)
+            patch_H = util.augment_img(patch_H, mode=mode)
+
+            # ---------------------------------
+            # HWC to CHW, numpy(uint) to tensor
+            # ---------------------------------
+            img_L = patch_H.copy()
+
+            # ---------------------------------
+            # set quality factor
+            # ---------------------------------
+            quality_factor = self.quality_factor
+
+            if self.is_color:  # color image
+                img_H = img_L.copy()
+                img_L = cv2.cvtColor(img_L, cv2.COLOR_RGB2BGR)
+                result, encimg = cv2.imencode('.jpg', img_L, [int(cv2.IMWRITE_JPEG_QUALITY), quality_factor])
+                img_L = cv2.imdecode(encimg, 1)
+                img_L = cv2.cvtColor(img_L, cv2.COLOR_BGR2RGB)
+            else:
+                if random.random() > 0.5:
+                    img_L = util.rgb2ycbcr(img_L)
+                else:
+                    img_L = cv2.cvtColor(img_L, cv2.COLOR_RGB2GRAY)
+                img_H = img_L.copy()
+                result, encimg = cv2.imencode('.jpg', img_L, [int(cv2.IMWRITE_JPEG_QUALITY), quality_factor])
+                img_L = cv2.imdecode(encimg, 0)
+
+            # ---------------------------------
+            # randomly crop a patch
+            # ---------------------------------
+            H, W = img_H.shape[:2]
+            if random.random() > 0.5:
+                rnd_h = random.randint(0, max(0, H - self.patch_size))
+                rnd_w = random.randint(0, max(0, W - self.patch_size))
+            else:
+                rnd_h = 0
+                rnd_w = 0
+            img_H = img_H[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size]
+            img_L = img_L[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size]
+        else:
+
+            H_path = self.paths_H[index]
+            L_path = H_path
+            # ---------------------------------
+            # set quality factor
+            # ---------------------------------
+            quality_factor = self.quality_factor_test
+
+            if self.is_color:  # color JPEG image deblocking
+                img_H = util.imread_uint(H_path, 3)
+                img_L = img_H.copy()
+                img_L = cv2.cvtColor(img_L, cv2.COLOR_RGB2BGR)
+                result, encimg = cv2.imencode('.jpg', img_L, [int(cv2.IMWRITE_JPEG_QUALITY), quality_factor])
+                img_L = cv2.imdecode(encimg, 1)
+                img_L = cv2.cvtColor(img_L, cv2.COLOR_BGR2RGB)
+            else:
+                img_H = cv2.imread(H_path, cv2.IMREAD_UNCHANGED)
+                is_to_ycbcr = True if img_L.ndim == 3 else False
+                if is_to_ycbcr:
+                    img_H = cv2.cvtColor(img_H, cv2.COLOR_BGR2RGB)
+                    img_H = util.rgb2ycbcr(img_H)
+
+                result, encimg = cv2.imencode('.jpg', img_H, [int(cv2.IMWRITE_JPEG_QUALITY), quality_factor])
+                img_L = cv2.imdecode(encimg, 0)
+
+        img_L, img_H = util.uint2tensor3(img_L), util.uint2tensor3(img_H)
+
+        return {'L': img_L, 'H': img_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_l.py

@@ -0,0 +1,43 @@
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+class DatasetL(data.Dataset):
+    '''
+    # -----------------------------------------
+    # Get L in testing.
+    # Only "dataroot_L" is needed.
+    # -----------------------------------------
+    # -----------------------------------------
+    '''
+
+    def __init__(self, opt):
+        super(DatasetL, self).__init__()
+        print('Read L in testing. Only "dataroot_L" is needed.')
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+
+        # ------------------------------------
+        # get the path of L
+        # ------------------------------------
+        self.paths_L = util.get_image_paths(opt['dataroot_L'])
+        assert self.paths_L, 'Error: L paths are empty.'
+
+    def __getitem__(self, index):
+        L_path = None
+
+        # ------------------------------------
+        # get L image
+        # ------------------------------------
+        L_path = self.paths_L[index]
+        img_L = util.imread_uint(L_path, self.n_channels)
+
+        # ------------------------------------
+        # HWC to CHW, numpy to tensor
+        # ------------------------------------
+        img_L = util.uint2tensor3(img_L)
+
+        return {'L': img_L, 'L_path': L_path}
+
+    def __len__(self):
+        return len(self.paths_L)

KAIR/data/dataset_plain.py

@@ -0,0 +1,85 @@
+import random
+import numpy as np
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+class DatasetPlain(data.Dataset):
+    '''
+    # -----------------------------------------
+    # Get L/H for image-to-image mapping.
+    # Both "paths_L" and "paths_H" are needed.
+    # -----------------------------------------
+    # e.g., train denoiser with L and H
+    # -----------------------------------------
+    '''
+
+    def __init__(self, opt):
+        super(DatasetPlain, self).__init__()
+        print('Get L/H for image-to-image mapping. Both "paths_L" and "paths_H" are needed.')
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = self.opt['H_size'] if self.opt['H_size'] else 64
+
+        # ------------------------------------
+        # get the path of L/H
+        # ------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+        self.paths_L = util.get_image_paths(opt['dataroot_L'])
+
+        assert self.paths_H, 'Error: H path is empty.'
+        assert self.paths_L, 'Error: L path is empty. Plain dataset assumes both L and H are given!'
+        if self.paths_L and self.paths_H:
+            assert len(self.paths_L) == len(self.paths_H), 'L/H mismatch - {}, {}.'.format(len(self.paths_L), len(self.paths_H))
+
+    def __getitem__(self, index):
+
+        # ------------------------------------
+        # get H image
+        # ------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+
+        # ------------------------------------
+        # get L image
+        # ------------------------------------
+        L_path = self.paths_L[index]
+        img_L = util.imread_uint(L_path, self.n_channels)
+
+        # ------------------------------------
+        # if train, get L/H patch pair
+        # ------------------------------------
+        if self.opt['phase'] == 'train':
+
+            H, W, _ = img_H.shape
+
+            # --------------------------------
+            # randomly crop the patch
+            # --------------------------------
+            rnd_h = random.randint(0, max(0, H - self.patch_size))
+            rnd_w = random.randint(0, max(0, W - self.patch_size))
+            patch_L = img_L[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size, :]
+            patch_H = img_H[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size, :]
+
+            # --------------------------------
+            # augmentation - flip and/or rotate
+            # --------------------------------
+            mode = random.randint(0, 7)
+            patch_L, patch_H = util.augment_img(patch_L, mode=mode), util.augment_img(patch_H, mode=mode)
+
+            # --------------------------------
+            # HWC to CHW, numpy(uint) to tensor
+            # --------------------------------
+            img_L, img_H = util.uint2tensor3(patch_L), util.uint2tensor3(patch_H)
+
+        else:
+
+            # --------------------------------
+            # HWC to CHW, numpy(uint) to tensor
+            # --------------------------------
+            img_L, img_H = util.uint2tensor3(img_L), util.uint2tensor3(img_H)
+
+        return {'L': img_L, 'H': img_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_plainpatch.py

@@ -0,0 +1,131 @@
+import os.path
+import random
+import numpy as np
+import torch.utils.data as data
+import utils.utils_image as util
+
+
+
+class DatasetPlainPatch(data.Dataset):
+    '''
+    # -----------------------------------------
+    # Get L/H for image-to-image mapping.
+    # Both "paths_L" and "paths_H" are needed.
+    # -----------------------------------------
+    # e.g., train denoiser with L and H patches
+    # create a large patch dataset first
+    # -----------------------------------------
+    '''
+
+    def __init__(self, opt):
+        super(DatasetPlainPatch, self).__init__()
+        print('Get L/H for image-to-image mapping. Both "paths_L" and "paths_H" are needed.')
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = self.opt['H_size'] if self.opt['H_size'] else 64
+
+        self.num_patches_per_image = opt['num_patches_per_image'] if opt['num_patches_per_image'] else 40
+        self.num_sampled = opt['num_sampled'] if opt['num_sampled'] else 3000
+
+        # -------------------
+        # get the path of L/H
+        # -------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+        self.paths_L = util.get_image_paths(opt['dataroot_L'])
+
+        assert self.paths_H, 'Error: H path is empty.'
+        assert self.paths_L, 'Error: L path is empty. This dataset uses L path, you can use dataset_dnpatchh'
+        if self.paths_L and self.paths_H:
+            assert len(self.paths_L) == len(self.paths_H), 'H and L datasets have different number of images - {}, {}.'.format(len(self.paths_L), len(self.paths_H))
+
+        # ------------------------------------
+        # number of sampled images
+        # ------------------------------------
+        self.num_sampled = min(self.num_sampled, len(self.paths_H))
+
+        # ------------------------------------
+        # reserve space with zeros
+        # ------------------------------------
+        self.total_patches = self.num_sampled * self.num_patches_per_image
+        self.H_data = np.zeros([self.total_patches, self.path_size, self.path_size, self.n_channels], dtype=np.uint8)
+        self.L_data = np.zeros([self.total_patches, self.path_size, self.path_size, self.n_channels], dtype=np.uint8)
+
+        # ------------------------------------
+        # update H patches
+        # ------------------------------------
+        self.update_data()
+
+
+    def update_data(self):
+        """
+        # ------------------------------------
+        # update whole L/H patches
+        # ------------------------------------
+        """
+        self.index_sampled = random.sample(range(0, len(self.paths_H), 1), self.num_sampled)
+        n_count = 0
+
+        for i in range(len(self.index_sampled)):
+            L_patches, H_patches = self.get_patches(self.index_sampled[i])
+            for (L_patch, H_patch) in zip(L_patches, H_patches):
+                self.L_data[n_count,:,:,:] = L_patch
+                self.H_data[n_count,:,:,:] = H_patch
+                n_count += 1
+
+        print('Training data updated! Total number of patches is:  %5.2f X %5.2f = %5.2f\n' % (len(self.H_data)//128, 128, len(self.H_data)))
+
+    def get_patches(self, index):
+        """
+        # ------------------------------------
+        # get L/H patches from L/H images
+        # ------------------------------------
+        """
+        L_path = self.paths_L[index]
+        H_path = self.paths_H[index]
+        img_L = util.imread_uint(L_path, self.n_channels)  # uint format
+        img_H = util.imread_uint(H_path, self.n_channels)  # uint format
+
+        H, W = img_H.shape[:2]
+
+        L_patches, H_patches = [], []
+
+        num = self.num_patches_per_image
+        for _ in range(num):
+            rnd_h = random.randint(0, max(0, H - self.path_size))
+            rnd_w = random.randint(0, max(0, W - self.path_size))
+            L_patch = img_L[rnd_h:rnd_h + self.path_size, rnd_w:rnd_w + self.path_size, :]
+            H_patch = img_H[rnd_h:rnd_h + self.path_size, rnd_w:rnd_w + self.path_size, :]
+            L_patches.append(L_patch)
+            H_patches.append(H_patch)
+
+        return L_patches, H_patches
+
+    def __getitem__(self, index):
+
+        if self.opt['phase'] == 'train':
+
+            patch_L, patch_H = self.L_data[index], self.H_data[index]
+
+            # --------------------------------
+            # augmentation - flip and/or rotate
+            # --------------------------------
+            mode = random.randint(0, 7)
+            patch_L = util.augment_img(patch_L, mode=mode)
+            patch_H = util.augment_img(patch_H, mode=mode)
+
+            patch_L, patch_H = util.uint2tensor3(patch_L), util.uint2tensor3(patch_H)
+
+        else:
+
+            L_path, H_path = self.paths_L[index], self.paths_H[index]
+            patch_L = util.imread_uint(L_path, self.n_channels)
+            patch_H = util.imread_uint(H_path, self.n_channels)
+
+            patch_L, patch_H = util.uint2tensor3(patch_L), util.uint2tensor3(patch_H)
+
+        return {'L': patch_L, 'H': patch_H}
+
+
+    def __len__(self):
+        
+        return self.total_patches

KAIR/data/dataset_sr.py

@@ -0,0 +1,197 @@
+import math
+import numpy as np
+import random
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+from basicsr.data.degradations import circular_lowpass_kernel, random_mixed_kernels
+from basicsr.utils import DiffJPEG, USMSharp
+from numpy.typing import NDArray
+from PIL import Image
+from utils.utils_video import img2tensor
+from torch import Tensor
+
+from data.degradations import apply_real_esrgan_degradations
+
+class DatasetSR(data.Dataset):
+    '''
+    # -----------------------------------------
+    # Get L/H for SISR.
+    # If only "paths_H" is provided, sythesize bicubicly downsampled L on-the-fly.
+    # -----------------------------------------
+    # e.g., SRResNet
+    # -----------------------------------------
+    '''
+
+    def __init__(self, opt):
+        super(DatasetSR, self).__init__()
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.sf = opt['scale'] if opt['scale'] else 4
+        self.patch_size = self.opt['H_size'] if self.opt['H_size'] else 96
+        self.L_size = self.patch_size // self.sf
+
+        # ------------------------------------
+        # get paths of L/H
+        # ------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+        self.paths_L = util.get_image_paths(opt['dataroot_L'])
+
+        assert self.paths_H, 'Error: H path is empty.'
+        if self.paths_L and self.paths_H:
+            assert len(self.paths_L) == len(self.paths_H), 'L/H mismatch - {}, {}.'.format(len(self.paths_L), len(self.paths_H))
+
+        self.jpeg_simulator = DiffJPEG()
+        self.usm_sharpener = USMSharp()
+
+    blur_kernel_list1 = ['iso', 'aniso', 'generalized_iso',
+                         'generalized_aniso', 'plateau_iso', 'plateau_aniso']
+    blur_kernel_list2 = ['iso', 'aniso', 'generalized_iso',
+                         'generalized_aniso', 'plateau_iso', 'plateau_aniso']
+    blur_kernel_prob1 = [0.45, 0.25, 0.12, 0.03, 0.12, 0.03]
+    blur_kernel_prob2 = [0.45, 0.25, 0.12, 0.03, 0.12, 0.03]
+    kernel_size = 21
+    blur_sigma1 = [0.05, 0.2]
+    blur_sigma2 = [0.05, 0.1]
+    betag_range1 = [0.7, 1.3]
+    betag_range2 = [0.7, 1.3]
+    betap_range1 = [0.7, 1.3]
+    betap_range2 = [0.7, 1.3]
+
+    def _decide_kernels(self) -> NDArray:
+        blur_kernel1 = random_mixed_kernels(
+            self.blur_kernel_list1,
+            self.blur_kernel_prob1,
+            self.kernel_size,
+            self.blur_sigma1,
+            self.blur_sigma1, [-math.pi, math.pi],
+            self.betag_range1,
+            self.betap_range1,
+            noise_range=None
+        )
+        blur_kernel2 = random_mixed_kernels(
+            self.blur_kernel_list2,
+            self.blur_kernel_prob2,
+            self.kernel_size,
+            self.blur_sigma2,
+            self.blur_sigma2, [-math.pi, math.pi],
+            self.betag_range2,
+            self.betap_range2,
+            noise_range=None
+        )
+        if self.kernel_size < 13:
+            omega_c = np.random.uniform(np.pi / 3, np.pi)
+        else:
+            omega_c = np.random.uniform(np.pi / 5, np.pi)
+        sinc_kernel = circular_lowpass_kernel(omega_c, self.kernel_size, pad_to=21)
+        return (blur_kernel1, blur_kernel2, sinc_kernel)
+
+    def __getitem__(self, index):
+
+        L_path = None
+        # ------------------------------------
+        # get H image
+        # ------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+        img_H = util.uint2single(img_H)
+
+        # ------------------------------------
+        # modcrop
+        # ------------------------------------
+        img_H = util.modcrop(img_H, self.sf)
+
+        # ------------------------------------
+        # get L image
+        # ------------------------------------
+        if self.paths_L:
+            # --------------------------------
+            # directly load L image
+            # --------------------------------
+            L_path = self.paths_L[index]
+            img_L = util.imread_uint(L_path, self.n_channels)
+            img_L = util.uint2single(img_L)
+
+        else:
+            # --------------------------------
+            # sythesize L image via matlab's bicubic
+            # --------------------------------
+            H, W = img_H.shape[:2]
+            img_L = util.imresize_np(img_H, 1 / self.sf, True)
+
+        src_tensor = img2tensor(img_L.copy(), bgr2rgb=False,
+                                float32=True).unsqueeze(0)
+
+        blur_kernel1, blur_kernel2, sinc_kernel = self._decide_kernels()
+        (img_L_2, sharp_img_L, degraded_img_L) = apply_real_esrgan_degradations(
+            src_tensor,
+            blur_kernel1=Tensor(blur_kernel1).unsqueeze(0),
+            blur_kernel2=Tensor(blur_kernel2).unsqueeze(0),
+            second_blur_prob=0.2,
+            sinc_kernel=Tensor(sinc_kernel).unsqueeze(0),
+            resize_prob1=[0.2, 0.7, 0.1],
+            resize_prob2=[0.3, 0.4, 0.3],
+            resize_range1=[0.9, 1.1],
+            resize_range2=[0.9, 1.1],
+            gray_noise_prob1=0.2,
+            gray_noise_prob2=0.2,
+            gaussian_noise_prob1=0.2,
+            gaussian_noise_prob2=0.2,
+            noise_range=[0.01, 0.2],
+            poisson_scale_range=[0.05, 0.45],
+            jpeg_compression_range1=[85, 100],
+            jpeg_compression_range2=[85, 100],
+            jpeg_simulator=self.jpeg_simulator,
+            random_crop_gt_size=256,
+            sr_upsample_scale=1,
+            usm_sharpener=self.usm_sharpener
+        )
+        # Image.fromarray((degraded_img_L[0] * 255).permute(
+        #     1, 2, 0).cpu().numpy().astype(np.uint8)).save(
+        #     "/home/cll/Desktop/degraded_L.png")
+        # Image.fromarray((img_L * 255).astype(np.uint8)).save(
+        #     "/home/cll/Desktop/img_L.png")
+        # Image.fromarray((img_L_2[0] * 255).permute(
+        #     1, 2, 0).cpu().numpy().astype(np.uint8)).save(
+        #     "/home/cll/Desktop/img_L_2.png")
+        # exit()
+
+        # ------------------------------------
+        # if train, get L/H patch pair
+        # ------------------------------------
+        if self.opt['phase'] == 'train':
+
+            H, W, C = img_L.shape
+
+            # --------------------------------
+            # randomly crop the L patch
+            # --------------------------------
+            rnd_h = random.randint(0, max(0, H - self.L_size))
+            rnd_w = random.randint(0, max(0, W - self.L_size))
+            img_L = img_L[rnd_h:rnd_h + self.L_size, rnd_w:rnd_w + self.L_size, :]
+
+            # --------------------------------
+            # crop corresponding H patch
+            # --------------------------------
+            rnd_h_H, rnd_w_H = int(rnd_h * self.sf), int(rnd_w * self.sf)
+            img_H = img_H[rnd_h_H:rnd_h_H + self.patch_size, rnd_w_H:rnd_w_H + self.patch_size, :]
+
+            # --------------------------------
+            # augmentation - flip and/or rotate + RealESRGAN modified degradations
+            # --------------------------------
+            mode = random.randint(0, 7)
+            img_L, img_H = util.augment_img(img_L, mode=mode), util.augment_img(img_H, mode=mode)
+
+
+        # ------------------------------------
+        # L/H pairs, HWC to CHW, numpy to tensor
+        # ------------------------------------
+        img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+
+        if L_path is None:
+            L_path = H_path
+
+        return {'L': img_L, 'H': img_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_srmd.py

@@ -0,0 +1,155 @@
+import random
+import numpy as np
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+from utils import utils_sisr
+
+
+import hdf5storage
+import os
+
+
+class DatasetSRMD(data.Dataset):
+    '''
+    # -----------------------------------------
+    # Get L/H/M for noisy image SR with Gaussian kernels.
+    # Only "paths_H" is needed, sythesize bicubicly downsampled L on-the-fly.
+    # -----------------------------------------
+    # e.g., SRMD, H = f(L, kernel, sigma), sigma is noise level
+    # -----------------------------------------
+    '''
+
+    def __init__(self, opt):
+        super(DatasetSRMD, self).__init__()
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.sf = opt['scale'] if opt['scale'] else 4
+        self.patch_size = self.opt['H_size'] if self.opt['H_size'] else 96
+        self.L_size = self.patch_size // self.sf
+        self.sigma = opt['sigma'] if opt['sigma'] else [0, 50]
+        self.sigma_min, self.sigma_max = self.sigma[0], self.sigma[1]
+        self.sigma_test = opt['sigma_test'] if opt['sigma_test'] else 0
+
+        # -------------------------------------
+        # PCA projection matrix
+        # -------------------------------------
+        self.p = hdf5storage.loadmat(os.path.join('kernels', 'srmd_pca_pytorch.mat'))['p']
+        self.ksize = int(np.sqrt(self.p.shape[-1]))  # kernel size
+
+        # ------------------------------------
+        # get paths of L/H
+        # ------------------------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])
+        self.paths_L = util.get_image_paths(opt['dataroot_L'])
+
+    def __getitem__(self, index):
+
+        # ------------------------------------
+        # get H image
+        # ------------------------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+        img_H = util.uint2single(img_H)
+
+        # ------------------------------------
+        # modcrop for SR
+        # ------------------------------------
+        img_H = util.modcrop(img_H, self.sf)
+
+        # ------------------------------------
+        # kernel
+        # ------------------------------------
+        if self.opt['phase'] == 'train':
+            l_max = 10
+            theta = np.pi*random.random()
+            l1 = 0.1+l_max*random.random()
+            l2 = 0.1+(l1-0.1)*random.random()
+
+            kernel = utils_sisr.anisotropic_Gaussian(ksize=self.ksize, theta=theta, l1=l1, l2=l2)
+        else:
+            kernel = utils_sisr.anisotropic_Gaussian(ksize=self.ksize, theta=np.pi, l1=0.1, l2=0.1)
+
+        k = np.reshape(kernel, (-1), order="F")
+        k_reduced = np.dot(self.p, k)
+        k_reduced = torch.from_numpy(k_reduced).float()
+
+        # ------------------------------------
+        # sythesize L image via specified degradation model
+        # ------------------------------------
+        H, W, _ = img_H.shape
+        img_L = utils_sisr.srmd_degradation(img_H, kernel, self.sf)
+        img_L = np.float32(img_L)
+
+        if self.opt['phase'] == 'train':
+            """
+            # --------------------------------
+            # get L/H patch pairs
+            # --------------------------------
+            """
+            H, W, C = img_L.shape
+
+            # --------------------------------
+            # randomly crop L patch
+            # --------------------------------
+            rnd_h = random.randint(0, max(0, H - self.L_size))
+            rnd_w = random.randint(0, max(0, W - self.L_size))
+            img_L = img_L[rnd_h:rnd_h + self.L_size, rnd_w:rnd_w + self.L_size, :]
+
+            # --------------------------------
+            # crop corresponding H patch
+            # --------------------------------
+            rnd_h_H, rnd_w_H = int(rnd_h * self.sf), int(rnd_w * self.sf)
+            img_H = img_H[rnd_h_H:rnd_h_H + self.patch_size, rnd_w_H:rnd_w_H + self.patch_size, :]
+
+            # --------------------------------
+            # augmentation - flip and/or rotate
+            # --------------------------------
+            mode = random.randint(0, 7)
+            img_L, img_H = util.augment_img(img_L, mode=mode), util.augment_img(img_H, mode=mode)
+
+            # --------------------------------
+            # get patch pairs
+            # --------------------------------
+            img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+
+            # --------------------------------
+            # select noise level and get Gaussian noise
+            # --------------------------------
+            if random.random() < 0.1:
+                noise_level = torch.zeros(1).float()
+            else:
+                noise_level = torch.FloatTensor([np.random.uniform(self.sigma_min, self.sigma_max)])/255.0
+                # noise_level = torch.rand(1)*50/255.0
+                # noise_level = torch.min(torch.from_numpy(np.float32([7*np.random.chisquare(2.5)/255.0])),torch.Tensor([50./255.]))
+    
+        else:
+
+            img_H, img_L = util.single2tensor3(img_H), util.single2tensor3(img_L)
+            noise_level = noise_level = torch.FloatTensor([self.sigma_test])
+
+        # ------------------------------------
+        # add noise
+        # ------------------------------------
+        noise = torch.randn(img_L.size()).mul_(noise_level).float()
+        img_L.add_(noise)
+
+        # ------------------------------------
+        # get degradation map M
+        # ------------------------------------
+        M_vector = torch.cat((k_reduced, noise_level), 0).unsqueeze(1).unsqueeze(1)
+        M = M_vector.repeat(1, img_L.size()[-2], img_L.size()[-1])
+
+        """
+        # -------------------------------------
+        # concat L and noise level map M
+        # -------------------------------------
+        """
+
+        img_L = torch.cat((img_L, M), 0)
+        L_path = H_path
+
+        return {'L': img_L, 'H': img_H, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_usrnet.py

@@ -0,0 +1,126 @@
+import random
+
+import numpy as np
+import torch
+import torch.utils.data as data
+import utils.utils_image as util
+from utils import utils_deblur
+from utils import utils_sisr
+import os
+
+from scipy import ndimage
+from scipy.io import loadmat
+# import hdf5storage
+
+
+class DatasetUSRNet(data.Dataset):
+    '''
+    # -----------------------------------------
+    # Get L/k/sf/sigma for USRNet.
+    # Only "paths_H" and kernel is needed, synthesize L on-the-fly.
+    # -----------------------------------------
+    '''
+    def __init__(self, opt):
+        super(DatasetUSRNet, self).__init__()
+        self.opt = opt
+        self.n_channels = opt['n_channels'] if opt['n_channels'] else 3
+        self.patch_size = self.opt['H_size'] if self.opt['H_size'] else 96
+        self.sigma_max = self.opt['sigma_max'] if self.opt['sigma_max'] is not None else 25
+        self.scales = opt['scales'] if opt['scales'] is not None else [1,2,3,4]
+        self.sf_validation = opt['sf_validation'] if opt['sf_validation'] is not None else 3
+        #self.kernels = hdf5storage.loadmat(os.path.join('kernels', 'kernels_12.mat'))['kernels']
+        self.kernels = loadmat(os.path.join('kernels', 'kernels_12.mat'))['kernels']  # for validation
+
+        # -------------------
+        # get the path of H
+        # -------------------
+        self.paths_H = util.get_image_paths(opt['dataroot_H'])  # return None if input is None
+        self.count = 0
+
+    def __getitem__(self, index):
+
+        # -------------------
+        # get H image
+        # -------------------
+        H_path = self.paths_H[index]
+        img_H = util.imread_uint(H_path, self.n_channels)
+        L_path = H_path
+
+        if self.opt['phase'] == 'train':
+
+            # ---------------------------
+            # 1) scale factor, ensure each batch only involves one scale factor
+            # ---------------------------
+            if self.count % self.opt['dataloader_batch_size'] == 0:
+                # sf = random.choice([1,2,3,4])
+                self.sf = random.choice(self.scales)
+                # self.count = 0  # optional
+            self.count += 1
+            H, W, _ = img_H.shape
+
+            # ----------------------------
+            # randomly crop the patch
+            # ----------------------------
+            rnd_h = random.randint(0, max(0, H - self.patch_size))
+            rnd_w = random.randint(0, max(0, W - self.patch_size))
+            patch_H = img_H[rnd_h:rnd_h + self.patch_size, rnd_w:rnd_w + self.patch_size, :]
+
+            # ---------------------------
+            # augmentation - flip, rotate
+            # ---------------------------
+            mode = np.random.randint(0, 8)
+            patch_H = util.augment_img(patch_H, mode=mode)
+
+            # ---------------------------
+            # 2) kernel
+            # ---------------------------
+            r_value = random.randint(0, 7)
+            if r_value>3:
+                k = utils_deblur.blurkernel_synthesis(h=25)  # motion blur
+            else:
+                sf_k = random.choice(self.scales)
+                k = utils_sisr.gen_kernel(scale_factor=np.array([sf_k, sf_k]))  # Gaussian blur
+                mode_k = random.randint(0, 7)
+                k = util.augment_img(k, mode=mode_k)
+
+            # ---------------------------
+            # 3) noise level
+            # ---------------------------
+            if random.randint(0, 8) == 1:
+                noise_level = 0/255.0
+            else:
+                noise_level = np.random.randint(0, self.sigma_max)/255.0
+
+            # ---------------------------
+            # Low-quality image
+            # ---------------------------
+            img_L = ndimage.filters.convolve(patch_H, np.expand_dims(k, axis=2), mode='wrap')
+            img_L = img_L[0::self.sf, 0::self.sf, ...]
+            # add Gaussian noise
+            img_L = util.uint2single(img_L) + np.random.normal(0, noise_level, img_L.shape)
+            img_H = patch_H
+
+        else:
+
+            k = self.kernels[0, 0].astype(np.float64)  # validation kernel
+            k /= np.sum(k)
+            noise_level = 0./255.0  # validation noise level
+
+            # ------------------------------------
+            # modcrop
+            # ------------------------------------
+            img_H = util.modcrop(img_H, self.sf_validation)
+
+            img_L = ndimage.filters.convolve(img_H, np.expand_dims(k, axis=2), mode='wrap')  # blur
+            img_L = img_L[0::self.sf_validation, 0::self.sf_validation, ...]  # downsampling
+            img_L = util.uint2single(img_L) + np.random.normal(0, noise_level, img_L.shape)
+            self.sf = self.sf_validation
+
+        k = util.single2tensor3(np.expand_dims(np.float32(k), axis=2))
+        img_H, img_L = util.uint2tensor3(img_H), util.single2tensor3(img_L)
+        noise_level = torch.FloatTensor([noise_level]).view([1,1,1])
+
+        return {'L': img_L, 'H': img_H, 'k': k, 'sigma': noise_level, 'sf': self.sf, 'L_path': L_path, 'H_path': H_path}
+
+    def __len__(self):
+        return len(self.paths_H)

KAIR/data/dataset_video_test.py

@@ -0,0 +1,382 @@
+import glob
+import torch
+from os import path as osp
+import torch.utils.data as data
+
+import utils.utils_video as utils_video
+
+
+class VideoRecurrentTestDataset(data.Dataset):
+    """Video test dataset for recurrent architectures, which takes LR video
+    frames as input and output corresponding HR video frames. Modified from
+    https://github.com/xinntao/BasicSR/blob/master/basicsr/data/reds_dataset.py
+
+    Supported datasets: Vid4, REDS4, REDSofficial.
+    More generally, it supports testing dataset with following structures:
+
+    dataroot
+    ├── subfolder1
+        ├── frame000
+        ├── frame001
+        ├── ...
+    ├── subfolder1
+        ├── frame000
+        ├── frame001
+        ├── ...
+    ├── ...
+
+    For testing datasets, there is no need to prepare LMDB files.
+
+    Args:
+        opt (dict): Config for train dataset. It contains the following keys:
+            dataroot_gt (str): Data root path for gt.
+            dataroot_lq (str): Data root path for lq.
+            io_backend (dict): IO backend type and other kwarg.
+            cache_data (bool): Whether to cache testing datasets.
+            name (str): Dataset name.
+            meta_info_file (str): The path to the file storing the list of test
+                folders. If not provided, all the folders in the dataroot will
+                be used.
+            num_frame (int): Window size for input frames.
+            padding (str): Padding mode.
+    """
+
+    def __init__(self, opt):
+        super(VideoRecurrentTestDataset, self).__init__()
+        self.opt = opt
+        self.cache_data = opt['cache_data']
+        self.gt_root, self.lq_root = opt['dataroot_gt'], opt['dataroot_lq']
+        self.data_info = {'lq_path': [], 'gt_path': [], 'folder': [], 'idx': [], 'border': []}
+
+        self.imgs_lq, self.imgs_gt = {}, {}
+        if 'meta_info_file' in opt:
+            with open(opt['meta_info_file'], 'r') as fin:
+                subfolders = [line.split(' ')[0] for line in fin]
+                subfolders_lq = [osp.join(self.lq_root, key) for key in subfolders]
+                subfolders_gt = [osp.join(self.gt_root, key) for key in subfolders]
+        else:
+            subfolders_lq = sorted(glob.glob(osp.join(self.lq_root, '*')))
+            subfolders_gt = sorted(glob.glob(osp.join(self.gt_root, '*')))
+
+        for subfolder_lq, subfolder_gt in zip(subfolders_lq, subfolders_gt):
+            # get frame list for lq and gt
+            subfolder_name = osp.basename(subfolder_lq)
+            img_paths_lq = sorted(list(utils_video.scandir(subfolder_lq, full_path=True)))
+            img_paths_gt = sorted(list(utils_video.scandir(subfolder_gt, full_path=True)))
+
+            max_idx = len(img_paths_lq)
+            assert max_idx == len(img_paths_gt), (f'Different number of images in lq ({max_idx})'
+                                                  f' and gt folders ({len(img_paths_gt)})')
+
+            self.data_info['lq_path'].extend(img_paths_lq)
+            self.data_info['gt_path'].extend(img_paths_gt)
+            self.data_info['folder'].extend([subfolder_name] * max_idx)
+            for i in range(max_idx):
+                self.data_info['idx'].append(f'{i}/{max_idx}')
+            border_l = [0] * max_idx
+            for i in range(self.opt['num_frame'] // 2):
+                border_l[i] = 1
+                border_l[max_idx - i - 1] = 1
+            self.data_info['border'].extend(border_l)
+
+            # cache data or save the frame list
+            if self.cache_data:
+                print(f'Cache {subfolder_name} for VideoTestDataset...')
+                self.imgs_lq[subfolder_name] = utils_video.read_img_seq(img_paths_lq)
+                self.imgs_gt[subfolder_name] = utils_video.read_img_seq(img_paths_gt)
+            else:
+                self.imgs_lq[subfolder_name] = img_paths_lq
+                self.imgs_gt[subfolder_name] = img_paths_gt
+
+        # Find unique folder strings
+        self.folders = sorted(list(set(self.data_info['folder'])))
+        self.sigma = opt['sigma'] / 255. if 'sigma' in opt else 0 # for non-blind video denoising
+
+    def __getitem__(self, index):
+        folder = self.folders[index]
+
+        if self.sigma:
+        # for non-blind video denoising
+            if self.cache_data:
+                imgs_gt = self.imgs_gt[folder]
+            else:
+                imgs_gt = utils_video.read_img_seq(self.imgs_gt[folder])
+
+            torch.manual_seed(0)
+            noise_level = torch.ones((1, 1, 1, 1)) * self.sigma
+            noise = torch.normal(mean=0, std=noise_level.expand_as(imgs_gt))
+            imgs_lq = imgs_gt + noise
+            t, _, h, w = imgs_lq.shape
+            imgs_lq = torch.cat([imgs_lq, noise_level.expand(t, 1, h, w)], 1)
+        else:
+        # for video sr and deblurring
+            if self.cache_data:
+                imgs_lq = self.imgs_lq[folder]
+                imgs_gt = self.imgs_gt[folder]
+            else:
+                imgs_lq = utils_video.read_img_seq(self.imgs_lq[folder])
+                imgs_gt = utils_video.read_img_seq(self.imgs_gt[folder])
+
+        return {
+            'L': imgs_lq,
+            'H': imgs_gt,
+            'folder': folder,
+            'lq_path': self.imgs_lq[folder],
+        }
+
+    def __len__(self):
+        return len(self.folders)
+
+
+class SingleVideoRecurrentTestDataset(data.Dataset):
+    """Single ideo test dataset for recurrent architectures, which takes LR video
+    frames as input and output corresponding HR video frames (only input LQ path).
+
+    More generally, it supports testing dataset with following structures:
+
+    dataroot
+    ├── subfolder1
+        ├── frame000
+        ├── frame001
+        ├── ...
+    ├── subfolder1
+        ├── frame000
+        ├── frame001
+        ├── ...
+    ├── ...
+
+    For testing datasets, there is no need to prepare LMDB files.
+
+    Args:
+        opt (dict): Config for train dataset. It contains the following keys:
+            dataroot_gt (str): Data root path for gt.
+            dataroot_lq (str): Data root path for lq.
+            io_backend (dict): IO backend type and other kwarg.
+            cache_data (bool): Whether to cache testing datasets.
+            name (str): Dataset name.
+            meta_info_file (str): The path to the file storing the list of test
+                folders. If not provided, all the folders in the dataroot will
+                be used.
+            num_frame (int): Window size for input frames.
+            padding (str): Padding mode.
+    """
+
+    def __init__(self, opt):
+        super(SingleVideoRecurrentTestDataset, self).__init__()
+        self.opt = opt
+        self.cache_data = opt['cache_data']
+        self.lq_root = opt['dataroot_lq']
+        self.data_info = {'lq_path': [], 'folder': [], 'idx': [], 'border': []}
+
+        self.imgs_lq = {}
+        if 'meta_info_file' in opt:
+            with open(opt['meta_info_file'], 'r') as fin:
+                subfolders = [line.split(' ')[0] for line in fin]
+                subfolders_lq = [osp.join(self.lq_root, key) for key in subfolders]
+        else:
+            subfolders_lq = sorted(glob.glob(osp.join(self.lq_root, '*')))
+
+        for subfolder_lq in subfolders_lq:
+            # get frame list for lq and gt
+            subfolder_name = osp.basename(subfolder_lq)
+            img_paths_lq = sorted(list(utils_video.scandir(subfolder_lq, full_path=True)))
+
+            max_idx = len(img_paths_lq)
+
+            self.data_info['lq_path'].extend(img_paths_lq)
+            self.data_info['folder'].extend([subfolder_name] * max_idx)
+            for i in range(max_idx):
+                self.data_info['idx'].append(f'{i}/{max_idx}')
+            border_l = [0] * max_idx
+            for i in range(self.opt['num_frame'] // 2):
+                border_l[i] = 1
+                border_l[max_idx - i - 1] = 1
+            self.data_info['border'].extend(border_l)
+
+            # cache data or save the frame list
+            if self.cache_data:
+                print(f'Cache {subfolder_name} for VideoTestDataset...')
+                self.imgs_lq[subfolder_name] = utils_video.read_img_seq(img_paths_lq)
+            else:
+                self.imgs_lq[subfolder_name] = img_paths_lq
+
+        # Find unique folder strings
+        self.folders = sorted(list(set(self.data_info['folder'])))
+
+    def __getitem__(self, index):
+        folder = self.folders[index]
+
+        if self.cache_data:
+            imgs_lq = self.imgs_lq[folder]
+        else:
+            imgs_lq = utils_video.read_img_seq(self.imgs_lq[folder])
+
+        return {
+            'L': imgs_lq,
+            'folder': folder,
+            'lq_path': self.imgs_lq[folder],
+        }
+
+    def __len__(self):
+        return len(self.folders)
+
+
+class VideoTestVimeo90KDataset(data.Dataset):
+    """Video test dataset for Vimeo90k-Test dataset.
+
+    It only keeps the center frame for testing.
+    For testing datasets, there is no need to prepare LMDB files.
+
+    Args:
+        opt (dict): Config for train dataset. It contains the following keys:
+            dataroot_gt (str): Data root path for gt.
+            dataroot_lq (str): Data root path for lq.
+            io_backend (dict): IO backend type and other kwarg.
+            cache_data (bool): Whether to cache testing datasets.
+            name (str): Dataset name.
+            meta_info_file (str): The path to the file storing the list of test
+                folders. If not provided, all the folders in the dataroot will
+                be used.
+            num_frame (int): Window size for input frames.
+            padding (str): Padding mode.
+    """
+
+    def __init__(self, opt):
+        super(VideoTestVimeo90KDataset, self).__init__()
+        self.opt = opt
+        self.cache_data = opt['cache_data']
+        if self.cache_data:
+            raise NotImplementedError('cache_data in Vimeo90K-Test dataset is not implemented.')
+        self.gt_root, self.lq_root = opt['dataroot_gt'], opt['dataroot_lq']
+        self.data_info = {'lq_path': [], 'gt_path': [], 'folder': [], 'idx': [], 'border': []}
+        neighbor_list = [i + (9 - opt['num_frame']) // 2 for i in range(opt['num_frame'])]
+
+        with open(opt['meta_info_file'], 'r') as fin:
+            subfolders = [line.split(' ')[0] for line in fin]
+        for idx, subfolder in enumerate(subfolders):
+            gt_path = osp.join(self.gt_root, subfolder, 'im4.png')
+            self.data_info['gt_path'].append(gt_path)
+            lq_paths = [osp.join(self.lq_root, subfolder, f'im{i}.png') for i in neighbor_list]
+            self.data_info['lq_path'].append(lq_paths)
+            self.data_info['folder'].append('vimeo90k')
+            self.data_info['idx'].append(f'{idx}/{len(subfolders)}')
+            self.data_info['border'].append(0)
+
+        self.pad_sequence = opt.get('pad_sequence', False)
+
+    def __getitem__(self, index):
+        lq_path = self.data_info['lq_path'][index]
+        gt_path = self.data_info['gt_path'][index]
+        imgs_lq = utils_video.read_img_seq(lq_path)
+        img_gt = utils_video.read_img_seq([gt_path])
+        img_gt.squeeze_(0)
+
+        if self.pad_sequence:  # pad the sequence: 7 frames to 8 frames
+            imgs_lq = torch.cat([imgs_lq, imgs_lq[-1:,...]], dim=0)
+
+        return {
+            'L': imgs_lq,  # (t, c, h, w)
+            'H': img_gt,  # (c, h, w)
+            'folder': self.data_info['folder'][index],  # folder name
+            'idx': self.data_info['idx'][index],  # e.g., 0/843
+            'border': self.data_info['border'][index],  # 0 for non-border
+            'lq_path': lq_path[self.opt['num_frame'] // 2]  # center frame
+        }
+
+    def __len__(self):
+        return len(self.data_info['gt_path'])
+
+
+class SingleVideoRecurrentTestDataset(data.Dataset):
+    """Single Video test dataset (only input LQ path).
+
+    Supported datasets: Vid4, REDS4, REDSofficial.
+    More generally, it supports testing dataset with following structures:
+
+    dataroot
+    ├── subfolder1
+        ├── frame000
+        ├── frame001
+        ├── ...
+    ├── subfolder1
+        ├── frame000
+        ├── frame001
+        ├── ...
+    ├── ...
+
+    For testing datasets, there is no need to prepare LMDB files.
+
+    Args:
+        opt (dict): Config for train dataset. It contains the following keys:
+            dataroot_gt (str): Data root path for gt.
+            dataroot_lq (str): Data root path for lq.
+            io_backend (dict): IO backend type and other kwarg.
+            cache_data (bool): Whether to cache testing datasets.
+            name (str): Dataset name.
+            meta_info_file (str): The path to the file storing the list of test
+                folders. If not provided, all the folders in the dataroot will
+                be used.
+            num_frame (int): Window size for input frames.
+            padding (str): Padding mode.
+    """
+
+    def __init__(self, opt):
+        super(SingleVideoRecurrentTestDataset, self).__init__()
+        self.opt = opt
+        self.cache_data = opt['cache_data']
+        self.lq_root = opt['dataroot_lq']
+        self.data_info = {'lq_path': [], 'folder': [], 'idx': [], 'border': []}
+        # file client (io backend)
+        self.file_client = None
+
+        self.imgs_lq = {}
+        if 'meta_info_file' in opt:
+            with open(opt['meta_info_file'], 'r') as fin:
+                subfolders = [line.split(' ')[0] for line in fin]
+                subfolders_lq = [osp.join(self.lq_root, key) for key in subfolders]
+        else:
+            subfolders_lq = sorted(glob.glob(osp.join(self.lq_root, '*')))
+
+        for subfolder_lq in subfolders_lq:
+            # get frame list for lq and gt
+            subfolder_name = osp.basename(subfolder_lq)
+            img_paths_lq = sorted(list(utils_video.scandir(subfolder_lq, full_path=True)))
+
+            max_idx = len(img_paths_lq)
+
+            self.data_info['lq_path'].extend(img_paths_lq)
+            self.data_info['folder'].extend([subfolder_name] * max_idx)
+            for i in range(max_idx):
+                self.data_info['idx'].append(f'{i}/{max_idx}')
+            border_l = [0] * max_idx
+            for i in range(self.opt['num_frame'] // 2):
+                border_l[i] = 1
+                border_l[max_idx - i - 1] = 1
+            self.data_info['border'].extend(border_l)
+
+            # cache data or save the frame list
+            if self.cache_data:
+                logger.info(f'Cache {subfolder_name} for VideoTestDataset...')
+                self.imgs_lq[subfolder_name] = utils_video.read_img_seq(img_paths_lq)
+            else:
+                self.imgs_lq[subfolder_name] = img_paths_lq
+
+        # Find unique folder strings
+        self.folders = sorted(list(set(self.data_info['folder'])))
+
+    def __getitem__(self, index):
+        folder = self.folders[index]
+
+        if self.cache_data:
+            imgs_lq = self.imgs_lq[folder]
+        else:
+            imgs_lq = utils_video.read_img_seq(self.imgs_lq[folder])
+
+        return {
+            'L': imgs_lq,
+            'folder': folder,
+            'lq_path': self.imgs_lq[folder],
+        }
+
+    def __len__(self):
+        return len(self.folders)

KAIR/data/dataset_video_train.py

@@ -0,0 +1,390 @@
+import numpy as np
+import os
+import random
+import torch
+from pathlib import Path
+import torch.utils.data as data
+
+import utils.utils_video as utils_video
+
+
+class VideoRecurrentTrainDataset(data.Dataset):
+    """Video dataset for training recurrent networks.
+
+    The keys are generated from a meta info txt file.
+    basicsr/data/meta_info/meta_info_XXX_GT.txt
+
+    Each line contains:
+    1. subfolder (clip) name; 2. frame number; 3. image shape, separated by
+    a white space.
+    Examples:
+    720p_240fps_1 100 (720,1280,3)
+    720p_240fps_3 100 (720,1280,3)
+    ...
+
+    Key examples: "720p_240fps_1/00000"
+    GT (gt): Ground-Truth;
+    LQ (lq): Low-Quality, e.g., low-resolution/blurry/noisy/compressed frames.
+
+    Args:
+        opt (dict): Config for train dataset. It contains the following keys:
+            dataroot_gt (str): Data root path for gt.
+            dataroot_lq (str): Data root path for lq.
+            dataroot_flow (str, optional): Data root path for flow.
+            meta_info_file (str): Path for meta information file.
+            val_partition (str): Validation partition types. 'REDS4' or
+                'official'.
+            io_backend (dict): IO backend type and other kwarg.
+
+            num_frame (int): Window size for input frames.
+            gt_size (int): Cropped patched size for gt patches.
+            interval_list (list): Interval list for temporal augmentation.
+            random_reverse (bool): Random reverse input frames.
+            use_hflip (bool): Use horizontal flips.
+            use_rot (bool): Use rotation (use vertical flip and transposing h
+                and w for implementation).
+
+            scale (bool): Scale, which will be added automatically.
+    """
+
+    def __init__(self, opt):
+        super(VideoRecurrentTrainDataset, self).__init__()
+        self.opt = opt
+        self.scale = opt.get('scale', 4)
+        self.gt_size = opt.get('gt_size', 256)
+        self.gt_root, self.lq_root = Path(opt['dataroot_gt']), Path(opt['dataroot_lq'])
+        self.filename_tmpl = opt.get('filename_tmpl', '08d')
+        self.filename_ext = opt.get('filename_ext', 'png')
+        self.num_frame = opt['num_frame']
+
+        keys = []
+        total_num_frames = [] # some clips may not have 100 frames
+        start_frames = [] # some clips may not start from 00000
+        train_folders = os.listdir(self.lq_root)
+        print("TRAIN FOLDER: ", train_folders[0])
+        with open(opt['meta_info_file'], 'r') as fin:
+            for line in fin:
+                folder, frame_num, _, start_frame = line.split(' ')
+                if folder in train_folders:
+                    keys.extend([f'{folder}/{i:{self.filename_tmpl}}' for i in range(int(start_frame), int(start_frame)+int(frame_num))])
+                    total_num_frames.extend([int(frame_num) for i in range(int(frame_num))])
+                    start_frames.extend([int(start_frame) for i in range(int(frame_num))])
+
+        # remove the video clips used in validation
+        if opt['name'] == 'REDS':
+            if opt['val_partition'] == 'REDS4':
+                val_partition = ['000', '011', '015', '020']
+            elif opt['val_partition'] == 'official':
+                val_partition = [f'{v:03d}' for v in range(240, 270)]
+            else:
+                raise ValueError(f'Wrong validation partition {opt["val_partition"]}.'
+                                 f"Supported ones are ['official', 'REDS4'].")
+        else:
+            val_partition = []
+
+        self.keys = []
+        self.total_num_frames = [] # some clips may not have 100 frames
+        self.start_frames = []
+        if opt['test_mode']:
+            for i, v in zip(range(len(keys)), keys):
+                if v.split('/')[0] in val_partition:
+                    self.keys.append(keys[i])
+                    self.total_num_frames.append(total_num_frames[i])
+                    self.start_frames.append(start_frames[i])
+        else:
+            for i, v in zip(range(len(keys)), keys):
+                if v.split('/')[0] not in val_partition:
+                    self.keys.append(keys[i])
+                    self.total_num_frames.append(total_num_frames[i])
+                    self.start_frames.append(start_frames[i])
+
+        # file client (io backend)
+        self.file_client = None
+        self.io_backend_opt = opt['io_backend']
+        self.is_lmdb = False
+        if self.io_backend_opt['type'] == 'lmdb':
+            self.is_lmdb = True
+            if hasattr(self, 'flow_root') and self.flow_root is not None:
+                self.io_backend_opt['db_paths'] = [self.lq_root, self.gt_root, self.flow_root]
+                self.io_backend_opt['client_keys'] = ['lq', 'gt', 'flow']
+            else:
+                self.io_backend_opt['db_paths'] = [self.lq_root, self.gt_root]
+                self.io_backend_opt['client_keys'] = ['lq', 'gt']
+
+        # temporal augmentation configs
+        self.interval_list = opt.get('interval_list', [1])
+        self.random_reverse = opt.get('random_reverse', False)
+        interval_str = ','.join(str(x) for x in self.interval_list)
+        print(f'Temporal augmentation interval list: [{interval_str}]; '
+                    f'random reverse is {self.random_reverse}.')
+
+    def __getitem__(self, index):
+        if self.file_client is None:
+            self.file_client = utils_video.FileClient(self.io_backend_opt.pop('type'), **self.io_backend_opt)
+
+        key = self.keys[index]
+        total_num_frames = self.total_num_frames[index]
+        start_frames = self.start_frames[index]
+        clip_name, frame_name = key.split('/')  # key example: 000/00000000
+
+        # determine the neighboring frames
+        interval = random.choice(self.interval_list)
+
+        # ensure not exceeding the borders
+        start_frame_idx = int(frame_name)
+        endmost_start_frame_idx = start_frames + total_num_frames - self.num_frame * interval
+        if start_frame_idx > endmost_start_frame_idx:
+            start_frame_idx = random.randint(start_frames, endmost_start_frame_idx)
+        end_frame_idx = start_frame_idx + self.num_frame * interval
+
+        neighbor_list = list(range(start_frame_idx, end_frame_idx, interval))
+
+        # random reverse
+        if self.random_reverse and random.random() < 0.5:
+            neighbor_list.reverse()
+
+        # get the neighboring LQ and GT frames
+        img_lqs = []
+        img_gts = []
+        for neighbor in neighbor_list:
+            if self.is_lmdb:
+                img_lq_path = f'{clip_name}/{neighbor:{self.filename_tmpl}}'
+                img_gt_path = f'{clip_name}/{neighbor:{self.filename_tmpl}}'
+            else:
+                img_lq_path = self.lq_root / clip_name / f'{neighbor:{self.filename_tmpl}}.{self.filename_ext}'
+                img_gt_path = self.gt_root / clip_name / f'{neighbor:{self.filename_tmpl}}.{self.filename_ext}'
+
+            # get LQ
+            img_bytes = self.file_client.get(img_lq_path, 'lq')
+            img_lq = utils_video.imfrombytes(img_bytes, float32=True)
+            img_lqs.append(img_lq)
+
+            # get GT
+            img_bytes = self.file_client.get(img_gt_path, 'gt')
+            img_gt = utils_video.imfrombytes(img_bytes, float32=True)
+            img_gts.append(img_gt)
+
+        # randomly crop
+        img_gts, img_lqs = utils_video.paired_random_crop(img_gts, img_lqs, self.gt_size, self.scale, img_gt_path)
+
+        # augmentation - flip, rotate
+        img_lqs.extend(img_gts)
+        img_results = utils_video.augment(img_lqs, self.opt['use_hflip'], self.opt['use_rot'])
+
+        img_results = utils_video.img2tensor(img_results)
+        img_gts = torch.stack(img_results[len(img_lqs) // 2:], dim=0)
+        img_lqs = torch.stack(img_results[:len(img_lqs) // 2], dim=0)
+
+        # img_lqs: (t, c, h, w)
+        # img_gts: (t, c, h, w)
+        # key: str
+        return {'L': img_lqs, 'H': img_gts, 'key': key}
+
+    def __len__(self):
+        return len(self.keys)
+
+
+class VideoRecurrentTrainNonblindDenoisingDataset(VideoRecurrentTrainDataset):
+    """Video dataset for training recurrent architectures in non-blind video denoising.
+
+    Args:
+        Same as VideoTestDataset.
+
+    """
+
+    def __init__(self, opt):
+        super(VideoRecurrentTrainNonblindDenoisingDataset, self).__init__(opt)
+        self.sigma_min = self.opt['sigma_min'] / 255.
+        self.sigma_max = self.opt['sigma_max'] / 255.
+
+    def __getitem__(self, index):
+        if self.file_client is None:
+            self.file_client = utils_video.FileClient(self.io_backend_opt.pop('type'), **self.io_backend_opt)
+
+        key = self.keys[index]
+        total_num_frames = self.total_num_frames[index]
+        start_frames = self.start_frames[index]
+        clip_name, frame_name = key.split('/')  # key example: 000/00000000
+
+        # determine the neighboring frames
+        interval = random.choice(self.interval_list)
+
+        # ensure not exceeding the borders
+        start_frame_idx = int(frame_name)
+        endmost_start_frame_idx = start_frames + total_num_frames - self.num_frame * interval
+        if start_frame_idx > endmost_start_frame_idx:
+            start_frame_idx = random.randint(start_frames, endmost_start_frame_idx)
+        end_frame_idx = start_frame_idx + self.num_frame * interval
+
+        neighbor_list = list(range(start_frame_idx, end_frame_idx, interval))
+
+        # random reverse
+        if self.random_reverse and random.random() < 0.5:
+            neighbor_list.reverse()
+
+        # get the neighboring GT frames
+        img_gts = []
+        for neighbor in neighbor_list:
+            if self.is_lmdb:
+                img_gt_path = f'{clip_name}/{neighbor:{self.filename_tmpl}}'
+            else:
+                img_gt_path = self.gt_root / clip_name / f'{neighbor:{self.filename_tmpl}}.{self.filename_ext}'
+
+            # get GT
+            img_bytes = self.file_client.get(img_gt_path, 'gt')
+            img_gt = utils_video.imfrombytes(img_bytes, float32=True)
+            img_gts.append(img_gt)
+
+        # randomly crop
+        img_gts, _ = utils_video.paired_random_crop(img_gts, img_gts, self.gt_size, 1, img_gt_path)
+
+        # augmentation - flip, rotate
+        img_gts = utils_video.augment(img_gts, self.opt['use_hflip'], self.opt['use_rot'])
+
+        img_gts = utils_video.img2tensor(img_gts)
+        img_gts = torch.stack(img_gts, dim=0)
+
+        # we add noise in the network
+        noise_level = torch.empty((1, 1, 1, 1)).uniform_(self.sigma_min, self.sigma_max)
+        noise = torch.normal(mean=0, std=noise_level.expand_as(img_gts))
+        img_lqs = img_gts + noise
+
+        t, _, h, w = img_lqs.shape
+        img_lqs = torch.cat([img_lqs, noise_level.expand(t, 1, h, w)], 1)
+
+        # img_lqs: (t, c, h, w)
+        # img_gts: (t, c, h, w)
+        # key: str
+        return {'L': img_lqs, 'H': img_gts, 'key': key}
+
+
+    def __len__(self):
+        return len(self.keys)
+
+
+class VideoRecurrentTrainVimeoDataset(data.Dataset):
+    """Vimeo90K dataset for training recurrent networks.
+
+    The keys are generated from a meta info txt file.
+    basicsr/data/meta_info/meta_info_Vimeo90K_train_GT.txt
+
+    Each line contains:
+    1. clip name; 2. frame number; 3. image shape, separated by a white space.
+    Examples:
+        00001/0001 7 (256,448,3)
+        00001/0002 7 (256,448,3)
+
+    Key examples: "00001/0001"
+    GT (gt): Ground-Truth;
+    LQ (lq): Low-Quality, e.g., low-resolution/blurry/noisy/compressed frames.
+
+    The neighboring frame list for different num_frame:
+    num_frame | frame list
+             1 | 4
+             3 | 3,4,5
+             5 | 2,3,4,5,6
+             7 | 1,2,3,4,5,6,7
+
+    Args:
+        opt (dict): Config for train dataset. It contains the following keys:
+            dataroot_gt (str): Data root path for gt.
+            dataroot_lq (str): Data root path for lq.
+            meta_info_file (str): Path for meta information file.
+            io_backend (dict): IO backend type and other kwarg.
+
+            num_frame (int): Window size for input frames.
+            gt_size (int): Cropped patched size for gt patches.
+            random_reverse (bool): Random reverse input frames.
+            use_hflip (bool): Use horizontal flips.
+            use_rot (bool): Use rotation (use vertical flip and transposing h
+                and w for implementation).
+
+            scale (bool): Scale, which will be added automatically.
+    """
+
+    def __init__(self, opt):
+        super(VideoRecurrentTrainVimeoDataset, self).__init__()
+        self.opt = opt
+        self.gt_root, self.lq_root = Path(opt['dataroot_gt']), Path(opt['dataroot_lq'])
+
+        with open(opt['meta_info_file'], 'r') as fin:
+            self.keys = [line.split(' ')[0] for line in fin]
+
+        # file client (io backend)
+        self.file_client = None
+        self.io_backend_opt = opt['io_backend']
+        self.is_lmdb = False
+        if self.io_backend_opt['type'] == 'lmdb':
+            self.is_lmdb = True
+            self.io_backend_opt['db_paths'] = [self.lq_root, self.gt_root]
+            self.io_backend_opt['client_keys'] = ['lq', 'gt']
+
+        # indices of input images
+        self.neighbor_list = [i + (9 - opt['num_frame']) // 2 for i in range(opt['num_frame'])]
+
+        # temporal augmentation configs
+        self.random_reverse = opt['random_reverse']
+        print(f'Random reverse is {self.random_reverse}.')
+
+        self.flip_sequence = opt.get('flip_sequence', False)
+        self.pad_sequence = opt.get('pad_sequence', False)
+        self.neighbor_list = [1, 2, 3, 4, 5, 6, 7]
+
+    def __getitem__(self, index):
+        if self.file_client is None:
+            self.file_client = utils_video.FileClient(self.io_backend_opt.pop('type'), **self.io_backend_opt)
+
+        # random reverse
+        if self.random_reverse and random.random() < 0.5:
+            self.neighbor_list.reverse()
+
+        scale = self.opt['scale']
+        gt_size = self.opt['gt_size']
+        key = self.keys[index]
+        clip, seq = key.split('/')  # key example: 00001/0001
+
+        # get the neighboring LQ and  GT frames
+        img_lqs = []
+        img_gts = []
+        for neighbor in self.neighbor_list:
+            if self.is_lmdb:
+                img_lq_path = f'{clip}/{seq}/im{neighbor}'
+                img_gt_path = f'{clip}/{seq}/im{neighbor}'
+            else:
+                img_lq_path = self.lq_root / clip / seq / f'im{neighbor}.png'
+                img_gt_path = self.gt_root / clip / seq / f'im{neighbor}.png'
+            # LQ
+            img_bytes = self.file_client.get(img_lq_path, 'lq')
+            img_lq = utils_video.imfrombytes(img_bytes, float32=True)
+            # GT
+            img_bytes = self.file_client.get(img_gt_path, 'gt')
+            img_gt = utils_video.imfrombytes(img_bytes, float32=True)
+
+            img_lqs.append(img_lq)
+            img_gts.append(img_gt)
+
+        # randomly crop
+        img_gts, img_lqs = utils_video.paired_random_crop(img_gts, img_lqs, gt_size, scale, img_gt_path)
+
+        # augmentation - flip, rotate
+        img_lqs.extend(img_gts)
+        img_results = utils_video.augment(img_lqs, self.opt['use_hflip'], self.opt['use_rot'])
+
+        img_results = utils_video.img2tensor(img_results)
+        img_lqs = torch.stack(img_results[:7], dim=0)
+        img_gts = torch.stack(img_results[7:], dim=0)
+
+        if self.flip_sequence:  # flip the sequence: 7 frames to 14 frames
+            img_lqs = torch.cat([img_lqs, img_lqs.flip(0)], dim=0)
+            img_gts = torch.cat([img_gts, img_gts.flip(0)], dim=0)
+        elif self.pad_sequence:  # pad the sequence: 7 frames to 8 frames
+            img_lqs = torch.cat([img_lqs, img_lqs[-1:,...]], dim=0)
+            img_gts = torch.cat([img_gts, img_gts[-1:,...]], dim=0)
+
+        # img_lqs: (t, c, h, w)
+        # img_gt: (c, h, w)
+        # key: str
+        return {'L': img_lqs, 'H': img_gts, 'key': key}
+
+    def __len__(self):
+        return len(self.keys)

KAIR/data/degradations.py

@@ -0,0 +1,145 @@
+from typing import Tuple
+
+import numpy as np
+import random
+import torch
+from numpy.typing import NDArray
+
+from basicsr.data.degradations import random_add_gaussian_noise_pt, random_add_poisson_noise_pt
+from basicsr.data.transforms import paired_random_crop
+from basicsr.utils import DiffJPEG, USMSharp
+from basicsr.utils.img_process_util import filter2D
+from torch import Tensor
+from torch.nn import functional as F
+
+
+def blur(img: Tensor, kernel: NDArray) -> Tensor:
+    return filter2D(img, kernel)
+
+
+def random_resize(
+    img: Tensor,
+    resize_prob: float,
+    resize_range: Tuple[int, int],
+    output_scale: float = 1
+) -> Tensor:
+    updown_type = random.choices(['up', 'down', 'keep'], resize_prob)[0]
+    if updown_type == 'up':
+        random_scale = np.random.uniform(1, resize_range[1])
+    elif updown_type == 'down':
+        random_scale = np.random.uniform(resize_range[0], 1)
+    else:
+        random_scale = 1
+    mode = random.choice(['area', 'bilinear', 'bicubic'])
+    out = F.interpolate(img, scale_factor=output_scale * random_scale, mode=mode)
+    return out
+
+
+def add_noise(
+    img: Tensor,
+    gray_noise_prob: float,
+    gaussian_noise_prob: float,
+    noise_range: Tuple[float, float],
+    poisson_scale_range: Tuple[float, float]
+) -> Tensor:
+    if np.random.uniform() < gaussian_noise_prob:
+        img = random_add_gaussian_noise_pt(
+            img, sigma_range=noise_range, clip=True, rounds=False,
+            gray_prob=gray_noise_prob)
+    else:
+        img = random_add_poisson_noise_pt(
+            img, scale_range=poisson_scale_range,
+            gray_prob=gray_noise_prob, clip=True, rounds=False)
+    return img
+
+
+def jpeg_compression_simulation(
+    img: Tensor,
+    jpeg_range: Tuple[float, float],
+    jpeg_simulator: DiffJPEG
+) -> Tensor:
+    jpeg_p = img.new_zeros(img.size(0)).uniform_(*jpeg_range)
+
+    # clamp to [0, 1], otherwise JPEGer will result in unpleasant artifacts
+    img = torch.clamp(img, 0, 1)
+    return jpeg_simulator(img, quality=jpeg_p)
+
+
+@torch.no_grad()
+def apply_real_esrgan_degradations(
+    gt: Tensor,
+    blur_kernel1: NDArray,
+    blur_kernel2: NDArray,
+    second_blur_prob: float,
+    sinc_kernel: NDArray,
+    resize_prob1: float,
+    resize_prob2: float,
+    resize_range1: Tuple[int, int],
+    resize_range2: Tuple[int, int],
+    gray_noise_prob1: float,
+    gray_noise_prob2: float,
+    gaussian_noise_prob1: float,
+    gaussian_noise_prob2: float,
+    noise_range: Tuple[float, float],
+    poisson_scale_range: Tuple[float, float],
+    jpeg_compression_range1: Tuple[float, float],
+    jpeg_compression_range2: Tuple[float, float],
+    jpeg_simulator: DiffJPEG,
+    random_crop_gt_size: 512,
+    sr_upsample_scale: float,
+    usm_sharpener: USMSharp
+):
+    """
+    Accept batch from batchloader, and then add two-order degradations
+    to obtain LQ images.
+
+    gt: Tensor of shape (B x C x H x W)
+    """
+    gt_usm = usm_sharpener(gt)
+    # from PIL import Image
+    # Image.fromarray((gt_usm[0].permute(1, 2, 0).cpu().numpy() * 255.).astype(np.uint8)).save(
+    #         "/home/cll/Desktop/GT_USM_orig.png")
+    orig_h, orig_w = gt.size()[2:4]
+
+    # ----------------------- The first degradation process ----------------------- #
+    out = blur(gt_usm, blur_kernel1)
+    out = random_resize(out, resize_prob1, resize_range1)
+    out = add_noise(out, gray_noise_prob1, gaussian_noise_prob1, noise_range, poisson_scale_range)
+    out = jpeg_compression_simulation(out, jpeg_compression_range1, jpeg_simulator)
+
+    # ----------------------- The second degradation process ----------------------- #
+    if np.random.uniform() < second_blur_prob:
+        out = blur(out, blur_kernel2)
+    out = random_resize(out, resize_prob2, resize_range2, output_scale=(1/sr_upsample_scale))
+    out = add_noise(out, gray_noise_prob2, gaussian_noise_prob2,
+                    noise_range, poisson_scale_range)
+
+    # JPEG compression + the final sinc filter
+    # We also need to resize images to desired sizes.
+    # We group [resize back + sinc filter] together
+    # as one operation.
+    # We consider two orders:
+    #   1. [resize back + sinc filter] + JPEG compression
+    #   2. JPEG compression + [resize back + sinc filter]
+    # Empirically, we find other combinations (sinc + JPEG + Resize)
+    # will introduce twisted lines.
+    if np.random.uniform() < 0.5:
+        # resize back + the final sinc filter
+        mode = random.choice(['area', 'bilinear', 'bicubic'])
+        out = F.interpolate(out, size=(orig_h // sr_upsample_scale,
+                                       orig_w // sr_upsample_scale), mode=mode)
+        out = blur(out, sinc_kernel)
+        out = jpeg_compression_simulation(out, jpeg_compression_range2, jpeg_simulator)
+    else:
+        out = jpeg_compression_simulation(out, jpeg_compression_range2, jpeg_simulator)
+        mode = random.choice(['area', 'bilinear', 'bicubic'])
+        out = F.interpolate(out, size=(orig_h // sr_upsample_scale,
+                                       orig_w // sr_upsample_scale), mode=mode)
+        out = blur(out, sinc_kernel)
+
+    # clamp and round
+    lq = torch.clamp((out * 255.0).round(), 0, 255) / 255.
+
+    (gt, gt_usm), lq = paired_random_crop([gt, gt_usm], lq, random_crop_gt_size, sr_upsample_scale)
+
+    return gt, gt_usm, lq

KAIR/data/select_dataset.py

@@ -0,0 +1,86 @@
+
+
+'''
+# --------------------------------------------
+# select dataset
+# --------------------------------------------
+# Kai Zhang (github: https://github.com/cszn)
+# --------------------------------------------
+'''
+
+
+def define_Dataset(dataset_opt):
+    dataset_type = dataset_opt['dataset_type'].lower()
+    if dataset_type in ['l', 'low-quality', 'input-only']:
+        from data.dataset_l import DatasetL as D
+
+    # -----------------------------------------
+    # denoising
+    # -----------------------------------------
+    elif dataset_type in ['dncnn', 'denoising']:
+        from data.dataset_dncnn import DatasetDnCNN as D
+
+    elif dataset_type in ['dnpatch']:
+        from data.dataset_dnpatch import DatasetDnPatch as D
+
+    elif dataset_type in ['ffdnet', 'denoising-noiselevel']:
+        from data.dataset_ffdnet import DatasetFFDNet as D
+
+    elif dataset_type in ['fdncnn', 'denoising-noiselevelmap']:
+        from data.dataset_fdncnn import DatasetFDnCNN as D
+
+    # -----------------------------------------
+    # super-resolution
+    # -----------------------------------------
+    elif dataset_type in ['sr', 'super-resolution']:
+        from data.dataset_sr import DatasetSR as D
+
+    elif dataset_type in ['srmd']:
+        from data.dataset_srmd import DatasetSRMD as D
+
+    elif dataset_type in ['dpsr', 'dnsr']:
+        from data.dataset_dpsr import DatasetDPSR as D
+
+    elif dataset_type in ['usrnet', 'usrgan']:
+        from data.dataset_usrnet import DatasetUSRNet as D
+
+    elif dataset_type in ['bsrnet', 'bsrgan', 'blindsr']:
+        from data.dataset_blindsr import DatasetBlindSR as D
+
+    # -------------------------------------------------
+    # JPEG compression artifact reduction (deblocking)
+    # -------------------------------------------------
+    elif dataset_type in ['jpeg']:
+        from data.dataset_jpeg import DatasetJPEG as D
+
+    # -----------------------------------------
+    # video restoration
+    # -----------------------------------------
+    elif dataset_type in ['videorecurrenttraindataset']:
+        from data.dataset_video_train import VideoRecurrentTrainDataset as D
+    elif dataset_type in ['videorecurrenttrainnonblinddenoisingdataset']:
+        from data.dataset_video_train import VideoRecurrentTrainNonblindDenoisingDataset as D
+    elif dataset_type in ['videorecurrenttrainvimeodataset']:
+        from data.dataset_video_train import VideoRecurrentTrainVimeoDataset as D
+    elif dataset_type in ['videorecurrenttestdataset']:
+        from data.dataset_video_test import VideoRecurrentTestDataset as D
+    elif dataset_type in ['singlevideorecurrenttestdataset']:
+        from data.dataset_video_test import SingleVideoRecurrentTestDataset as D
+    elif dataset_type in ['videotestvimeo90kdataset']:
+        from data.dataset_video_test import VideoTestVimeo90KDataset as D
+
+    # -----------------------------------------
+    # common
+    # -----------------------------------------
+    elif dataset_type in ['plain']:
+        from data.dataset_plain import DatasetPlain as D
+
+    elif dataset_type in ['plainpatch']:
+        from data.dataset_plainpatch import DatasetPlainPatch as D
+
+    else:
+        raise NotImplementedError('Dataset [{:s}] is not found.'.format(dataset_type))
+
+    dataset = D(dataset_opt)
+    print('Dataset [{:s} - {:s}] is created.'.format(dataset.__class__.__name__, dataset_opt['name']))
+    return dataset

KAIR/docs/README_SwinIR.md

@@ -0,0 +1,194 @@
+# SwinIR: Image Restoration Using Shifted Window Transformer
+[paper](https://arxiv.org/abs/2108.10257)
+**|** 
+[supplementary](https://github.com/JingyunLiang/SwinIR/releases/tag/v0.0)
+**|** 
+[visual results](https://github.com/JingyunLiang/SwinIR/releases/tag/v0.0)
+**|** 
+[original project page](https://github.com/JingyunLiang/SwinIR)
+**|**
+[online Colab demo](https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb)
+
+[![arXiv](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2108.10257)
+[![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/SwinIR?style=social)](https://github.com/JingyunLiang/SwinIR)
+[![download](https://img.shields.io/github/downloads/JingyunLiang/SwinIR/total.svg)](https://github.com/JingyunLiang/SwinIR/releases)
+[ <a href="https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>](https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb)
+
+> Image restoration is a long-standing low-level vision problem that aims to restore high-quality images from low-quality images (e.g., downscaled, noisy and compressed images). While state-of-the-art image restoration methods are based on convolutional neural networks, few attempts have been made with Transformers which show impressive performance on high-level vision tasks. In this paper, we propose a strong baseline model SwinIR for image restoration based on the Swin Transformer. SwinIR consists of three parts: shallow feature extraction, deep feature extraction and high-quality image reconstruction. In particular, the deep feature extraction module is composed of several residual Swin Transformer blocks (RSTB), each of which has several Swin Transformer layers together with a residual connection. We conduct experiments on three representative tasks: image super-resolution (including classical, lightweight and real-world image super-resolution), image denoising (including grayscale and color image denoising) and JPEG compression artifact reduction. Experimental results demonstrate that SwinIR outperforms state-of-the-art methods on different tasks by up to 0.14~0.45dB, while the total number of parameters can be reduced by up to 67%.
+
+
+### Dataset Preparation
+
+Training and testing sets can be downloaded as follows. Please put them in `trainsets` and `testsets` respectively.
+
+| Task                 | Training Set | Testing Set|       
+| :---                 | :---:        |     :---:      |
+| classical/lightweight image SR          | [DIV2K](https://cv.snu.ac.kr/research/EDSR/DIV2K.tar) (800 training images) or DIV2K +[Flickr2K](https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar) (2650 images) | set5 + Set14 + BSD100 + Urban100 + Manga109 [download all](https://drive.google.com/drive/folders/1B3DJGQKB6eNdwuQIhdskA64qUuVKLZ9u) |
+| real-world image SR          | SwinIR-M (middle size): [DIV2K](https://cv.snu.ac.kr/research/EDSR/DIV2K.tar) (800 training images) +[Flickr2K](https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar) (2650 images) + [OST](https://openmmlab.oss-cn-hangzhou.aliyuncs.com/datasets/OST_dataset.zip) (10324 images, sky,water,grass,mountain,building,plant,animal) <br /> SwinIR-L (large size): DIV2K + Flickr2K + OST + [WED](http://ivc.uwaterloo.ca/database/WaterlooExploration/exploration_database_and_code.rar)(4744 images) + [FFHQ](https://drive.google.com/drive/folders/1tZUcXDBeOibC6jcMCtgRRz67pzrAHeHL) (first 2000 images, face) + Manga109 (manga) + [SCUT-CTW1500](https://universityofadelaide.box.com/shared/static/py5uwlfyyytbb2pxzq9czvu6fuqbjdh8.zip) (first 100 training images, texts) <br /><br />  ***We use the first practical degradation model [BSRGAN, ICCV2021  ![GitHub Stars](https://img.shields.io/github/stars/cszn/BSRGAN?style=social)](https://github.com/cszn/BSRGAN) for real-world image SR** | [RealSRSet+5images](https://github.com/JingyunLiang/SwinIR/releases/download/v0.0/RealSRSet+5images.zip) | 
+| color/grayscale image denoising      | [DIV2K](https://cv.snu.ac.kr/research/EDSR/DIV2K.tar) (800 training images) + [Flickr2K](https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar) (2650 images) + [BSD500](http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/BSR/BSR_bsds500.tgz) (400 training&testing images) + [WED](http://ivc.uwaterloo.ca/database/WaterlooExploration/exploration_database_and_code.rar)(4744 images) |  grayscale: Set12 + BSD68 + Urban100 <br />  color: CBSD68 + Kodak24 + McMaster + Urban100 [download all](https://github.com/cszn/FFDNet/tree/master/testsets) | 
+| JPEG compression artifact reduction  | [DIV2K](https://cv.snu.ac.kr/research/EDSR/DIV2K.tar) (800 training images) + [Flickr2K](https://cv.snu.ac.kr/research/EDSR/Flickr2K.tar) (2650 images) + [BSD500](http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/BSR/BSR_bsds500.tgz) (400 training&testing images) + [WED](http://ivc.uwaterloo.ca/database/WaterlooExploration/exploration_database_and_code.rar)(4744 images) |  grayscale: Classic5 +LIVE1 [download all](https://github.com/cszn/DnCNN/tree/master/testsets) |
+
+
+### Training
+To train SwinIR, run the following commands. You may need to change the `dataroot_H`, `dataroot_L`, `scale factor`, `noisel level`, `JPEG level`, `G_optimizer_lr`, `G_scheduler_milestones`, etc. in the json file for different settings. 
+
+
+
+```python
+# 001 Classical Image SR (middle size)
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/swinir/train_swinir_sr_classical.json  --dist True
+
+# 002 Lightweight Image SR (small size)
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/swinir/train_swinir_sr_lightweight.json  --dist True
+
+# 003 Real-World Image SR (middle size)
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/swinir/train_swinir_sr_realworld_psnr.json  --dist True
+# before training gan, put the PSNR-oriented model into superresolution/swinir_sr_realworld_x4_gan/models/
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/swinir/train_swinir_sr_realworld_gan.json  --dist True
+
+# 004 Grayscale Image Deoising (middle size)
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/swinir/train_swinir_denoising_gray.json  --dist True
+
+# 005 Color Image Deoising (middle size)
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/swinir/train_swinir_denoising_color.json  --dist True
+
+# 006 JPEG Compression Artifact Reduction (middle size)
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_psnr.py --opt options/swinir/train_swinir_car_jpeg.json  --dist True
+```
+
+You can also train above models using `DataParallel` as follows, but it will be slower.
+```python
+# 001 Classical Image SR (middle size)
+python main_train_psnr.py --opt options/swinir/train_swinir_sr_classical.json
+
+...
+```
+
+
+Note:
+
+1, We fine-tune X3/X4/X8 (or noise=25/50, or JPEG=10/20/30) models from the X2 (or noise=15, or JPEG=40) model, so that total_iteration can be halved to save training time. In this case, we halve the initial learning rate and lr_milestones accordingly. This way has similar performance as training from scratch.
+
+2, For SR, we use different kinds of `Upsampler` in classical/lightweight/real-world image SR for the purpose of fair comparison with existing works.
+
+3, We did not re-train the models after cleaning the codes. Feel free to open an issue if you meet any problems. 
+
+## Testing
+Following command will download the [pretrained models](https://github.com/JingyunLiang/SwinIR/releases/tag/v0.0) and put them in `model_zoo/swinir`. All visual results of SwinIR can be downloaded [here](https://github.com/JingyunLiang/SwinIR/releases/tag/v0.0).
+
+If you are too lazy to prepare the datasets, please follow the guide in the [original project page](https://github.com/JingyunLiang/SwinIR#testing-without-preparing-datasets), where you can start testing in a minute. We also provide an [online Colab demo for real-world image SR  <a href="https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>](https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb) for comparison with [the first practical degradation model BSRGAN (ICCV2021)  ![GitHub Stars](https://img.shields.io/github/stars/cszn/BSRGAN?style=social)](https://github.com/cszn/BSRGAN) and a recent model [RealESRGAN](https://github.com/xinntao/Real-ESRGAN). Try to test your own images on Colab!
+
+```bash
+# 001 Classical Image Super-Resolution (middle size)
+# Note that --training_patch_size is just used to differentiate two different settings in Table 2 of the paper. Images are NOT tested patch by patch.
+# (setting1: when model is trained on DIV2K and with training_patch_size=48)
+python main_test_swinir.py --task classical_sr --scale 2 --training_patch_size 48 --model_path model_zoo/swinir/001_classicalSR_DIV2K_s48w8_SwinIR-M_x2.pth --folder_lq testsets/set5/LR_bicubic/X2 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task classical_sr --scale 3 --training_patch_size 48 --model_path model_zoo/swinir/001_classicalSR_DIV2K_s48w8_SwinIR-M_x3.pth --folder_lq testsets/set5/LR_bicubic/X3 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task classical_sr --scale 4 --training_patch_size 48 --model_path model_zoo/swinir/001_classicalSR_DIV2K_s48w8_SwinIR-M_x4.pth --folder_lq testsets/set5/LR_bicubic/X4 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task classical_sr --scale 8 --training_patch_size 48 --model_path model_zoo/swinir/001_classicalSR_DIV2K_s48w8_SwinIR-M_x8.pth --folder_lq testsets/set5/LR_bicubic/X8 --folder_gt testsets/set5/HR
+
+# (setting2: when model is trained on DIV2K+Flickr2K and with training_patch_size=64)
+python main_test_swinir.py --task classical_sr --scale 2 --training_patch_size 64 --model_path model_zoo/swinir/001_classicalSR_DF2K_s64w8_SwinIR-M_x2.pth --folder_lq testsets/set5/LR_bicubic/X2 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task classical_sr --scale 3 --training_patch_size 64 --model_path model_zoo/swinir/001_classicalSR_DF2K_s64w8_SwinIR-M_x3.pth --folder_lq testsets/set5/LR_bicubic/X3 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task classical_sr --scale 4 --training_patch_size 64 --model_path model_zoo/swinir/001_classicalSR_DF2K_s64w8_SwinIR-M_x4.pth --folder_lq testsets/set5/LR_bicubic/X4 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task classical_sr --scale 8 --training_patch_size 64 --model_path model_zoo/swinir/001_classicalSR_DF2K_s64w8_SwinIR-M_x8.pth --folder_lq testsets/set5/LR_bicubic/X8 --folder_gt testsets/set5/HR
+
+
+# 002 Lightweight Image Super-Resolution (small size)
+python main_test_swinir.py --task lightweight_sr --scale 2 --model_path model_zoo/swinir/002_lightweightSR_DIV2K_s64w8_SwinIR-S_x2.pth --folder_lq testsets/set5/LR_bicubic/X2 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task lightweight_sr --scale 3 --model_path model_zoo/swinir/002_lightweightSR_DIV2K_s64w8_SwinIR-S_x3.pth --folder_lq testsets/set5/LR_bicubic/X3 --folder_gt testsets/set5/HR
+python main_test_swinir.py --task lightweight_sr --scale 4 --model_path model_zoo/swinir/002_lightweightSR_DIV2K_s64w8_SwinIR-S_x4.pth --folder_lq testsets/set5/LR_bicubic/X4 --folder_gt testsets/set5/HR
+
+
+# 003 Real-World Image Super-Resolution (use --tile 400 if you run out-of-memory)
+# (middle size)
+python main_test_swinir.py --task real_sr --scale 4 --model_path model_zoo/swinir/003_realSR_BSRGAN_DFO_s64w8_SwinIR-M_x4_GAN.pth --folder_lq testsets/RealSRSet+5images
+
+# (larger size + trained on more datasets)
+python main_test_swinir.py --task real_sr --scale 4 --large_model --model_path model_zoo/swinir/003_realSR_BSRGAN_DFOWMFC_s64w8_SwinIR-L_x4_GAN.pth --folder_lq testsets/RealSRSet+5images
+
+
+# 004 Grayscale Image Deoising (middle size)
+python main_test_swinir.py --task gray_dn --noise 15 --model_path model_zoo/swinir/004_grayDN_DFWB_s128w8_SwinIR-M_noise15.pth --folder_gt testsets/set12
+python main_test_swinir.py --task gray_dn --noise 25 --model_path model_zoo/swinir/004_grayDN_DFWB_s128w8_SwinIR-M_noise25.pth --folder_gt testsets/set12
+python main_test_swinir.py --task gray_dn --noise 50 --model_path model_zoo/swinir/004_grayDN_DFWB_s128w8_SwinIR-M_noise50.pth --folder_gt testsets/set12
+
+
+# 005 Color Image Deoising (middle size)
+python main_test_swinir.py --task color_dn --noise 15 --model_path model_zoo/swinir/005_colorDN_DFWB_s128w8_SwinIR-M_noise15.pth --folder_gt testsets/McMaster
+python main_test_swinir.py --task color_dn --noise 25 --model_path model_zoo/swinir/005_colorDN_DFWB_s128w8_SwinIR-M_noise25.pth --folder_gt testsets/McMaster
+python main_test_swinir.py --task color_dn --noise 50 --model_path model_zoo/swinir/005_colorDN_DFWB_s128w8_SwinIR-M_noise50.pth --folder_gt testsets/McMaster
+
+
+# 006 JPEG Compression Artifact Reduction (middle size, using window_size=7 because JPEG encoding uses 8x8 blocks)
+python main_test_swinir.py --task jpeg_car --jpeg 10 --model_path model_zoo/swinir/006_CAR_DFWB_s126w7_SwinIR-M_jpeg10.pth --folder_gt testsets/classic5
+python main_test_swinir.py --task jpeg_car --jpeg 20 --model_path model_zoo/swinir/006_CAR_DFWB_s126w7_SwinIR-M_jpeg20.pth --folder_gt testsets/classic5
+python main_test_swinir.py --task jpeg_car --jpeg 30 --model_path model_zoo/swinir/006_CAR_DFWB_s126w7_SwinIR-M_jpeg30.pth --folder_gt testsets/classic5
+python main_test_swinir.py --task jpeg_car --jpeg 40 --model_path model_zoo/swinir/006_CAR_DFWB_s126w7_SwinIR-M_jpeg40.pth --folder_gt testsets/classic5
+```
+
+---
+
+## Results
+<details>
+<summary>Classical Image Super-Resolution (click me)</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/classic_image_sr.png">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/classic_image_sr_visual.png">
+</p>
+</details>
+
+<details>
+<summary>Lightweight Image Super-Resolution</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/lightweight_image_sr.png">
+</p>
+</details>
+
+<details>
+<summary>Real-World Image Super-Resolution</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/real_world_image_sr.png">
+</p>
+</details>
+
+
+|&nbsp;&nbsp;&nbsp; Real-World Image (x4)|[BSRGAN, ICCV2021](https://github.com/cszn/BSRGAN)|[Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN)|SwinIR (ours)|
+|      :---      |     :---:        |        :-----:         |        :-----:         | 
+|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_LR.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_BSRGAN.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_realESRGAN.jpg">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/ETH_SwinIR.png">
+|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_LR.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_BSRGAN.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_realESRGAN.png">|<img width="200" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/OST_009_crop_SwinIR.png">|
+
+<details>
+<summary>Grayscale Image Deoising</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/gray_image_denoising.png">
+</p>
+</details>
+
+<details>
+<summary>Color Image Deoising</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/color_image_denoising.png">
+</p>
+</details>
+
+<details>
+<summary>JPEG Compression Artifact Reduction</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/SwinIR/main/figs/jepg_compress_artfact_reduction.png">
+</p>
+</details>
+
+
+
+Please refer to the [paper](https://arxiv.org/abs/2108.10257) and the [original project page](https://github.com/JingyunLiang/SwinIR)
+for more results.
+
+
+## Citation
+    @article{liang2021swinir,
+        title={SwinIR: Image Restoration Using Swin Transformer},
+        author={Liang, Jingyun and Cao, Jiezhang and Sun, Guolei and Zhang, Kai and Van Gool, Luc and Timofte, Radu},
+        journal={arXiv preprint arXiv:2108.10257}, 
+        year={2021}
+    }

KAIR/docs/README_VRT.md

@@ -0,0 +1,191 @@
+# [VRT: A Video Restoration Transformer](https://github.com/JingyunLiang/VRT)
+[arxiv](https://arxiv.org/abs/2201.12288)
+**|** 
+[supplementary](https://github.com/JingyunLiang/VRT/releases/download/v0.0/VRT_supplementary.pdf)
+**|** 
+[pretrained models](https://github.com/JingyunLiang/VRT/releases)
+**|** 
+[visual results](https://github.com/JingyunLiang/VRT/releases)
+**|** 
+[original project page](https://github.com/JingyunLiang/VRT)
+
+[![arXiv](https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg)](https://arxiv.org/abs/2201.12288)
+[![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/VRT?style=social)](https://github.com/JingyunLiang/VRT)
+[![download](https://img.shields.io/github/downloads/JingyunLiang/VRT/total.svg)](https://github.com/JingyunLiang/VRT/releases)
+![visitors](https://visitor-badge.glitch.me/badge?page_id=jingyunliang/VRT)
+[ <a href="https://colab.research.google.com/gist/JingyunLiang/deb335792768ad9eb73854a8efca4fe0#file-vrt-demo-on-video-restoration-ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>](https://colab.research.google.com/gist/JingyunLiang/deb335792768ad9eb73854a8efca4fe0#file-vrt-demo-on-video-restoration-ipynb)
+
+This is the readme of "VRT: A Video Restoration Transformer"
+([arxiv](https://arxiv.org/pdf/2201.12288.pdf), [supp](https://github.com/JingyunLiang/VRT/releases/download/v0.0/VRT_supplementary.pdf), [pretrained models](https://github.com/JingyunLiang/VRT/releases), [visual results](https://github.com/JingyunLiang/VRT/releases)). VRT ahcieves state-of-the-art performance **(up to 2.16dB)** in
+- video SR (REDS, Vimeo90K, Vid4 and UDM10)
+- video deblurring (GoPro, DVD and REDS)
+- video denoising (DAVIS and Set8)
+
+<p align="center">
+  <a href="https://github.com/JingyunLiang/VRT/releases">
+    <img width=30% src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/teaser_vsr.gif"/>
+    <img width=30% src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/teaser_vdb.gif"/>
+    <img width=30% src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/teaser_vdn.gif"/>
+  </a>
+</p>
+
+---
+
+> Video restoration (e.g., video super-resolution) aims to restore high-quality frames from low-quality frames. Different from single image restoration, video restoration generally requires to utilize temporal information from multiple adjacent but usually misaligned video frames. Existing deep methods generally tackle with this by exploiting a sliding window strategy or a recurrent architecture, which either is restricted by frame-by-frame restoration or lacks long-range modelling ability. In this paper, we propose a Video Restoration Transformer (VRT) with parallel frame prediction and long-range temporal dependency modelling abilities. More specifically, VRT is composed of multiple scales, each of which consists of two kinds of modules: temporal mutual self attention (TMSA) and parallel warping. TMSA divides the video into small clips, on which mutual attention is applied for joint motion estimation, feature alignment and feature fusion, while self-attention is used for feature extraction. To enable cross-clip interactions, the video sequence is shifted for every other layer. Besides, parallel warping is used to further fuse information from neighboring frames by parallel feature warping. Experimental results on three tasks, including video super-resolution, video deblurring and video denoising, demonstrate that VRT outperforms the state-of-the-art methods by large margins (**up to 2.16 dB**) on nine benchmark datasets.
+<p align="center">
+  <img width="800" src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/framework.jpeg">
+</p>
+
+#### Contents
+
+1. [Requirements](#Requirements)
+1. [Quick Testing](#Quick-Testing)
+1. [Training](#Training)
+1. [Results](#Results)
+1. [Citation](#Citation)
+1. [License and Acknowledgement](#License-and-Acknowledgement)
+
+
+## Requirements
+> - Python 3.8, PyTorch >= 1.9.1
+> - Requirements: see requirements.txt
+> - Platforms: Ubuntu 18.04, cuda-11.1
+
+## Quick Testing
+Following commands will download [pretrained models](https://github.com/JingyunLiang/VRT/releases) and [test datasets](https://github.com/JingyunLiang/VRT/releases) **automatically** (except Vimeo-90K testing set). If out-of-memory, try to reduce `--tile` at the expense of slightly decreased performance.
+
+You can also try to test it on Colab[ <a href="https://colab.research.google.com/gist/JingyunLiang/deb335792768ad9eb73854a8efca4fe0#file-vrt-demo-on-video-restoration-ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="google colab logo"></a>](https://colab.research.google.com/gist/JingyunLiang/deb335792768ad9eb73854a8efca4fe0#file-vrt-demo-on-video-restoration-ipynb), but the results may be slightly different due to `--tile` difference.
+```bash
+# download code
+git clone https://github.com/JingyunLiang/VRT
+cd VRT
+pip install -r requirements.txt
+
+# 001, video sr trained on REDS (6 frames), tested on REDS4
+python main_test_vrt.py --task 001_VRT_videosr_bi_REDS_6frames --folder_lq testsets/REDS4/sharp_bicubic --folder_gt testsets/REDS4/GT --tile 40 128 128 --tile_overlap 2 20 20
+
+# 002, video sr trained on REDS (16 frames), tested on REDS4
+python main_test_vrt.py --task 002_VRT_videosr_bi_REDS_16frames --folder_lq testsets/REDS4/sharp_bicubic --folder_gt testsets/REDS4/GT --tile 40 128 128 --tile_overlap 2 20 20
+
+# 003, video sr trained on Vimeo (bicubic), tested on Vid4 and Vimeo
+python main_test_vrt.py --task 003_VRT_videosr_bi_Vimeo_7frames --folder_lq testsets/Vid4/BIx4 --folder_gt testsets/Vid4/GT --tile 32 128 128 --tile_overlap 2 20 20
+python main_test_vrt.py --task 003_VRT_videosr_bi_Vimeo_7frames --folder_lq testsets/vimeo90k/vimeo_septuplet_matlabLRx4/sequences --folder_gt testsets/vimeo90k/vimeo_septuplet/sequences --tile 8 0 0 --tile_overlap 0 20 20
+
+# 004, video sr trained on Vimeo (blur-downsampling), tested on Vid4, UDM10 and Vimeo
+python main_test_vrt.py --task 004_VRT_videosr_bd_Vimeo_7frames --folder_lq testsets/Vid4/BDx4 --folder_gt testsets/Vid4/GT --tile 32 128 128 --tile_overlap 2 20 20
+python main_test_vrt.py --task 004_VRT_videosr_bd_Vimeo_7frames --folder_lq testsets/UDM10/BDx4 --folder_gt testsets/UDM10/GT --tile 32 128 128 --tile_overlap 2 20 20
+python main_test_vrt.py --task 004_VRT_videosr_bd_Vimeo_7frames --folder_lq testsets/vimeo90k/vimeo_septuplet_BDLRx4/sequences --folder_gt testsets/vimeo90k/vimeo_septuplet/sequences --tile 8 0 0 --tile_overlap 0 20 20
+
+# 005, video deblurring trained and tested on DVD
+python main_test_vrt.py --task 005_VRT_videodeblurring_DVD --folder_lq testsets/DVD10/test_GT_blurred --folder_gt testsets/DVD10/test_GT --tile 12 256 256 --tile_overlap 2 20 20
+
+# 006, video deblurring trained and tested on GoPro
+python main_test_vrt.py --task 006_VRT_videodeblurring_GoPro --folder_lq testsets/GoPro11/test_GT_blurred --folder_gt testsets/GoPro11/test_GT --tile 18 192 192 --tile_overlap 2 20 20
+
+# 007, video deblurring trained on REDS, tested on REDS4
+python main_test_vrt.py --task 007_VRT_videodeblurring_REDS --folder_lq testsets/REDS4/blur --folder_gt testsets/REDS4/GT --tile 12 256 256 --tile_overlap 2 20 20
+
+# 008, video denoising trained on DAVIS (noise level 0-50) and tested on Set8 and DAVIS
+python main_test_vrt.py --task 008_VRT_videodenoising_DAVIS --sigma 10 --folder_lq testsets/Set8 --folder_gt testsets/Set8 --tile 12 256 256 --tile_overlap 2 20 20
+python main_test_vrt.py --task 008_VRT_videodenoising_DAVIS --sigma 10  --folder_lq testsets/DAVIS-test --folder_gt testsets/DAVIS-test --tile 12 256 256 --tile_overlap 2 20 20
+
+# test on your own datasets (an example)
+python main_test_vrt.py --task 001_VRT_videosr_bi_REDS_6frames --folder_lq testsets/your/own --tile 40 128 128 --tile_overlap 2 20 20
+```
+
+**All visual results of VRT can be downloaded [here](https://github.com/JingyunLiang/VRT/releases)**.
+
+
+## Training
+The training and testing sets are as follows (see the [supplementary](https://github.com/JingyunLiang/VRT/releases) for a detailed introduction of all datasets). For better I/O speed, use commands like `python scripts/data_preparation/create_lmdb.py --dataset reds` to convert `.png` datasets to `.lmdb` datasets.
+
+Note: You do **NOT need** to prepare the datasets if you just want to test the model. `main_test_vrt.py` will download the testing set automaticaly.
+
+
+| Task                                                          |                                                                                                                                                                                                                                    Training Set                                                                                                                                                                                                                                     |                                                                                                                                                                                                                                                                                 Testing Set                                                                                                                                                                                                                                                                                  |        Pretrained Model and Visual Results of VRT  |
+|:--------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|    :---:      |
+| video SR (setting 1, BI)                                      |                                                                                 [REDS sharp & sharp_bicubic](https://seungjunnah.github.io/Datasets/reds.html) (266 videos, 266000 frames: train + val except REDS4)   <br  /><br  /> *Use  [regroup_reds_dataset.py](https://github.com/cszn/KAIR/tree/master/scripts/data_preparation/regroup_reds_dataset.py) to regroup and rename REDS val set                                                                                 |                                                                                                                                                                                                                                                           REDS4 (4 videos, 400 frames: 000, 011, 015, 020 of REDS)                                                                                                                                                                                                                                                           | [here](https://github.com/JingyunLiang/VRT/releases) |
+| video SR (setting 2 & 3, BI & BD)                             |    [Vimeo90K](http://data.csail.mit.edu/tofu/dataset/vimeo_septuplet.zip) (64612 seven-frame videos as in `sep_trainlist.txt`)  <br  /><br  /> * Use [generate_LR_Vimeo90K.m](https://github.com/cszn/KAIR/tree/master/scripts/matlab_scripts/generate_LR_Vimeo90K.m) and [generate_LR_Vimeo90K_BD.m](https://github.com/cszn/KAIR/tree/master/scripts/matlab_scripts/generate_LR_Vimeo90K_BD.m) to generate LR frames for bicubic and blur-downsampling VSR, respectively.    |                                                                       Vimeo90K-T (the rest 7824 7-frame videos) + [Vid4](https://drive.google.com/file/d/1ZuvNNLgR85TV_whJoHM7uVb-XW1y70DW/view) (4 videos) + [UDM10](https://www.terabox.com/web/share/link?surl=LMuQCVntRegfZSxn7s3hXw&path=%2Fproject%2Fpfnl) (10 videos)  <br  /><br  /> *Use [prepare_UDM10.py](https://github.com/cszn/KAIR/tree/master/scripts/data_preparation/prepare_UDM10.py) to regroup and rename the UDM10 dataset                                                                        | [here](https://github.com/JingyunLiang/VRT/releases) |
+| video deblurring (setting 1, motion blur)                     |                                                                                            [DVD](http://www.cs.ubc.ca/labs/imager/tr/2017/DeepVideoDeblurring/DeepVideoDeblurring_Dataset.zip) (61 videos, 5708 frames)  <br  /><br  /> *Use [prepare_DVD.py](https://github.com/cszn/KAIR/tree/master/scripts/data_preparation/prepare_DVD.py) to regroup and rename the dataset.                                                                                             |                                                                                                                                                                              DVD (10 videos, 1000 frames)             <br  /><br  /> *Use [evaluate_video_deblurring.m](https://github.com/cszn/KAIR/tree/master/scripts/matlab_scripts/evaluate_video_deblurring.m) for final evaluation.                                                                                                                                                                              | [here](https://github.com/JingyunLiang/VRT/releases) |
+| video deblurring (setting 2, motion blur)                     |                                                                                             [GoPro](http://data.cv.snu.ac.kr:8008/webdav/dataset/GOPRO/GOPRO_Large.zip) (22 videos, 2103 frames)  <br  /><br  /> *Use [prepare_GoPro_as_video.py](https://github.com/cszn/KAIR/tree/master/scripts/data_preparation/prepare_GoPro_as_video.py) to regroup and rename the dataset.                                                                                              |                                                                                                                                                                                  GoPro (11 videos, 1111 frames)  <br  /><br  /> *Use [evaluate_video_deblurring.m](https://github.com/cszn/KAIR/tree/master/scripts/matlab_scripts/evaluate_video_deblurring.m) for final evaluation.                                                                                                                                                                                   | [here](https://github.com/JingyunLiang/VRT/releases) |
+| video deblurring (setting 3, motion blur)                     |                                                         [REDS sharp & blur](https://seungjunnah.github.io/Datasets/reds.html) (266 videos, 266000 frames: train & val except REDS4)   <br  /><br  /> *Use  [regroup_reds_dataset.py](https://github.com/cszn/KAIR/tree/master/scripts/data_preparation/regroup_reds_dataset.py) to regroup and rename REDS val set. Note that it shares the same HQ frames as in VSR.                                                          |                                                                                                                                                                                                                                                           REDS4 (4 videos, 400 frames: 000, 011, 015, 020 of REDS)                                                                                                                                                                                                                                                           | [here](https://github.com/JingyunLiang/VRT/releases) |
+| video denoising (Gaussian noise)                              |                                                                                                                                             [DAVIS-2017](https://data.vision.ee.ethz.ch/csergi/share/davis/DAVIS-2017-Unsupervised-trainval-480p.zip) (90 videos, 6208 frames)  <br  /><br  /> *Use all files in DAVIS/JPEGImages/480p                                                                                                                                              |                                                                                                              [DAVIS-2017-test](https://github.com/JingyunLiang/VRT/releases) (30 videos) + [Set8](https://www.dropbox.com/sh/20n4cscqkqsfgoj/AABGftyJuJDwuCLGczL-fKvBa/test_sequences?dl=0&subfolder_nav_tracking=1) (8 videos: tractor, touchdown, park_joy and sunflower selected from DERF + hypersmooth, motorbike, rafting and snowboard from GOPRO_540P)                                                                                                               | [here](https://github.com/JingyunLiang/VRT/releases) |
+
+Run following commands for training:
+```bash
+# download code
+git clone https://github.com/cszn/KAIR
+cd KAIR
+pip install -r requirements.txt
+
+# 001, video sr trained on REDS (6 frames), tested on REDS4
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/001_train_vrt_videosr_bi_reds_6frames.json  --dist True
+
+# 002, video sr trained on REDS (16 frames), tested on REDS4
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/002_train_vrt_videosr_bi_reds_16frames.json  --dist True
+
+# 003, video sr trained on Vimeo (bicubic), tested on Vid4 and Vimeo
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/003_train_vrt_videosr_bi_vimeo_7frames.json  --dist True
+
+# 004, video sr trained on Vimeo (blur-downsampling), tested on Vid4, Vimeo and UDM10
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/004_train_vrt_videosr_bd_vimeo_7frames.json  --dist True
+
+# 005, video deblurring trained and tested on DVD
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/005_train_vrt_videodeblurring_dvd.json  --dist True
+
+# 006, video deblurring trained and tested on GoPro
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/006_train_vrt_videodeblurring_gopro.json  --dist True
+
+# 007, video deblurring trained on REDS, tested on REDS4
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/007_train_vrt_videodeblurring_reds.json  --dist True
+
+# 008, video denoising trained on DAVIS (noise level 0-50) and tested on Set8 and DAVIS
+python -m torch.distributed.launch --nproc_per_node=8 --master_port=1234 main_train_vrt.py --opt options/vrt/008_train_vrt_videodenoising_davis.json  --dist True
+```
+Tip: The training process will terminate automatically at 20000 iteration due to a bug. Just resume training after that.
+<details>
+<summary>Bug</summary>
+Bug: PyTorch DistributedDataParallel (DDP) does not support `torch.utils.checkpoint` well. To alleviate the problem, set `find_unused_parameters=False` when `use_checkpoint=True`. If there are other errors, make sure that unused parameters will not change during training loop and set `use_static_graph=True`.
+
+If you find a better solution, feel free to pull a request. Thank you.
+</details>
+
+## Results
+We achieved state-of-the-art performance on video SR, video deblurring and video denoising. Detailed results can be found in the [paper](https://arxiv.org/abs/2201.12288).
+
+<details>
+<summary>Video Super-Resolution (click me)</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/vsr.jpeg">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/vsr_visual.jpeg">
+</p>
+</details>
+
+<details>
+<summary>Video Deblurring</summary>
+<p align="center">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/vdb_dvd_gopro.jpeg">
+  <img width="900" src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/vdb_visual.jpeg">
+  <img width="350" src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/vdb_reds.jpeg">
+</p>
+</details>
+
+<details>
+<summary>Video Denoising</summary>
+<p align="center">
+  <img width="350" src="https://raw.githubusercontent.com/JingyunLiang/VRT/main/assets/vdn.jpeg">
+</p>
+</details>
+
+
+## Citation
+    @article{liang2022vrt,
+        title={VRT: A Video Restoration Transformer},
+        author={Liang, Jingyun and Cao, Jiezhang and Fan, Yuchen and Zhang, Kai and Ranjan, Rakesh and Li, Yawei and Timofte, Radu and Van Gool, Luc},
+        journal={arXiv preprint arXiv:2201.12288},
+        year={2022}
+    }
+
+
+## License and Acknowledgement
+This project is released under the CC-BY-NC license. We refer to codes from [KAIR](https://github.com/cszn/KAIR), [BasicSR](https://github.com/xinntao/BasicSR), [Video Swin Transformer](https://github.com/SwinTransformer/Video-Swin-Transformer) and [mmediting](https://github.com/open-mmlab/mmediting). Thanks for their awesome works. The majority of VRT is licensed under CC-BY-NC, however portions of the project are available under separate license terms: KAIR is licensed under the MIT License, BasicSR, Video Swin Transformer and mmediting are licensed under the Apache 2.0 license.

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_095438.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": null,
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "trainsets/REDS/train_sharp_with_val.lmdb",
+      "dataroot_lq": "trainsets/REDS/train_sharp_bicubic_with_val.lmdb",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "lmdb"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "testsets/REDS4/GT",
+      "dataroot_lq": "testsets/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_095450.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": null,
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "trainsets/REDS/train_sharp_with_val.lmdb",
+      "dataroot_lq": "trainsets/REDS/train_sharp_bicubic_with_val.lmdb",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "lmdb"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "testsets/REDS4/GT",
+      "dataroot_lq": "testsets/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_095518.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": null,
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "trainsets/REDS/train_sharp_with_val.lmdb",
+      "dataroot_lq": "trainsets/REDS/train_sharp_bicubic_with_val.lmdb",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "lmdb"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "testsets/REDS4/GT",
+      "dataroot_lq": "testsets/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_101636.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": null,
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/val/val_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/val/val_sharp_bicubic",
+      "meta_info_file": "",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_101949.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "/home/cll/dev/KAIR/model_zoo/vrt/",
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/val/val_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/val/val_sharp_bicubic",
+      "meta_info_file": "",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_102114.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "/home/cll/dev/KAIR/model_zoo/vrt/",
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/val/val_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/val/val_sharp_bicubic",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_102214.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "/home/cll/dev/KAIR/model_zoo/vrt/001_VRT_videosr_bi_REDS_6frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/val/val_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/val/val_sharp_bicubic",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_104612.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "/home/cll/dev/KAIR/model_zoo/vrt/001_VRT_videosr_bi_REDS_6frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/train/train_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/train/train_sharp_bicubic/X4",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_105219.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "/home/cll/dev/KAIR/model_zoo/vrt/001_VRT_videosr_bi_REDS_6frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/train/train_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/train/train_sharp_bicubic/X4",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 6,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_105304.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "/home/cll/dev/KAIR/model_zoo/vrt/001_VRT_videosr_bi_REDS_6frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/train/train_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/train/train_sharp_bicubic/X4",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 4,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      6,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/001_train_vrt_videosr_bi_reds_6frames/options/001_train_vrt_videosr_bi_reds_6frames_220311_105340.json

@@ -0,0 +1,201 @@
+{
+  "task": "001_train_vrt_videosr_bi_reds_6frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "/home/cll/dev/KAIR/model_zoo/vrt/001_VRT_videosr_bi_REDS_6frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "log": "experiments/001_train_vrt_videosr_bi_reds_6frames",
+    "options": "experiments/001_train_vrt_videosr_bi_reds_6frames/options",
+    "models": "experiments/001_train_vrt_videosr_bi_reds_6frames/models",
+    "images": "experiments/001_train_vrt_videosr_bi_reds_6frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainDataset",
+      "dataroot_gt": "/home/cll/datasets/REDS/train/train_sharp",
+      "dataroot_lq": "/home/cll/datasets/REDS/train/train_sharp_bicubic/X4",
+      "meta_info_file": "data/meta_info/meta_info_REDS_GT.txt",
+      "filename_tmpl": "08d",
+      "filename_ext": "png",
+      "val_partition": "REDS4",
+      "test_mode": false,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": 4,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": false,
+      "use_hflip": true,
+      "use_rot": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "/home/cll/Desktop/REDS4/GT",
+      "dataroot_lq": "/home/cll/Desktop/REDS4/sharp_bicubic",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      6,
+      64,
+      64
+    ],
+    "window_size": [
+      2,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 2,
+    "deformable_groups": 12,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": true,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 40,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/001_train_vrt_videosr_bi_reds_6frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/003_train_vrt_videosr_bi_vimeo_7frames/options/003_train_vrt_videosr_bi_vimeo_7frames_220311_095626.json

@@ -0,0 +1,198 @@
+{
+  "task": "003_train_vrt_videosr_bi_vimeo_7frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "model_zoo/vrt/002_VRT_videosr_bi_REDS_16frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "log": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "options": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/options",
+    "models": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/models",
+    "images": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainVimeoDataset",
+      "dataroot_gt": "trainsets/vimeo90k",
+      "dataroot_lq": "trainsets/vimeo90k",
+      "meta_info_file": "data/meta_info/meta_info_Vimeo90K_train_GT.txt",
+      "io_backend": {
+        "type": "file"
+      },
+      "num_frame": -1,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": true,
+      "use_hflip": true,
+      "use_rot": true,
+      "pad_sequence": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "testsets/Vid4/GT",
+      "dataroot_lq": "testsets/Vid4/BIx4",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      8,
+      64,
+      64
+    ],
+    "window_size": [
+      8,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 4,
+    "deformable_groups": 16,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": false,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 32,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/003_train_vrt_videosr_bi_vimeo_7frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/003_train_vrt_videosr_bi_vimeo_7frames/options/003_train_vrt_videosr_bi_vimeo_7frames_220311_101027.json

@@ -0,0 +1,198 @@
+{
+  "task": "003_train_vrt_videosr_bi_vimeo_7frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "model_zoo/vrt/002_VRT_videosr_bi_REDS_16frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "log": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "options": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/options",
+    "models": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/models",
+    "images": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainVimeoDataset",
+      "dataroot_gt": "trainsets/vimeo90k",
+      "dataroot_lq": "trainsets/vimeo90k",
+      "meta_info_file": "data/meta_info/meta_info_Vimeo90K_train_GT.txt",
+      "io_backend": {
+        "type": "file"
+      },
+      "num_frame": -1,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": true,
+      "use_hflip": true,
+      "use_rot": true,
+      "pad_sequence": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "testsets/Vid4/GT",
+      "dataroot_lq": "testsets/Vid4/BIx4",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      8,
+      64,
+      64
+    ],
+    "window_size": [
+      8,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 4,
+    "deformable_groups": 16,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": false,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 32,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/003_train_vrt_videosr_bi_vimeo_7frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/003_train_vrt_videosr_bi_vimeo_7frames/options/003_train_vrt_videosr_bi_vimeo_7frames_220311_101042.json

@@ -0,0 +1,198 @@
+{
+  "task": "003_train_vrt_videosr_bi_vimeo_7frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "model_zoo/vrt/002_VRT_videosr_bi_REDS_16frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "log": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "options": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/options",
+    "models": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/models",
+    "images": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainVimeoDataset",
+      "dataroot_gt": "trainsets/vimeo90k",
+      "dataroot_lq": "trainsets/vimeo90k",
+      "meta_info_file": "data/meta_info/meta_info_Vimeo90K_train_GT.txt",
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": true,
+      "use_hflip": true,
+      "use_rot": true,
+      "pad_sequence": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "testsets/Vid4/GT",
+      "dataroot_lq": "testsets/Vid4/BIx4",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      8,
+      64,
+      64
+    ],
+    "window_size": [
+      8,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 4,
+    "deformable_groups": 16,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": false,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 32,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/003_train_vrt_videosr_bi_vimeo_7frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/experiments/003_train_vrt_videosr_bi_vimeo_7frames/options/003_train_vrt_videosr_bi_vimeo_7frames_220311_101058.json

@@ -0,0 +1,198 @@
+{
+  "task": "003_train_vrt_videosr_bi_vimeo_7frames",
+  "model": "vrt",
+  "gpu_ids": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7
+  ],
+  "dist": false,
+  "find_unused_parameters": false,
+  "use_static_graph": true,
+  "scale": 4,
+  "n_channels": 3,
+  "path": {
+    "root": "experiments",
+    "pretrained_netG": "model_zoo/vrt/002_VRT_videosr_bi_REDS_16frames.pth",
+    "pretrained_netE": null,
+    "task": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "log": "experiments/003_train_vrt_videosr_bi_vimeo_7frames",
+    "options": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/options",
+    "models": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/models",
+    "images": "experiments/003_train_vrt_videosr_bi_vimeo_7frames/images",
+    "pretrained_optimizerG": null
+  },
+  "datasets": {
+    "train": {
+      "name": "train_dataset",
+      "dataset_type": "VideoRecurrentTrainVimeoDataset",
+      "dataroot_gt": "trainsets/vimeo90k",
+      "dataroot_lq": "trainsets/vimeo90k",
+      "meta_info_file": "data/meta_info/meta_info_Vimeo90K_train_GT.txt",
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "gt_size": 256,
+      "interval_list": [
+        1
+      ],
+      "random_reverse": true,
+      "use_hflip": true,
+      "use_rot": true,
+      "pad_sequence": true,
+      "dataloader_shuffle": true,
+      "dataloader_num_workers": 32,
+      "dataloader_batch_size": 8,
+      "phase": "train",
+      "scale": 4,
+      "n_channels": 3
+    },
+    "test": {
+      "name": "test_dataset",
+      "dataset_type": "VideoRecurrentTestDataset",
+      "dataroot_gt": "testsets/Vid4/GT",
+      "dataroot_lq": "testsets/Vid4/BIx4",
+      "cache_data": true,
+      "io_backend": {
+        "type": "disk"
+      },
+      "num_frame": -1,
+      "phase": "test",
+      "scale": 4,
+      "n_channels": 3
+    }
+  },
+  "netG": {
+    "net_type": "vrt",
+    "upscale": 4,
+    "img_size": [
+      8,
+      64,
+      64
+    ],
+    "window_size": [
+      8,
+      8,
+      8
+    ],
+    "depths": [
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      8,
+      4,
+      4,
+      4,
+      4,
+      4,
+      4
+    ],
+    "indep_reconsts": [
+      11,
+      12
+    ],
+    "embed_dims": [
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      120,
+      180,
+      180,
+      180,
+      180,
+      180,
+      180
+    ],
+    "num_heads": [
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6,
+      6
+    ],
+    "spynet_path": "model_zoo/vrt/spynet_sintel_final-3d2a1287.pth",
+    "pa_frames": 4,
+    "deformable_groups": 16,
+    "nonblind_denoising": false,
+    "use_checkpoint_attn": false,
+    "use_checkpoint_ffn": false,
+    "no_checkpoint_attn_blocks": [],
+    "no_checkpoint_ffn_blocks": [],
+    "init_type": "default",
+    "scale": 4
+  },
+  "train": {
+    "G_lossfn_type": "charbonnier",
+    "G_lossfn_weight": 1.0,
+    "G_charbonnier_eps": 1e-09,
+    "E_decay": 0,
+    "G_optimizer_type": "adam",
+    "G_optimizer_lr": 0.0004,
+    "G_optimizer_betas": [
+      0.9,
+      0.99
+    ],
+    "G_optimizer_wd": 0,
+    "G_optimizer_clipgrad": null,
+    "G_optimizer_reuse": true,
+    "fix_iter": 20000,
+    "fix_lr_mul": 0.125,
+    "fix_keys": [
+      "spynet",
+      "deform"
+    ],
+    "total_iter": 300000,
+    "G_scheduler_type": "CosineAnnealingWarmRestarts",
+    "G_scheduler_periods": 300000,
+    "G_scheduler_eta_min": 1e-07,
+    "G_regularizer_orthstep": null,
+    "G_regularizer_clipstep": null,
+    "G_param_strict": false,
+    "E_param_strict": true,
+    "checkpoint_test": 5000,
+    "checkpoint_save": 5000,
+    "checkpoint_print": 200,
+    "F_feature_layer": 34,
+    "F_weights": 1.0,
+    "F_lossfn_type": "l1",
+    "F_use_input_norm": true,
+    "F_use_range_norm": false,
+    "G_scheduler_restart_weights": 1
+  },
+  "val": {
+    "save_img": false,
+    "pad_seq": false,
+    "flip_seq": false,
+    "center_frame_only": false,
+    "num_frame_testing": 32,
+    "num_frame_overlapping": 2,
+    "size_patch_testing": 128
+  },
+  "opt_path": "options/vrt/003_train_vrt_videosr_bi_vimeo_7frames.json",
+  "is_train": true,
+  "merge_bn": false,
+  "merge_bn_startpoint": -1,
+  "num_gpu": 8,
+  "rank": 0,
+  "world_size": 1
+}

KAIR/image_degradation.py

@@ -0,0 +1,106 @@
+import math
+import os
+
+import numpy as np
+from basicsr.data.degradations import circular_lowpass_kernel, random_mixed_kernels
+from basicsr.utils import DiffJPEG, USMSharp
+from numpy.typing import NDArray
+from PIL import Image
+from torch import Tensor
+from torch.nn import functional as F
+
+from data.degradations import apply_real_esrgan_degradations
+from utils.utils_video import img2tensor
+
+
+blur_kernel_list1 = ['iso', 'aniso', 'generalized_iso',
+                     'generalized_aniso', 'plateau_iso', 'plateau_aniso']
+blur_kernel_list2 = ['iso', 'aniso', 'generalized_iso',
+                     'generalized_aniso', 'plateau_iso', 'plateau_aniso']
+blur_kernel_prob1 = [0.45, 0.25, 0.12, 0.03, 0.12, 0.03]
+blur_kernel_prob2 = [0.45, 0.25, 0.12, 0.03, 0.12, 0.03]
+kernel_size = 21
+blur_sigma1 = [0.05, 0.2]
+blur_sigma2 = [0.05, 0.1]
+betag_range1 = [0.7, 1.3]
+betag_range2 = [0.7, 1.3]
+betap_range1 = [0.7, 1.3]
+betap_range2 = [0.7, 1.3]
+
+
+
+
+def degrade_imgs(src_folder: str, dst_folder: str, degrade_scale: float, start_size: int) -> None:
+    src_img_filenames = os.listdir(src_folder)
+    jpeg_simulator = DiffJPEG()
+    usm_sharpener = USMSharp()
+    for src_img_filename in src_img_filenames:
+        src_img = Image.open(os.path.join(src_folder, src_img_filename))
+
+        src_tensor = img2tensor(np.array(src_img), bgr2rgb=False,
+                                float32=True).unsqueeze(0) / 255.0
+        orig_h, orig_w = src_tensor.size()[2:4]
+        print("SRC TENSOR orig size: ", src_tensor.size())
+        if orig_h != start_size or orig_w != start_size:
+            src_tensor = F.interpolate(src_tensor, size=(start_size, start_size), mode='bicubic')
+            print("SRC TENSOR new size: ", src_tensor.size())
+
+        blur_kernel1, blur_kernel2, sinc_kernel = _decide_kernels()
+        (src, src_sharp, degraded_img) = apply_real_esrgan_degradations(
+            src_tensor,
+            blur_kernel1=Tensor(blur_kernel1).unsqueeze(0),
+            blur_kernel2=Tensor(blur_kernel2).unsqueeze(0),
+            second_blur_prob=0.4,
+            sinc_kernel=Tensor(sinc_kernel).unsqueeze(0),
+            resize_prob1=[0.2, 0.7, 0.1],
+            resize_prob2=[0.3, 0.4, 0.3],
+            resize_range1=[0.9, 1.1],
+            resize_range2=[0.9, 1.1],
+            gray_noise_prob1=0.2,
+            gray_noise_prob2=0.2,
+            gaussian_noise_prob1=0.2,
+            gaussian_noise_prob2=0.2,
+            noise_range=[0.01, 0.2],
+            poisson_scale_range=[0.05, 0.45],
+            jpeg_compression_range1=[85, 100],
+            jpeg_compression_range2=[85, 100],
+            jpeg_simulator=jpeg_simulator,
+            random_crop_gt_size=start_size,
+            sr_upsample_scale=1,
+            usm_sharpener=usm_sharpener
+        )
+
+        # print(src.size())
+        # print(src_sharp.size())
+        # print(degraded_img.size())
+        # print(torch.max(src))
+        # print(torch.max(src_sharp))
+        # print(torch.max(degraded_img))
+        # print(torch.min(src))
+        # print(torch.min(src_sharp))
+        # print(torch.min(degraded_img))
+        # Image.fromarray((src[0] * 255.0).permute(1, 2, 0).cpu().numpy().astype(np.uint8)).save(
+        #     "/home/cll/Desktop/TEST_IMAGE1.png")
+        # Image.fromarray((src_sharp[0] * 255.0).permute(
+        #     1, 2, 0).cpu().numpy().astype(np.uint8)).save(
+        #         "/home/cll/Desktop/TEST_IMAGE2.png")
+
+        Image.fromarray((degraded_img[0] * 255.0).permute(
+            1, 2, 0).cpu().numpy().astype(np.uint8)).save(
+                os.path.join(dst_folder, src_img_filename))
+        print("SAVED %s: " % src_img_filename)
+
+        # Image.fromarray((src_tensor[0] * 255.0).permute(
+        #     1, 2, 0).cpu().numpy().astype(np.uint8)).save(
+        #         os.path.join(dst_folder, src_img_filename))
+        # print("SAVED %s: " % src_img_filename)
+
+
+if __name__ == "__main__":
+    SRC_FOLDER = "/home/cll/Desktop/sr_test_GT_HQ"
+    OUTPUT_RESOLUTION_SCALE = 1
+    DST_FOLDER = "/home/cll/Desktop/sr_test_degraded_LQ_512"
+    # DST_FOLDER = "/home/cll/Desktop/sr_test_GT_512"
+    os.makedirs(DST_FOLDER, exist_ok=True)
+
+    degrade_imgs(SRC_FOLDER, DST_FOLDER, OUTPUT_RESOLUTION_SCALE, 512)

KAIR/main_challenge_sr.py

@@ -0,0 +1,174 @@
+import os.path
+import logging
+import time
+from collections import OrderedDict
+import torch
+
+from utils import utils_logger
+from utils import utils_image as util
+# from utils import utils_model
+
+
+'''
+This code can help you to calculate:
+`FLOPs`, `#Params`, `Runtime`, `#Activations`, `#Conv`, and `Max Memory Allocated`.
+
+- `#Params' denotes the total number of parameters. 
+- `FLOPs' is the abbreviation for floating point operations. 
+- `#Activations' measures the number of elements of all outputs of convolutional layers. 
+- `Memory' represents maximum GPU memory consumption according to the PyTorch function torch.cuda.max_memory_allocated().
+- `#Conv' represents the number of convolutional layers. 
+- `FLOPs', `#Activations', and `Memory' are tested on an LR image of size 256x256.
+
+For more information, please refer to ECCVW paper "AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results".
+
+# If you use this code, please consider the following citations:
+
+@inproceedings{zhang2020aim,
+  title={AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results},
+  author={Kai Zhang and Martin Danelljan and Yawei Li and Radu Timofte and others},
+  booktitle={European Conference on Computer Vision Workshops},
+  year={2020}
+}
+@inproceedings{zhang2019aim,
+  title={AIM 2019 Challenge on Constrained Super-Resolution: Methods and Results},
+  author={Kai Zhang and Shuhang Gu and Radu Timofte and others},
+  booktitle={IEEE International Conference on Computer Vision Workshops},
+  year={2019}
+}
+
+CuDNN (https://developer.nvidia.com/rdp/cudnn-archive) should be installed.
+
+For `Memery` and `Runtime`, set 'print_modelsummary = False' and 'save_results = False'.
+'''
+
+
+
+
+def main():
+
+    utils_logger.logger_info('efficientsr_challenge', log_path='efficientsr_challenge.log')
+    logger = logging.getLogger('efficientsr_challenge')
+
+#    print(torch.__version__)               # pytorch version
+#    print(torch.version.cuda)              # cuda version
+#    print(torch.backends.cudnn.version())  # cudnn version
+
+    # --------------------------------
+    # basic settings
+    # --------------------------------
+    model_names = ['msrresnet', 'imdn']
+    model_id = 1                  # set the model name
+    sf = 4
+    model_name = model_names[model_id]
+    logger.info('{:>16s} : {:s}'.format('Model Name', model_name))
+
+    testsets = 'testsets'         # set path of testsets
+    testset_L = 'DIV2K_valid_LR'  # set current testing dataset; 'DIV2K_test_LR'
+    testset_L = 'set12'
+
+    save_results = True
+    print_modelsummary = True     # set False when calculating `Max Memery` and `Runtime`
+
+    torch.cuda.set_device(0)      # set GPU ID
+    logger.info('{:>16s} : {:<d}'.format('GPU ID', torch.cuda.current_device()))
+    torch.cuda.empty_cache()
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+    # --------------------------------
+    # define network and load model
+    # --------------------------------
+    if model_name == 'msrresnet':
+        from models.network_msrresnet import MSRResNet1 as net
+        model = net(in_nc=3, out_nc=3, nc=64, nb=16, upscale=4)  # define network
+        model_path = os.path.join('model_zoo', 'msrresnet_x4_psnr.pth')  # set model path
+    elif model_name == 'imdn':
+        from models.network_imdn import IMDN as net
+        model = net(in_nc=3, out_nc=3, nc=64, nb=8, upscale=4, act_mode='L', upsample_mode='pixelshuffle')  # define network
+        model_path = os.path.join('model_zoo', 'imdn_x4.pth')            # set model path
+
+    model.load_state_dict(torch.load(model_path), strict=True)
+    model.eval()
+    for k, v in model.named_parameters():
+        v.requires_grad = False
+    model = model.to(device)
+
+    # --------------------------------
+    # print model summary
+    # --------------------------------
+    if print_modelsummary:
+        from utils.utils_modelsummary import get_model_activation, get_model_flops
+        input_dim = (3, 256, 256)  # set the input dimension
+
+        activations, num_conv2d = get_model_activation(model, input_dim)
+        logger.info('{:>16s} : {:<.4f} [M]'.format('#Activations', activations/10**6))
+        logger.info('{:>16s} : {:<d}'.format('#Conv2d', num_conv2d))
+
+        flops = get_model_flops(model, input_dim, False)
+        logger.info('{:>16s} : {:<.4f} [G]'.format('FLOPs', flops/10**9))
+
+        num_parameters = sum(map(lambda x: x.numel(), model.parameters()))
+        logger.info('{:>16s} : {:<.4f} [M]'.format('#Params', num_parameters/10**6))
+
+    # --------------------------------
+    # read image
+    # --------------------------------
+    L_path = os.path.join(testsets, testset_L)
+    E_path = os.path.join(testsets, testset_L+'_'+model_name)
+    util.mkdir(E_path)
+
+    # record runtime
+    test_results = OrderedDict()
+    test_results['runtime'] = []
+
+    logger.info('{:>16s} : {:s}'.format('Input Path', L_path))
+    logger.info('{:>16s} : {:s}'.format('Output Path', E_path))
+    idx = 0
+
+    start = torch.cuda.Event(enable_timing=True)
+    end = torch.cuda.Event(enable_timing=True)
+
+    for img in util.get_image_paths(L_path):
+
+        # --------------------------------
+        # (1) img_L
+        # --------------------------------
+        idx += 1
+        img_name, ext = os.path.splitext(os.path.basename(img))
+        logger.info('{:->4d}--> {:>10s}'.format(idx, img_name+ext))
+
+        img_L = util.imread_uint(img, n_channels=3)
+        img_L = util.uint2tensor4(img_L)
+        torch.cuda.empty_cache()
+        img_L = img_L.to(device)
+
+        start.record()
+        img_E = model(img_L)
+        # img_E = utils_model.test_mode(model, img_L, mode=2, min_size=480, sf=sf)  # use this to avoid 'out of memory' issue.
+        # logger.info('{:>16s} : {:<.3f} [M]'.format('Max Memery', torch.cuda.max_memory_allocated(torch.cuda.current_device())/1024**2))  # Memery
+        end.record()
+        torch.cuda.synchronize()
+        test_results['runtime'].append(start.elapsed_time(end))  # milliseconds
+
+
+#        torch.cuda.synchronize()
+#        start = time.time()
+#        img_E = model(img_L)
+#        torch.cuda.synchronize()
+#        end = time.time()
+#        test_results['runtime'].append(end-start)  # seconds
+
+        # --------------------------------
+        # (2) img_E
+        # --------------------------------
+        img_E = util.tensor2uint(img_E)
+
+        if save_results:
+            util.imsave(img_E, os.path.join(E_path, img_name+ext))
+    ave_runtime = sum(test_results['runtime']) / len(test_results['runtime']) / 1000.0
+    logger.info('------> Average runtime of ({}) is : {:.6f} seconds'.format(L_path, ave_runtime))
+
+
+if __name__ == '__main__':
+
+    main()

KAIR/main_download_pretrained_models.py

@@ -0,0 +1,141 @@
+import argparse
+import os
+import requests
+import re
+
+
+"""
+How to use:
+download all the models:
+    python main_download_pretrained_models.py --models "all"  --model_dir "model_zoo"
+
+download DnCNN models:
+    python main_download_pretrained_models.py --models "DnCNN" --model_dir "model_zoo"
+
+download SRMD models:
+    python main_download_pretrained_models.py --models "SRMD" --model_dir "model_zoo"
+
+download BSRGAN models:
+    python main_download_pretrained_models.py --models "BSRGAN" --model_dir "model_zoo"
+
+download FFDNet models:
+    python main_download_pretrained_models.py --models "FFDNet" --model_dir "model_zoo"
+
+download DPSR models:
+    python main_download_pretrained_models.py --models "DPSR" --model_dir "model_zoo"
+
+download SwinIR models:
+    python main_download_pretrained_models.py --models "SwinIR" --model_dir "model_zoo"
+
+download VRT models:
+    python main_download_pretrained_models.py --models "VRT" --model_dir "model_zoo"
+    
+download other models:
+    python main_download_pretrained_models.py --models "others" --model_dir "model_zoo"
+
+------------------------------------------------------------------
+
+download 'dncnn_15.pth' and 'dncnn_50.pth'
+    python main_download_pretrained_models.py --models "dncnn_15.pth dncnn_50.pth" --model_dir "model_zoo"
+
+------------------------------------------------------------------
+
+download DnCNN models and 'BSRGAN.pth'
+    python main_download_pretrained_models.py --models "DnCNN BSRGAN.pth" --model_dir "model_zoo"
+
+"""
+
+
+def download_pretrained_model(model_dir='model_zoo', model_name='dncnn3.pth'):
+    if os.path.exists(os.path.join(model_dir, model_name)):
+        print(f'already exists, skip downloading [{model_name}]')
+    else:
+        os.makedirs(model_dir, exist_ok=True)
+        if 'SwinIR' in model_name:
+            url = 'https://github.com/JingyunLiang/SwinIR/releases/download/v0.0/{}'.format(model_name)
+        elif 'VRT' in model_name:
+            url = 'https://github.com/JingyunLiang/VRT/releases/download/v0.0/{}'.format(model_name)
+        else:
+            url = 'https://github.com/cszn/KAIR/releases/download/v1.0/{}'.format(model_name)
+        r = requests.get(url, allow_redirects=True)
+        print(f'downloading [{model_dir}/{model_name}] ...')
+        open(os.path.join(model_dir, model_name), 'wb').write(r.content)
+        print('done!')
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--models',
+                        type=lambda s: re.split(' |, ', s),
+                        default = "dncnn3.pth",
+                        help='comma or space delimited list of characters, e.g., "DnCNN", "DnCNN BSRGAN.pth", "dncnn_15.pth dncnn_50.pth"')
+    parser.add_argument('--model_dir', type=str, default='model_zoo', help='path of model_zoo')
+    args = parser.parse_args()
+
+    print(f'trying to download {args.models}')
+
+    method_model_zoo = {'DnCNN': ['dncnn_15.pth', 'dncnn_25.pth', 'dncnn_50.pth', 'dncnn3.pth', 'dncnn_color_blind.pth', 'dncnn_gray_blind.pth'],
+                        'SRMD': ['srmdnf_x2.pth', 'srmdnf_x3.pth', 'srmdnf_x4.pth', 'srmd_x2.pth', 'srmd_x3.pth', 'srmd_x4.pth'],
+                        'DPSR': ['dpsr_x2.pth', 'dpsr_x3.pth', 'dpsr_x4.pth', 'dpsr_x4_gan.pth'],
+                        'FFDNet': ['ffdnet_color.pth', 'ffdnet_gray.pth', 'ffdnet_color_clip.pth', 'ffdnet_gray_clip.pth'],
+                        'USRNet': ['usrgan.pth', 'usrgan_tiny.pth', 'usrnet.pth', 'usrnet_tiny.pth'],
+                        'DPIR': ['drunet_gray.pth', 'drunet_color.pth', 'drunet_deblocking_color.pth', 'drunet_deblocking_grayscale.pth'],
+                        'BSRGAN': ['BSRGAN.pth', 'BSRNet.pth', 'BSRGANx2.pth'],
+                        'IRCNN': ['ircnn_color.pth', 'ircnn_gray.pth'],
+                        'SwinIR': ['001_classicalSR_DF2K_s64w8_SwinIR-M_x2.pth', '001_classicalSR_DF2K_s64w8_SwinIR-M_x3.pth', 
+                                   '001_classicalSR_DF2K_s64w8_SwinIR-M_x4.pth', '001_classicalSR_DF2K_s64w8_SwinIR-M_x8.pth', 
+                                   '001_classicalSR_DIV2K_s48w8_SwinIR-M_x2.pth', '001_classicalSR_DIV2K_s48w8_SwinIR-M_x3.pth', 
+                                   '001_classicalSR_DIV2K_s48w8_SwinIR-M_x4.pth', '001_classicalSR_DIV2K_s48w8_SwinIR-M_x8.pth', 
+                                   '002_lightweightSR_DIV2K_s64w8_SwinIR-S_x2.pth', '002_lightweightSR_DIV2K_s64w8_SwinIR-S_x3.pth', 
+                                   '002_lightweightSR_DIV2K_s64w8_SwinIR-S_x4.pth', '003_realSR_BSRGAN_DFO_s64w8_SwinIR-M_x4_GAN.pth', 
+                                   '003_realSR_BSRGAN_DFO_s64w8_SwinIR-M_x4_PSNR.pth', '004_grayDN_DFWB_s128w8_SwinIR-M_noise15.pth', 
+                                   '004_grayDN_DFWB_s128w8_SwinIR-M_noise25.pth', '004_grayDN_DFWB_s128w8_SwinIR-M_noise50.pth', 
+                                   '005_colorDN_DFWB_s128w8_SwinIR-M_noise15.pth', '005_colorDN_DFWB_s128w8_SwinIR-M_noise25.pth', 
+                                   '005_colorDN_DFWB_s128w8_SwinIR-M_noise50.pth', '006_CAR_DFWB_s126w7_SwinIR-M_jpeg10.pth', 
+                                   '006_CAR_DFWB_s126w7_SwinIR-M_jpeg20.pth', '006_CAR_DFWB_s126w7_SwinIR-M_jpeg30.pth', 
+                                   '006_CAR_DFWB_s126w7_SwinIR-M_jpeg40.pth'],
+                        'VRT': ['001_VRT_videosr_bi_REDS_6frames.pth', '002_VRT_videosr_bi_REDS_16frames.pth',
+                                   '003_VRT_videosr_bi_Vimeo_7frames.pth', '004_VRT_videosr_bd_Vimeo_7frames.pth',
+                                   '005_VRT_videodeblurring_DVD.pth', '006_VRT_videodeblurring_GoPro.pth',
+                                   '007_VRT_videodeblurring_REDS.pth', '008_VRT_videodenoising_DAVIS.pth'],
+                        'others': ['msrresnet_x4_psnr.pth', 'msrresnet_x4_gan.pth', 'imdn_x4.pth', 'RRDB.pth', 'ESRGAN.pth', 
+                                   'FSSR_DPED.pth', 'FSSR_JPEG.pth', 'RealSR_DPED.pth', 'RealSR_JPEG.pth']
+                        }
+
+    method_zoo = list(method_model_zoo.keys())
+    model_zoo = []
+    for b in list(method_model_zoo.values()):
+        model_zoo += b
+
+    if 'all' in args.models:
+        for method in method_zoo:
+            for model_name in method_model_zoo[method]:
+                download_pretrained_model(args.model_dir, model_name)
+    else:
+        for method_model in args.models:
+            if method_model in method_zoo:  # method, need for loop
+                for model_name in method_model_zoo[method_model]:
+                    if 'SwinIR' in model_name:
+                        download_pretrained_model(os.path.join(args.model_dir, 'swinir'), model_name)
+                    elif 'VRT' in model_name:
+                        download_pretrained_model(os.path.join(args.model_dir, 'vrt'), model_name)
+                    else:
+                        download_pretrained_model(args.model_dir, model_name)
+            elif method_model in model_zoo:  # model, do not need for loop
+                if 'SwinIR' in method_model:
+                    download_pretrained_model(os.path.join(args.model_dir, 'swinir'), method_model)
+                elif 'VRT' in method_model:
+                    download_pretrained_model(os.path.join(args.model_dir, 'vrt'), method_model)
+                else:
+                    download_pretrained_model(args.model_dir, method_model)
+            else:
+                print(f'Do not find {method_model} from the pre-trained model zoo!')
+
+
+
+
+
+
+
+
+       

KAIR/main_test_dncnn.py

@@ -0,0 +1,203 @@
+import os.path
+import logging
+import argparse
+
+import numpy as np
+from datetime import datetime
+from collections import OrderedDict
+# from scipy.io import loadmat
+
+import torch
+
+from utils import utils_logger
+from utils import utils_model
+from utils import utils_image as util
+
+
+'''
+Spyder (Python 3.6)
+PyTorch 1.1.0
+Windows 10 or Linux
+
+Kai Zhang (cskaizhang@gmail.com)
+github: https://github.com/cszn/KAIR
+        https://github.com/cszn/DnCNN
+
+@article{zhang2017beyond,
+  title={Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising},
+  author={Zhang, Kai and Zuo, Wangmeng and Chen, Yunjin and Meng, Deyu and Zhang, Lei},
+  journal={IEEE Transactions on Image Processing},
+  volume={26},
+  number={7},
+  pages={3142--3155},
+  year={2017},
+  publisher={IEEE}
+}
+
+% If you have any question, please feel free to contact with me.
+% Kai Zhang (e-mail: cskaizhang@gmail.com; github: https://github.com/cszn)
+
+by Kai Zhang (12/Dec./2019)
+'''
+
+"""
+# --------------------------------------------
+|--model_zoo          # model_zoo
+   |--dncnn_15        # model_name
+   |--dncnn_25
+   |--dncnn_50
+   |--dncnn_gray_blind
+   |--dncnn_color_blind
+   |--dncnn3
+|--testset            # testsets
+   |--set12           # testset_name
+   |--bsd68
+   |--cbsd68
+|--results            # results
+   |--set12_dncnn_15  # result_name = testset_name + '_' + model_name
+   |--set12_dncnn_25
+   |--bsd68_dncnn_15
+# --------------------------------------------
+"""
+
+
+def main():
+
+    # ----------------------------------------
+    # Preparation
+    # ----------------------------------------
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--model_name', type=str, default='dncnn_25', help='dncnn_15, dncnn_25, dncnn_50, dncnn_gray_blind, dncnn_color_blind, dncnn3')
+    parser.add_argument('--testset_name', type=str, default='set12', help='test set, bsd68 | set12')
+    parser.add_argument('--noise_level_img', type=int, default=15, help='noise level: 15, 25, 50')
+    parser.add_argument('--x8', type=bool, default=False, help='x8 to boost performance')
+    parser.add_argument('--show_img', type=bool, default=False, help='show the image')
+    parser.add_argument('--model_pool', type=str, default='model_zoo', help='path of model_zoo')
+    parser.add_argument('--testsets', type=str, default='testsets', help='path of testing folder')
+    parser.add_argument('--results', type=str, default='results', help='path of results')
+    parser.add_argument('--need_degradation', type=bool, default=True, help='add noise or not')
+    parser.add_argument('--task_current', type=str, default='dn', help='dn for denoising, fixed!')
+    parser.add_argument('--sf', type=int, default=1, help='unused for denoising')
+    args = parser.parse_args()
+
+    if 'color' in args.model_name:
+        n_channels = 3        # fixed, 1 for grayscale image, 3 for color image
+    else:
+        n_channels = 1        # fixed for grayscale image
+    if args.model_name in ['dncnn_gray_blind', 'dncnn_color_blind', 'dncnn3']:
+        nb = 20               # fixed
+    else:
+        nb = 17               # fixed
+
+    result_name = args.testset_name + '_' + args.model_name     # fixed
+    border = args.sf if args.task_current == 'sr' else 0        # shave boader to calculate PSNR and SSIM
+    model_path = os.path.join(args.model_pool, args.model_name+'.pth')
+
+    # ----------------------------------------
+    # L_path, E_path, H_path
+    # ----------------------------------------
+
+    L_path = os.path.join(args.testsets, args.testset_name) # L_path, for Low-quality images
+    H_path = L_path                               # H_path, for High-quality images
+    E_path = os.path.join(args.results, result_name)   # E_path, for Estimated images
+    util.mkdir(E_path)
+
+    if H_path == L_path:
+        args.need_degradation = True
+    logger_name = result_name
+    utils_logger.logger_info(logger_name, log_path=os.path.join(E_path, logger_name+'.log'))
+    logger = logging.getLogger(logger_name)
+
+    need_H = True if H_path is not None else False
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+    # ----------------------------------------
+    # load model
+    # ----------------------------------------
+
+    from models.network_dncnn import DnCNN as net
+    model = net(in_nc=n_channels, out_nc=n_channels, nc=64, nb=nb, act_mode='R')
+    # model = net(in_nc=n_channels, out_nc=n_channels, nc=64, nb=nb, act_mode='BR')  # use this if BN is not merged by utils_bnorm.merge_bn(model)
+    model.load_state_dict(torch.load(model_path), strict=True)
+    model.eval()
+    for k, v in model.named_parameters():
+        v.requires_grad = False
+    model = model.to(device)
+    logger.info('Model path: {:s}'.format(model_path))
+    number_parameters = sum(map(lambda x: x.numel(), model.parameters()))
+    logger.info('Params number: {}'.format(number_parameters))
+
+    test_results = OrderedDict()
+    test_results['psnr'] = []
+    test_results['ssim'] = []
+
+    logger.info('model_name:{}, image sigma:{}'.format(args.model_name, args.noise_level_img))
+    logger.info(L_path)
+    L_paths = util.get_image_paths(L_path)
+    H_paths = util.get_image_paths(H_path) if need_H else None
+
+    for idx, img in enumerate(L_paths):
+
+        # ------------------------------------
+        # (1) img_L
+        # ------------------------------------
+
+        img_name, ext = os.path.splitext(os.path.basename(img))
+        # logger.info('{:->4d}--> {:>10s}'.format(idx+1, img_name+ext))
+        img_L = util.imread_uint(img, n_channels=n_channels)
+        img_L = util.uint2single(img_L)
+
+        if args.need_degradation:  # degradation process
+            np.random.seed(seed=0)  # for reproducibility
+            img_L += np.random.normal(0, args.noise_level_img/255., img_L.shape)
+
+        util.imshow(util.single2uint(img_L), title='Noisy image with noise level {}'.format(args.noise_level_img)) if args.show_img else None
+
+        img_L = util.single2tensor4(img_L)
+        img_L = img_L.to(device)
+
+        # ------------------------------------
+        # (2) img_E
+        # ------------------------------------
+
+        if not args.x8:
+            img_E = model(img_L)
+        else:
+            img_E = utils_model.test_mode(model, img_L, mode=3)
+
+        img_E = util.tensor2uint(img_E)
+
+        if need_H:
+
+            # --------------------------------
+            # (3) img_H
+            # --------------------------------
+
+            img_H = util.imread_uint(H_paths[idx], n_channels=n_channels)
+            img_H = img_H.squeeze()
+
+            # --------------------------------
+            # PSNR and SSIM
+            # --------------------------------
+
+            psnr = util.calculate_psnr(img_E, img_H, border=border)
+            ssim = util.calculate_ssim(img_E, img_H, border=border)
+            test_results['psnr'].append(psnr)
+            test_results['ssim'].append(ssim)
+            logger.info('{:s} - PSNR: {:.2f} dB; SSIM: {:.4f}.'.format(img_name+ext, psnr, ssim))
+            util.imshow(np.concatenate([img_E, img_H], axis=1), title='Recovered / Ground-truth') if args.show_img else None
+
+        # ------------------------------------
+        # save results
+        # ------------------------------------
+
+        util.imsave(img_E, os.path.join(E_path, img_name+ext))
+
+    if need_H:
+        ave_psnr = sum(test_results['psnr']) / len(test_results['psnr'])
+        ave_ssim = sum(test_results['ssim']) / len(test_results['ssim'])
+        logger.info('Average PSNR/SSIM(RGB) - {} - PSNR: {:.2f} dB; SSIM: {:.4f}'.format(result_name, ave_psnr, ave_ssim))
+
+if __name__ == '__main__':
+
+    main()