Super-Resolution Image Restoration Using Convolutional Neural Network

Authors

  • O. Yu. Nedzelskyi National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute", Kyiv, Ukraine
  • N. О. Lashchevska National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute", Kyiv, Ukraine https://orcid.org/0000-0003-2148-115X

DOI:

https://doi.org/10.20535/RADAP.2023.91.79-86

Keywords:

super resolution, convolutional neural networks, signal-to-noise ratio, MSE loss, VGG loss, sampling reduction factor, encoder, decoder, desubpixel

Abstract

The main goal of the super resolution method is to create a higher resolution image from a lower resolution image. High-resolution images provide a high pixel density, hence more detail in the original image. The need for high resolution is widespread in computer vision techniques, pattern recognition applications, or general image analysis. However, high-resolution images are not always available. This is due to the fact that the conversion processes and processing methods require ultra-powerful processes, and the equipment for obtaining high-resolution images is expensive. These problems can be overcome by using image processing algorithms that are relatively inexpensive, which has led to the concept of super-resolution. This has the advantage that it can cost less and existing low-resolution imaging systems are readily available. High resolution is essential in medical imaging for diagnosis. Many applications require zooming into a specific image area, where high resolution becomes essential, such as surveillance, forensics, and satellite imaging. The method is presented in this paper, using a convolutional neural network to reproduce super-resolution images, directly performs the conversion from a low-resolution image to an image similar to the original. To speed up the output time, the proposed method performs most computational operations in low-resolution space, while reducing the sampling does not lead to information loss. The main task of the neural network is to reconstruct the distorted image and find the ideal reconstruction function, according to which, in fact, a neural network of a simple structure creates high-quality images with better performance, such as resolution, signal-to-noise ratio, with less time spent on image restoration. During the experiment, we determined an algorithm by which the proposed neural network can reconstruct any image with different types of distortion. The super-resolution method is implemented using the python 3.6 programming language and the tensorflow and tensorlayer software modules for convolutional neural networks. Graphical data of signal-to-noise ratio, structural similarity, and loss plots are obtained using the tensorboardX module.

References

References

Yang J. and Huang T. (2017). Image Super-Resolution: Historical Overview and Future Challenges. in Super-Resolution Imaging, CRC Press, pp. 1–34. doi: 10.1201/9781439819319-1.

Kim P. (2017). Convolutional Neural Network. in MATLAB Deep Learning, Berkeley, CA: Apress, pp. 121–147. doi: 10.1007/978-1-4842-2845-6.

Li Z., Liu F., Yang W., Peng S., and Zhou J. (2022). A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects. IEEE Trans Neural Netw Learn Syst, Vol. 33, No. 12, pp. 6999–7019. doi: 10.1109/TNNLS.2021.3084827.

Albawi S., Mohammed T. A., and Al-Zawi S. (2017). Understanding of a convolutional neural network. 2017 International Conference on Engineering and Technology (ICET), pp. 1–6. doi: 10.1109/ICEngTechnol.2017.8308186.

Zhao N., Wei Q., Basarab A., Dobigeon N., Kouame D., and Tourneret J.-Y. (2016). Fast Single Image Super-Resolution Using a New Analytical Solution l2-l2 Problems. IEEE Transactions on Image Processing, Vol. 25, No. 8, pp. 3683–3697. doi: 10.1109/TIP.2016.2567075.

Ignatov A. et al. (2018). PIRM Challenge on Perceptual Image Enhancement on Smartphones: Report. Cornell University. doi: 10.48550/arXiv.1810.01641.

Fattal R. (2007). Image upsampling via imposed edge statistics. ACM SIGGRAPH 2007 papers, p. 95. doi: 10.1145/1275808.1276496.

Zhang Y., Zhao D., Zhang J., Xiong R. and Gao W. (2011). Interpolation-Dependent Image Downsampling. IEEE Transactions on Image Processing, Vol. 20, No. 11, pp. 3291–3296. doi: 10.1109/TIP.2011.2158226.

Bayar B. and Stamm M. C. (2016). A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer. Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, pp. 5–10. doi: 10.1145/2909827.2930786.

Ide H. and Kurita T. (2017). Improvement of learning for CNN with ReLU activation by sparse regularization. 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2684–2691. doi: 10.1109/IJCNN.2017.7966185.

Ngernplubpla J. and Chitsobhuk O. (2019). Neuro-fuzzy profile clustering in image enhancement. 2019 7th International Electrical Engineering Congress (iEECON), pp. 1–4. doi: 10.1109/iEECON45304.2019.8938965.

Govil, R. (2000). Neural Networks in Signal Processing. In: Ruan, D. (eds) Fuzzy Systems and Soft Computing in Nuclear Engineering. Studies in Fuzziness and Soft Computing, Vol 38. doi: 10.1007/978-3-7908-1866-6_11.

Shi W. et al. (2016). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1874–1883. doi: 10.1109/CVPR.2016.207.

Ignatov A., Kobyshev N., Timofte R., and Vanhoey K. (2017). DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks. 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3297–3305. doi: 10.1109/ICCV.2017.355.

Downloads

Published

2023-03-30

How to Cite

Недзельський, . О. Ю. and Лащевська, Н. О. (2023) “Super-Resolution Image Restoration Using Convolutional Neural Network”, Visnyk NTUU KPI Seriia - Radiotekhnika Radioaparatobuduvannia, (91), pp. 79-86. doi: 10.20535/RADAP.2023.91.79-86.

Issue

No. 91 (2023)

Section

Computing methods in radio electronics

License

Copyright (c) 2023 Олександр Недзельський, Наталія Лащевська

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).