Enhancing the Informativeness of Multispectral Images by means of Multimodal Image Fusion
DOI:
https://doi.org/10.20535/RADAP.2018.73.40-49Keywords:
image fusion, multispectral monitoring, image informativeness, wavelet transform, regression analysisAbstract
The research is devoted to the problem of combining graphic information from sensors of various physical nature in multispectral monitoring systems using methods of multimodal image fusion. Each of the sensors of the multispectral monitoring system allows to obtain digital images of the observed scene in different ranges of electromagnetic radiation. In this paper, we consider a two-spectral monitoring system, the frst sensor of which operates in the visible range, and the second in the infrared. The main problem with multimodal image fusion is that each partial sensor of the multispectral monitoring system represents specifc characteristics of the environment (brightness, thermal or radar contrasts of objects, etc). Another, no less important problem is the different spatial resolution of sensors of different physical nature and the inconsistency of their felds of view. Therefore, the problem of effective multimodal image fusion is not trivial. As a criterion for the effectiveness of combining graphical information in a single fused image, which should contain the maximum available useful information from various sensors, the informativeness of this image is chosen. A quantitative assessment of image informativeness is proposed to be performed using an improved method based on multicriteria analysis of image parameters. Multimodal image fusion is performed using the proposed method based on the discrete wavelet transform with the formation of low-frequency wavelet coefcients of the resulting wavelet spectrum by analyzing the regression communication model between the input images from different sensors. Confrmed experimentally that the proposed method of image fusion makes it possible to synthesize more informative multispectral images than known algorithms. The application of proposed method of image fusion for monitoring objects in difcult observation conditions (smoke, fog, low illumination) allows to increase the efciency of the multispectral monitoring system and signifcantly reduce the amount of redundant information coming to the operator of the system.References
Перелік посилань
Zheng Y. Image Fusion and Its Applications / Y. Zheng, ed. - Publisher: InTech. - 2011. - 252,p.
Фролов В. Н. Методы информационного совмещения изображений в многоканальных оптико-электронных системах / В. Н. Фролов, В. А. Тупиков, В. А. Павлова, В. А. Александров // Известия ТулГУ. Технические науки. - 2016. - № 11-3. - С. 95–104.
Zheng Y. An Advanced Image Fusion Algorithm Based on Wavelet Transform: Incorporation with PCA and Morphological Processing / Y. Zheng, E. A. Essock, B. C. Hansen // Proc. SPIE 5298. - 2004. - pp. 177-187.
Гривачевський А. П. Аналіз методів комплексування сигналів парціальних спектральних каналів у системах моніторингу об’єктів та сцен / А. П. Гривачевський // Вісник Національного університету "Львівська політехніка". Радіоелектроніка та телекомунікації. - 2015. - № 818. - С. 55-61.
Kondratov P. O. Detection and allotment of the objects based on multispectral monitoring / P. O. Kondratov, A. H. Ohanesyan, V. F. Tkachenko, I. N. Prudyus, L. V. Lazko, A. P. Hryvachevskyi // Modern Problems of Radio Engineering, Telecommunications and Computer Science (TCSET), Lviv. -2016. - pp. 259-262.
Пат. 94566 UA Україна, МПК G01S 13/00, G01J 3/29. Багатоспектральний виявляч наземних об’єктів / А. М. Зубков, І. Н. Прудиус, А. В. Д’яков, С. А. Мартиненко, Д. О. Мимріков, А. А. Щерба; заявник і власник патенту Національний університет «Львівська політехніка». - № а2010158036; заявл. 28.12.10; опубл. 10.05.2011, Бюл. № 9.
Бондаренко А. Аппаратно-программная реализация мультиспектральной системы улучшенного видения / А. Бондаренко, М. Бондаренко // Современная электроника. - 2017. - № 1. - С. 32-37.
Hryvachevskyi A. P. The influence of destabilizing factors in the high resolution multispectral imaging systems / A. P. Hryvachevskyi, S. E. Fabirovskyy, L. V. Lazko, V. F. Tkachenko // International Conference on Information and Telecommunication Technologies and Radio Electronics (UkrMiCo), Odessa. -2017. - pp. 1-4.
Гривачевський А. П. Суміщення зображень сформованих сенсорами різної фізичної природи в процесі комплексування сигналів в мультиспектральних системах моніторингу / А. П. Гривачевський, C. Є. Фабіровський // Вісник Національного університету "Львівська політехніка". Радіоелектроніка та телекомунікації. - 2017. - № 874. - С. 73-80.
Sahoo P. K. Image registration using mutual information with correlation for medical image / P. K. Sahoo, U. C. Pati // Global Conference on Communication Technologies (GCCT), Thuckalay. - 2015. - pp. 34-38.
Faycal S. A refined automatic co-registration method for high-resolution optical and sar images by maximizing mutual information / Saidi Faycal, Chen Jie, Wang Pengbo // IEEE International Conference on Signal and Image Processing (ICSIP). - 2016. - pp. 231-235.
Wang Zhou A universal image quality index / Zhou Wang, A. C. Bovik // IEEE Signal Processing Letters. Vol. 9, No. 3. -2002. - pp. 81-84
Wang Zhou Image Qualifty Assessment: From Error Visibility to Structural Similarity / Zhou Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli // IEEE Transactions on Image Processing. Vol. 13, Issue 4. - 2004. - pp. 600-612.
Бондаренко М. А. Оценка информативности комбинированных изображений в мультиспектральных системах технического зрения / М. А. Бондаренко, В. И. Дрыкин // Программные системы и вычислительные методы. - 2016. - № 1. - С. 64-79.
Романов Ю. Н. Оценка качества цифровых изображений / А. П. Богданов, Ю. Н. Романов // Механика, управление и информатика. - 2012. - № 9. - С. 218-226.
Кривошеев М. И. Международные стандарты по цифровому телевизионному вещанию / М. И. Кривошеев, В. Федунин // Связь и телекоммуникации. - 2007. - № 8. - С. 28-36.
Seltman H. J. Experimental Design and Analysis / H. J. Seltman. - 2015. - 414,p.
References
Zheng Y. (2011) Image Fusion and Its Applications. InTech, 252 p. DOI: 10.5772/691
Frolov V.N., Tupikov V.A., Pavlova V.A. and Aleksandrov V.A. (2016) Informational image fusion methods in multichannel optoelectronic systems. Izvestiya Tul'skogo gosudarstvennogo universiteta. Tekhnicheskie nauki, No. 11-3, pp. 95-104.
Zheng Y., Essock E.A. and Hansen B.C. (2004) An advanced image fusion algorithm based on wavelet transform: incorporation with PCA and morphological processing. Image Processing: Algorithms and Systems III. DOI: 10.1117/12.523966
Hryvachevskyi A. P. (2015) Analysis of the methods of signal data fusion of partial spectral channels in the monitoring systems of objects and scenes. Bulletin of the Lviv Polytechnic National University. Radioelectronics and Telecommunications, No. 818, pp. 55-61.
Kondratov P., Ohanesyan A., Tkachenko V., Pradyus I., Lazko L. and Hryvachevskyi A. (2016) Detection and allotment of the objects based on multispectral monitoring. 2016 13th International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science (TCSET). DOI: 10.1109/tcset.2016.7452030
Zubkov A. M., Prudyus I. N., Diakonov A. V., Martynenko S. A., Mymrikov D. O. and Sherba A. A. (2011) Multispectral detector of ground objects, Pat. UA94566.
Bondarenko A. V. and Bondarenko M. A. (2017) Apparatno-programmnaya realizatsiya mul'tispektral'noi sistemy uluchshennogo videniya [Hardware-software implementation of the multispectral system of improved vision]. Sovremennaya elektronnika, No. 1, pp. 32-37.
Hryvachevskyi A., Fabirovskyy S., Lazko L. and Tkachenko V. (2017) The influence of destabilizing factors in the high resolution multispectral imaging systems. 2017 International Conference on Information and Telecommunication Technologies and Radio Electronics (UkrMiCo). DOI: 10.1109/ukrmico.2017.8095371
Hryvachevskyi A. P. and Fabirovskyy S. E. (2017) Matching up of images which formed by sensors of different physical nature in the process of signal fusion in multispectral monitoring systems. Bulletin of the Lviv Polytechnic National University. Radioelectronics and Telecommunications, No. 874, pp. 73-80.
Sahoo P.K. and Pati U.C. (2015) Image registration using mutual information with correlation for medical image. 2015 Global Conference on Communication Technologies (GCCT). DOI: 10.1109/gcct.2015.7342619
Saidi F., Chen J. and Wang P. (2016) A refined automatic co-registration method for high-resolution optical and sar images by maximizing mutual information. 2016 IEEE International Conference on Signal and Image Processing (ICSIP). DOI: 10.1109/siprocess.2016.7888258
Wang Z. and Bovik A. (2002) A universal image quality index. IEEE Signal Processing Letters, Vol. 9, Iss. 3, pp. 81-84. DOI: 10.1109/97.995823
Wang Z., Bovik A., Sheikh H. and Simoncelli E. (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, Vol. 13, Iss. 4, pp. 600-612. DOI: 10.1109/tip.2003.819861
Bondarenko M. A. and Drynkin V. N. (2016) Assessment of the information content of image fusion in multispectral vision systems Software systems and computational methods), No. 1, pp. 64-79. DOI: 10.7256/2305-6061.2016.1.18047
Bogdanov A. P. and Romanov Yu. N. (2012) Digital images quality assessment. Mekhanika, upravlenie i informatika), No 9, pp. 218-226.
Krivosheev M. and Fedunin V. (2007) International standards for digital television broadcasting Elektronika: Nauka, Tekhnologiya, Biznes, No 8, pp. 28-36.
Seltman H. J. (2015) Experimental Design and Analysis), 227 p.
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
