The Object Tracker for Infrared and Visual Bands based on Channel-Independent Spatially-Regularized Discriminative Correlation Filter
DOI:
https://doi.org/10.20535/RADAP.2020.83.5-16Keywords:
visual object tracking, multispectral images, discriminative correlation filters (DCF), alternating direction method of multipliers (ADMM)Abstract
The method of visual object tracking intended for the application on multispectral video sequences is considered.
Introduction. The possible techniques of multispectral information fusion for visual object tracking are considered and the use of feature based fusion approach is justified. The tracker is suggested to be implemented using the discriminative correlation filters (DCF), since this approach is known to provide the compromise in terms of tracking quality and speed.
Theoretic results. The method for channel-independent discriminative correlation filter with spatial regularization calculation based on the use of alternating direction method of multipliers (ADMM) is proposed. The calculation of DCF filter and the object localization is suggested to be performed in special feature space, which employs the multichannel FHOG features and the features that are based on the backprojection of object weighted histogram. In particular, we propose to calculate the mentioned features for each channel of the respective frame of the multispectral video sequence with subsequent concatenation of obtained features into a single tensor, which forms the joint feature space.
Conclusions. Using the VOT Challenge RGBT2019 subchallenge, it was shown that the implementation of the suggested method is competitive in terms of tracking robustness with more sophisticated approaches, including the ones that are based on the convolutional neural networks. During the experiments, it was additionally established that the increasing of context-background information gives slight tracking quality improvement compared to the basic method implementation, even when only FHOG features are used.
References
Перелік посилань
Multi-Sensor Image Fusion and Its Applications / Ed. by R. S. Blum, Z. Liu. — CRC Press, 2006. — 528 p. ISBN: 9780849334177.
Image Fusion and Its Applications / Ed. by Y. Zheng. — InTechOpen, 2011. — 242 p. ISBN: 9789533071824.
Гривачевський А. П. Пiдвищення iнформативностi мультиспектральних зображень шляхом мультимодального комплексування / А. П. Гривачевський, І. Н. Прудиус // Вісник НТУУ <<КПІ>>. Серія Радіотехніка, Радіоапаратобудування. — 2018. — № 73. — C. 40--49. DOI: 10.20535/RADAP.2018.73.40-49.
Maggio E., Cavallaro A. Video tracking. Theory and Practice. — John Wiley & Sons Ltd, 2011. — 294 p. ISBN: 9780470749647.
Kristan M. The Visual Object Tracking VOT2015 challenge results / M.Kristan, J.Matas, A.Leonardis, M.Felsberg та ін. // IEEE International Conference on Computer Vision Workshop (ICCVW), . — 2015. — P. 564-586.
Kristan M. The Visual Object Tracking VOT2016 challenge results / M.Kristan, A.Leonardis, J.Matas та ін. // European Conference on Computer Vision — ECCV 2016 Workshops. ECCV 2016. Lecture Notes in Computer Science. — Vol. 9914. — 2016. — pp. 777-823.
Kristan M. The visual object tracking VOT2017 challenge results / M.Kristan, A.Leonardis, J.Matas, M.Felsberg та ін. // ICCV2017 Workshops, Workshop on visual object tracking challenge. — 2017.
Kristan M. The sixth Visual Object Tracking VOT2018 challenge results / M.Kristan, A.Leonardis, J.Matas, M.Felsberg та ін. // Computer Vision — ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science. — Vol. 11129. — 2018. — P. 3-53.
Nam H. Learning multi-domain convolutional neural networks for visual tracking / H.Nam, B.Han // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — 2016. — P. 4293-4302.
Danelljan M. ECO: Efficient convolution operators for tracking / M.Danelljan, G.Bhat, F.S.Khan, M.Felsberg // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — 2017. — P. 6931-6939.
Bertinetto L. Fully-convolutional siamese networks for object tracking / L.Bertinetto, J.Valmadre, J.F.Henriques, A.Vedaldi, P.H.S.Torr // European Conference on Computer Vision — ECCV 2016 Workshops. ECCV 2016. Lecture Notes in Computer Science. — Vol. 9914. — 2016. — P. 850-865.
Henriques J.F. High-speed tracking with kernelized correlation filters / J.F.Henriques, R.Caseiro, P.Martins, J.Batista // IEEE Trans. Pattern Anal. Mach. Intell. (PAMI). — Vol. 37, Iss. 3. — 2015. — P. 583-596.
Kristan M. The Seventh Visual Object Tracking VOT2019 Challenge Results / M.Kristan, J.Matas, A.Leonardis, M.Felsberg та ін. // 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). — 2019.
Bolme D.S. Visual Object Tracking using Adaptive Correlation Filters / D.S.Bolme, R.J.Beveridge, B.A.Draper, Y.M.Lui // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — 2010. — P. 2544-2550.
Galoogahi H.K. Correlation Filters with Limited Boundaries / H.K.Galoogahi, T.Sim, S.Lucey // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — 2015. — P. 4630-4638.
Danelljan M. Learning spatially regularized correlation filters for visual tracking / M.Danelljan, G.Häger, F.Khan, M.Felsberg // IEEE International Conference on Computer Vision (ICCV). — 2015. — P. 4310-4318.
Galoogahi H.K. Learning background-aware correlation filters for visual tracking / H.K.Galoogahi, A.Fagg, S.Lucey // IEEE International Conference on Computer Vision (ICCV). — 2017. — P. 1144-1152.
Lukezic A. Discriminative correlation filter tracker with channel and spatial reliability / A.Lukezic, T.Vojir, L.C.Zajc, J.Matas, M.Kristan // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — 2017. — P. 6309-6318.
Danelljan M. Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking / M.Danelljan, A.Robinson, F.S.Khan, M.Felsberg // 14th European Conference on Computer Vision — ECCV 2016. ECCV 2016. Lecture Notes in Computer Science. — Vol. 9909. — 2016. — P. 472-488.
Feng L. Learning spatial-temporal regularized correlation filters for visual tracking / L.Feng, C.Tian, W.Zuo, L.Zhang, M.-H.Yang // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — 2018. — P. 4904-4913.
Varfolomieiev A. Channel-independent spatially regularized discriminative correlation filter for visual object tracking / A. Varfolomieiev // Journal of Real-Time Image Processing (RTIP). — 2020. — P. 1-11.
Boyd S. Distributed optimization and statistical learning via the Alternating Direction Method of Multipliers / S.Boyd, N.Parikh, E.Chu, B.Peleato, J.Eckstein // Foundations and Trends in Machine Learning. — Vol. 3, Iss. 1. — 2010. — P. 1-122. DOI: 10.1561/2200000016
Messerschmitt D. Stationary points of a real-valued function of a complex variable. Tech. Report, EECS, U.C. Berkeley. — 2006.
Felzenszwalb P.F. Object Detection with Discriminatively Trained Part Based Models / P.F.Felzenszwalb, R.B.Girshick, D.McAllester, D.Ramanan // IEEE Trans. Pattern Anal. Mach. Intell. (PAMI). — Vol. 32, Iss. 9. — 2010. — P. 1627-1645.
Bertinetto L. Staple: Complementary learners for real-time tracking / L.Bertinetto, J.Valmadre, S.Golodetz, O.Miksik, P.H.S.Torr // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — 2016. — P. 1401-1409.
Varfolomieiev A. Modification of the KCF tracking method for implementation on embedded hardware platforms / A.Varfolomieiev, O.Lysenko // International Conference Radio Electronics & Info Communications (UkrMiCo), Kyiv. — 2016.
References
Blum R.S., Liu Z. (2006) Multi-Sensor Image Fusion and Its Applications. CRC Press , 528 p. ISBN: 9780849334177.
Zheng Y. (2011) Image Fusion and Its Applications. InTechOpen, 242 p. ISBN: 9789533071824.
Hryvachevskyi A.P., Prudyus I.N. (2018) Enhancing the Informativeness of Multi-spectral Images by means of Multimodal Image Fusion. Visnik NTUU KPI. Ser. Radioteh. radioaparatobuduv., Iss. 73, pp. 40-49. DOI: 10.20535/RADAP.2018.73.40-49.
Maggio E., Cavallaro A. (2011) Video tracking. Theory and Practice. John Wiley & Sons Ltd, 294 p. ISBN: 9780470749647.
Kristan M., Matas J., Leonardis A., Felsberg M. et al (2015) The Visual Object Tracking VOT2015 challenge results. IEEE International Conference on Computer Vision Workshop (ICCVW)>, pp. 564-586. DOI: 10.1109/ICCVW.2015.79.
Kristan M., Leonardis A., Matas J. et al (2016) The Visual Object Tracking VOT2016 challenge results. European Conference on Computer Vision - ECCV 2016 Workshops. ECCV 2016. Lecture Notes in Computer Science, vol. 9914, pp. 777-823. DOI: 10.1007/978-3-319-48881-3_54.
Kristan M., Leonardis A., Matas J., Felsberg M. et al (2017) The visual object tracking VOT2017 challenge results. ICCV2017 Workshops, Workshop on visual object tracking challenge. DOI: 10.1109/ICCVW.2017.230.
Kristan M., Leonardis A., Matas J., Felsberg M. et al (2018) The sixth Visual Object Tracking VOT2018 challenge results. Computer Vision - ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, vol. 11129, pp. 3-53. DOI: 10.1007/978-3-030-11009-3_1.
Nam H., Han B. (2016) Learning multi-domain convolutional neural networks for visual tracking. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4293-4302. DOI: 10.1109/CVPR.2016.465.
Danelljan M., Bhat G., Khan F.S., Felsberg M. (2017) ECO: Efficient convolution operators for tracking. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6931-6939. DOI: 10.1109/CVPR.2017.733.
Bertinetto L., Valmadre J., Henriques J.F., Vedaldi A., Torr P.H.S. (2016) Fully-convolutional siamese networks for object tracking. European Conference on Computer Vision - ECCV 2016 Workshops. ECCV 2016. Lecture Notes in Computer Science, Vol. 9914, pp. 850-865. DOI: 10.1007/978-3-319-48881-3_56.
Henriques J.F., Caseiro R., Martins P., Batista J. (2015) High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), Vol. 37, Iss. 3, pp. 583-596. DOI: 10.1109/TPAMI.2014.2345390.
Kristan M., Matas J., Leonardis A., Felsberg M. et al (2019) The Seventh Visual Object Tracking VOT2019 Challenge Results. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). DOI: 10.1109/ICCVW.2019.00276.
Bolme D.S., Beveridge R.J., Draper B.A., Lui Y.M. (2010) Visual Object Tracking using Adaptive Correlation Filters. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2544-2550. DOI: 10.1109/CVPR.2010.5539960.
Galoogahi H.K., Sim T., Lucey S. (2015) Correlation Filters with Limited Boundaries. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4630-4638. DOI: 10.1109/CVPR.2015.7299094.
Danelljan M., Häger G., Khan F., Felsberg M. (2015) Learning spatially regularized correlation filters for visual tracking. IEEE International Conference on Computer Vision (ICCV), pp. 4310-4318. DOI: 10.1109/ICCV.2015.490.
Galoogahi H.K., Fagg A., Lucey S. (2017) Learning background-aware correlation filters for visual tracking. IEEE International Conference on Computer Vision (ICCV), pp. 1144-1152. DOI: 10.1109/ICCV.2017.129.
Lukezic A., Vojir T., Zajc L. C., Matas J., Kristan M. (2017) Discriminative correlation filter tracker with channel and spatial reliability. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6309-6318. DOI: 10.1007/s11263-017-1061-3.
Danelljan M., Robinson A., Khan F.S. and Felsberg M. (2016) Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking. 14th European Conference on Computer Vision - ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, Vol. 9909, pp. 472-488. DOI: 10.1007/978-3-319-46454-1_29.
Li F., Tian C., Zuo W., Zhang Lei, Yang M.-H. (2018) Learning spatial-temporal regularized correlation filters for visual tracking. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4904-4913. DOI: 10.1109/CVPR.2018.00515.
Varfolomieiev A. (2020) Channel-independent spatially regularized discriminative correlation filter for visual object tracking. Journal of Real-Time Image Processing (RTIP), DOI: 10.1007/s11554-020-00967-y.
Boyd S., Parikh N., Chu E, Peleato B., Eckstein J. (2010) Distributed optimization and statistical learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, Vol. 3, Iss. 1, pp. 1-122. DOI: 10.1561/2200000016.
Messerschmitt D. (2006) Stationary points of a real-valued function of a complex variable. Tech. Report, EECS, U.C. Berkeley.
Felzenszwalb P.F., Girshick R.B., McAllester D., Ramanan D. (2010) Object Detection with Discriminatively Trained Part Based Models. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), Vol. 32, Iss. 9, pp. 1627-1645. DOI: 10.1109/TPAMI.2009.167.
Bertinetto L., Valmadre J., Golodetz S., Miksik O., Torr P.H.S. (2016) Staple: Complementary learners for real-time tracking. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1401-1409. DOI: 10.1109/CVPR.2016.156.
Varfolomieiev A., Lysenko O. (2016) Modification of the KCF tracking method for implementation on embedded hardware platforms. International Conference Radio Electronics & Info Communications (UkrMiCo). DOI: 10.1109/UkrMiCo.2016.7739644.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 А. Ю. Варфоломєєв , Є. В. Короткий
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
