Moving object recognition with optical flow-based fusion of thermal and visible videos

Jing Chen; Chao Liang; Yu Lan; Youtian Du

doi:10.1117/12.2635673

6 May 2022 Moving object recognition with optical flow-based fusion of thermal and visible videos

Jing Chen, Chao Liang, Yu Lan, Youtian Du

Proceedings Volume 12256, International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2022); 122562A (2022) https://doi.org/10.1117/12.2635673
Event: 2022 International Conference on Electronic Information Engineering, Big Data and Computer Technology, 2022, Sanya, China

Abstract

Thermal and visible videos can provide complementary information in the moving object detection and recognition. However, most previous approaches focus on the detection and recognition of moving objects from visible videos. In this paper, we present a two-stage approach to moving object recognition by jointly utilizing the thermal and visible videos. In the first stage, we extract the static appearance and the optical flow of moving objects from both sources of videos based on deep networks and generate the bounding box proposals of moving objects. In this stage, two sources of video frames need to be first registered to cover the same scenes with the same resolution. In the second stage, we design a deep network to recognize the categories of the object proposals generated in the first stage and thus obtain the recognition results. Combining the thermal and visible information for recognizing moving objects can improve the performance especially in the low light conditions. To evaluate the proposed approach, we build a thermal-visible video dataset consisting of 200 video pairs. Experimental results demonstrate the effectiveness of the proposed approach.

Citation Download Citation

Jing Chen, Chao Liang, Yu Lan, and Youtian Du "Moving object recognition with optical flow-based fusion of thermal and visible videos", Proc. SPIE 12256, International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2022), 122562A (6 May 2022); https://doi.org/10.1117/12.2635673

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available