A novel automated method for doing registration and 3D reconstruction from multi-modal RGB/IR image sequences

Richard Kirby; Ross Whitaker

doi:10.1117/12.2237165

19 September 2016 A novel automated method for doing registration and 3D reconstruction from multi-modal RGB/IR image sequences

Richard Kirby, Ross Whitaker

Author Affiliations +

Proceedings Volume 9974, Infrared Sensors, Devices, and Applications VI; 99740O (2016) https://doi.org/10.1117/12.2237165
Event: SPIE Optical Engineering + Applications, 2016, San Diego, California, United States

Abstract

In recent years, the use of multi-modal camera rigs consisting of an RGB sensor and an infrared (IR) sensor have become increasingly popular for use in surveillance and robotics applications. The advantages of using multi-modal camera rigs include improved foreground/background segmentation, wider range of lighting conditions under which the system works, and richer information (e.g. visible light and heat signature) for target identification. However, the traditional computer vision method of mapping pairs of images using pixel intensities or image features is often not possible with an RGB/IR image pair. We introduce a novel method to overcome the lack of common features in RGB/IR image pairs by using a variational methods optimization algorithm to map the optical flow fields computed from different wavelength images. This results in the alignment of the flow fields, which in turn produce correspondences similar to those found in a stereo RGB/RGB camera rig using pixel intensities or image features. In addition to aligning the different wavelength images, these correspondences are used to generate dense disparity and depth maps. We obtain accuracies similar to other multi-modal image alignment methodologies as long as the scene contains sufficient depth variations, although a direct comparison is not possible because of the lack of standard image sets from moving multi-modal camera rigs. We test our method on synthetic optical flow fields and on real image sequences that we created with a multi-modal binocular stereo RGB/IR camera rig. We determine our method's accuracy by comparing against a ground truth.

Conference Presentation

Citation Download Citation

Richard Kirby and Ross Whitaker "A novel automated method for doing registration and 3D reconstruction from multi-modal RGB/IR image sequences", Proc. SPIE 9974, Infrared Sensors, Devices, and Applications VI, 99740O (19 September 2016); https://doi.org/10.1117/12.2237165

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available