Efficient dynamic fusion for multispectral pedestrian detection

Anqi Cao; Haipeng Li; Jian Yang; Xiaofei Mi; Zhong Chen

doi:10.1117/12.2692440

7 March 2024 Efficient dynamic fusion for multispectral pedestrian detection

Anqi Cao, Haipeng Li, Jian Yang, Xiaofei Mi, Zhong Chen

Proceedings Volume 13085, MIPPR 2023: Automatic Target Recognition and Navigation; 1308508 (2024) https://doi.org/10.1117/12.2692440
Event: Twelfth International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR2023), 2023, Wuhan, China

Abstract

When in an environment with insufficient light, RGB images cannot provide clear pedestrian information, while multispectral thermal imaging can provide accurate positioning information. Therefore, multispectral object detection is more reliable and robust in the open world. Most fusion methods for existing multispectral object detection are static, that is, after the network is trained, all the modalities of each piece of data are input into the network to perform static inference. However, when the light is good enough or the light is very dark, multi-spectral input will bring unnecessary noise interference and computational redundancy. Therefore, a dynamic fusion network (EDFNet) is proposed to selectively fuse RGB and thermal data, so that the network can efficiently perform multispectral fusion and improve the accuracy of object detection. The gating function included in the fusion network is able to perform modality-level decisions based on multimodal features, enabling dynamic inference on data. The Extensive experiments on multiple datasets demonstrate that the proposed fusion method can reduce the computation costs and obtain comparable detection performance.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Anqi Cao, Haipeng Li, Jian Yang, Xiaofei Mi, and Zhong Chen "Efficient dynamic fusion for multispectral pedestrian detection", Proc. SPIE 13085, MIPPR 2023: Automatic Target Recognition and Navigation, 1308508 (7 March 2024); https://doi.org/10.1117/12.2692440

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Object detection

RGB color model

Feature fusion

Image fusion

Thermography

Data fusion

Feature extraction

Show All Keywords

Keywords/Phrases

Search In:

Publication Years