Visible and infrared image fusion based on attention and multiscale residuals

Zhongxu Xiang; Wentie Yang; Zuoshuai Wang; Yidong Xu; Vladimir Grischenko; Vladimir Korochentsev

doi:10.1117/12.3033768

13 June 2024 Visible and infrared image fusion based on attention and multiscale residuals

Zhongxu Xiang, Wentie Yang, Zuoshuai Wang, Yidong Xu, Vladimir Grischenko, Vladimir Korochentsev

Proceedings Volume 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024); 1318010 (2024) https://doi.org/10.1117/12.3033768
Event: International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 2024, Guangzhou, China

Abstract

Image fusion is a significant research area, related to a specific fusion task and has broad application prospects. Most of the existing image fusion algorithms are based on the pixel level, and although the convolutional network with more layers has a powerful feature extraction capability, its complexity increases with the deepening of the network. The dependency of the local range of the image is also not fully utilized in this process by the convolutional network, which makes the fused image detail information lost. In this paper, we propose a local-attention mechanism based network with multi-scale residuals to fuse visible and infrared images. The network consists of two key parts: encoder-decoder, fusion strategy. During the network training process, we utilize a phased training approach, where an automatic codec is first trained for conducting feature extraction, local feature enhancement and feature reconstruction. In the fusion stage, the coder trained in the first step is utilized to extract the two light image features, and then these features are fused by a multi-scale residual network, respectively. Finally, the fused features are inverse encoding to acquire the fused image. The experimental comparison results show that our fusion solution outperforms the existing fusion methods in both visual perception and objective evaluation.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Zhongxu Xiang, Wentie Yang, Zuoshuai Wang, Yidong Xu, Vladimir Grischenko, and Vladimir Korochentsev "Visible and infrared image fusion based on attention and multiscale residuals", Proc. SPIE 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 1318010 (13 June 2024); https://doi.org/10.1117/12.3033768

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image fusion

Education and training

Feature fusion

Image enhancement

Feature extraction

Infrared imaging

Infrared radiation

Show All Keywords

Keywords/Phrases

Search In:

Publication Years