Suitable matching area selection (SMAS) is one of the key technologies for aircraft scene matching navigation (SMN). Recently, deep neural networks have been applied to SMAS due to their powerful feature extraction capabilities, and have achieved significant performance improvements compared to hand-crafted suitable matching indicators. However, SMAS solely based on deep networks does not make full use of existing image suitability information. Therefore, this paper constructs a dual-branch multi-modal fusion network to maximize the utilization of image suitability features to improve SMAS performance. The network contains two parallel channels, one of which takes hand-crafted suitable matching indicators as input, and uses natural language processing networks to obtain deep suitable matching parameter vectors, and the other employs deep networks to extract deep features from original image patches. Finally, the two types of multi-modal information are fused to produce multi-modal suitable matching features (MMSMF), which is input to the output layer to predict the matching probability. By setting a matching probability threshold, we can distinguish between suitable and unsuitable matching image patches. Compared with traditional hand-crafted suitable matching indicators and deep networks, MMSMF can yield richer suitability feature representation. The public Sentinel dataset named SEN1-2 is used to evaluate the performance of MMSMF. Experimental results show its advantages over other representative methods.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.