Semantic segmentation of multisensor remote sensing imagery with deep ConvNets and higher-order conditional random fields

Yansong Liu; Sankaranarayanan Piramanayagam; Sildomar T. Monteiro; Eli Saber

doi:10.1117/1.JRS.13.016501

11 January 2019 Semantic segmentation of multisensor remote sensing imagery with deep ConvNets and higher-order conditional random fields

Yansong Liu, Sankaranarayanan Piramanayagam, Sildomar T. Monteiro, Eli Saber

Author Affiliations +

Journal of Applied Remote Sensing, Vol. 13, Issue 1, 016501 (January 2019). https://doi.org/10.1117/1.JRS.13.016501

Abstract

Aerial images acquired by multiple sensors provide comprehensive and diverse information of materials and objects within a surveyed area. The current use of pretrained deep convolutional neural networks (DCNNs) is usually constrained to three-band images (i.e., RGB) obtained from a single optical sensor. Additional spectral bands from a multiple sensor setup introduce challenges for the use of DCNN. We fuse the RGB feature information obtained from a deep learning framework with light detection and ranging (LiDAR) features to obtain semantic labeling. Specifically, we propose a decision-level multisensor fusion technique for semantic labeling of the very-high-resolution optical imagery and LiDAR data. Our approach first obtains initial probabilistic predictions from two different sources: one from a pretrained neural network fine-tuned on a three-band optical image, and another from a probabilistic classifier trained on LiDAR data. These two predictions are then combined as the unary potential using a higher-order conditional random field (CRF) framework, which resolves fusion ambiguities by exploiting the spatial–contextual information. We utilize graph cut to efficiently infer the final semantic labeling for our proposed higher-order CRF framework. Experiments performed on three benchmarking multisensor datasets demonstrate the performance advantages of our proposed method.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Yansong Liu, Sankaranarayanan Piramanayagam, Sildomar T. Monteiro, and Eli Saber "Semantic segmentation of multisensor remote sensing imagery with deep ConvNets and higher-order conditional random fields," Journal of Applied Remote Sensing 13(1), 016501 (11 January 2019). https://doi.org/10.1117/1.JRS.13.016501

Received: 24 September 2018; Accepted: 10 December 2018; Published: 11 January 2019

ACCESS THE FULL ARTICLE

JOURNAL ARTICLE
23 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY