Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening

Abstract

The fusion of high spatial resolution panchromatic (PAN) data with simultaneously acquired multispectral (MS) data with the lower spatial resolution is a hot topic, which is often called pansharpening. In this article, we exploit the combination of machine learning techniques and fusion schemes introduced to address the pansharpening problem. In particular, deep convolutional neural networks (DCNNs) are proposed to solve this issue. The latter is combined first with the traditional component substitution and multiresolution analysis fusion schemes in order to estimate the nonlinear injection models that rule the combination of the upsampled low-resolution MS image with the extracted details exploiting the two philosophies. Furthermore, inspired by these two approaches, we also developed another DCNN for pansharpening. This is fed by the direct difference between the PAN image and the upsampled low-resolution MS image. Extensive experiments conducted both at reduced and full resolutions demonstrate that this latter convolutional neural network outperforms both the other detail injection-based proposals and several state-of-the-art pansharpening methods.

Schematic Diagram of the Proposed Method

Downloads

Full paper: click here

TensorFlow code: click here

Reference

@ARTICLE{fusionnet,
  author={Liang-Jian Deng, Gemine Vivone, Cheng Jin and Jocelyn Chanussot},
  journal={IEEE Transactions on Geoscience and Remote Sensing}, 
  title={Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening}, 
  year={2021},
  volume={59},
  number={8},
  pages={6995-7010},
  doi={10.1109/TGRS.2020.3031366}}