Chapter 14
Image Fusion

Satellite sensors present an important diversity in terms of characteristics. Some provide a high spatial resolution while other focus on providing several spectral bands. The fusion process brings the information from different sensors with different characteristics together to get the best of both worlds.

Most of the fusion methods in the remote sensing community deal with the pansharpening technique. This fusion combines the image from the PANchromatic sensor of one satellite (high spatial resolution data) with the multispectral (XS) data (lower resolution in several spectral bands) to generate images with a high resolution and several spectral bands. Several advantages make this situation easier:

This case is well-studied in the literature and many methods exist. Only very few are available in OTB now but this should evolve soon.

14.1 Simple Pan Sharpening

A simple way to view the pan-sharpening of data is to consider that, at the same resolution, the panchromatic channel is the sum of the XS channel. After putting the two images in the same geometry, after orthorectification (see chapter 12) with an oversampling of the XS image, we can proceed to the data fusion.

The idea is to apply a low pass filter to the panchromatic band to give it a spectral content (in the Fourier domain) equivalent to the XS data. Then we normalize the XS data with this low-pass panchromatic and multiply the result with the original panchromatic band.

The process is described on figure 14.1.


XS

PAN

Low Pass

/

×

PAN + XS

Figure 14.1: Simple pan-sharpening procedure.


The source code for this example can be found in the file
Examples/Fusion/PanSharpeningExample.cxx.

Here we are illustrating the use of the otb::SimpleRcsPanSharpeningFusionImageFilter for PAN-sharpening. This example takes a PAN and the corresponding XS images as input. These images are supposed to be registered.

The fusion operation is defined as

-----XS---- PAN
Filtered(PAN )
(14.1)

Figure 14.2 shows the result of applying this PAN sharpening filter to a Quickbird image.


PIC PIC PIC

Figure 14.2: Result of applying the otb::SimpleRcsPanSharpeningFusionImageFilter to orthorectified Quickbird image. From left to right : original PAN image, original XS image and the result of the PAN sharpening


We start by including the required header and declaring the main function:

#include "otbVectorImage.h" 
#include "otbImageFileReader.h" 
#include "otbImageFileWriter.h" 
#include "otbSimpleRcsPanSharpeningFusionImageFilter.h"
int main(int argc, char argv[]) 
{

We declare the different image type used here as well as the image reader. Note that, the reader for the PAN image is templated by an otb::Image while the XS reader uses an otb::VectorImage .

  typedef otb::Image<double, 2>                   ImageType; 
  typedef otb::VectorImage<double, 2>             VectorImageType; 
  typedef otb::ImageFileReader<ImageType>         ReaderType; 
  typedef otb::ImageFileReader<VectorImageType>   ReaderVectorType; 
  typedef otb::VectorImage<unsigned short int, 2> VectorIntImageType; 
 
  ReaderVectorType::Pointer readerXS = ReaderVectorType::New(); 
  ReaderType::Pointer       readerPAN = ReaderType::New();

We pass the filenames to the readers

  readerPAN->SetFileName(argv[1]); 
  readerXS->SetFileName(argv[2]);

We declare the fusion filter an set its inputs using the readers:

  typedef otb::SimpleRcsPanSharpeningFusionImageFilter 
  <ImageType, VectorImageType, VectorIntImageType> FusionFilterType; 
  FusionFilterType::Pointer fusion = FusionFilterType::New(); 
  fusion->SetPanInput(readerPAN->GetOutput()); 
  fusion->SetXsInput(readerXS->GetOutput());

And finally, we declare the writer and call its Update() method to trigger the full pipeline execution.

  typedef otb::ImageFileWriter<VectorIntImageType> WriterType; 
  WriterType::Pointer writer = WriterType::New(); 
  writer->SetFileName(argv[3]); 
  writer->SetInput(fusion->GetOutput()); 
  writer->Update();

14.2 Bayesian Data Fusion

The source code for this example can be found in the file
Examples/Fusion/BayesianFusionImageFilter.cxx.

The following example illustrates the use of the otb::BayesianFusionFilter . The Bayesian data fusion relies on the idea that variables of interest, denoted as vector Z, cannot be directly observed. They are linked to the observable variables Y through the following error-like model.

Y = g(Z)+ E
(14.2)

where g(Z) is a set of functionals and E is a vector of random errors that are stochastically independent from Z. This algorithm uses elementary probability calculus, and several assumptions to compute the data fusion. For more explication see Fasbender, Radoux and Bogaert’s publication [41]. Three images are used :

Let’s look at the minimal code required to use this algorithm. First, the following header defining the otb::BayesianFusionFilter class must be included.

#include "otbBayesianFusionFilter.h"

The image types are now defined using pixel types and particular dimension. The panchromatic image is defined as an otb::Image and the multispectral one as otb::VectorImage .

  typedef double InternalPixelType; 
  const unsigned int Dimension = 2; 
  typedef otb::Image<InternalPixelType, Dimension>       PanchroImageType; 
  typedef otb::VectorImage<InternalPixelType, Dimension> MultiSpecImageType;

The Bayesian data fusion filter type is instantiated using the images types as a template parameters.

  typedef otb::BayesianFusionFilter<MultiSpecImageType, 
      MultiSpecImageType, 
      PanchroImageType, 
      OutputImageType> 
  BayesianFusionFilterType;

Next the filter is created by invoking the New() method and assigning the result to a itk::SmartPointer .

  BayesianFusionFilterType::Pointer bayesianFilter = 
    BayesianFusionFilterType::New();

Now the multi spectral image, the interpolated multi spectral image and the panchromatic image are given as inputs to the filter.

  bayesianFilter->SetMultiSpect(multiSpectReader->GetOutput()); 
  bayesianFilter->SetMultiSpectInterp(multiSpectInterpReader->GetOutput()); 
  bayesianFilter->SetPanchro(panchroReader->GetOutput()); 
 
  writer->SetInput(bayesianFilter->GetOutput());

The BayesianFusionFilter requires defining one parameter : λ. The λ parameter can be used to tune the fusion toward either a high color consistency or sharp details. Typical λ value range in [0.5,1[, where higher values yield sharper details. by default λ is set at 0.9999.

  bayesianFilter->SetLambda(atof(argv[9]));

The invocation of the Update() method on the writer triggers the execution of the pipeline. It is recommended to place update calls in a try/catch block in case errors occur and exceptions are thrown.

  try 
    { 
    writer->Update(); 
    } 
  catch (itk::ExceptionObject& excep) 
    { 
    std::cerr << "Exception caught !" << std::endl; 
    std::cerr << excep << std::endl; 
    }

Let’s now run this example using as input the images multiSpect.tif , multiSpectInterp.tif and panchro.tif provided in the directory Examples/Data. The results obtained for 2 different values for λ are shown in figure 14.3.


PIC PIC PIC

Figure 14.3: Input images used for this example (European Space Imaging).



PIC PIC

Figure 14.4: Fusion results for the Bayesian Data Fusion filter for λ = 0.5 on the left and λ = 0.9999 on the right.