SAR2Optical

This project aims to test the effectiveness of transcoding Synthetic Aperture Radar (SAR) imagery to optical ones using the conditional generative adversarial network (cGAN) technique, and its practicality in waterbody mapping applications using SAR imagery. The main objective is to explore whether this approach can solve the challenge of the similarity of backscattering properties between waterbodies and certain sandy land covers in arid regions in SAR imagery. This similarity creates a perplexing situation for image interpretation. Traditional visual analysis methods often struggle to differentiate between these two land cover types accurately. This leads to ambiguity in identifying and delineating waterbodies and sandy areas, constraining the precision of various applications like environmental monitoring, land use planning, and disaster management.

This project leverages the power of a cutting-edge technique called Conditional Generative Adversarial Network (cGAN) to transform Synthetic Aperture Radar (SAR) images into visually interpretable optical representations. A cGAN is a type of deep learning architecture consisting of two neural networks: a generator and a discriminator working together to generate realistic data from a given input. The generator network learns to translate the complex patterns and features present in SAR images into corresponding optical characteristics. Through an adversarial training process, the discriminator evaluates the generated optical images, providing feedback to the generator, which refines its output iteratively. This dynamic interplay between the generator and discriminator results in the generation of realistic and visually coherent optical images from SAR inputs.

Table of Contents

• Study Area
• Experiment 1: Transcoding Sentinel-1 SAR Image VV Band to NIR Band of Sentinel-2
  • Dataset
  • Dataset Samples
  • Training Configuration
  • Results Samples
    • Temporal Variations
    • New Study Area
• Experiment 2: Transcoding Sentinel-1 SAR Image VV Band + DEM to NIR Band of Sentinel-2 + Water Mask
  • Dataset
  • Training Configuration
  • Results Samples
    • Temporal Variations
    • New Study Area
• Installation
• Citing
• Contributing
• License

Study Area

The study area consists of seven distinct water bodies situated within Egypt, spanning across its vast desert landscape. The choice of these waterbodies is deliberate. Their strategic location amidst sandy terrain provides an ideal and complex setting for the research problem.

Experiment 1: Transcoding Sentinel-1 SAR Image VV Band to NIR Band of Sentinel-2

The first experiment focuses on transcoding Sentinel-1 (SAR) imagery from the VV (vertical-vertical) polarization band to the Near-Infrared (NIR) band of Sentinel-2 optical imagery. The goal is to explore the feasibility of utilizing NIR data, which is sensitive to water content, to enhance the discrimination between waterbodies and sand landcover in arid regions.

1.1 Dataset :

In this experiment, a pair of SAR-Optical datasets were created from the Sentinel-1 SAR VV band and the NIR band of Sentinel-2 satellite imagery. The training dataset comprises paired images covering the designated study area, acquired in 2020, with a high spatial resolution of 30 meters. The testing consists of two main categories:

1. New study areas: This category includes images from study areas not part of the training dataset to evaluate the model's ability to generalize and produce accurate results in unfamiliar landscapes.
2. Temporal Variations: This included images from the same study area captured in different years to examine how well the model handles changes over time within a specific location. It's a crucial test to ensure the model's consistency and adaptability across various temporal contexts.

1.2 Dataset Samples:

Source: Sentinel-1 SAR VV Band (Toushka Lakes, Southern Egypt), Acquisition Date: 06-12-2021

Target: Sentinel-2 NIR Band (Toushka Lakes, Southern Egypt), Acquisition Date: 04-12-2021

1.3 Training Configuration :

The generator network architecture utilized UNet-style blocks, combining downsampling (UNetDown) and upsampling (UNetUp) layers. The downsampling layers incorporated 2D convolutional operations, Leaky Rectified Linear Unit (LeakyReLU) activation with a slope of 0.2, optional dropout for regularization, and instance normalization for stability during training. Upsampling layers utilized transpose convolution operations, ReLU activation, and instance normalization. Skip connections were established between corresponding layers in the encoder and decoder sections, enhancing the flow of gradient information during backpropagation.

The discriminator network consists of multiple convolutional blocks, each performing downsampling operations. These blocks use 2D convolutional layers, leaky rectified linear unit (LeakyReLU) activation, and optional instance normalization.

Component	Architecture
Input Channels	1 (Generator), 2 (Discriminator)
Output Channels	1 (for both Generator and Discriminator)
Patch Size	512 x 512
Training Epochs	50
GPU	NVIDIA GeForce GTX 1660 Ti
Generator	UNet Architecture
Downsampling Layers	2D Convolution, LeakyReLU (slope: 0.2), Optional Dropout, Instance Normalization
Upsampling Layers	Transpose Convolution, ReLU Activation, Instance Normalization
Discriminator	Convolutional Blocks with LeakyReLU (slope: 0.2) and Instance Normalization
Optimizer (Generator)	Adam (lr=0.0002, betas=(0.5, 0.999))
Optimizer (Discriminator)	Adam (lr=0.0002, betas=(0.5, 0.999))

1.4 Results Samples (Patch Level):

1.4.1 Temporal Variations Catagory:

Wadi El Rayan Lakes (Southwest of Cairo, Egypt)

Sentinel-2, Acquisition Date: 09-12-2021

In the showcased samples, the generator model effectively translates waterbodies from SAR imagery to the optical NIR band. While successful in this task, challenges persist, particularly in distinguishing certain land cover types, such as sandy areas, which exhibit backscattering patterns similar to water bodies.

1.4.2 New Study Area Catagory:

These lakes are located along Egypt's western border shared with Libya

Sentinel-2, Acquisition Date: 03-12-2021

In this testing dataset category, the model exhibited poor performance in distinguishing both waterbodies and other land cover types. Challenges still persist in accurately classifying these categories.

Experiment 2: Transcoding Sentinel-1 SAR Image VV Band + DEM to NIR Band of Sentinel-2 + Water Mask

In this experiment, the generator architecture is enhanced by integrating digital elevation data with the VV band of SAR images as inputs. The output includes the NIR band optical data and a water mask extracted from the VV band of the SAR image. This modification represents a significant step forward, combining multiple data sources to improve the accuracy and richness of our model's output.

2.1 Dataset

For this experiment, the same dataset is utilized as in the first study along with incorporating digital elevation models specific to each study area. By integrating these elevation data alongside the VV band of SAR images, our model generated outputs comprising NIR band optical data and water masks extracted from the SAR imagery.

2.2 Training Configuration

Component	Architecture
Input Channels	2 (Generator), 4 (Discriminator)
Output Channels	2 (Generator), 1 Discriminator)
Patch Size	512 x 512
Training Epochs	100
GPU	NVIDIA GeForce GTX 1660 Ti
Generator	UNet Architecture
Downsampling Layers	2D Convolution, LeakyReLU (slope: 0.2), Optional Dropout, Instance Normalization
Upsampling Layers	Transpose Convolution, ReLU Activation, Instance Normalization
Discriminator	Convolutional Blocks with LeakyReLU (slope: 0.2) and Instance Normalization
Optimizer (Generator)	Adam (lr=0.0002, betas=(0.5, 0.999))
Optimizer (Discriminator)	Adam (lr=0.0002, betas=(0.5, 0.999))

2.3 Results Samples (Patch Level):

2.3.1 Temporal Variations Catagory:

Toushka Lakes, Southern Egypt

Sentinel-2, Acquistation Date 04-12-2021

Including the digital elevation model in the second dataset and training the model for 100 epochs, double the epochs of the first experiment, unfortunately, does not achieve significant improvements. While the model effectively classifies most water and land cover types, it struggles with challenging land cover like sand, which shares similar backscatter properties with water bodies. Despite this limitation, the visually accurate generated water masks underscore the superiority of the supervised learning approach over the self-supervised approach (cGAN) utilized in this project.

2.3.2 New Study Area Catagory:

Siwa Lakes, Western Egypt

Sentinel-2 , Acquisation Date 04-12-2021

Installation

1. Clone the repository:

   git clone https://github.com/MuhammedM294/SAR2Optical

2. Change to the project directory:

   cd SAR2Optical

3. Setting up an environment to run the project:

     conda create --n <environment-name> 
     conda activate <environment-name>

4. Install the required dependencies::

    pip install -r requirements.txt

Citing

{
  Author = {Muhammed Abdelaal},
  Title = {SAR-to-Optical Image transcoding for Mapping Water Bodies in Arid Region},
  Year = {2023},
  Publisher = {GitHub},
  Journal = {GitHub repository},
  Howpublished = {\url{https://github.com/MuhammedM294/SAR2Optical}}
}

Contributing

Contributions to this project are welcome. If you find any issues or have suggestions for improvements, please open a new issue or submit a pull request.

License

This project is licensed under the MIT License.

---

Project based on the cookiecutter data science project template.