How to eliminate cloud shadows in aerial imaging and photogrammerty

We have some problems with shadows from the clouds in the images taken from the air. Clouds intersect with the sun's rays, casting shadows onto the landscape below. This is unavoidable, so we often end up with images like this. We need to remove the shadows to prepare the data for mapping and 3D visualization.

What is the best way to get rid of shadows in clouds in image editing? Usually, we need to create a mask and apply a filter to a specific area of the image, like the shadowed region. We also need to separate the objects that should not be affected by the filter. You can do that in different layers too. However, this is a manual approach, which is slow and imprecise. Also, shadows often have unclear edges. This makes it hard to define the mask and apply it to the image.

Our approach is based on classic image processing ideas, without machine learning (ML) or artificial intelligence (AI). This algorithm works well with soft shadows, especially when there isn't a clear edge. This is common in images taken from the air and in images made from photographs. These images often need to have shadows removed before other processing. We expect that computations on the NVIDIA GeForce RTX 4090 would offer performance of 5-7 GPix/s or more for 16-bit RGB images with high resolutions. This is not possible for any AI-based system.

This algorithm is only for soft cloud shadows. It's not meant to remove deep shadows. The picture below shows an example of soft shadows and the result of the processing.

Source JPG image is on the left, the result of Aerial Clarity is on the right

Main Points

• Automatic cloud shadow suppression for aerial images (soft shadows only)
• The image resolution could be very high, the 245.7 MPix (IMX811 with a resolution of 19,200 × 12,800) works well
• Hight performance because the algorithm is parallel and it's implemented on CUDA

Algorithm Overview

The cloud shadow suppression involves these main steps:

• Shadow Region Detection: Image statistics and pixel intensity analysis.
• Filter Parameters: Compute transform parameters to be applied.
• Shadow Compensation: Use local illumination correction techniques to brighten shadows.
• Seamless Blending: Blend corrected shadow regions back into the original image to maintain natural transitions without visible artifacts.

Classic vs AI-based Approach for Shadow Removal

Machine learning approaches need training data and often struggle with generalized shadow types. This classic computer vision approach uses only pixel-level image characteristics, statistics and deterministic filtering. This results in the following:

• Pros:
  • Fast GPU execution, even for very high image resolutions
  • No need for data gathering, annotating and training
  • Robust for soft and diffuse shadows, which are typical in aerial imagery and photogrammetry
• Cons:
  • Can't be applied to deep shadows
  • Shadow elimination may result in the presence of minor traces on the image
  • Less adaptable to varied conditions in comparison with AI models

Therefore, it is a good choice for real-time or very large-scale aerial and photogrammetry applications where speed and reliability are important.

Roadmap

• Further algorithm development to handle stronger shadows
• Software optimization both on CPU and on GPU
• Algorithm integration into RAW-to-RGB processing pipeline on CUDA
• Compatibility with Phase One IIQ format (Phase One ImageSDK)
• Integration with third-party software