Thin cloud removal fusing full spectral and spatial features for Sentinel-2 imagery

Multispectral remote sensing images are widely used for monitoring the globe. Although thin clouds can affect all optical bands, the influences of thin clouds differ with band wavelength. When processing multispectral bands at different resolutions, many methods only remove thin clouds in visible/near-infrared bands or rescale multiresolution bands to the same resolution and then process them together. The former cannot make full use of multispectral information, and in the latter, the rescaling process will introduce noise. In this article, a deep-learning-based thin cloud removal method that fuses full spectral and spatial features in original Sentinel-2 bands is proposed, named CR4S2. A multi-input and output architecture is designed for better fusing information in all bands and reconstructing the background at original resolutions. In addition, two parallel downsampling residual blocks are designed to transfer features extracted from different depths to the bottom of the network. Experiments were conducted on a new globally distributed Sentinel-2 thin cloud removal dataset called WHUS2-CRv. The results show that the best averaged peak signal-to-noise ratio, structural similarity index measurement, normalized root-mean-square error, and spectral angle mapper of the proposed method over 12 bands in all 20 testing images were 39.55, 0.9443, 0.0245, and 2.5676°, respectively. Compared with baseline methods, the proposed CR4S2 method can better restore not only the spatial features but also spectral features. This indicates that the proposed method is very promising for removing thin clouds in multispectral remote sensing images at different resolutions.