Activities for Land Cover/Use Mapping
Tool 1a: Semi-Automated Pre-Processing of Sentinel-2 Data for LU/LC Classification, on a Cloud-Based Platform (Tool 1b, below)
Monitoring of land cover and land cover change over large scales is only feasible with large-scale remote sensing datasets. Moderate resolution optical remote sensing instruments have been the mainstay of national-scale monitoring (e.g. Landsat), but new data from Sentinel-2 offers an increased resolution, faster re-visit times and guarantees of data continuity that make it attractive for future land cover monitoring. The SMFM Sentinel-2 mosaicking tool (sen2mosaic) will generate wall-to-wall mosaics at regional to national scales using composite images from multiple satellite overpasses to create season-specific cloud-free images suitable for land cover classification.
For this, the SMFM team has created sen2mosaic which is a set of tools to aid in the production of large-scale cloud-free seasonal mosaic products from Sentinel-2 data.
Building cloud-free mosaics of Sentinel-2 data for land cover mapping is difficult, with existing tools still under-development and hard to use. The goal of these tools are to streamline this processing chain with a set of straightforward command line tools. This repository contains four command-line based scripts to perform the following tasks:
Executing the sen2cor tool to perform atmospheric correction, and performing simple improvements to its cloud mask.
Building a mosaic of cloud-free outputs with sen2three.
Mosaicking cloud-free Sentinel-2 .SAFE files into larger GeoTIFF files that are suitable for image classification.
Downloading of Sentinel-2 Tiles
Image downloading is handled by a batch-processing script that accesses images from the Application Program Interface (API) within the Copernicus Open Access Hub, which offers better performance than the web interface for large downloads. The script builds upon an existing open source Python library named sentinelsat. The user is prompted to specify the tile name, date range and maximum cloud cover options to automate the selection of the most appropriate images. For the case where sen2mosaic is operated from a cloud server, where data is stored locally, this script may not be required.
Pre-processing of Sentinel-2 Tiles
Pre-processing of Sentinel-2 images is performed with the open source sen2cor toolset developed by ESA. Sen2cor is a toolset to perform atmospheric correction and cloud masking of Sentinel-2 data. Sen2cor requires a level 1C (top of atmosphere reflectance) tile as input, and generates a level 2A file (atmospherically corrected, cloud masked) file as output. Sen2mosaic assists in batch processing by accepting multiple Sentinel-2 1C tiles as inputs, and automates large pre-processing tasks with parallel processing of multiple tiles simultaneously.
The cloud masks produced by sen2cor are conservative (commonly leaving in light clouds in imagery), frequently confuse cloud shadows and dark features, and can leave artefacts around image borders. A range of logical corrections using buffers and reclassification were developed and implemented to obtain an improved mosaic quality (Figure below). These are based on a modification of the methods developed by GMV (2016), used in the development of 2016 cloud-free mosaics in Mozambique. The corrections applied to the cloud mask are:
Reclassification of ‘dark feature’ to ‘cloud shadow’;
Reclassification of ‘cloud shadow’ not within 1800 metres of ‘cloud’ to ‘water’ (not masked);
Dilation of ‘cloud shadow’, ‘medium probability cloud’ and ‘high probability cloud’ pixels by 180 metres to remove residual light clouds;
Erosion of the outer 300 metres of image tiles, which removes border artefacts whilst retaining image overlap.
For the case where sen2mosaic is operated from a cloud server where level 2A Sentinel-2 data is stored locally, the user may opt to only run the mask correction elements of this script.
Improving existing methods for pre-processing Sentinel 1&2 data...
Existing Tools ...
... and with sen2mosaic
Mosaic outputs without modifications to the sen2cor cloud mask (left), and after modifications (right). Without modification to the cloud masks, prominent artefacts are present around light clouds and at scene boundaries.
Generation of Cloud-Free Composite Images
Atmospherically corrected Sentinel-2 tiles contain gaps where clouds have been removed from the images, and tiles that are split across the satellite swath need to be combined to create a seamless image. Sen2three is an ESA toolset for creating composite images from Sentinel-2 data, combining multiple images from each to create a single gap-filled image using available clear pixels.
Image compositing is based on the ESA tool sen2three. Sen2three is a memory intensive program, hence a series of conditions that encourage the user to input only necessary images are implemented to minimise the processing memory required.
Multi-tile Mosaicking in the User-Friendly GeoTiff Format
The final step generates a seamless mosaic with composite images re-projected to a single coordinate reference system, including the conversion to the user friendly and flexible GeoTiff format. The script also outputs VRT files, which are virtual raster file that can be used for straightforward visualisation in open source GIS software such as QGIS. This script uses the Geospatial Data Abstraction Library (GDAL), a widely used open source Python library for dealing with geospatial data.
User Interface and Usage
Sen2mosaic will be made available as a set of four scripts to perform the processes of downloading, pre-processing, compositing and mosaicking. These scripts will be usable from the Linux command line, or by a more advanced user by importing functions in Python.
National-scale outputs from sen2mosaic are shown in Figure, below.
A 10 metres resolution mosaic of Sentinel-2 data generated by sen2mosaic for the extent of Zambia, using images for the early dry season (May – June) 2017.
Documentation and worked examples are available online:
Tool 1b: Semi-Automated Pre-Processing of Sentinel-1 Data for LU/LC Classification, on a Cloud-Based Platform
Land cover mapping most commonly makes use of optical data, which is widely available and comparatively easy to work with. Using additional data from radar instruments can provide useful and complementary inputs to land cover classifications, but uptake has been limited by the limited availability of free data and onerous pre-processing requirements. With the launch of the Sentinel-1 radar sensors with free and open-access data there is an opportunity to leverage new datasets to improve land cover classifications. The SMFM Sentinel-1 mosaicking tool (sen1mosaic) will assist in the generation of large-scale Sentinel-1 mosaic products suitable for use in land cover classification tasks.
During the inception phase, it was agreed to test the utility of Sentinel-1 data for improving land cover classifications, and develop a fully functional tool if the outputs were judged to be useful (SMFM Inception Report, Section 6.2.1). In practice, it was found that the burden of testing on partner countries would be difficult to manage and that SMFM Tool 3 will likely require pre-processing of Sentinel-1 data anyway, therefore the complete tool to pre-process Sentinel-1 data (i.e. Tool 1b) has been developed in advance of a testing phase.
For this, the SMFM team has created sen1mosaic which is a set of tools to aid in the production of large-scale cloud-free seasonal mosaic products from Sentinel-1 data.
Building cloud-free mosaics of Sentinel-1 data for land cover mapping is difficult, with existing tools still under-development and hard to use. The goal of these tools are to streamline this processing chain with a set of straightforward command line tools. This repository contains four command-line based scripts to perform the following tasks:
Executing the sen1cor tool to perform atmospheric correction, and performing simple improvements to its cloud mask.
Building a mosaic of processed outputs with sen2three.
Mosaicking cloud-free Sentinel-1 .SAFE files into larger GeoTIFF files that are suitable for image classification.
Downloading of Sentinel-1 Tiles
Like sen2mosaic, sen1mosaic uses the sentinelsat library to download images. Sen1mosaic uses Ground Range Detected (GRD) images collected in the Interferometric Wide (IW) swath mode, the main acquisition mode of Sentinel-1 and the most applicable to land use studies. Data from Sentinel-1 are not provided in a standard tiling grid like Sentinel-2, and images are acquired from both ascending and descending paths. The tool requires the user to input a bounding box in latitude/longitude, and downloads all images that meet criteria of date ranges and specification of ascending or descending paths. For the case where sen1mosaic is operated from a cloud server, where data is stored locally, this script may not be required.
Pre-processing of Sentinel-1 Tiles
Image pre-processing is performed using the Sentinel Application Platform (SNAP) graph processing tool. SNAP is an open source ESA program which has functions for pre-processing and analysis of satellite imagery, including a range of Sentinel-1 specific functions.
Image pre-processing takes the following steps using SNAP functions:
Application of orbit file, which improves image georeferencing through precise orbit state data.
Thermal noise removal applies corrections for thermal noise, which is particularly problematic in the cross-polarised (VH) channel (optional)
Calibration from digital numbers to pixel values directly related to radar backscatter in the scene.
Slice assembly stitches together consecutive satellite scenes to produce a continuous image for each satellite pass.
Multi-looking (optional) reduces noise and processing load by degrading image resolution.
Radiometric terrain flattening (optional) corrects for the tendency for slopes facing towards the sensor appearing brighter than those facing away.
Filtering (optional) further reduces image noise from speckle. We apply a Refined Lee filter, a radar-specific speckle filter which aims to preserve edges between field boundaries in images.
Geometric terrain correction aligns pixels with their geographical location, and corrects for the tendency of topographic features to appear to lean towards the sensor (foreshortening).
Translation and mosaicking aligns multiple radar passes to a common Universal Transverse Mercator (UTM) geographical grid. We output processed images in units of Decibels.
Example outputs from sen1mosaic (VV backscatter) showing processing options of (a) short processing chain, (b) full processing chain and (c) full processing chain with speckle filtering. This example shows Mount Meru in Northern Tanzania
A number of processing steps in sen1mosaic are optional. Speckle filtering can be activated or deactivated to accommodate multiple user’s needs, and the multi-looking step will be customisable to allow specification of output resolutions. As this processing chain is computationally intensive, a separate short processing chain (omitting thermal noise removal and radiometric terrain flattening) can be optionally run to generate test outputs rapidly. A range of outputs using different processing chains are shown in Figure above.
Sentinel-1 border artefacts from the (a) azimuth (along track) and (b) range (across track) directions, which are removed by sen1mosaic.
Sentinel-1 data outputs processed in SNAP are prone to image artefacts that can interfere with image classification (Figure above). Image sub-setting is used to eliminate border noise and minimise along-track artefacts whilst retaining the overlap between satellite overpasses.
For an advanced user, the tool can also support the use of customised processing chains complied in SNAP.
Multi-image Mosaicking in the User-Friendly GeoTiff Format
A mosaicking script very similar to that used by sen2mosaic (Section 2.2) is used. The main difference is that in place of the compositing and mosaicking steps of sen2mosaic, sen1mosaic generates outputs of the mean average, minimum, maximum, and standard deviation of each pixel in input images, which together describe the backscatter properties and its temporal variability of each pixel which can be used as inputs in land cover classifications. The output is provided in GeoTiff format, in the UTM zone and resolution specified. Two additional VRT files are provided for rapid visualisation of outputs in open source GIS software (e.g. Figure below).
Highlight from a regional-scale mosaic of Sentinel-1 data for the period May-June 2017 for Zambezia Province of Mozambique, generated with sen1mosaic.
User Interface and Usage
Sen1mosaic will be made available as a set of three scripts to perform the processes of downloading, pre-processing, and mosaicking. These scripts will be usable from the Linux command line, or by a more advanced user by importing functions in Python. The scripts required to operate sen1mosaic will mirror sen2mosaic, and will therefore require similar user expertise to be executed.
Documentation and worked examples are available online: