In general terms, the main outcomes of the COMMONS Project are:
- the development of a library of specific signatures of changes that occur between two or more Sentinel-1 - Sentinel-2 acquisitions.
- the exploitation of the library to detect changes of land cover due to landslides and flood occurrence, snow and forest coverage changes, and agriculture evolutions.
In the following, an specific summary of the main achievements for each application is described.
Agriculture main results
We implemented and developed a procedure is to estimate a phenological stage value at a parcel level combining information coming from Sentinel-1 and Sentinel-2 images. The proposed method is based on the exploitation of a library that describes the phenological stage evolution in terms of the radar backscatter and optical NDVI for a parcel crop cycle. This allows detecting spatial and temporal divergences from the “expected behaviour” on the parcel at a particular acquisition date. Figure 1 (a) and (b) shows Radarsat-2 mean and Stdv HV/HH for a particular AgriSAR 2009 parcel along the crop cycle vs (a) Doy and (b) phenological stage with their associated (grey and blue respectively) model.
The generation of the library is based on the phenological stage ground truth data of the parcel along the crop cycle. To that matter the Agrisar 2009 data was employed, where radarsat-2 and RapidEye images were acquired over different test parcels where ground truth data was acquired. HV/VV ratio and NDVI through a crop cycle over the parcel test were modelled to generate the corresponding magnitude vs. phenological stage pdf (library). The phenological stage of a new image was then calculated as the maximum probability stage. A previous unimodality test is performed on the ratio and the NDVI to detect and discard anomaly spatial behaviour on the parcels.
The study was extended to ESA’s Sentinel-1 and Sentinel-2 data (Figure 2). Nevertheless, Sentinel analysis was not completely done due to the late availability of a complete set of Sentinel-1 and Sentinel-2 data set and the lack of phenological stage ground truth. The performed tests were done using the day of year referred to the beginning and the end of the crop cycle.
Agriculture main conclusions
The procedure performed well for estimating the phenological stage and possible anomalies of the investigated crops at a parcel level. The goodness of the results depends on the temporal sampling of the crop cycle as the natural phenological cycle is not linear. Overall errors on the phenological stage were on the order of the 10% of the phenological stage range. Both radar and optical signatures are complementary. Nevertheless, joint analysis at a pdf simultaneous level required more temporally overlapping images.
Sentinel-1 and Sentinel-2 series are sensitive and complementary to parcel’s crop cycle phenology and thus valid for the application of the developed methodology. Even though that spatial resolution and the SNR performances for Sentinel-1 and Sentinel-2 are lower than Radarsat-2 and RapidEye, the done tests show that Sentinel’s characteristics fit the requirements to apply the developed methodology.
Figure 1. Radarsat-2 mean and Stdv HV/HH for a particular AgriSAR 2009 parcel along the crop cycle vs (a) Doy and (b) phenological stage with their associated model.
Figure 2. (a) Sentinel-1 VH/VV, VH and VV backscatter coefficient evolution of one AgriSAR 2009 parcel and (b) Sentinel-2 NDVI evolution.
Forest main results
The concept of synergistically use Sentinel-1 and Sentinel-2 data for forest change detection has been successfully applied in a complex mountain environment in the Central Alps (South Tyrol, Italy). To setup the library, pixel-based harmonic functions were fitted to each band of the Earth Observation data time series (Sentinel-1 and Landsat-8 as a substitute for Sentinel-2, Figure 3). Model parameters and model accuracy estimates (differences between actual and modeled values as well as integrated differences across all bands and time (RMSE)) were added to the library.
Figure 3. Examples of modeling the spectral-temporal variability for Sentinel-1A and Landsat timeseries for different polarizations, spectral bands and forest types.
Based on the library content, backscatter and spectral values were estimated for each new Sentinel acquisition and consequently compared to the actual data. Therefore, model parameters and accuracy estimates were resampled from Landsat-8 data to Sentinel-2 data. A synthetic change detection data set was established which introduced change events by combining time series of forest and grasslands considering different local incidence angles and change event timings (Figure 4). The change detection approach was then applied to the synthetic data set and a defined set of change features (RMSE > 5 times mean of modelling period for 3 consecutive observations).
Figure 4. Examples of synthetic changes in the Sentinel timeseries and their corresponding RMSE values between actual and predicted values.
The synergistic use of Sentinel-1 and Sentinel-2 data resulted in an overall change detection accuracy of 73.2% (kappa = 0.64) outperforming the use of Sentinel-2 data only (65.5%, kappa = 0.54) and Sentinel-1 data only (70.6%, kappa = 0.61). With respect to the timeliness of the change detection results were less clear. Sentinel-1 and Sentinel-2 data detected 77% pixels at the time of change event and 89% within 2 consecutive acquisitions. Sentinel-1 data only resulted in a slightly better results (80% of pixels detected at the time of the change event, 95% within 2 consecutive acquisitions) while Sentinel-2 data only were less accurate on the exact timing of the change event (67% of pixels detected) but most accurate in when considering also the 2 consecutive acquisitions (97%).
Forest main conclusions
The synergistic use of Sentinel-1 and Sentinel-2 data outperforms the use of Sentinel-1 or Sentinel-2 data alone in detecting forest changes while the accuracy of the change detection timeliness was less clear. Two main aspects need to be highlighted that might have impacted the results and introduced an underestimation of accuracy: the library for Sentinel-2 data was based on Landsat-8 data and a synthetic change detection set was used to validate the results.
Overall, the underlying concept of two independent time-series fitting modules proves to be a highly flexible system which has the potential for global applicability. The system is:
- adaptable to Sentinel-1 (A+B) and Sentinel-2 (A+B) data availability and forest phenology.
- extendable to any other Earth Observation data set (e.g. Landsat8).
- tuneable to different user needs (sensitivity of change detection capacity vs easily implementable in a cloud processing environment.
Snow main results
During the project, the effectiveness of the combination of Sentinel-1 and Sentinel-2 for snow cover and snow status identification has been demonstrated. In detail, the combination of S1 and S2 allows:
- The improvement of S1 wet snow detection;
- The improvement of S2 snow cover detection;
- The derivation of wet and dry snow maps.
The analysis of obtained results highlight that the high spatial and temporal resolution provides by the Sentinel-1 and Sentinel-2 allows the monitoring of snow evolution in complex terrain with a rich level of detail (Figure 5). Moreover, the high temporal resolution of the Sentinel 1 data allowed the definition of transitions in backscattering that identify precise phases related to the different ground conditions that are: soil freezing/soil moisture fluctuation/wetsnow.
Figure 5. North East Alps, 29 April 2016: (a) VH-NDSI false color composition (b) result obtained by the proposed approach.
Snow main conclusions
The combination of multi-source information under the same ground conditions generally reduces the total coverage of the data. This renders the combination of Sentinel 1 and Sentinel 2 suitable for monitoring snow evolution in small and medium size catchments.
The proper exploitation of the multisource information provided by the SAR and optical sensors requires that the acquisitions should be performed simultaneously. Nonetheless, this very strict requirement is not feasible using a virtual constellation (i.e., a constellation made up of sensors on board of different spacecraft) and therefore a maximum time delay between the two acquisitions for which the situation on the ground is unchanged has been assumed equal to one day.
In order to mitigate the possibly high cloud coverage, Landsat-8 has been successfully added to the Sentinels virtual constellation. In detail, the use of Sentinel-1 and 2 in full configuration i.e., A and B and the use of Landsat-8 allows the acquisition of up to 72 multisource images per years over the Alps which is a suitable number of observations for snow monitoring.
The combination of SAR and optical data requires a high processing effort due to the large size of the data. As shown during the project this drawback can be mitigated by using the proposed fusion approaches i.e., geophysical level fusion supposing that the optical snow cover maps and the wet snow maps are already made available from third party data providers such as www.cryoland.eu.
Landslides main results
We implemented and delivered a procedure to generate event landslide inventory maps combining information recorded by Sentinel-1 and Sentinel-2 images. The classification method recognizes changes caused by new event landslides querying an a-priori knowledge (a library) of the statistical behaviour of similar changes that occurred in the past in similar geo-environmental conditions.
The library is prepared using a training dataset of images where changes are measured through the LogRatio (LR) index in consecutive Sentinel-1 images, and through a Principal Component Analysis (PCA), a Spectral Angle (SA) Index and changes in Normalized Differential Vegetation Index (ΔNDVI) in consecutive Sentinel-2 images. The combination of Sentinel-1 and Sentinel-2 images takes place in the library when the changes occurred in the different land cover classes are statistically modelled together in the indexes and put in a covariance matrix that represents the training model for the classification. Using as models the elements of the library, a Montecarlo framework generates synthetic training dataset to iteratively train a supervised Maximum Likelihood classifier and obtain potential classifications of new images once the new stack of changes is obtained (Figure 6 (a)). The final ensemble of classifications allows obtaining a single classification together with an estimation of the uncertainty of it that comes from the use of training datasets that are not selected in the image to classify.
This approach was primarily implemented with the purpose to train in an automatic way, the Maximum Llikelihood classifier with two hypothesized advantages: (i) avoid to select training areas every time that a new event is to map (this process is very time consuming) making easier the systematic use of the Sentinel images to map changes caused by landslides, (ii) to estimate the error of the final classification.
To test the procedure we mapped the event landslides occurred in Myanmar (Tozang area) in July 2015, event triggered by strong rainfall (Figure 6 (b)). We compared the obtained map with an inventory map obtained through expert-driven photo-interpretation.
Figure 6. Event landslide in Tozang, area, Myanmar, July 2015. (a) stack of changes including ΔNDVI, PCA, and LR, (b) in red, polygons mapping landslides obtained using the method proposed in the project.
Landslides main conclusions
According to the results of the comparison, the classifier performed numerically well because false positive and false negative in the final landslide map represent respectively 0.59%, and 1.04% of the entire classified area.
In clear sky conditions, the contribute of the optical component was statistically predominant in the achievement of the final results, while the SAR component contributed in very local and marginal situation, in the face of an increase of the complexity in the preparation of the library. The bi-temporal approach does not fully exploits the potentialities offered by the Sentinel-1 and a combined approach, multi-temporal SAR-based for event identification, and bi-temporal optical-(eventually SAR)-based (Mondini, 2017) promises to increment the capacities of system to provide information about landslide events.
Floods main results
The same procedure developed for the landslide was used also for generating maps of flooded areas. Changes were measured through Log-ratio (LR) index in consecutive Sentinel-1 images and through a Principal Component Analysis (PCA) and Normalized Difference Water Index (NDWI) in consecutive Sentinel-2 images. The combination of these indices (LR, PCA and NDWI) within a covariance matrix allows us to determine a training model for the classification. Repeating the procedure used to produce landslide maps, the classification of new images is obtained iteratively training a supervised Maximum Likelihood (ML) classifier through a synthetic dataset of signatures generated in a Montecarlo framework based on the signatures derived by the combination of the three indices. The final outcome is represented by an ensemble of classifications providing the inundated pixels and the uncertainty of them.
To test the procedure, we mapped the flooded areas associated to the event occurred in Missouri (USA) in December 2015 (Figure 7).
Since the closest post-event Sentinel-1 image was acquired eight days later the peak when the extension of the flood was already drastically reduced, while the optical images were acquired at the time of the maximum flows, we trained the classification model using only optical information present in the library parcelling out the SAR component.
The extension of the flooded area obtained by the procedure was compared to a reference flooded map (benchmark) obtained by (i) using another classification method (i.e. Support Vector Machine, SVM), and (ii) through a visual interpretation of the optical image acquired during the event, both filtering the permanent water bodies.
Figure 7. Classification of change (blu) and no change (green) obtained by the proposed approach for the flood event of the December 2015 in Missouri river (USA).
Floods main conclusions
The good performance of the procedure is confirmed by the results obtained in terms of True Negative Rate (0.92 for the SVM benchmark and 0.94 for the heuristic benchmark) and True Positive Rate (0.61 for the SVM benchmark and 0.71 for the heuristic benchmark). The goodness of the result is partially due to the flood event, the time of acquisition and the selected study area.
The fusion can be applied only if the two sensors observe the same phenomenon at the same time, hence to produce a flood map the satellite images should be acquired during (or close to) the maximum extension of the inundation.
The separated use of optical and SAR sensors can give good results even if test on multi-temporal SAR images should be carried out.
Other flood events and study areas should be investigated to test the transferability of the procedure and to validate the conclusions.
Scientific Communications in Workshops:
ESA living planet symposium. Library based change detection methodologies combining Sentinel-1 and Sentinel-2 data. Mondini, Alessandro (1); Notarnicola, Claudia (2); Blanco, Pablo (3); Tarpanelli, Angelica (1); Rossi, Mauro (1); Sonnenschein, Ruth (2); Marin, Carlo (2) 1: IRPI, CNR, Italy; 2: EURAC, Italy; 3: Altamira Information, Spain
Tarpanelli A., Mondini A., Rossi M. (2015) "FLOOD MAPPING BY COUPLING SAR AND OPTICAL IMAGES: POTENTIAL APPLICATION FOR JOINED USE OF SENTINEL-1 AND SENTINEL-2", Le giornate dell'Idrologia 2015: Idrologia di bacino e rischi naturali: monitoraggio, previsione, prevenzione e mitigazione in un contesto di cambiamenti globali, 6-8 October 2015, Perugia, Italy.
Tarpanelli A., Mondini A., Rossi M. (2016) "FLOOD MAPPING THROUGH THE JOINED USE OF SENTINEL-1 AND SENTINEL-2", Living Planet Symposium, 9-13 May 2016, Prague, Czech Republic.
Land Cover Changes investigation using Sentinel 1 and 2: the project COMMOnS (IGU-IGC Beijing 8.2016)
Spatial autocorrelation changes in multitemporal SAR images for landslide event detection modelling and early warning of landslides, new methods and technologies (3rd Joint Seminar Korea-Italy Florence 4.2017)
Marin, C., Callegari, M., Notarnicola, C., “A novel multitemporal approach to wet snow retrieval with Sentinel-1 images”, SPIE Remote Sensing, Edinburgh, United Kingdom, 26 - 29 September 2016
Marin, C., Callegari, M., Greifeneder, F., Notarnicola, C., Zebisch, M., “Monitoring Alpine Water Resources using Satellite Data“, Virtual Alpine Observatory Symposium 2017, Bolzano, Italy, 28 - 30 March 2017
Combined use of Sentinel-1, Sentinel-2 and Landsat 8 images for near-time forest change detection. R. Sonnenschein, C. Marin, C. Notarnicola, M. Davidson. Multitemp 2017.
Measures of Spatial Autocorrelation Changes in Multitemporal SAR Images for event landslides Detection. Submitted to Remote Sensing.
Carlo Marin, Mattia Callegari, Claudia Notarnicola, and Malcolm Davidson, Combination of SAR and Optical Data for Snow Monitoring During the Melting Season, In preparation.