Journal cover Journal topic
Atmospheric Measurement Techniques An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 3.400 IF 3.400
  • IF 5-year value: 3.841 IF 5-year
  • CiteScore value: 3.71 CiteScore
  • SNIP value: 1.472 SNIP 1.472
  • IPP value: 3.57 IPP 3.57
  • SJR value: 1.770 SJR 1.770
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 70 Scimago H
    index 70
  • h5-index value: 49 h5-index 49
Discussion papers
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.

Submitted as: research article 24 Sep 2019

Submitted as: research article | 24 Sep 2019

Review status
This discussion paper is a preprint. It is a manuscript under review for the journal Atmospheric Measurement Techniques (AMT).

A review and framework for the evaluation of pixel-level uncertainty estimates in satellite aerosol remote sensing

Andrew M. Sayer1,2, Yves Govaerts3, Pekka Kolmonen4, Antti Lipponen4, Marta Luffarelli3, Tero Mielonen4, Falguni Patadia1,2, Thomas Popp5, Adam C. Povey6, Kerstin Stebel7, and Marcin L. Witek8 Andrew M. Sayer et al.
  • 1GESTAR, Universities Space Research Association, Columbia, MD, USA
  • 2NASA Goddard Space Flight Center, Greenbelt, MD, USA
  • 3Rayference, Brussels, Belgium
  • 4Finnish Meteorological Institute, Atmospheric Research Centre of Eastern Finland, Kuopio, Finland
  • 5Deutsches Zentrum für Luft-und Raumfahrt e. V. (DLR), Deutsches Fernerkundungsdatenzentrum (DFD), 82234 Oberpfaffenhofen, Germany
  • 6National Centre for Earth Observation, University of Oxford, Oxford, OX1 3PU, UK
  • 7NILU - Norwegian Institute for Air Research, NO-2007 Kjeller, Norway
  • 8Jet Propulsion Laboratory, California Institute of Technology, 4800 Oak Grove Drive, Pasadena, CA 91109, USA

Abstract. Recent years have seen the increasing inclusion of per-retrieval prognostic (predictive) uncertainty estimates within satellite aerosol optical depth (AOD) data sets, providing users with quantitative tools to assist in optimal use of these data. Prognostic estimates contrast with diagnostic (i.e. relative to some external truth) ones, which are typically obtained using sensitivity and/or validation analyses. Up to now, however, the quality of these uncertainty estimates has not been routinely assessed. This study presents a review of existing prognostic and diagnostic approaches for quantifying uncertainty in satellite AOD retrievals, and presents a general framework to evaluate them, based on the expected statistical properties of ensembles of estimated uncertainties and actual retrieval errors. It is hoped that this framework will be adopted as a complement to existing AOD validation exercises; it is not restricted to AOD and can in principle be applied to other quantities for which a reference validation data set is available. This framework is then applied to assess the uncertainties provided by several satellite data sets (seven over land, five over water), which draw on methods from the empirical to sensitivity analyses to formal error propagation, at 12 Aerosol Robotic Network (AERONET) sites. The AERONET sites are divided into those where it is expected that the techniques will perform well, and those for which some complexity about the site may provide a more severe test. Overall all techniques show some skill in that larger estimated uncertainties are generally associated with larger observed errors, although they are sometimes poorly calibrated (i.e. too small/large in magnitude). No technique uniformly performs best. For powerful formal uncertainty propagation approaches such as Optimal Estimation the results illustrate some of the difficulties in appropriate population of the covariance matrices required by the technique. When the data sets are confronted by a situation strongly counter to the retrieval forward model (e.g. potential mixed land/water surfaces, or aerosol optical properties outside of the family of assumptions), some algorithms fail to provide a retrieval, while others do but with a quantitatively unreliable uncertainty estimate. The discussion suggests paths forward for refinement of these techniques.

Andrew M. Sayer et al.
Interactive discussion
Status: final response (author comments only)
Status: final response (author comments only)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
Andrew M. Sayer et al.
Andrew M. Sayer et al.
Total article views: 483 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
376 107 0 483 2 2
  • HTML: 376
  • PDF: 107
  • XML: 0
  • Total: 483
  • BibTeX: 2
  • EndNote: 2
Views and downloads (calculated since 24 Sep 2019)
Cumulative views and downloads (calculated since 24 Sep 2019)
Viewed (geographical distribution)  
Total article views: 343 (including HTML, PDF, and XML) Thereof 340 with geography defined and 3 with unknown origin.
Country # Views %
  • 1
No saved metrics found.
No discussed metrics found.
Latest update: 13 Dec 2019
Publications Copernicus
Short summary
Satellite measurements of the Earth are routinely processed to estimate of useful quantities; one example is the amount of atmospheric aerosols (which are particles such as mineral dust, smoke, volcanic ash, or sea spray). As with all measurements and inferred quantities, there is some degree of uncertainty in this process. There are various methods to estimate these uncertainties. A related question is: how reliable are these estimates? This paper presents a method to assess them.
Satellite measurements of the Earth are routinely processed to estimate of useful quantities;...