Journal cover Journal topic
Geoscientific Instrumentation, Methods and Data Systems An interactive open-access journal of the European Geosciences Union

Journal metrics

  • IF value: 1.023 IF 1.023
  • IF 5-year<br/> value: 1.557 IF 5-year
  • CiteScore<br/> value: 0.86 CiteScore
  • SNIP value: indexed SNIP
  • SJR value: indexed SJR
  • IPP value: indexed IPP
  • h5-index value: 10 h5-index 10
Geosci. Instrum. Method. Data Syst., 3, 187-200, 2014
© Author(s) 2014. This work is distributed under
the Creative Commons Attribution 3.0 License.
Research article
25 Sep 2014
A framework for benchmarking of homogenisation algorithm performance on the global scale
K. Willett1, C. Williams2, I. T. Jolliffe3, R. Lund4, L. V. Alexander5, S. Brönnimann6, L. A. Vincent7, S. Easterbrook8, V. K. C. Venema9, D. Berry10, R. E. Warren11, G. Lopardo12, R. Auchmann6, E. Aguilar13, M. J. Menne2, C. Gallagher4, Z. Hausfather14, T. Thorarinsdottir15, and P. W. Thorne16 1Met Office Hadley Centre, FitzRoy Road, Exeter, UK
2National Climatic Data Center, Ashville, NC, USA
3Exeter Climate Systems, University of Exeter, Exeter, UK
4Department of Mathematical Sciences, Clemson University, Clemson, SC, USA
5ARC Centre of Excellence for Climate System Science and Climate Change Research Centre, University of New South Wales, Sydney, Australia
6Oeschger Center for Climate Change Research & Institute of Geography, University of Bern, Bern, Switzerland
7Climate Research Division, Science and Technology Branch, Environment Canada, Toronto, Canada
8Department of Computer Science, University of Toronto, Toronto, Canada
9Meteorologisches Institut, University of Bonn, Bonn, Germany
10National Oceanography Centre, Southampton, UK
11College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK
12Istituto Nazionale di Ricerca Metrologica (INRiM), Torino, Italy
13Centre for Climate Change, Universitat Rovira i Virgili, Tarragona, Spain
14Berkeley Earth, Berkeley, CA, USA
15Norwegian Computing Center, Oslo, Norway
16Nansen Environmental and Remote Sensing Center, Bergen, Norway
Abstract. The International Surface Temperature Initiative (ISTI) is striving towards substantively improving our ability to robustly understand historical land surface air temperature change at all scales. A key recently completed first step has been collating all available records into a comprehensive open access, traceable and version-controlled databank. The crucial next step is to maximise the value of the collated data through a robust international framework of benchmarking and assessment for product intercomparison and uncertainty estimation. We focus on uncertainties arising from the presence of inhomogeneities in monthly mean land surface temperature data and the varied methodological choices made by various groups in building homogeneous temperature products. The central facet of the benchmarking process is the creation of global-scale synthetic analogues to the real-world database where both the "true" series and inhomogeneities are known (a luxury the real-world data do not afford us). Hence, algorithmic strengths and weaknesses can be meaningfully quantified and conditional inferences made about the real-world climate system. Here we discuss the necessary framework for developing an international homogenisation benchmarking system on the global scale for monthly mean temperatures. The value of this framework is critically dependent upon the number of groups taking part and so we strongly advocate involvement in the benchmarking exercise from as many data analyst groups as possible to make the best use of this substantial effort.

Citation: Willett, K., Williams, C., Jolliffe, I. T., Lund, R., Alexander, L. V., Brönnimann, S., Vincent, L. A., Easterbrook, S., Venema, V. K. C., Berry, D., Warren, R. E., Lopardo, G., Auchmann, R., Aguilar, E., Menne, M. J., Gallagher, C., Hausfather, Z., Thorarinsdottir, T., and Thorne, P. W.: A framework for benchmarking of homogenisation algorithm performance on the global scale, Geosci. Instrum. Method. Data Syst., 3, 187-200,, 2014.
Publications Copernicus