From Observations to Service Delivery: Challenges and Opportunities

by Adrian Simmons*

How observations are processed is vital to the provision of monitoring and forecasting services for weather, air quality and climate. Assimilation of observational data into comprehensive forecast models is the established way of exploiting observations for weather forecasting beyond a few hours ahead. Advances in global observation since the 1970s have been accompanied by considerable progress in modelling and data assimilation, resulting in very substantial improvements in the accuracy of forecasts.

Applying data assimilation to historical observational records in the process of reanalysis helps put forecasting improvements in context and complements the direct use of observations for climate monitoring. Modelling and assimilation are also supporting development of services for air-quality assessment and forecasting, and providing information on the forcing of climate change.

Landmark achievement

In 1971, Taroh Matsuno published a landmark paper that elucidated the dynamics of the stratospheric sudden warming phenomenon. Originally detected in radiosonde observations over Berlin by Scherhag (1952), the phenomenon was mapped for the 1957 event by Teweles (1958), and first examined in a comprehensive general circulation model by Miyakoda et al. (1970). Improved observational coverage was in place by 1979 for the primary observing year of the Global Atmospheric Research Programme (GARP), and significant progress had been made by then in both modelling and assimilation of observational data. This was demonstrated by studies of the predictability of the substantial warming events of that year (Bengtsson et al., 1982a; Simmons and Strüfing, 1983).

Subsequent improvements in both observing and forecasting systems (discussed further for tropospheric forecasts later in this article) made it possible in September 2005 to predict a week or so in advance the first major warming ever to be observed in the stratosphere of the southern hemisphere (Simmons et al., 2005). The event was notable not only for its rarity, but also for a substantial effect on the Antarctic ozone hole that was likewise well forecast (Eskes et al., 2005). Already weaker than usual as a result prior activity, the dynamical vortex and accompanying ozone hole both split into two, with one part short-lived and the other re-establishing itself over the pole, but much weaker than hitherto.

Analyses of the multisensor satellite record show that southern polar ozone levels were higher in October 2002 than in any October since the early 1980s (van der A et al., 2010). Variability of the polar vortex also resulted in relatively high ozone over the Antarctic in September 2010 and low ozone over the Arctic in March 2011. Monitoring and forecasting stratospheric ozone and the related surface UV radiation have become a service activity, for example as undertaken within the pilot atmospheric service of Europe’s Global Monitoring for Environment and Security (GMES) initiative (Hollingsworth et al., 2008).

Prof. Matsuno’s work four decades ago can thus be viewed as a milestone in the progression over time from early observation through theoretical understanding and modelling to an enhanced capability for monitoring, prediction and the delivery of associated services. Observations, however, play a much more immediate role in supporting the delivery of services for weather, air quality and climate. This may occur through several routes, as indicated schematically in Figure 1.

Figure 1 — Routes from observation to service delivery.

Use of observations

Observations are used directly and vitally in monitoring local weather, in “nowcasting” predictions for the coming few hours, in establishing weather and climate records on a station-by-station basis for conventional data, and in providing synoptic overviews in the case of satellite data. They also have direct application for the validation and calibration of other types of observation and of products from numerical prediction. They are needed to link with various types of socio-economic and ecosystem data to develop assessments of impacts of weather and climate, of climate variability and change in particular, and to develop appropriate mitigative and adaptive responses.

Observations may also be combined in a rather direct way through some form of gridding or analysis to produce regional or global fields, often based on a particular type of observation or a few related types. Use of the hand-analysed weather chart may be becoming increasingly rare, but observations are routinely processed numerically in a quite direct manner to map climate, its short-term anomalies and long-term trends.

An example is provided by the HadCRUT3 dataset (Brohan et al., 2006), which is one of the primary datasets used in assessing global temperature. It combines analyses of monthly averages of surface air temperature measurements from land stations with sea surface temperature analyses based on measurements from ships and buoys.

Data assimilation provides a sequence of analyses of atmospheric, oceanic and land-surface conditions. It uses information from the latest observations to adjust a “background” model forecast initiated from the preceding analysis in the sequence. The model carries information from earlier observations forward in time. Information is spread in space and from one variable to another by the model forecast and through the background-error structures used in the adjustment process.

The set of observations may comprise many different types of measurement, each with its own accuracy and spatial distribution. The use of data assimilation to provide atmospheric initial condition for numerical weather forecasting is long established, and its use for analysis of oceanic observations now supports forecasting for monthly, seasonal and longer time ranges.

Extension to include additional trace chemical and aerosol species is adding capability for air-quality monitoring and forecasting. Moreover, reanalyses of the observations made over past decades with fixed modern assimilation systems provides a widely used record of weather and climate. The use of data assimilation in the progression from observations to service delivery is the focus of this article.

Evolution of the observing system and medium-range weather forecasting

Following pioneering studies of medium-range prediction by Miyakoda et al. (1972), and the launch the same year of the satellite NOAA-2 carrying the first in an operational series of vertical profiling infrared radiometers, the first global weather prediction system became operational in September 1974 at the US National Meteorological Center, the forerunner of today’s National Centers for Environmental Prediction (Shuman, 1989). The European Centre for Medium-Range Weather Forecasts (ECMWF) followed in August 1979. The data assimilation system developed by ECMWF was used not only for its early operational forecasting, but also to produce analyses of the First GARP Global Experiment (FGGE) observations (Bengtsson et al., 1982b). A second set of analyses of the FGGE data was produced by K. Miyakoda and colleagues at the Geophysical Fluid Dynamics Laboratory (GFDL) at Princeton University (Ploshay et al., 1992). These analyses were used in many research studies over the next few years, but the limitations of a one-year sampling period and rapid development of global modelling led to more widespread use of analyses produced by the operational global systems.

This lasted until the emergence in the mid-1990s of products of more uniform quality covering longer periods, provided by reanalysis, most notably by the National Centers for Environmental Protection (NCEP, Kalnay et al. 1996) and ECMWF, and later the Japan Meteorological Agency (Onogi et al, 2007) and others. Here, results from ECMWF’s two most recent reanalyses, ERA-40 (Uppala et al., 2005) and ERA-Interim (Dee et al., 2011), and from ECMWF operations, are utilised to illustrate some impacts of changes in observing and forecasting systems on medium-range forecasting and climate monitoring.

ERA-40 covered the period 1958–2001. It used an assimilating model with a ~125 km quasi-uniform grid, 60-level vertical resolution and a three-dimensional variational (3D-Var) analysis. Otherwise, it used a version of the ECMWF forecasting system that was operational in 2001. ERA-Interim runs from 1979 to the present, and uses ~80 km horizontal resolution, 60-level vertical resolution, and otherwise a 2006 version of the ECMWF system, including four-dimensional variational (4D-Var) analysis. Forecasts to 10 days ahead have been carried out twice daily from 00 and 12 UTC for both ERA-40 and ERA-Interim. Results from the 12 UTC forecasts can be compared with those from ECMWF operations that have been archived since 1 January 1980.

Over this period, horizontal resolution of the operational system has ranged from about 210 km to its present 16 km, vertical resolution has increased from 15 to 91 levels, and there have been numerous changes to the forecasting model and the way observational data are assimilated.

Miyakoda et al. (1972) introduced the anomaly correlation of the 500 hPa height field, the correlation between forecast and analysed anomalies of the field, as a measure of the skill of their northern hemispheric forecasts. This measure has proved useful over the years in providing a headline measure of how forecasting performance has evolved, a measure in broad agreement with subjective assessment of the synoptic forecast. Figure 2 shows time series of this measure from ERA-40 and ERA-Interim at forecast ranges of three, five and seven days, computed for regions encompassing Europe and Australia/New Zealand. Observational coverage for these regions is good enough to have reasonable confidence in the quality of the verifying analysis for the pre-satellite as well as the satellite era.

Figure 2 — 500 hPa height anomaly correlation (%) for forecast ranges of 3, 5 and 7 days, computed over Europe (35–75°N, 12.5°W–42.5°E; left) and Australia/New Zealand (45–12.5°S, 120–175°E; upper) from ERA-40 (1958–2001; upper and lower) and ERA-Interim (1979–2011; lower). Twelve-month running means of the monthly averages of values for forecasts carried out daily from 12 UTC are shown.

Looking first at the ERA-40 results shown in the upper panels, forecast skill increases rather steadily over the period for Europe, which benefits from being located downstream of relative good observational coverage over North America and from the Atlantic weather ships prior to the advent of satellite soundings. (Uppala et al. 2005) showed improvement over time to be larger for North America, which, lying downstream of the broad expanse of the Pacific Ocean, benefited more than Eurasia from the introduction of satellite data coverage.

Better data coverage and forecasting systems

The consequence of improved data coverage over the oceans is seen most strikingly in the ERA-40 results shown for the Australasian region in Figure 2. Here, the accuracy of the forecasts is very much poorer than elsewhere prior to establishment of the observing system for FGGE in 1979. Forecast scores in fact decline early in the period, which may be a consequence of a degradation of observational coverage after the International Geophysical Year, 1958. Scores begin to pick up only in the 1970s, most likely due to the assimilation of Vertical Temperature Profile Radiometer (VTPR) radiance data, and subsequently jump substantially around the end of 1978; the improvement amounts to about a two-day gain in predictive capability in the medium range.

Corresponding ERA-Interim results (up to July 2011) are presented in the lower panels of Figure 2. They show improvement over ERA-40, more so over Australasia than Europe, a result expected from the change from 3D-Var to 4D-Var analysis. Improvement over time continues beyond the time of ERA-40, and again is rather larger for Australasia. Skill levels for the two regions are similar for the most recent years.

Results for the entire extratropics are shown in Figure 3 for ECMWF operations and ERA-Interim. Shading indicates the difference is scores between the northern and southern hemispheres. The left-hand panel for operations is adapted and extended from a plot first published by Simmons and Hollingsworth (2002). It shows a substantial and continuing improvement in forecast skill over time.

Figure 3 — 500 hPa height anomaly correlation (%) for forecast ranges of 3, 5, 7 and 10 days, computed over the extratropical northern and southern hemispheres from ECMWF operations (1980–2011; left) and ERA-Interim (1979–2011; right). Twelve-month running means of the monthly averages of values for forecasts carried out daily from 12 UTC are shown. Shading depicts the extent of the differences between the two hemispheres.

For the northern hemisphere, the level of skill achieved at the three-day range in 1980 is today achieved at around the six-day range, and such improvement by about one day per decade is seen also at other forecast ranges. Improvement is larger still for the southern hemisphere, where the principal extra gain was between the mid 1990s and early 2000s, after which the difference in average performance between the two hemispheres has been generally small.

The forecasting system improvements over this period, in particular the introduction and subsequent refinement of variational assimilation of radiance data, have been discussed by Simmons and Hollingsworth (2002).

The plot for ERA-Interim in the right-hand panel shows much less trend than that for ECMWF operations, showing the net operational improvement since 1980 to have been predominantly due to improvement of the forecasting system rather than improvement of the observing system. Nevertheless, the inter-hemispheric differences for ERA-Interim do decline over the first two decades, and both hemispheres show an improvement over the last decade or so that is not much less than half that seen for the operational forecasts.

In this regard it should be noted that ERA-Interim’s use of a fixed 2006 version of the ECMWF forecasting system means that it is unable to assimilate data from the high-resolution infrared sounder (IASI) and scatterometer (ASCAT) on Europe’s Metop satellite, launched in October that year. The ERA-Interim results thus do not measure the full impact of observing-system changes. They also, like those from operations, are subject to variations in the predictability of atmospheric flow regimes. This is the likely explanation for the larger difference in scores between the hemispheres seen for both operations and ERA-Interim for the last two years.

The observing-system changes since the late 1990s that have contributed to the improvement seen in the ERA-Interim forecasts include launches of several new satellite instruments. Among them are the build up starting in 1998 to five AMSU-A instruments operating on various satellites, the first of the high-resolution infrared sounders, AIRS, MODIS instruments from which high-latitude winds are derived by tracking features in successive polar images, and a number of orbiting receivers providing temperature information though measurement of the occultation of GPS signals.

The period also saw quite substantial increases for several types of in situ observation. Figure 4 shows, for example, sharp rises in the numbers of near-tropopause temperature data from aircraft and surface-pressure data from drifting buoys and land SYNOP stations, and a more gradual rise in radiosonde temperature measurements for the free troposphere. Radiosonde ascents dropped from an average of 1 626 per day in 1979 (much the same Figure as in 1958) to 1 189 per day in 2001, but numbers have increased since then, and currently typically exceed 1 300 each day. Moreover, there has been an increase in the amount of significant-level data transmitted, and improvements in the quality of the data.

Figure 4 — Number of observations available daily to ERA-Interim from 1979 onward, for four types of in situ data.

Some other aspects of forecasting

A good forecast of the evolving synoptic situation, such as measured by the height anomaly correlation for extratropical prediction, is a prerequisite for an accurate forecast of the associated weather, but not a guarantee. Weather conditions may be associated with sub-synoptic-scale dynamical systems, local topography, boundary-layer characteristics and so forth, and this brings additional requirements for observations in support of the delivery of forecasting services.

This includes not only additional atmospheric observations needed to predict the smaller scales of motion in the short range, but also observations of the land-surface conditions such as snow cover and soil moisture that can influence local near-surface weather elements in medium-and longer-range predictions. Observations of land and ice conditions, as well as open ocean conditions, are needed more fundamentally for the longer time ranges over which they provide the underlying basis for atmospheric predictability.

Notwithstanding the substantial progress made in estimating atmospheric states for numerical prediction, some degree of uncertainty in initial conditions cannot be avoided. The improvement in deterministic numerical forecasting over the past 20 years has been accompanied by the development of probabilistic forecasting systems based on the use of ensembles of lower resolution forecasts that sample uncertainty in initial conditions and modelling (Molteni et al., 1996: Toth and Kalnay, 1997). A recent set of tropical cyclone forecasts is presented here to illustrate a few points. The storm in question, named Songda, occurred over the western Pacific in May 2011.

Figure 5 shows forecasts from 00 UTC on 23 May, 12 UTC on 25 May and 00 UTC on 28 May. The left-hand panels show track forecasts, with circles denoting the reported positions at 12-hourly intervals up to the start time of each forecast. The storm was quite young on 23 May, and the ensemble indicates a spread in possible forecast tracks. The deterministic forecast from this date is for a track that is a little to the left of the observed track by 27 May, as is the case for the most likely track inferred from the ensemble.

Figure 5 — ECMWF forecasts of tropical cyclone Songda made from start times of 00 UTC 23 May (upper), 12 UTC 25 May (middle) and 00 UTC 28 May 2011 (lower). The left-hand plots show track forecasts, with colours indicating the probability (%) based on ensemble forecasts that Songda will pass within 120 km of the point in question, and the black lines denoting high resolution deterministic forecasts. The right-hand panels show central pressures from the high resolution forecast (red), the ensemble control forecast (green) and the ensemble itself (blue), for which the box denotes the range between the 25th and 75th percentiles and the vertical line denotes the full range of values.

A cluster of several ensemble members does however indicate the possibility of a curvature of track closer to that observed. Much less spread is seen in the subsequent forecasts, all of which depict rapid movement following curvature, with filling and transition to an extratropical system over southern Japan.

The right-hand panels of Figure 5 show corresponding forecasts of central surface pressures. Here there is an evident effect of the difference in horizontal resolution, which is some 32 km for the ensemble members compared with 16 km for the deterministic forecast. The deterministic forecast produces lower central pressures than any ensemble member when the storm is at its strongest, although the reported minimum pressure was a further 20 hPa or so lower still.

Recent years have seen a substantial improvement of capability for tropical cyclone prediction using global systems. The annual-mean position error of ECMWF’s operational three-day deterministic forecasts has been reduced from about 350 km to 200 km since 2005, and the mean absolute error of central pressure has been reduced from about 25 hPa to under 15 hPa over the same period. Whilst these recent improvements have been ascribed principally to finer horizontal resolution and better parameterization of convection (Fiorino, 2009), prediction has also benefitted from the use of 4D-Var and additional observations such as from scatterometers (Isaksen and Janssen, 2004). The performance of global models also benefits from improvements made to their prediction of the larger scale environment in which the tropical cyclone is embedded.

Challenges nevertheless remain in the use of observations in the vicinity of intense, small-scale systems. Errors in position, intensity and horizontal scale in background forecasts can cause rejection of good data by quality-control processes that check for unusually large differences between background forecasts and observations. On the other hand, lax quality control of a single misreporting buoy can cause a significant degradation of the analysis of a tropical cyclone.

Good observations made just outside a strong vortex may undesirably weaken a background-model vortex that is too large in scale and already too weak due to limited resolution. Here, progress is being made with the use of ensemble data assimilation methods that provide a dynamical estimate of background errors for use both within the assimilation system itself and in constructing better perturbed initial conditions for ensemble forecasting.

Monitoring climate variability and change

Effective use of multiple sources of data to initialize numerical forecasts requires that various biases in the observations be corrected prior to or during the data assimilation. These biases can be much larger than the standard deviations of differences between observed and background values and can vary between instruments of the same type, from one satellite to another for example.

The effort devoted to observational bias correction in weather prediction systems has also brought benefit to the consistency over time of fields from reanalysis, increasing their value for study of decadal climate variability and change.

Figure 6 demonstrates the stability and temporal consistency of the ERA-Interim reanalysis for global-mean mid-tropospheric temperature. The background temperatures from the data assimilation are largely consistent with both radiosonde observations (top panel) and with bias-corrected radiance measurements from the Microwave Sounding Unit (MSU) instruments flown on successive National Oceanic and Atmospheric Association (NOAA) satellites (centre panel).

Figure 6 — Global-mean background departures between ERA-Interim and radiosonde temperatures (K; upper) for pressures in the range from 275 to 775 hPa, and between ERA-Interim and MSU channel-2 brightness temperatures (K; middle). The global-mean bias corrections (K) derived for the MSU data are shown in the lower panel. Colouring is used in the middle and lower panels to denote results from each of the satellites that carried the MSU instrument, from TIROS-N to NOAA-14.

The radiosonde bias correction comprises long-term homogeneity adjustments derived by Haimberger et al. (2008) and separate adjustments for annual variations in the bias due to solar heating. These adjustments are more significant at upper tropospheric and stratospheric levels.

The bias corrections for the MSU data (lower panel) are produced by ERA-Interim’s variational analysis (Dee and Uppala, 2009), and account for calibration differences, orbital drifts and various other instrument errors, and for systematic errors in the radiative transfer model utilised by the assimilation system. The corrections are much larger than the mean discrepancies between the background forecasts and the observations.

The consistency with the adjusted radiosonde and MSU data gives some confidence in the low-frequency variability and trends in upper-air temperature from ERA-Interim. It is important, however, to compare the MSU corrections with independent estimates of biases in these data, and to assess the fit of the reanalysis to other types of data. Dee and Uppala (2009) discuss independent evidence for the rapid variation in the bias of NOAA-14 seen around 2002 in Figure 6.

They also show evidence of a detrimental effect on long-term trend estimates for upper tropospheric temperature due to assimilation of increasing amounts of data from aircraft for which no bias correction was applied. Assimilating recently available GPS radio occultation data improves the fit of the reanalysis to radiosonde temperatures at the tropopause and in the lower stratosphere (Poli et al., 2010), but thereby introduces a small artificial trend in the reanalysis.

There are also issues concerning upper stratospheric temperature, where routine data to anchor bias corrections of the uppermost satellite sounding channels is lacking.

Considerable progress has been made in this area, but challenges still remain.

Temperature and precipitation anomalies

Much of the practical demand for climate information and services naturally concerns conditions at or near the earth’s surface. Figure 7 shows maps of anomalies in temperature and precipitation for 2010, from ERA-Interim and from direct analysis of the observational record based on monthly station data (CRUTEM3, Brohan et al., 2006; 2.5° Global Precipitation Climatology Centre (GPCC) version 5 and monitoring product.

Figure 7 — Anomalies for 2010 relative to 1979–2010 in surface air temperature (K; left) from ERA-Interim (upper) and CRUTEM3 (lower), and in precipitation (mm/day; right) from ERA-Interim (upper) and GPCC (lower). Values are plotted over land for grid-squares with a complete monthly data record for 2010 and no more than 12 missing months from 1979 to 2009. For GPCC, it is also required that there be at least one station per grid-box.

The ERA-Interim temperature field is based on a separate analysis of the synoptic record using background fields derived from the 4D-Var upper-air analysis (Simmons et al., 2004, 2010). In contrast, precipitation data are not assimilated over land; values shown here are derived from forecasts in the range from 12–24 hours ahead carried out every 12 hours.

CRUTEM3 and GPCC values are shown only for grid squares for which station data are available to contribute to the analysis, and only a limited number of missing months are allowed in defining the 1979–2010 reference for computing anomalies.

It is clear from Figure 7 that ERA-Interim captures the anomaly patterns in temperature and rainfall that are indicated by the CRUTEM3 and GPCC analyses, and indeed as have been reported directly from the records of individual observing stations and summarised in the WMO Statement on the Status of Global Climate in 2010.

Particularly evident in the annual means are warm conditions over much of Canada, northern Africa and parts of Asia, and below average temperatures over north-western Europe, north-central Asia and eastern Australia. Rainfall is substantially above average over this part of Australia, Indonesia and north-western South America, while drought over Brazil is indicated by above average temperatures and below average rainfall.

Extremes such as the heatwave and drought over western Russia and devastating summer rains over Pakistan show less clearly in these annual means. Reanalysis provides a comprehensive record of atmospheric and surface fields for study of the processes involved in these events.

Figure 7 shows that ERA-Interim gives spatially coherent patterns where station data are unavailable or cannot be used in the CRUTEM3 or GPCC analyses. It should be noted that ERA-Interim does assimilate observational data for these regions. It uses many synoptic observations of surface air temperature from stations for which long time series of monthly climatic data are not available for use in CRUTEM3, and many different types of surface and upper-air observations influence its precipitation estimates. Nevertheless, these regions tend to be ones where data coverage is sparser than elsewhere, and where modelling of atmospheric processes is more challenging.

Extra care is needed in using reanalysis results for these regions, and quite aside from any initiatives taken to improve observational coverage, it is important that as much of the data that has been taken for these regions is made available in the formats needed by the different types of analysis. This includes both recovery of historic data and more prompt and widespread transmission of recent data, in particular through timely updating of World Weather Records in the case of input to analyses such as CRUTEM3 and timely transmission of monthly precipitation data to the Global Precipitation Climatology Centre.

Variations in annual-mean anomalies from 1979 to 2010 for parts of central Europe, southeastern Australia and eastern Africa are presented in Figure 8. Area averages are formed using all available values from CRUTEM3 and all GPCC values for grid squares that include at least one observing station. The GPCC data are used here at 1° resolution. Monthly-mean temperature and precipitation from ERA-Interim are mapped onto the CRUTEM3 and GPCC grids respectively, and used only where CRUTEM3 and GPCC provide data, as in Simmons et al. (2010).

Figure 8 — Annual anomalies in surface air temperature (K; left) and precipitation (mm/day; right) from ERA-Interim (broad bars) and respectively CRUTEM3 and GPCC (narrow bars), over central Europe (10-25°E, 45-55°N; top), south-eastern Australia (135-150°E, 30-40°S; middle) and eastern Africa (30-40°E, 15°N-10°S; bottom).

The central European region is very well observed, and CRUTEM3 and ERA-Interim temperatures are in particularly close agreement here. Temperature variations are larger than for the other regions considered, and show an abundance of relatively warm years later in the period. Precipitation anomalies are generally smaller than for the other regions, with agreement between ERA-Interim and GPCC mostly within about 0.1 mm/day. This includes the pronounced dry anomaly of 2003 and wet anomaly of 2010.

South-eastern Australia is also relatively well observed, and interannual variations in temperature are captured well. There is nevertheless a clear shift in temperature values around 1990. ERA-Interim anomalies are warm compared with CRUTEM3 for all earlier years, and cool for all subsequent years. Agreement is in fact much better for precipitation, with ERA-Interim capturing quite well the dry years of the early 1980s and the 2000s, and the wet years of 1992 and 2010, though rather underestimating the 2010 anomaly compared with GPCC. This suggests that the shift in temperature analysis arises from a change in the near-surface temperature data used by ERA-Interim (or perhaps CRUTEM3) rather than from a more general degradation of the background fields of the reanalysis.

The more sparsely observed east African region poses much more of a challenge. There is broad agreement between CRUTEM3 and ERA-Interim as regards the overall warming trend, with both datasets identifying 2009 and 2010 as the warmest years of the period. Discrepancies are however of the order of 0.5 K for 1991–1994 and 2002–2003.

Marked multiannual discrepancies in east African rainfall anomalies are seen for the first part of the period, with ERA-Interim relatively wet compared with GPCC for the first nine years and dry for the next eight, by a quite substantial margin. ERA-Interim nevertheless detects the drought year of 1984 as the driest in its first decade. Agreement is better in the second half of the period, where both ERA-Interim and GPCC show anomalously high rainfall in the flood years of 1997, 1998 and 2006, and very dry condition accompanying the warmth of the past two years. Repeating the comparison using full data coverage for ERA-Interim rather than the same coverage as GPCC gives much the same picture.

Services for atmospheric composition

Evolving capabilities for atmospheric observation, modelling and data assimilation have led to the emergence in recent years of new services relating to trace constituents important for air quality and climate, with end-user applications in areas such as health and solar energy supply. The constituents comprise the long-lived greenhouse gases, the faster reacting gases affecting air quality and the aerosols that affect air quality and climate forcing. The various types of constituent interact one with the other, and influence weather through radiative processes and the interactions between aerosol, cloud and precipitation. Monitoring and forecasting of stratospheric ozone and surface UV radiation have already been discussed.

Constituent forecasting or monitoring can be carried out with some degree of success using chemical transport models with prescribed or parameterized surface emissions and meteorological data supplied from numerical weather prediction or reanalysis. However, there is an increasing focus on the use of data assimilation and more integrated modelling approaches.

This has been prompted in part by availability of data such as from Europe’s ERS-2, ENVISAT and Metop satellites, from the US series of EOS satellites and from the Japanese GOSAT mission. ECMWF and its partners in GMES atmospheric-service projects are developing integrated global modelling for the meteorological, chemical and particulate variables, and utilising it in 4D-Var data assimilation for joint analysis of trace species and meteorology.

Prototype global operational services provide forecasting and reanalysis, and support regional air-quality prediction. The regional systems have also begun to use reanalysis for air-quality assessment over Europe, with interpretation provided by source-receptor modelling. This integration of pilot atmospheric-service provision exists within the broader integration of the GMES programme, which includes corresponding services for ocean and land, the provision of the required space-based observation through the forthcoming Sentinel series of satellites, and coordination of in situ observation.

It is beyond the scope of this article to go much further into this and other initiatives, but one new challenge merits mention. It concerns the estimation either of emissions or of net surface fluxes. Indeed, the primary motivation for developing data assimilation for the long-lived greenhouse gases is to improve knowledge of the variations in space and time of the surface fluxes, as basic monitoring of concentrations of the gases is already provided by analysis of samples collected from the surface flask network.

For carbon dioxide, NOAA’s Carbon-Tracker (Peters et al., 2007) provides service delivery for surface fluxes and atmospheric distributions based on use of surface flask and tower observations, transport modelling and data assimilation. Methane flux estimation for the pilot GMES atmospheric service also employs transport modelling and data assimilation for the flux inversion (Bergamaschi et al., 2009), but uses methane analyses from the comprehensive integrated global system for input data, in addition to surface measurements.

The integrated system currently assimilates only SCIAMACHY data on methane, and although in this case the two-step approach does not bring evident benefit over direct use of the SCIAMACHY data in the inversion, it is expected to be beneficial when methane data from multiple satellites are used. A similar approach has been developed for carbon dioxide, for which promising results are being obtained using GOSAT data, as discussed by Chevalier at the 16th World Meteorological Congress side event.

The GMES system in general utilises specified or parameterized emissions for its range of species. The highly variable emissions from wildfires are based on assimilating satellite observations of fire radiative power. This component of the system and its analysis and forecasting of aerosols from the fires over western Russia in July and August 2010 are discussed by Kaiser et al. (2011). The system currently does not cater by default for the emissions from volcanic eruptions other than by picking up any consequential increases in sulphur dioxide and aerosols captured in the assimilated observations.

This is illustrated in Figure 9 for the recent eruption of the Icelandic volcano Grimsvötn. The initial injection of sulphur dioxide was well captured by OMI satellite observations assimilated into the GMES system, as shown for 12 UTC 22 May, and confirmed by a detection algorithm using IASI data (Clarisse et al., 2008). Only very limited data for the eastward tip of the plume were available for the next 12-hour assimilation period, and in the absence of an emission source in the assimilating model, the subsequent 12-hour background forecast significantly underestimated the intensity and westward extension of the plume, which was enhanced considerably by assimilating the OMI data that had good coverage again for the 12 UTC analysis for 23 May.

Figure 9 — Column SO2 (Dobson units) for the eruption of Grimsvötn in May 2011. The colour shading in the left panels show analyses for 12 UTC on 22 May (top) and 23 May (bottom), and the 12-hour background forecast for the 12 UTC 23 May analysis (middle). The OMI data available for assimilation are shown in the right panels, with grey dots denoting values less than 2 Dobson units. Dots in the left-hand panels denote an independent detection of SO2 from IASI data.

Manual intervention to specify unusual emission levels is fundamental to the use of the transport models operated by the Volcanic Ash Advisory Centres for provision of service to aviation, or by WMO’s Regional Specialized Meteorological Centres for Environmental Emergency Response for other types of event. There is, however, a broader interest in developing the 4D-Var approach so that it determines not only initial atmospheric distributions but also adjustments to emissions over the assimilation period for the faster varying species (Elbern et al., 2007).

This offers prospects for accounting for short-term fluctuations or trends in emissions of species that are currently prescribed from inventories, and for reducing effects of deficiencies in the parameterization of natural emissions, in addition to estimating the emissions from isolated extreme events.

The latter pose other challenges such as that of specifying background errors for atypical conditions. Ensemble approaches would also be appropriate in recognition of the inevitable remaining uncertainty in emissions, although the additional computational costs of chemical and aerosol modelling are a limiting factor in extending this approach from weather forecasting.

Conclusions

Observations are vital to the provision of weather, air-quality and climate services, but how they are used in the route from making the original measurement to delivering the service to users is vital too. This article has focused on one aspect of this, the use of data assimilation.

Much has been achieved over the past three decades in extracting an increasing amount of information from the observations made over the period, but much remains to be addressed in improving monitoring and prediction. Not least is the continued development of the assimilation and forecasting systems essential to extract full benefit from the investment made in observation.

Important requirements for continuing to improve in situ and satellite observation for atmosphere, ocean and land have not been discussed in this article, and must not be downplayed. However, irrespective of the progress made in this regard there are continuing challenges to be met in improving the use of those observations that have already been made or are being made.

Opportunities to recover past data and to ensure prompt and widespread access to all data need to be exploited to the fullest possible extent, catering for the different ways in which observations will be used for various purposes. This includes discovery and digitisation of older ground-based observations and reprocessing of earlier satellite data.

Continued development and comparison of alternative approaches to producing datasets for climate monitoring is especially important in giving confidence, quantifying uncertainty and identifying issues to be resolved, especially where data coverage is poor.

Acknowledgments

Colleagues past and present are thanked for their collaborations since the 1970s. Particular thanks go to current members of the ERA and GMES projects, especially Paul Poli and Antje Inness for providing the material for three figures. Funding was provided by the European Union’s Seventh Framework Programme.

References

van der A, R.J., M.A.F Allaart and H.FJ. Eskes, 2010: “Multisensor reanalysis of total ozone”. Atmospheric Chemistry and Physics 10, pp. 11277-11294.

Bengtsson, L., M. Kanamitsu, P. Kållberg and S. Uppala, 1982a: “FGGE Research Activities at ECMWF”. Bulletin of the American Meteorological Society 63, pp. 277-303.

Bengtsson, L., M. Kanamitsu, P. Kållberg and S. Uppala, 1982b: FGGE 4 Dimensional Data Assimilation at ECMWF”. Bulletin of the American Meteorological Society 63, pp. 29-43.

Bergamaschi, P., C. Frankenberg, J.F. Meirink, M. Krol, M.G. Villani, S. Houweling, F. Dentener, E.J. Dlugokencky, J.B. Miller, L.V. Gatti, A. Engel, and I. Levin, 2009: “Inverse Modeling of Global and Regional CH4 Emissions using SCIAMACHY Satellite Retrievals”. Journal of Geophysical Research 114, doi:10.1029/2009JD012287.

Brohan, P., J.J. Kennedy, I. Harris, S.F.B. Tett and P.D. Jones, 2006: “Uncertainty Estimates in Regional and Global Observed Temperature Changes: a new dataset from 1850”. Journal of Geophysical Research 111, D12106, doi:10.1029/2005JD006548.

Clarisse, L., P.F. Coheur, A.J. Prata, D. Hurtmans, A. Razavi, T. Phulpin, J. Hadji-Lazaro and C. Clerbaux, 2008: “Tracking and quantifying volcanic SO2 with IASI, the September 2007 eruption at Jebel at Tair”. Atmospheric Chemistry and Physics 8, pp. 7723-7734.

Dee, D.P., and S. Uppala, 2009: “Variational bias correction of satellite radiance data in the ERA-Interim reanalysis”. Quarterly Journal of the Royal Meteorological Society 135, pp. 1830-1841.

Dee, D.P., S.M. Uppala, A.J. Simmons, P. Berrisford, P. Poli, S. Kobayashi, U. Andrae, M.A. Balmaseda, G. Balsamo, P. Bauer, P. Bechtold, A.C.M. Beljaars, L. van de Berg, J. Bidlot, N. Bormann, C. Delsol, R. Dragani, M. Fuentes, A.J. Geer, L. Haimberger, S./B. Healy, H. Hersbach, E.V. Hólm, L. Isaksen, P. Kållberg, M. Köhler, M. Matricardi, A.P. McNally, B.M. Monge-Sanz, J.-J. Morcrette, B.-K. Park, C. Peubey, P. de Rosnay, C. Tavolato, J.-N. Thépaut and F. Vitart, 2011: “The ERA-Interim reanalysis: configuration and performance of the data assimilation system”. Quarterly Journal of the Royal Meteorological Society 137, pp. 553-597.

Elbern, H., A. Strunk, H. Schmidt and O. Talagrand, 2007: “Emission rate and chemical state estimation by 4-dimensional variational inversion”. Atmospheric Chemistry and Physics 7, pp. 3749-3769.

Eskes, H., A. Segers and P. van Velthoven, 2005: “Ozone Forecasts of the Stratospheric Polar Vortex–Splitting Event in September 2002”. Journal of the Atmospheric Sciences 62, pp. 812–821.

Fiorino, M., 2009: “Record-setting performance of the ECMWF IFS in medium-range tropical cyclone track prediction”. ECMWF Newsletter 118, pp. 20-27.

Haimberger L., C. Tavolato and S. Sperka, 2008: “Toward elimination of the warm bias in historic radiosonde temperature records: Some new results from a comprehensive intercomparison of upper-air data”. Journal of Climate 21, pp. 4587-4606.

Hollingsworth, A., R. J. Engelen, C. Textor, A. Benedetti, O. Boucher, F. Chevallier, A. Dethof, H. Elbern, H. Eskes, J. Flemming, C. Granier, J. W. Kaiser, J.-J. Morcrette, P. Rayner, V.-H. Peuch, L. Rouil, M.G. Schultz, A.J. Simmons and the GEMS Consortium, 2008: “Toward a monitoring and forecasting system for atmospheric composition: The Gems Project”. Bulletin of the American Meteorological Society 89, pp. 1147-1164.

Isaksen, L., P.A.E.M. Janssen, 2004: “Impact of ERS scatterometer winds in ECMWF’s assimilation system”. Quarterly Journal of the Royal Meteorological Society 130, pp. 1793-1814.

Kaiser, J.W., A. Heil, M.O. Andreae, A. Benedetti, N. Chubarova, L. Jones, J.-J. Morcrette, M. Razinger, M.G. Schultz, M. Suttie, M. and G.R. van der Werf, 2011: “Biomass burning emissions estimated with a global fire assimilation system based on observed fire radiative power”. Biogeosciences 8(4), pp. 7339-7398.

Kalnay, E., M. Kanamitsu, R. Kistler, W. Collins, D. Deaven, L. Gandin, M. Iredell, S. Saha, G. White, J. Woollen, Y., Zhu, M. Chelliah, W. Ebisuzaki, W. Higgins, J. Janowiak, K.C. Mo, C. Ropelewski, J. Wang, A. Leetmaa, R. Reynolds, R. Jenne and D. Joseph, 1996: “The NCEP/NCAR 40-year reanalysis project”. Bulletin of the American Meteorological Society 77, pp. 437-471.

Matsuno, T., 1971: “A dynamical model of the stratospheric sudden warming”. Journal of the Atmospheric Sciences 28, pp. 1479-1494.

Miyakoda, K. , R.F. Strickler, and G.D. Hembree, 1970: “Numerical simulation of the breakdown of a polar-night vortex in the stratosphere”. Journal of the Atmospheric Sciences 27, pp. 139-154.

Miyakoda, K., G.D. Hembree, R.F. Strickler and I. Shulman, 1972: “Cumulative Results of Extended Forecast Experiments I. Model Performance for Winter Cases”. Monthly Weather Review 100, pp. 836-855.

Molteni, F., R. Buizza, T.N. Palmer and T. Petroliagis, 1996: “The ECMWF Ensemble Prediction System: Methodology and validation”. Quarterly Journal of the Royal Meteorological Society 122, pp. 73-119.

Onogi, K., J. Tsutsui, H. Koide, M. Sakamoto, S. Kobayashi, H. Hatsushika, T. Matsumoto, N. Yamazaki, H. Kamahori, K. Takahashi, S. Kadokura, K. Wada, K. Kato, R. Oyama, T. Ose, N. Mannoji and R. Taira, 2007: “The JRA-25 Reanalysis”. Journal of the Meteorological Society of Japan 85, pp. 369-432.

Peters W., A.R. Jacobson, C. Sweeney, A.E. Andrews, T.J. Conway, K. Masarie, J.B. Miller, L.M.P. Bruhwiler, G. Pétron, A.I. Hirsch, D.E.J. Worthy, G.R. van der Werf, J.T. Randerson, P.O. Wennberg, M.C. Krol and P.P. Tans, 2007: “An atmospheric perspective on North American carbon dioxide exchange: CarbonTracker”. Proceedings of The National Academy of Sciences USA 104, pp. 18925 18930.

Ploshay, J.J., R. White and K. Miyakoda, 1992: “FGGE reanalysis at GFDL”. Monthly Weather Review 120, pp. 1747-1763.

Poli, P., S.B. Healy, D.P. Dee, 2010. Assimilation of Global Positioning System radio occultation data in the ECMWF ERA–Interim reanalysis. Quarterly Journal of the Royal Meteorological Society 136, pp. 1972–1990.

Scherhag, R., 1952: “Die explosionsartigen Stratosphärenerwärmungen des Spätwinters 1951/52”. Berichte des deutschen Wetterdienstes in der US-Zone 6, No. 38, pp. 51-63.

Shuman, F.G., 1989: “History of numerical weather prediction at the National Meteorological Center”. Weather and Forecasting 4, pp. 286-296.

Simmons, A.J., and R. Strüfing, 1983: “Numerical forecasts of stratospheric warming events using a model with a hybrid vertical coordinate”. Quarterly Journal of the Royal Meteorological Society 109, pp. 81-111.

Simmons, A.J. and A. Hollingsworth, 2002: “Some aspects of the improvement in skill of numerical weather prediction”. Quarterly Journal of the Royal Meteorological Society 128, pp. 647-677.

Simmons, A.J., P.D. Jones, V. da Costa Bechtold, A.C.M. Beljaars, P.W. Kållberg, S. Saarinen, S.M. Uppala, P. Viterbo and N. Wedi, 2004: “Comparison of trends and low-frequency variability in CRU, ERA-40 and NCEP/NCAR analyses of surface air temperature”. The Journal of Geophysical Research 109, D24115, doi:10.1029/2004JD005306.

Simmons, A.J., M. Hortal, G. Kelly, A. McNally, A. Untch and S. Uppala, 2005: “ECMWF analyses and forecasts of stratospheric winter polar vortex break-up: September 2002 in the southern hemisphere and related events”. Journal of Atmospheric Science 62, pp. 668-689.

Simmons, A. J., K.M. Willett, P.D. Jones, P.W. Thorne and D.P. Dee, 2010: “Low-frequency variations in surface atmospheric humidity, temperature, and precipitation: Inferences from reanalyses and monthly gridded observational data sets”. The Journal of Geophysical Research, 115, D01110, doi:10.1029/2009JD012442.

Teweles, S., 1958: “Anomalous warming of the stratosphere over North America in early 1957”. Monthly Weather Review 86, pp. 377-396.

Toth, Z., and E. Kalnay, 1997: “Ensemble Forecasting at NCEP and the Breeding Method”. Monthly Weather Review 125, pp. 3297-3319.

Uppala, S.M., P.W. Kållberg, A.J. Simmons, U. Andrae, V. da Costa Bechtold, M. Fiorino, J.K Gibson, J. Haseler, A. Hernandez, G.A. Kelly, X. Li, K. Onogi, S. Saarinen, N. Sokka, R.P. Allan, E. Andersson, K. Arpe, M.A. Balmaseda, A.C.M. Beljaars, L. van de Berg, J. Bidlot, N. Bormann, S. Caires, F. Chevallier, A. Dethof, M. Dragosavac, M. Fisher, M. Fuentes, S. Hagemann, E. Hólm, B.J. Hoskins, L. Isaksen, P.A.E.M. Janssen, R. Jenne, A.P. McNally, J.-F. Mahfouf, J.-J. Morcrette, N.A Rayner, R.W. Saunders, P. Simon, A. Sterl, K.E. Trenberth, A. Untch, D. Vasiljevic, P. Viterbo, and J. Woollen, 2005: “The ERA-40 Reanalysis”. Quarterly Journal of the Royal Meteorological Society 131, pp. 2961-3012.

__________

European Centre for Medium-Range Weather Forecasts, Shinfield Park, Reading RG2 9AX, UK
adrian.simmons[at]ecmwf.int