No Member Left Behind - Part 2: Development partners’ perspectives on overcoming sustainability challenges in observing networks and data exchange - lessons learned

Over the last two decades, development agencies[1] have invested hundreds of millions of US dollars in projects aimed at improving meteorological observing networks in developing countries. Their goal was, and remains, to assist those developing countries that cannot meet commitments to consistently operate and maintain their national observing networks and data exchange.  Weather, water and climate services rely on a consistent, coordinated worldwide system for real-time gathering and exchange of observations, and all WMO Members are committed to contributing to this exchange. The failure of any Member to meet these commitments adversely impacts the quality of weather and climate monitoring and prediction products both locally and globally.

Persistent capacity gaps have led to a growing number of development projects aimed at strengthening meteorological observing networks. However, the results have often been suboptimal. This article highlights some of the main reasons why observing networks supported by development agencies often fail to gain permanent traction in developing countries, and it offers a few examples of pathways for improving the support.  

Lack of surface-based observations - a persistent global problem

Despite several decades of significant investments in strengthening the meteorological sector in developing countries, many areas of the globe remain far from the goal of continuous, robust, real-time international exchange of surface-based observations. Figure 1 shows the international exchange of in situ observations of surface pressure – a key input variable for Earth system numerical modeling – as of 9 September 2021. The situation is dire, especially in areas with observing stations shown in black (no observations exchanged), red (sporadic exchange of observations) or with too few stations altogether. Not only will it be nearly impossible to provide high-quality forecast products in those areas; it will also be difficult to assess how good the forecasts are since there are no observations against which they can be verified. Satellite observations can help ensure a realistic model representation of large-scale atmospheric dynamics in the upper layers of the atmosphere, but cannot be used to verify forecasts of surface weather. Without the exchange of surface-based observations, the rest of the meteorological value chain (see Article 1, fig. 1) has little to build on.

Figure 1. Surface pressure observations received by global NWP Centres on 9 Sept 2021. (source: WIGOS Data Quality Monitoring System)

The WMO community has been concerned about the lack of observations from developing countries for decades, and many attempts have been made to address the problem. However, despite these efforts, in many places the data gap has been growing. For instance, the number of radiosonde observations over Africa provided to the global models decreased by roughly 50% between 2015 and early 2020. Why have substantial investments in observing systems not translated into increased observational data sharing?

Lack of a global approach

A significant part of the shortfall in results can be attributed to the lack of a global approach. Development projects are typically single-country focused and, therefore, whenever such projects include an observing system component, it will be focused on the national observing infrastructure. However, the action that is needed to establish a functioning data exchange is rarely purely national. Rather it involves collaboration with – and sometimes investment in – systems and entities working outside the country, for example Regional Telecommunication Hubs, Global Information System Centres and Regional WIGOS Centres. Single-country projects in general, therefore, cannot address data exchange issues.

Since observations that are not exchanged have little impact on prediction, this leads to a lack of incentive for the National Meteorological and Hydrological Services (NMHSs) to maintain and operate the observing networks once the projects are completed and the support ceases.

An overly narrow approach hampers other project types. Last-mile projects (such as early warning systems) rely heavily on the use of global model data. While the importance of these data is well understood by project implementers, the role of local observations in the global models generally is not. The critical link between the availability of local observations and the local quality of model data is generally not recognized, nor is the importance of observations for forecast verification. Furthermore,  the observations that are most important for weather forecasts in smaller countries often come from outside their borders. Single-country, last-mile focused projects generally have no coordination with similar projects in neighbouring countries, and implementing an observing network in a single country without any guarantee that the surrounding countries will do likewise is likely to provide limited value. Ongoing failure to address this problem of missing international coordination of observing system activities has been highly detrimental to the availability of radiosonde observations, especially over Africa.

Lack of appropriate measure of success

While the lack of observations from developing countries is recognized and frequently cited in project rationales and design documents, the problem of missing observations is often mistakenly interpreted as being a problem of missing observing stations (Box 1).  However, since observational data exchange is the ultimate goal, metrics of success for observing system projects should be defined accordingly, and not in terms of local installation and operation of stations.

Box 1 –  Distinguishing between observing networks and observational data exchange Hydromet development project in Malawi: A comprehensive network of 50 state-of-the-art Automated Weather Stations (AWS) was installed in Malawi with the support of the United Nations Development Programme (UNDP). The installation was completed in 2019, and the final evaluation rated the project as successful, i.e. all stations operating and delivering observations.  However, the initial Global Basic Observing Network (GBON) gap analysis undertaken by WMO in 2020 showed observations being exchanged internationally only sporadically from a single station in Malawi, and none from the AWS network. WMO in collaboration with UNDP conducted an internal assessment of the situation that concluded that “while the equipment on the ground is functioning and providing data to the national servers, there is a technical challenge that is still preventing the connection of this data to the regional and global servers housed by WMO”.  Upon further investigation by WMO starting in 2021, it was found that no observations from the AWS network were available at the NMHS headquarters, that due to increasing national capacity and budgetary constraints no observations were being internationally exchanged, that the network was unable to deliver the data in WMO standard (BUFR) data format, and that the telecommunication capabilities were inadequate. CREWS Western Africa project: In 2020, five countries in Western Africa (Burkina Faso, Chad, Mali, Niger and Togo) carried out assessments of their observation infrastructure and data management systems. These efforts, part of a multi-year regional investment by the Climate Risk and Early Warning Systems (CREWS) Initiative,[2] were carried out to strengthen the countries' access to essential data for predictions and risk information for effective early warnings. The assessments showed that of there were 341 observation stations with pressure sensors available in the five countries but only 60 (17%) were registered in the WMO Observing Systems Capability Analysis and Review (OSCAR) Surface station database. This limits the international exchange of data and therefore the quality of forecast products available in the countries. The low level of contribution comes from historical factors: many of the stations  were established primarily to provide data for predicting food insecurity and are therefore not connected to other regional and global systems. Capacity and resourcing were also identified as challenges. The five countries have started addressing the issue. Measures include development of maintenance plans for observing infrastructures, updating metadata in the OSCAR database and connecting stations to the WMO Information System (WIS), a process that is now simplified by Internet-based connectivity. In Burkina Faso, a step change was achieved. The figure below reflects a before and after context of the number of stations registered in OSCAR Surface database in Burkina Faso between April and August 2021. These efforts are being scaled-up, to cover all 24 countries in West and Central Africa, building on the effective model of South-South sustainable cooperation with financial support by CREWS. Number of surface stations (blue dots) registered in the WMO Observing Systems Capability Analysis and Review (OSCAR) database before and after the CREWS project

Lack of structural adjustment

Automatic weather stations (AWSs) are seen by donors and implementing entities as a modern, high-efficiency, low-cost means of providing surface-based meteorological data, but they often fail to gain traction in developing countries. NMHSs in developing countries continue to rely on manual observations made by human observers and transmitted by outdated communication methods even after AWS networks have been installed. There are structural reasons for this, as well as institutional barriers that are not easily removable via short-term project approaches.

Lack of coordinated and integrated implementation approach

A recurring issue faced by many developing Members is having several development partners independently of each other attempting to address the issue of missing observations via separate projects within their country. Many developing countries thus find themselves with disparate observing networks relying on vendor support from different donor countries, providing data in different proprietary formats, and requiring separate stocks of spare parts, etc. Such systems are difficult to sustain, even for NMHSs in developed countries.

Another coordination problem stems from the lack of recognition of the role of NMHSs in international data exchange. NMHSs act as the national node in the international exchange of observations as per WMO regulations and practice. However, in some cases, implementing entities only recognized the critical role of the NMHS in the data exchange after all project resources had been expended – hardware purchased and installed – but no data was flowing. Insufficient institutional, technical or financial support had been foreseen for the NMHS. This led to a lack of incentives, and as a result, no observational data were exchanged.

Lack of a realistic financing model undermines sustainability

In developing countries, in particular the Small Island Developing States (SIDS) and Least Developed Countries (LDCs), the lack of observations is often closely linked to the lack of ability to fund the necessary observing networks. Figure 2 shows the horizontal density of national observing networks (left panel), and available financial resources, measured by Gross Domestic Product (GDP) per km² surface area (right panel); a larger surface area implies a larger observing remit. Since SIDS often have Exclusive Economic Zones (EEZ) that are many times larger than their land areas, calculations for SIDS were made including both EEZ and land areas. The difference in “ability to pay” between rich and poor countries is striking: The richest countries make more than one million times more money per km² than the poorest countries. Scarce local resources lead to scarcity of observations, as shown by the similarity between the left and right panels in figure 2.

Figure 2. Ability to pay versus the ability to observe: Left panel: density of observations by nation (red do not meet requirements). Right panel: National GDP/km2 of surface area; darker colors (blue and purple) show fewer resources per unit area. (Source: WMO, 2021)

Finally, commercial approaches to generate revenues to cover the cost of certain government services are often extremely difficult to reconcile with the need for free and unrestricted international exchange of observations. Due to the role of observations at the beginning of the value chain and to  international data sharing agreements, observational data are difficult for national governments to monetize, and a wealth of economic analyses have shown that doing so would severely limit the use and, therefore, also the impact of the data.[3] However, in their quest for revenue, some national governments have attempted to limit the freedom of their NMHSs to exchange data, including observations.

A new support mechanism for observing networks in  developing countries - the Systematic Observations Financing Facility (SOFF)  

In many parts of the world, even with improved management and practices, it is unlikely that countries can sustain and operate adequate observation networks on their own (see box 2) . Recognizing this, the global community under the leadership of WMO, UNDP and UNEP and its partners in the Alliance for Hydromet Development are establishing a new financing mechanism, the Systematic Observations Financing Facility (SOFF).

SOFF is a dedicated mechanism that will provide long-term grants and technical assistance, with a focus on SIDS and LDCs, to enable sustained compliance with the Global Basic Observing Network (GBON - see article 11) requirements. SOFF will (i) deploy a global approach with sustained international data exchange as a measure of success; (ii) provide long-term finance toward sustained data sharing results; (iii) enhance technical competency through peer-to-peer advisory, harnessing the operational experience of the most advanced national meteorological services around the globe; and (iv) leverage partners’ knowledge and resources.

SOFF will focus exclusively on the initial part of the meteorological value chain (see Article 2), while working in partnership with other development agencies that focus on other links in the chain, to help ensure that its investments will ultimately translate into end-user benefits. SOFF funding will be embedded within larger hydromet/climate projects. This will ensure that countries will be further supported in developing the capacity to effectively use improved forecast and climate products to create adaptation and resilience development benefits.

SOFF will be structured as a “UN coalition fund”. WMO, the UN Development Programme (UNDP) and the UN Environment Programme (UNEP) will co-create the fund and the UN Multi-Partner Trust  Fund Office will administer SOFF funds.

SOFF has a well-defined theory of change. SOFF support will be provided in three consecutive phases with outputs designed to achieve sustained GBON compliance. This in turn will contribute to the ultimate goal of strengthened climate adaptation and resilient development through improved weather forecasts, early warning systems and climate information services crucial to saving lives and fostering economic prosperity. The three phases of SOFF support include:

• The Readiness Phase - beneficiary countries – SIDS, LDCs and other Official Development Assistance (ODA)-eligible countries – will be able to access analytical and advisory assistance provided by national meteorological services as peer advisors to define their GBON gap, and develop a GBON National Contribution Plan.
• The Investment Phase - SIDS and LDCs will receive grants for investments and advisory support to establish the stations network of stations and strengthen human and institutional capacity for GBON compliance.
• The Compliance Phase - SIDS and LDCs will receive results-based grants in support of operation and maintenance expenses for GBON data data-sharing compliant stations.

SOFF will be operationalized in three periods implemented over 10 years designed to achieve sustained GBON compliance of all SIDS and LDCs and provide technical assistance on GBON to all developing countries. The Facility will be legally established under the UN Multi-Partner Trust Fund Office by the end of October 2021. The creation of SOFF will be announced at the 26th session of the Conference Of Parties  (COP26) to the UN Framework Convention on Climate Change in a high-level event jointly with the initial funders. SOFF is envisaged to become operational by mid-2022.

Achieving sustained compliance   with   the   GBON   regulations   and   hence  a sustained  improvement  in  the  international  exchange  of  observational data will require substantial investments, strengthened  capacity  and  long-term resources for operation and maintenance  in  many  countries. To  close  their GBON gaps, the observations in SIDS and LDCs need to increase 28 times over their current levels for surface stations and 12 times for upper air stations. Reaching this ambitious target with the  urgency  needed  requires  an  accelerated  and  dedicated  international    effort. SOFF responds to this critical need.

Box 2. The unique sustainability challenges of observation networks in Pacific SIDS countries - UNEP/GCF programme In November 2020, the Green Climate Fund (GCF) approved a UNEP programme for five Pacific SIDS: Cook Islands, Niue, Palau, Republic of Marshall Islands (RMI), and Tuvalu, with a total value of US$ 49.9 million. The initiative supports the development of integrated climate and ocean information services, people-centred hydromet services and multi-hazard early warning systems. The five countries were selected as initial case studies for the GBON country gap analyses. The objectives will be achieved through four inter-related components: (i) a sustainable business delivery model for climate, hydromet, and early warning services; (ii) strengthened observations meeting GBON requirements and impact-based forecasting; (iii) improved community preparedness, response capabilities and resilience to climate risks, including forecast-based financing; and (iv) enhanced regional cooperation and knowledge management for climate services. The GCF Board and the Independent Technical Advisory Panel assessment of the programme considered that compliance with GBON was an innovative approach that strengthened the programme’s value proposition and noted the sustainability challenges for the proposed networks in the programme countries. As Pacific SIDS, the countries face unique challenges in assuring the sustainability of their hydrometeorological observation networks. The current expectation that each country should provide the resources to sustainably operate and maintain the observation network within their national territory (including ocean zones) is impracticable for Pacific SIDS with low incomes, small land masses and vast ocean areas. For example, the land area of the Marshall Islands (RMI) (181 km2) constitutes just 0.009 % of its EEZ (2 131 000 km2). The small size, remoteness and insularity of the countries pose a significant challenge to transport logistics. The cost of travel, transactions and general operations in the Pacific region are comparatively higher than in other parts of the world. Communication with outer islands can also be expensive and unreliable. This in turn translates to increased costs at each stage of network investment, operation, maintenance and replacement. In addition, the environmental conditions (i.e., warm temperatures, high humidity, and salt winds) in hot tropical locations such as the South Pacific are often unfavorable for meteorological sensors and automatic equipment – cheap weather stations often fail within 12 months. This necessitates the installation of more sophisticated and robust equipment to ensure accurate operation over long periods with only limited maintenance, which is more economical in the long-term but requires more significant up-front costs. Disruption and damage caused by increasingly frequent or intense extreme weather events, brought about by climate change, further impede sustainable operations. Yet without systematic in situ observations across widely dispersed outer islands, local forecast products cannot be validated, and Pacific SIDS cannot implement timely actions to reduce extreme climate impacts.

Summary

All WMO Members are committed to international data exchange,  however, structural, political and financial constraints currently prevent some of the developing country  Members from fully living up to their commitment under the WMO Convention.  The new WMO Unified Data Policy, and associated initiatives like the GBON regulations and SOFF, provide an opportunity for WMO, development partners and the Members of the Alliance for Hydromet Development to help developing country Members to address these issues for the mutual benefit of all. This will lead to a dramatic increase in the amount of observational data that are exchanged internationally, and therefore also to significantly improved model products for monitoring and prediction. The new policy also, for the first time, clearly articulates the principle that developing Members, in return for their observations, must be given free and unrestricted access to the model products that are  supported by their observations. This will help improve the service delivery capabilities of all WMO Members in all areas of Earth system monitoring and prediction.