Towards effective NAMAs: tapping knowledge from policy evaluation

The origin and future of NAMAs

Nationally Appropriate Mitigation Actions (NAMAs) are sets of development policies and actions aimed at reducing greenhouse gas emissions, which were first mentioned in the Bali Action Plan in 2007. NAMAs are voluntary actions defined by developing countries and they have become one of the key topics on the agenda of international negotiations as the means for boosting social, economic and environmental transformations necessary to promote low carbon development in developing countries.

The ‘intended nationally determined contributions (INDCs)’ for the agreement in Paris in 2015 will be a next important step in this process. All Parties are expected to define their contributions - including developing countries. This will likely also provide a further boost to international support for NAMA implementation. The fact that the Green Climate Fund is now operational and initial capitalization is almost reaching the goal of 10 billion US$ also points towards enhanced funding, which makes it even more important to address some of the challenges NAMA implementation faces today.

Going beyond project based activities

While the NAMA concept has been under discussion for some years, concrete implementation still need to take off. The NAMA Status Report in 2013 identified the lack of clarity of the concept as one of the main reasons for the relatively slow implementation of NAMAs. Given that there is no internationally agreed definition, each country is free to define NAMAs as they see appropriate. The full list of NAMAs submitted to the UNFCCC  is a good indication for the broad interpretation of the term. Three types of NAMAs are emerging, differing in level of detail and concreteness, as shown below.



It is, however, important to realise that NAMAs are in itself not a new concept. Fundamentally, it is about good development policies and actions that now have the additional - and sometimes leading - objective to reduce GHG emissions. So what is new then? NAMAs can provide new sources of funding through for example the Green Climate Fund as well as many bilateral and other multilateral sources. For the international climate community it is also relatively new to think about large-scale, structured mitigation in developing countries beyond project-based activities. This leads to many approaches that are still fundamentally rooted in project-style thinking, rather than aiming at a smart design and implementation of policies. This is driven not only by existing experience - or lack thereof, but also by the complexity of determining the impacts of policies.

Understanding the effectiveness of NAMAs

The international climate community has years of experience with assessing the effectiveness of project-based activities under the CDM. Assessing the effectiveness for NAMAs, however, becomes more complex as the scope of mitigation actions under NAMAs is broadened, both in geographic coverage and the types of activities. Especially for policy NAMAs the CDM experience is of limited use when looking for methodologies to assess their effectiveness.

While being faces with similar questions, policy-based NAMAs require a different approach to solve issues, such as:

• How to determine the contribution of a policy NAMA towards achieving emission reductions?
• What is the causal relationship between the policy NAMA and the achieved reductions?
• How to set a standard for determining policy NAMA effectiveness in “diffused” sectors like households and transport or for cross-cutting policies, such as carbon taxes?
• What needs to be monitored in order to be able to determine if a policy NAMA is effective or not?

Discussions around baseline setting and additionally within the CDM demonstrate the challenges already faced at a project level. At a policy level these questions are even becoming more complex to answer. Determining cause-and-effect relationships and assessing policy interaction are typical problems faced in evaluating effectiveness. Data availability and quality beyond project activities become crucial challenges.

Involving the policy evaluation community

Extensive experience with these issues exists in the wider policy evaluation community. Substantial effort has been put into the development of both ex-post and ex-ante policy evaluation methodologies, e.g. in relation to energy efficiency, renewable energy and GHG emission policies and programs. This experience can be valuable input into the efforts to move towards the implementation of effective NAMAs. In doing so, potential funders of NAMAs can be provided with crucial information to enable, ex-ante, the prioritisation of NAMAs looking for international support in line with their own selection criteria and, ex-post, assess the cost-effectiveness of their investment. At the same time, robust policy evaluation can support implementing countries to enhance the overall effectiveness of their policy NAMAs, thus maximizing GHG reduction and sustainable development benefits.

A smart policy design process

A policy design process in the ideal case starts with clear objectives and the identification of barriers that hamper meeting these objectives. Barrier refers to any factor that inhibits the effectiveness of the policy. This is taken into account in designing and formulating the policy in such a way that barriers are addressed to a maximum extent. Policy implementation is followed by a continuous cycle involving monitoring and evaluations and possible reformulation of the policy to increase effectiveness of the policies. A successful policy design process includes an explicit understanding and logic of the causal relationship between the policy (NAMA) and policy objectives, like emission reductions to be achieved. In other words: How is the policy instrument (or a package of instruments) expected to achieve its impacts and who is expected to take action at which point in time. This relation is “translated” into a monitoring plan holding information for the key indicators that needs to be collected to assess its effectiveness. Experiences from a large number of ex-post evaluation studies have resulted in a listing of ingredients for effective policies: typical circumstances in which to apply different types of policies to improve energy efficiency and GHG emission and characteristics that determine the success of the policy.

Key challenges in policy impact evaluation

Various challenges exist when assessing the impact of policies. Experiences from policy evaluation can help to address these challenges. Some of these include:

• Setting a baseline: The baseline represents - in the case of policy NAMAs - the level of GHG emissions that most likely would have been emitted in the absence of the policy being assessed. Like in CDM, the baseline is not a historical reference point but an assumption about what would have happened in the implementation period if the assessed policy were not implemented. Attributing changes in emissions to specific policies and actions is not straightforward. GHG emissions can change due to a variety of factors, including: the policy or action being assessed; other policies or actions that affect the same emissions sources; and various external drivers that affect emissions, such as changes in economic activity, population, energy prices, weather, autonomous technological improvements, and structural shifts in the economy. In policy evaluations it is crucial to be very transparent on what is included in the baseline and what not, which assumptions are applied to determine the level of GHG emission for the baseline and what the uncertainties are in the calculations. Experiences from the policy evaluation community allow identifying key non-policy drivers for baseline development and robust methods for making projections and analysing uncertainty related to such projections.
   
• Establishing cause-and-effect relationships: For investment projects - like most of the CDM - it is relatively clear how the cause leads to the GHG effect. For the installation of, for example, a facility to capture landfill gas will reduce methane emissions from landfills and the amount can be estimated relatively accurately using available data on the landfill and the installation specifications. For policies this is often not quite as straight forward. Policies can impact investment and consumer behaviour in different ways - some more, some less direct. For directly impacting policies, such as regulations, the cause-and-effect relationship is rather clear, while this is less the case for indirect measures, such as taxes, incentives or research and development initiatives. Unlike the CDM, not only the relationship between the policy and the intended GHG effect needs to be considered, but also potential indirect and often unintended effects that can reinforce or counteract the policy’s mitigation goals. A range of methods and tools supports analysts in determining these relationships, like for example logical modelling. For many policies there are established relationships that can be adjusted based on the specific national circumstances.
   
• Evaluating multi benefits: NAMAs are not only about reducing greenhouse gas emissions but aim at contributing to broader sustainable development goals. Listing and assessing these multi benefits can increase stakeholder buy-in, enhance motivation of targeted actors, expand financial support for policies, and improve the cost-benefit ratio for policies.
  
  A good example for the multiple benefits is provided by the recently published IEA overview on how investment in energy efficiency provides benefits to different stakeholders. Improving energy efficiency in households can - next to GHG emission reduction – for example results in increase in disposable income by households. Energy efficiency improvement in industry is often accompanied by important increases in productivity of the firm, reduced air pollutant emissions and waste and improved public and labour health. Policy evaluation in the past did not focus on GHG emissions, but aimed to assess the broad spectrum of potential impacts of policies to provide governments with robust analysis for decision-making. A wide range of methods is available to evaluate different types of effects and to support an integrated assessment of multiple impacts.
   
• Determining success: For investment projects, success is normally easy to determine, with the planned installations up and running. For policies, success is much harder to assess, and depending on the type of policy implemented. A framework that includes the monitoring of inputs, activities and intermediate outcomes can facilitate a better understanding of successes and problems and help fine-tune policies to make them more effective. Policy evaluation is working with a multitude of indicators, depending on the type and scope of the policy. This allows targeted data collection to inform decision makers and provide confidence on the effectiveness of policies.

Concluding remarks

Policies are an essential element in delivering the needed social, economic and environmental transformations necessary to promote low carbon development in developing countries. To tap the potentials provided by smart policy making, a robust analysis of expected impacts and sound monitoring of achievements is key. Using insights and methodologies available from the community of policy evaluators can help to provide confidence in the effectiveness of policies proposed as NAMAs. Better understanding of expected impacts can enhance successful implementation, facilitate peer-to-peer exchange and unlock international support.
Experiences from many years of policy evaluation at different levels of jurisdiction can support governments in identifying the most promising policies to achieve the given objectives within the national circumstances. This starts with identifying favourable circumstances for different policy instruments in general to the detailed analysis of expected impacts in the given national context. The tools, methods and experience from the policy evaluation community can support governments in selecting, implementing and monitoring policy NAMAs to ensure effectiveness and create multiple benefits for the climate as well as the national communities.

Ingredients for effective policies

^{Source: Harmelink et al (2008)}