89147 Water Papers June 30, 2014 THE HYDROPOWER SUSTAINABILITY ASSESSMENT PROTOCOL FOR USE BY WORLD BANK CLIENTS Lessons Learned and Recommendations Rikard Liden, Senior Hydropower Specialist Kimberly Lyon, Water Policy Specialist Approving Manager William Rex, Acting Sector Manager, Water Unit (TWIWA) Contact Information This paper is available online at www.worldbank.org/water. Authors may also be contacted through World Bank Water at worldbankwater@worldbank.org. Disclaimer—World Bank © 2014 The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This volume is a product of the staff of the International Bank for Reconstruction and Development/The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978- 750-8400, fax 978-750-4470, http://www.copy-right.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Acknowledgments This report was prepared by Rikard Liden, Senior Hydropower Specialist, and Kimberly Lyon, Water Policy Specialist, TWIWA, World Bank, as part of the Economic and Sector Work (ESW) Piloting the Hydropower Sustainability Assessment Protocol for the Trung Son Hydropower Project in Vietnam. The writing of the report and the work conducted to inform it was heavily supported by the Vietnam Energy Team, which included: Franz Gerner, Lead Energy Specialist, EASVS; Ky Hong Tran, Energy Specialist, EASVS; Thi Ba Chu, Energy Specialist, EASVS; and Jennifer Sara, Sector Manager, EASVS. The report has benefited from a thorough review process, which refined its focus and content, and which included invaluable input from a reference group of World Bank staff with knowledge of the Protocol and hydropower development. The peer reviewers and members of the reference group included Jean-Michel Devernay, Chief Technical Specialist for Hydropower, TWI; Julia Bucknall, Sector Manager for Energy, SASDE; William Rex, Lead Water Resources Specialist, TWIWA; Richard Spencer, Lead Energy Specialist, SASDE; Peter Leonard, Regional Safeguards Adviser, EASDE; Daryl Fields, Senior Water Resources Specialist, ECSAR; Pravin Karki, Senior Hydropower Specialist, SASDE; Ruth Tiffer-Sotomayor, Senior Environmental Specialist, LCSEN; Amal Talbi, Senior Water & Sanitation Specialist, MNSWA; and Pablo Cardinale, Principal Environmental Specialist, CESI2 (IFC). In addition, both Emmanuel Boulet, Environmental Lead Specialist, Inter-American Development Bank and Cameron Ironside, Program Director, International Hydropower Association gave valuable input as external peer reviewers. The team expresses its gratitude to Vietnam Electricity and Trung Son Hydropower Company for the extra time and input given to inform the World Bank on this report as part of the application of the protocol for the Trung Son Hydropower Project. A group of Accredited Assessors for the Protocol—Bernt Rydgren, Jörg Hartmann, Aida Khalil, and Doug Smith—conducted the assessment in Vietnam. Dr. Hartmann further provided input to the report on the experience from previous assessments. The Management Entity of the Protocol is acknowledged and thanked for providing data and information on the Protocol and previous assessments. This work was supported and financed by the Water Partnership Program. CONTENTS Foreword .......................................................................................................................................... vi Abbreviations & Acronyms ........................................................................................................... vii Executive Summary ...................................................................................................................... viii 1 Introduction ................................................................................................................................ 1 2 About the Hydropower Sustainability Assessment Protocol ................................................ 2 2.1 What Is the Protocol? ................................................................................................... 2 2.2 How Was It Developed? ............................................................................................... 2 2.3 How Is It Managed? ...................................................................................................... 4 2.4 What Is Entailed in a Protocol Assessment? ............................................................. 5 2.5 How Are Projects Scored? ........................................................................................... 7 2.6 How Does the Protocol Compare with Standards? ................................................... 8 2.6.1 The Protocol and World Bank Policies and Procedures .................................... 9 3 Experience with the Protocol: The Trung Son Hydropower Project, Vietnam.................... 10 3.1 Rationale...................................................................................................................... 10 3.2 About the Trung Son Project ..................................................................................... 10 3.3 Process and Timelines ............................................................................................... 12 3.4 Objective Setting ........................................................................................................ 13 3.5 Outcomes of the Assessment ................................................................................... 13 3.6 Client Feedback .......................................................................................................... 14 3.7 Lessons Learned ........................................................................................................ 15 3.7.1 Training and Client Buy-in ................................................................................ 15 3.7.2 Resource Requirements .................................................................................. 16 3.7.3 Project Cycle and Timing of the Assessment .................................................. 17 3.7.4 Document Management .................................................................................. 17 3.7.5 Accredited Assessors ...................................................................................... 18 3.7.6 Complementarity with World Bank Policies and Procedures ........................... 18 4 Review of Other Experiences with the Protocol ................................................................... 20 4.1 Overview of Previous Assessments ......................................................................... 20 4.2 Feedback from the Developers .................................................................................. 20 4.2.1 Objectives ........................................................................................................ 21 4.2.2 Motivations for Adoption .................................................................................. 21 4.2.3 Resource Requirements .................................................................................. 21 4.2.4 Expectations and Outcomes ............................................................................ 22 4.2.5 Value Added and Uses of the Protocol ............................................................ 22 4.3 Feedback from the Accredited Assessors ............................................................... 23 4.3.1 Requirements of a Good Assessment ............................................................. 23 4.3.2 The Protocol in Developing Countries ............................................................. 24 4.3.3 Training and Capacity Building ........................................................................ 24 5 The Views of World Bank Staff on the Protocol ................................................................... 25 6 Conclusions: Opportunities and Challenges ........................................................................ 26 7 Recommendations ................................................................................................................... 30 7.1 On the Use of the Protocol by Public Sector Clients of the World Bank ............ 30 7.2 On the Use of the Protocol in the World Bank (IBRD/IDA) Project Cycle .............. 31 7.3 On the Governance of the Protocol .......................................................................... 33 8 References ................................................................................................................................ 34 Annex 1: Questionnaire for Accredited Assessors ..................................................................... 35 Annex 2: Questionnaire for Sustainability Partners.................................................................... 36 iv LIST OF TABLES, FIGURES, AND PHOTOS Tables Table 2.1: List of Protocol Topics ................................................................................................. 3 Table 2.2: Steps in a Protocol Assessment .................................................................................. 6 Table 2.3: Comparison of World Bank Safeguards and the Protocol ........................................... 9 Table 3.1: Main Characteristics of the Trung Son Hydropower Project ...................................... 10 Table 3.2: Topic Allocation for the Trung Son Project ................................................................. 12 Table 4.1: Main Protocol Assessments to Date .......................................................................... 20 Figures Figure 2.1: Evolution of the Protocol.............................................................................................. 2 Figure 2.2: Structure of the Hydropower Sustainability Assessment Council ................................ 4 Figure 2.3: Example of a Sustainability Profile for an Operations Stage Assessment .................. 8 Figure 3.1: Map of the Trung Son Hydropower Project ............................................................... 11 Figure 3.2: Timeline of Protocol Assessment for Trung Son Hydropower Project ....................... 13 Figure 3.3: Protocol Manuals According to Project Stage ........................................................... 17 Figure 6.1: Types of Applications According to Client Objectives................................................ 28 Photos Photo 2.1: Assessors with Construction Manager, Jirau Project ................................................... 7 Photo 2.2: Assessors with Supervision Consultants, Trung Son Project ...................................... 7 Photo 3.1: Protocol Training in Vietnamese and English ............................................................ 15 Photo 3.2: Planning During Pre-Assessment Visit, Trung Son.................................................... 15 Photo 4.1: Assessor with Sheep Farmers, Blanda Project ......................................................... 22 Photo 4.2: Generating Dialogue in the Jirau Project ................................................................... 22 v FOREWORD The World Bank’s value proposition for sustainable hydropower development is to help its clients: 1) strengthen their management of the hydropower sector; 2) do the right projects; and 3) do the projects right. Towards these objectives, the World Bank supports the Hydropower Sustainability Assessment Protocol, a position that is articulated in the Energy Strategy (Towards a Sustainable Energy Future for All—Directions for the World Bank’s Energy Sector, 2013). The Protocol was launched in 2011 by the multi-stakeholder Hydropower Sustainability Assessment Forum (HSAF) and is rapidly being recognized as an important tool for sustainable hydropower development among a wide range of actors involved in the sector. Beyond the first movers, however, the reach of the Protocol is relatively limited, and questions remain as to how it works, the nature of its costs and requirements, and how it relates to existing procedures and standards for large infrastructure development. This report aims to address these questions. It is based on demand from the World Bank Group’s Hydropower Community of Practice to analyze the complementarity and relevance of the Protocol vis-à-vis the World Bank’s policies and procedures, as well as to provide guidance on the Protocol’s application by World Bank clients. The Protocol as a complete tool is in its adolescence. Of its four stages, (Early Stage, Preparation, Implementation, and Operation), some have been used more widely than others. There is also more to be done before the geographical distribution of the tool’s use and its coverage in countries of different income levels are representative of the sector. Consequently, this report offers some practical guidance during this early rollout phase with the aim of increasing the body of knowledge and experiences available to inform the use of the Protocol going forward. The Protocol’s uptake in low- and middle-income countries, where much of the remaining hydropower potential exists, has been slow. This presents an opportunity for the World Bank to raise awareness about the Protocol, particularly through its sector-level engagement, to encourage sustainable hydropower development beyond its own portfolio of projects. This report seeks to provide timely recommendations on the use of the Protocol by World Bank client countries and with respect to the World Bank’s role in the further development of the Protocol. The Authors, June 2014 vi ABBREVIATIONS & ACRONYMS EDF Electricity of France (Électricité de France) ESBR Sustainable Energy of Brazil (Energia Sustentável do Brasil) ESMF Environmental and Social Management Framework ESW Economic and Sector Work EVN Vietnam Electricity GIZ German Society for International Cooperation (Deutsche Gesellschaft für Internationale Zusammenarbeit) GW Gigawatt HSAF Hydropower Sustainability Assessment Forum IBRD International Bank for Reconstruction and Development IFC International Finance Corporation IHA International Hydropower Association IUCN International Union for Conservation of Nature MW Megawatt NGO Non-governmental Organization PCN Project Concept Note PHAP Public Health Action Plan TNC The Nature Conservancy TSHPCo Trung Son Hydropower Company UN United Nations WCD World Commission on Dams WWF World Wide Fund for Nature vii EXECUTIVE SUMMARY The Hydropower Sustainability Assessment Protocol is a framework for assessing the sustainability of hydropower projects with a consistent, globally-applicable methodology. Its objective is to improve the quality and public acceptance of hydropower. It is the product of 30 months of review and engagement by the multi-stakeholder Hydropower Sustainability Assessment Forum (HSAF), which was constituted in 2007 of representatives from industry, civil society, donors, developing country governments, and commercial and development banks. The Protocol is early in its global rollout, with the first set of official assessments taking place primarily in advanced hydropower sectors. After finalization of the Protocol, the Hydropower Sustainability Assessment Council was created. It consists of seven sectoral chambers, which elect a chair to sit on the Governance Committee, and a “Management Entity” for day-to-day implementation of the Protocol. The central office of the International Hydropower Association (IHA) is currently serving as Management Entity. The Protocol itself is not a standard; it is an auditing tool used to measure the environmental, social, technical, financial, and economic aspects of a project’s sustainability performance. A deliberate decision was made by the HSAF to design the tool without a pass or fail result; instead, it sought to provide a status baseline and a benchmarking mechanism for continuous improvement. The policies and procedures of the World Bank, like those of the Protocol, seek to promote good practices in hydropower projects. They are also among the sustainability regimes informing the principles that underpin the Protocol. For this reason, there are many parallels between Protocol topics and the issues covered by the World Bank Environmental and Social Safeguards and Performance Standards. The Protocol, however, is fundamentally different in purpose; it measures the sustainability of a specific project at a specific moment in time. It can, thus, be viewed as complementary to but not substitutable for the World Bank’s policies and procedures, which also seek to reduce risk to the lender. Guidance on how the Protocol can be used by World Bank clients is based on the suitability of the tool in developing country contexts, the tool’s value proposition for clients, its cost and ease of use, and its potential impact on project performance. Main observations and conclusions: • The Protocol is a useful tool for guiding the development of sustainable hydropower in developing countries. It emerged from a multi-stakeholder forum and is based on more than 100 years of lessons learned. It has also benefitted from the input of hydropower practitioners, acknowledging the site-specificity of hydropower and its challenges. • It is suitable for the identification of areas of improvement in hydropower projects in a variety of localities and at various stages of project development. The experiences of developers having applied the Protocol to date indicate that assessments deliver value for money and that findings are conducive to management action. In this regard, World Bank clients could benefit from the process and findings of Protocol assessments conducted on a voluntary basis. • The Protocol could potentially be used in client capacity building, however the tool’s manuals are complex and insufficient. A training program must, therefore, be built around them, including other materials relevant to the topic of sustainable hydropower. viii • The Protocol has a range of other potential uses, including incremental improvement in project components and providing a transparent framework for stakeholder dialogue and conflict resolution. • Assessor experience and training are essential in maintaining the quality and consistency of assessments and capacity building. Given the Protocol documents’ complexity and the extreme site-specificity of hydropower development, the use of Accredited Assessors having participated in previous assessments is essential for quality assurance. Despite the rigid methodology of the Protocol, assessor judgment and discretion is very important for the scoring of topics, especially when dealing with overlapping issues and the double counting of gaps. Practical considerations and constraints for use by World Bank clients: • Because of the heavy reliance on evidence provided by the developer for the triangulation of results, client ownership is essential to a successful assessment, and an assessment should not be conducted unless there is full commitment from the developer. • The Protocol was formulated based on the experience and norms of hydropower development and operation primarily in developed countries. Thus, basic good practice as defined by the Protocol may not reflect normal practice in a given client country, especially in countries with weaker institutions or where national regulations differ greatly from international norms. Managing expectations is, therefore, key to successful implementation and follow-through on identified gaps. • For many clients, achieving basic good practice on all sustainability topics will be difficult in the short term; as such, it is important to emphasize the process of continuous improvement rather than present the Protocol as an absolute measure for comparison between countries. • Similarly, to encourage uptake of the Protocol in developing countries, where much of the untapped hydropower potential resides, it may be necessary to diversify the tool’s uses beyond full assessments to include capacity building and guidance during project preparation. Protocol assessments will be more useful for clients if coupled with advisory services or technical assistance to help address identified gaps. • One challenge in conducting full Protocol assessments in developing countries will be to mobilize human and financial resources. Although the resources needed for a Protocol assessment are small compared to the cost of developing a hydropower project, the experience of Protocol use to present has shown that a significant investment of time is needed for planning, logistics, and document management in the lead-up to an assessment. Such assessments, whether publicly disclosed or not, require training conducted in the language of the target country with respect to the tool’s purpose and application. Following an assessment, there will also be costs associated with addressing the gaps identified in the process. The Protocol as an instrument for the World Bank: • The Protocol is not a replacement for World Bank policies and procedures; they are potentially complementary. The Protocol offers a snapshot of a project at a particular moment in time. While this provides a good summary of project performance, it is no substitute for the extensive body of policies and procedures that were formulated with the specific aim of informing World Bank investment decisions. The Protocol does, however, complement the World Bank’s general Safeguards with content that is hydropower-specific. ix • Further work should be done before formulating specific guidance for the World Bank’s use of the Protocol in projects with private sector clients. In the case of public sector clients, many potential uses of the Protocol in the project cycle have been identified. The Protocol could serve as a comprehensive reference document during country strategy frameworks and/or project preparation, and possibly as a performance monitoring tool to complement project supervision. • The Protocol has the potential to enhance capacity-building programs and sector-level engagement supported by the World Bank by consolidating in one suite of tools the international good practices that have been developed over many years of investment in the hydropower sector. • When considering the use of the Protocol in a World Bank-supported project, it is important to consider the specific objectives, risks, and timing. The Protocol will be useful if it can reinforce project preparation and/or supervision; it is likely to have more value during early preparation and less value during the short, intensive period of project appraisal. • As the Protocol gains momentum and clients are increasingly aware of its utility, World Bank staff will need to be trained on what the Protocol is, how it works, and how it may be most valuable for World Bank clients in order to provide timely and good-quality advice. World Bank and the governance of the Protocol • As members in the chamber for financial institutions,1 the World Bank and the International Finance Corporation (IFC) have an opportunity to influence the further development of the Protocol in a way that favors its use by client countries. To have a significant impact on the global sector, the Protocol must increase its uptake by developing countries, ensure diverse applicability by developers with varying needs, and develop training modules on the broader principles of sustainable hydropower. • To ensure the continued success of the Protocol, the World Bank should explore how the tool’s financial sustainability can be strengthened in order to maintain high quality and continued updating to keep pace with the changing global landscape. 1The full name of the chamber is the “Development, Public or Commercial Banks, Financial Organisations, and Pri- vate Investors/Investment Funds Chamber.” x The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 1 INTRODUCTION As a mature technology, hydropower has by far the largest installed capacity of any renewable source of electricity generation. The sector has seen more than a century of commercial success and has also been the subject of intense criticism by civil society for the negative impacts of some projects on people and nature. In 1998, in the face of escalating pressure, the World Commission on Dams (WCD) was established by the World Bank and the International Union for Conservation of Nature (IUCN) to review the development effectiveness of large dams and establish a comprehensive set of guidelines for the design, implementation, and operation of dams and their decommissioning. Following the final report of the WCD, the hydropower industry took many steps to operationalize its key recommendations and to improve environmental and social management of hydropower globally. In an effort to improve its performance and provide a consistent approach for assessment, the industry partnered with civil society, policymakers, and financiers around the principles of sustainable hydropower. The output of this process is the multi-stakeholder Hydropower Sustainability Assessment Protocol (hereafter referred to as “the Protocol”). In an era defined by increased climate variability, and as demand for energy security drives the development of new hydropower resources, the issues surrounding environmental and social sustainability remain prominent. Thus, the Protocol, as the first hydropower-specific assessment tool, has emerged as a useful reference document but also as a forum in which to continually raise issues surrounding the performance of the hydropower sector. The Protocol was developed to measure and guide the performance of hydropower projects against globally applicable criteria for environmental, social, financial, and technical sustainability. Early in its rollout across the sector, it is a suite of four tools that correspond to various stages of the project cycle. Given that many of the remaining undeveloped hydropower resources are in low- and middle-income countries, the Protocol presents relevant guidance for the responsible preparation, implementation, and operation of these future projects. The main purpose of this report is to reflect on the applicability of the Protocol in developing country contexts, based on the available cases, and offer direction on how it can be used to improve the performance of hydropower projects in World Bank client countries. This report also seeks to clarify the complementarity of the Protocol with respect to World Bank policies and procedures and provide input as to the World Bank’s role in the governance of the Protocol. The lessons and recommendations presented in this report are based on a desk review of previous Protocol assessments; consultation with World Bank staff; structured interviews with Accredited Assessors and developers from previous assessments; as well as direct observations during the pilot assessment of the Trung Son Hydropower Project in Vietnam. While the primary audience of this report is the management and staff of the World Bank, it is intended to address many of the questions faced by other multilateral development banks and donors supporting hydropower development worldwide. 1 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 2 THE HYDROPOWER SUSTAINABILITY ASSESSMENT PROTOCOL 2.1 What Is the Protocol? The Hydropower Sustainability Assessment Protocol is a framework for assessing the sustainability of hydropower projects with a consistent, globally-applicable methodology. Its objective is to improve environmental, social, and technical aspects of hydropower development and gain more public acceptance of hydropower. The Protocol is a series of assessment manuals corresponding to four different stages in the hydropower project cycle: Early Stage; Preparation; Implementation (including construction); and Operation. It contains definitions of good practice and best practice for over 20 sustainability topics that combine environmental, social, technical, and economic/financial perspectives. Not all topics, however, are relevant to every stage of the project cycle (See Table 2.1). The Early Stage2 tool applies a methodology different from that of the other three tools in that it is not a scoring protocol. It is also the only one that, at the time of writing this report, had not yet been used. The Protocol is not a certification scheme for sustainable hydropower, nor is it a replacement for assessments of environmental and social impacts. Rather, the Protocol assessment is a snapshot of project performance with the on-site assessment itself taking place over a matter of weeks. It is undertaken by a team of Accredited Assessors, headed by a Lead Assessor, who visit the project site, review relevant documents, conduct interviews with stakeholders, and prepare a final assessment report. 2.2 How Was It Developed? The Protocol is the result of a decade of work, culminating in a 30-month-long process of review and engagement by the multi-stakeholder Hydropower Sustainability Assessment Forum (HSAF). The HSAF was initiated in 2007 following a meeting of the World Wide Fund for Nature (WWF), the Nature Conservancy (TNC), and the International Hydropower Association (IHA) regarding support for the IHA Sustainability Assessment Protocol of 2006. Figure 2.1. Evolution of the Protocol               ‡:RUOG ‡81(3 ‡,+$ ‡,QLWLDO ‡0XOWL ‡2IILFLDO ‡3URWRFRO ‡+\GURSRZHU &RPPLVVLRQ 'DPV  6XVWDLQDELOLW\ 6XVWDLQDELOLW\ VWDNHKROGHU 3URWRFRO 7UDLQLQJ 6XVWDLQDELOLW\ RQ'DPV 'HYHORSPHQW *XLGHOLQHV $VVHVVPHQW )RUXP /DXQFK $VVHVVPHQWV $VVHVVPHQW )LQDO5HSRUW 3URMHFW 3URWRFRO 5HILQHVWKH %HJLQ &RXQFLO 3URWRFRO &RQVWLWXWHG 2 The nine topics covered under an Early Stage assessment are: Demonstrated Need; Options Assessment; Policies & Plans; Political Risks; Institutional Capacity; Technical Issues & Risks; Social Issues & Risks; Environmental Issues & Risks; and Economic & Financial Issues & Risks. 2 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Table 2.1. List of Protocol Topics (IHA 2014) Sustainability Topics Preparation Implementation Operation Technical Siting and Design • - - Hydrological Resource • - • Reservoir Planning / Preparation and • • • Filling / Management Infrastructure Safety • • • Asset reliability and efficiency - - • Environmental Downstream Flow Regimes • • • Erosion and Sedimentation • • • Water Quality • • • Waste, Noise, and Air Quality - • - Biodiversity and Invasive Species • • • Social Project-Affected Communities and • • • Livelihoods Cultural Heritage • • • Indigenous Peoples • • • Resettlement • • • Public Health • • • Labor and Working Conditions • • • Economic and Economic Viability • - - Financial Financial Viability • • • Project Benefits • • • Procurement • • - Integrative Demonstrated Need and Strategic Fit • - - Communications and Consultation • • • Governance • • • Environmental and Social Impact • • • Assessment and Management / Environmental and Social Issues Management Integrated Project Management • • • The HSAF members included representatives of developed and developing country governments, the hydropower industry, social and environmental NGOs, and commercial and development banks. This group was tasked with recommending enhancements to the IHA’s assessment tool, which had been developed, in response to the findings of the World Commission on Dams, to provide a standardized means of assessing the sustainability of hydropower projects. It was agreed that the IHA Protocol should be developed in a more inclusive way, with a greater variety of stakeholders, and so the HSAF was established. The World Bank served as an observer to the HSAF during this period. In August 2009, a draft version of the Hydropower Sustainability Assessment Protocol was released, and after a consultation and trial period, the final Protocol was compiled and adopted by the IHA in November 2010. 3 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 2.3 How Is It Managed? After finalization of the Protocol, the Hydropower Sustainability Assessment Council (hereafter “the Council”) was created. It consists of seven sectoral chambers, which elect a chair to sit on the Governance Committee, and a secretariat or “Management Entity” for day-to-day implementation of the Protocol. The IHA’s central office currently assumes the role of Management Entity. The structure is loosely modeled on other sectoral sustainability initiatives, such as the Forest and Marine Stewardship Councils and the Roundtables on Sustainable Biofuels, Palm Oil, and Soy. The responsibilities of the Council chambers include representing their constituencies’ interests in the Council; participating in Council dialogue; and providing guidance to their Governance Committee representatives (Hydropower Sustainability Assessment Protocol 2011). As per Figure 2.2, the Chambers each represent one of the stakeholder groups present in the HSAF. To invite feedback from industry members and to advance the rollout of the Protocol globally, the IHA established the Sustainability Partners initiative, in which leading companies partner with IHA to receive training on the Protocol and conduct a number of assessments. At the time of this report, there were a total of 15 Sustainability Partners (IHA 2014), for which the majority of protocol assessments to date have been carried out. Going forward, the Council and Governance Committee are seeking to define an operating model for the Protocol that ensures its financial sustainability. Currently, the Protocol is supported with funding from the IHA, with some revenue generated on a percentage basis from official assessments. Figure 2.2. Structure of the Hydropower Sustainability Assessment Council (IHA 2014) 4 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Even though the Protocol was produced in the context of a multi-stakeholder process, it is perceived by some—including organizations that are opposed to hydropower—as being controlled too much by the hydropower industry. This may affect its uptake in the short term, but as the tool finds a broader constituency and demonstrates its value in more projects, this is expected to be less of a limitation. The Protocol is free to download, but the Governance Committee has created a quality control structure to ensure consistent and reliable results. The Terms and Conditions for Use of the Protocol differentiate between commercial and non-commercial uses. For all commercial uses, where it is the intention to pay a third party for training, consulting, or assessments, the Terms and Conditions require that only Accredited Assessors be used. The Terms and Conditions also differentiate between official and unofficial assessments. For an assessment to be recognized as official, it must use an Accredited Assessor, obtain the full support of the project sponsor, and comply with the other guidelines, as determined by the Council. For any claims to be made about the results of an assessment, it must have been an official assessment, and the results must be publicly disclosed (Hydropower Sustainability Assessment Protocol 2011). 2.4 What Is Entailed in a Protocol Assessment? The on-site assessment is the cornerstone of a Protocol assessment, lasting approximately one week, depending on the size, location, and complexity of the project. The entire process, however, can take several months. This is partly due to the preparation time required and partly due to scheduling difficulties. Currently, there is a small cohort of Accredited Assessors in high demand,3 but as the number of assessors grows, this is expected to reduce the large time gaps between different stages in the assessment process. Once the Project Owner has decided to carry out a Protocol assessment, the Project Owner engages with the Management Entity of the Protocol, which is currently the IHA. The IHA identifies a team of assessors and negotiates a tentative schedule and costs. The cost of conducting an assessment is typically borne by the developer. Prior to the actual assessment, the Lead Assessor and Lead Trainer visit the project site to consider the logistical requirements, the time needed for the on-site assessment, and the size of the team required. In preparing for the assessment, the project sponsor appoints a “single point of contact”, who is the main counterpart to the assessor team. During the pre-assessment visit, the Lead Assessor works with the single point of contact and the rest of the local support team to identify potential interviewees and relevant documentation. The single point of contact is also a key participant during training, which, in the Sustainability Partner model, typically lasts a full week. The on-site assessment, which may occur some months after the pre-assessment visit, is an intensive period of interviewing and site inspection. For a smaller project, the assessors may conduct up to 40 interviews and review more than 100 documents. For the largest projects to date, they have held over 100 interviews and reviewed more than 400 documents. At the end of the on-site assessment, the assessors determine initial scores for the project and make a request for additional evidence, if needed. After considering the additional evidence, the assessor team 3At the time of writing this report, there were six Accredited Assessors with a further three awaiting accreditation and a further number of trainee assessors in various stages of assessor training. 5 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Table 2.2. Steps in a Protocol Assessment Concept Project Owner Protocol Management Entity • Decision to carry out a Protocol Assessment • Submit proposal to Project Owner with tentative • Procure consultancy services from the Manage- schedule and costs ment Entity • Procure individual consultants to make up asses- sor team Pre-Assessment Visit Project Owner Assessor Team • Refine objectives for assessment • Carry out training on the Protocol • Identify primary point of contact for assessment • Translate materials into national language, if • Identify local support team for assessment necessary • Guide local support team in identification of docu- mentary evidence and interviewees • Plan an intermediate “readiness visit,” if necessary Preparation for On-Site Assessment Project Owner Assessor team • Arrange interviews • Begin review of available documentary evidence • Select and share relevant documentation with assessor team • Arrange logistics for on-site assessment, includ- ing local travel, accommodation, translation, etc. On-Site Assessment Interviews and site inspection Post-Assessment Project Owner Assessor Team • Review draft report and provide one round of • Prepare draft report comments • Upon receipt of comments, prepare final assess- • Prepare management plan to address gaps ment report identified • Publish final report on hydrosustainability.org (optional) prepares a draft report, which is circulated for one round of comments to the Project Owner, and then compiles a final assessment report, which may or may not be published at the discretion of the Project Owner. For all of the assessments to date but one, the Project Owner (developer/operator) has been the only client stakeholder. In instances where a financier such as the World Bank is involved, there are some additional steps. For example, during the assessment for the Trung Son Hydropower Project in Vietnam (discussed further below), the World Bank arranged financing for the assessment, assisted 6 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Photo 2.1. Assessors with Construction Photo 2.2. Assessors with Supervision Manager, Jirau Project Consultants, Trung Son Project with the logistical arrangements, supplied documentary evidence, acted as an observer during the on-site assessment, and provided feedback on the draft report. 2.5 How Are Projects Scored? A project is typically scored in 20 sustainability topics; there may be more or fewer topics depending on the stage of the project and whether certain topics are determined as not relevant. For example, if there are no persons resettled over the course of a project, the assessors would consider the topic on Resettlement not relevant, and the project would not be assessed on that topic. For each topic, two sets of scoring criteria are used: basic good practice (score of 3) and proven best practice (score of 5). A project must attain basic good practice on a particular topic before it can be scored against proven best practice for that topic. In order to achieve a score of 3 on a topic, a project must satisfy all of the scoring statements with no “significant gaps.” If one significant gap against basic good practice is noted, the project receives a score of 2, and if there are two or more significant gaps, the project receives a score of 1. If a project has met all of the criteria to achieve the status of basic good practice for a particular topic, it is then assessed at the level of proven best practice. If there are no significant gaps against these criteria, the project will receive a score of 5. A single significant gap will earn it a score of 4. If two more or more significant gaps have been noted against proven best practice, the score for that topic will remain at 3. The “significance” of a gap is determined based on the severity of the risk that it poses to the project and whether or not plans are in place and there is sufficient time to address the gap before it can have a major impact on the overall project. The issue of significance is an area of focus in the training and support materials for Accredited Assessors. The final scores for the project are displayed in a “spider diagram” that shows the individual scores for each topic (Figure 2.3). Scores for topics cannot be combined or averaged to present a single sustainability index. This is important, as the topics vary greatly in their implications for a project. 7 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Figure 2.3. Example of a Sustainability Profile for an Operations Stage Assessment (IHA 2014) For example, a significant gap against basic good practice in the field of Communications & Consultation will not have the same impact as a significant gap on the topic of Infrastructure Safety. 2.6 How Does the Protocol Compare with Standards? The Protocol itself is not a standard; it is an auditing tool that is designed to measure all aspects of a hydropower project’s sustainability performance, providing a mechanism for continuous improvement. A deliberate decision was made by the HSAF to design the tool without a pass or fail result. While there is no pass or fail during an assessment, the scoring levels of 3 and 5 (basic good practice and proven best practice respectively) represent two levels of sustainability that are considered acceptable and desirable within a particular topic. Because no single scoring index exists and the topics are not weighted according to their importance, it is not currently possible to say that a project is “sustainable” or “unsustainable” across all the relevant topics. Furthermore, because the scoring is based on a snapshot at a particular moment in time, there are limitations in extrapolating from a Protocol assessment whether good practices employed in a project will be sustainable in the longer term. 8 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 2.6.1 The Protocol and World Bank Policies and Procedures The World Bank’s Safeguards are among the sustainability regimes that informed the principles underpinning the Protocol. For this reason, there are many parallels between the Protocol topics and the issues covered by the Safeguards and Performance Standards. The Protocol, however, is fundamentally different in purpose. These conclusions have been borne out in a study commissioned by the Hydropower Sustainability Assessment Forum to compare the approach and content of the Protocol with those of comparator systems, most notably those of the World Bank and International Finance Corporation (Wenban-Smith 2010). The Protocol can, thus, be viewed as complementary to but not substitutable for the World Bank’s policies and procedures. Table 2.3. Comparison of World Bank Safeguards and the Protocol Hydropower Sustainability Assessment World Bank Safeguards Protocol Characterization Operational Policies: Statements Suite of four assessment tools. of policy objectives and operational principles. Bank Procedures: Mandatory procedures to be followed by the World Bank and its borrowing clients. Purpose Seek improved project design and Help hydropower developers understand how ensure that World Bank-financed they match up to good industry practices and projects avoid, mitigate, or compensate identify areas for improvement. for any negative impacts on people and their environment. Applicability Required for World Bank-financed Any hydropower project at any stage of projects. development or operation. Sectoral coverage Safeguards apply to all investment The protocol is specifically designed for projects and are not sector- or industry- application to hydropower projects. specific. Compliance Safeguards involve specific standards The protocol is used voluntarily by project or procedures that have to be met by developers and operators, and while it is implementing agencies. Compliance is supposed to be applied by trained and monitored by the World Bank. certified teams, no specific compliance is required. 9 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 3 EXPERIENCE WITH THE PROTOCOL: THE TRUNG SON HYDROPOWER PROJECT, VIETNAM 3.1 Rationale To inform the guidance presented in this report, it was judged necessary to carry out a pilot assessment to gain first-hand knowledge about the process, challenges, and benefits of conducting an official Protocol assessment of a World Bank-supported project. The Trung Son Hydropower Project was identified as an ideal candidate due to its size and design as a storage project, but also in light of a request from the project sponsor for capacity building to bring its hydropower projects up to international standards. 3.2 About the Trung Son Project The Trung Son Hydropower Project is a multi-purpose endeavor that aims to supply low-cost electric power to Vietnam while also delivering benefits from flood control and irrigation. When completed, Trung Son will have an installed capacity of 260 MW and will generate an average of 1,019 GW of energy per year, which will help to meet the growing demand for energy in an urbanizing and industrializing Vietnam. The project is located on the Ma River in the Quan Hoa district of Thanh Hoa Province. The reservoir will have an estimated volume of 384.5 million cubic meters and will cover area of 13.13 km2 in both the Thanh Hoa and Son La provinces. The project is owned by Trung Son Hydropower Company Limited (TSHPCo), which is a wholly owned subsidiary company under the state-owned Vietnam Electricity (EVN). The total cost of the project is expected to be $411 million, of which $330 million is being financed by an IBRD investment loan. The project consists of four components: (i) Dam and Ancillary Construction; (ii) Transmission Line; (iii) Social and Environment Impact Mitigation; and (iv) Capacity Development and Scale-up. The project also includes a Resettlement and Livelihoods Development Program and a Public Health Action Plan (PHAP) that is planned to cover a ten-year period (more details on the components are available on the website http://www.trungsonhp.vn/en). Table 3.1. Main Characteristics of the Trung Son Hydropower Project Installed Capacity 260 MW (4 x 65 MW Francis Reservoir Volume 384.5 million m3 turbines) Average Generation 1,019 GWh/year Flood Control Volume 112 million m3 Avoided Emissions 1 million tonnes/year CO2 Transmission Line 220-kV (65 km long) equivalent Dam Height 84.5 m Access Road 20.4 km Reservoir Area 13.13 km2 Total Impacted Persons 10,600 10 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Figure 3.1. Map of the Trung Son Hydropower Project (World Bank 2011) 11 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 3.3 Process and Timelines The entire Protocol assessment, including the preparation prior to the on-site assessment and the finalization of the report following the assessment, took place over a period of one year. The assessment required coordination among multiple actors: EVN, TSHPCo, the Accredited Assessors, and the World Bank.4 In September 2013, a pre-assessment mission was carried out at the project site with the purpose of setting objectives, performing logistical planning for the on-site assessment, and conducting training for EVN and TSHPCo on the use of the Protocol. A subset of the team of Accredited Assessors, together with the World Bank, organized a one-day workshop in Hanoi at EVN headquarters to introduce the Protocol to EVN staff. During this visit, the assessor team also prepared an interim evidence register, including desired documentation and interviews with key actors in the project. The Trung Son project was assessed against all topics contained in the Implementation stage tool, with each Accredited Assessor being responsible for a group of topics. By the end of the assessment, the Accredited Assessors5 reviewed 200 pieces of documentary evidence, held 72 interviews, and took more than 100 photos. Table 3.2. Topic Allocation for the Trung Son Project I-1 Communications and Consultation I-11 Indigenous Peoples I-2 Governance I-12 Labor and Working Conditions I-3 Environmental and Social Issues Management I-13 Cultural Heritage I-4 Integrated Project Management I-14 Public Health I-5 Infrastructure Safety I-15 Biodiversity and Invasive Species I-6 Financial Viability I-16 Erosion and Sedimentation I-7 Project Benefits I-17 Water Quality I-8 Procurement I-18 Waste, Noise, and Air Quality I-9 Project-Affected Communities and Livelihoods I-19 Reservoir Preparation and Filling I-10 Resettlement I-20 Downstream Flow Regimes 4The World Bank team for the pilot assessment consisted of: Rikard Liden (Senior Hydropower Specialist, Water Practice); Kimberly Lyon (Operations Analyst, Water Practice); Franz Gerner (Task Team Leader, Trung Son Hy- dropower Project); Tran Hong Ky (Co-Task Team Leader, Trung Son Hydropower Project); and Thi Ba Chu (Energy Specialist, Vietnam Country Office). 5The Lead Assessor for the Trung Son assessment was Dr. Bernt Rydgren, Senior Consultant, ÅF Industry, Hydro- power. The co-assessors were: Dr. Jörg Hartmann, Independent Consultant; Aida Khalil, Sustainability Specialist, IHA; and Doug Smith, Senior Sustainability Specialist, IHA. The assessor team also had one trainee assessor: Elisa Xiao, Manager for Social and Resettlement Services, Environmental Resources Management China, Ltd. 12 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Figure 3.2. Timeline of Protocol Assessment for Trung Son Hydropower Project   0D\-XO\ $XJXVW 6HSWHPEHU 1RYHPEHU -DQXDU\ )HEUXDU\ $SULO 0D\ -XQH  &RQFHSW ,+$ 3UH 5HDGLQHVV 2Q6LWH 'UDIW &RPPHQWV )LQDO $FWLRQ3ODQ 'HYHORSHG 3URFXUHG $VVHVVPHQW 9LVLW $VVHVVPHQW 5HSRUW 3URYLGHGE\ 5HSRUW 'HYHORSHG  ZLWK(91 9LVLWDQG 'HOLYHUHG :%DQG 'HOLYHUHG E\76+3&R 7UDLQLQJ (91 In January 2014, the official assessment was carried out with a team of four Accredited Assessors and one trainee assessor over a period of eight days. At the end of the on-site assessment, the assessor team summarized its findings and presented initial scores for each topic. A draft report was then prepared and shared, and comments were provided separately by EVN, TSHPCo, and the World Bank. 3.4 Objective Setting Based on the experience gained through previous Protocol assessments, the assessors placed a lot of emphasis on the importance of objective setting during the pre-appraisal visit. During this exercise, a project sponsor would also typically identify priority aspects of the project on which to focus, helping to determine management’s response to the assessment outcomes. For EVN and TSHPCo, there was general interest in testing the Protocol in Vietnam and seeing a snapshot of project performance to compare how the results of the external assessment reflected their own understanding of the implementation of the Trung Son project. The objectives of the assessment, as agreed between EVN and TSHPCo, were to: (i) assess the sustainability performance of the Trung Son Project and identify opportunities to improve performance during the construction phase; and (ii) benchmark the project’s performance and the practices of EVN against international practice in hydropower, as defined by the Protocol. As this was the first time an official Protocol assessment had ever been conducted with the direct involvement of a project financier—i.e. the World Bank—great care was taken during the early phases of the process to emphasize the objectives of EVN and TSHPCo. While the World Bank had its own internal objective to understand how the Protocol could support its engagement with clients on sustainable hydropower, the World Bank team made clear to EVN and TSHPCo that the Protocol was not being applied to the project as a due diligence or official supervision tool according to Bank standards. 3.5 Outcomes of the Assessment The assessment report produced for an official assessment is structured according to the template specified by the Governance Committee of the Protocol. It first presents a “sustainability profile” of the project with scores for each topic plotted on a radar chart, as well as a table of the “significant gaps” identified. The report then provides a detailed evaluation of each topic and annex containing 13 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients supporting evidence. Examples of publicly disclosed assessment reports can be found on the Protocol website at http://www.hydrosustainability.org/Protocol-Assessments.aspx. Although the assessment for the Trung Son project was focused specifically on the project’s sustainability performance, the corporate-level performance of EVN and TSHPCo was relevant under several topics. It was concluded that the “project performance for the Trung Son project has generally been of a high standard, with only a few exceptions.” It was also affirmed that the project was helping to elevate sustainability standards of hydropower in Vietnam. Despite its solid design, the assessment identified “a number of interrelated problems with the practical management and monitoring of various issues,” most of which could still be “successfully rectified if proper and timely management attention [was] paid to them.” For many of the gaps that were identified, the source was the “late procurement of consultants and experts to assist TSHPCo’s own staff with the implementation of studies and monitoring programmes.” For example, delays in procuring the independent social and environmental consultant resulted in the late start of monitoring, which affected the topics Environmental and Social Issues Management, Erosion and Sedimentation, and Water Quality. Another source of gaps was the existence of parallel processes and documents, i.e. those originating in the national context and those developed in accordance with the World Bank’s policies and procedures. For example, the report stated that “the quality of the environmental and social action plans is generally high . . .” but that the “managing for and monitoring against two separate sets of plans and requirements (Vietnamese government and World Bank) is confusing.” The report concludes that the supervision is less effective than it could have been had the two sets of plans been fully integrated. The topics for which the project had the best performance were: Integrated Project Management; Infrastructure Safety; Procurement; Public Health; Biodiversity and Invasive Species; Reservoir Preparation and Filling; and Downstream Flow Regimes. 3.6 Client Feedback EVN, TSHPCo, and the World Bank expected that the Trung Son project would score highly. In the words of the Assessors, the project represented “a solid design, which could significantly elevate Vietnamese standards for hydropower development.” A major incentive for carrying out an assessment on the Trung Son project was, therefore, to show that it would perform well against internationally accepted criteria. Upon completion of the assessment, however, the developer’s initial reaction to the scores was that the demands of the Protocol were much higher than anticipated. While EVN and TSHPCo found the assessment useful in identifying areas of improvement, they did not think that the results fully captured the dynamic nature of project implementation. The report was strongly focused on facts observed during the assessment, regardless of whether the gaps were previously recognized by the developer. While this is the expectation of a third-party assessment, it was discouraging to the developer that efforts to address known gaps, especially when complicated procedures were required, had not been reflected in the assessment report. This was regarded as an area in which the assessment report could be improved. 14 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Other aspects of the Protocol assessment that presented challenges for the Trung Son teams were the criteria for Best Proven Practice on some topics that required actions outside the project’s area of influence. For example, on Cultural Heritage, the scoring statement includes a clause stating that “contributions to addressing cultural heritage issues beyond those impacts caused by the project are achieved or are on track to be achieved.” The team considered that such impacts beyond the project’s boundary had no limits, potentially representing an extraordinary expense for a developer. At the end of the assessment, there was a high degree of overlap between the gaps identified through the Protocol assessment and those identified during the regular course of supervision by the World Bank project team. It was concluded, however, that the template for the Protocol assessment report could be improved. While gaps are easily extracted from the report, it proved difficult to translate the gaps identified into specific actions that project management should take to correct areas of weakness. In addition, the report’s executive summary heavily utilized the terminology of the Protocol methodology, which Accredited Assessors are trained to recognize but which would be difficult for persons less familiar with the Protocol’s language to understand. In general, the report would benefit from much greater elaboration when it comes to the process of scoring, especially related to overlapping issues and double counting of gaps, to bolster the assessment report as a standalone document without need for accompanying guidance materials. 3.7 Lessons Learned From the experience of using the Protocol in the Trung Son project, many lessons can be drawn about the practicalities of using the tool. Even though this assessment tested the Implementation Stage tool only, most lessons can be generalized for all the Protocol tools. 3.7.1 Training and Client Buy-in Training on the Protocol itself was very important for the Trung Son assessment, given that the tool had never before been used by either a Vietnamese developer or by a World Bank task team. The barrier was especially high since none of the Protocol documents was available in Vietnamese; only those staff with strong English language skills would have had the opportunity to review them in Photo 3.1. Protocol Training in Vietnamese Photo 3.2. Planning During Pre-Assessment and English Visit, Trung Son 15 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients advance. After the one-day workshop in Hanoi, which was conducted in both languages, the level of understanding of the Protocol by EVN had increased significantly, but some gaps remained. Because of the unique relationship between the World Bank and EVN (as the mother company) and TSHPCo (as a subsidiary), it was not clear in the early stages of the assessment what the most appropriate distribution of roles might be. As a result, the training workshop was attended primarily by EVN staff with little representation by TSHPCo staff, who were responsible for the project’s day- to-day implementation. TSHPCo was fully involved for the remainder of the pre-assessment visit, but since most of this time was spent covering much of the material presented to EVN, less time was available to ensure that all stakeholders had a full appreciation for how the Protocol would work in practice. These training workshops also typically include a self-assessment by the project sponsor. For Trung Son, due to time constraints, the overall training as well as the self-assessment exercise were shortened. Based on previous assessments with Sustainability Partners, it is likely that the overall experience with the Protocol would have been easier had all the stakeholders from EVN and TSHPCo been able to self-assess on some topics, with the assistance of the Lead Trainer, and better understand ahead of the assessment how the scoring worked and the way in which conclusions would be drawn from the evidence available. This exercise would have helped inform the expectations of EVN, TSHPCo, and the World Bank as to the project’s likely performance during the assessment. 3.7.2 Resource Requirements While the on-site assessment took place in just eight days, the entire process—from concept development to receipt of the final assessment report—took several months. It was estimated that the primary points of contact from EVN and TSHPCo committed approximately 20 percent of their time, and the World Bank project staff in the Vietnam Country Office committed about 10 percent of their time to the assessment in the months leading up to and following the assessment. Given that the official assessor team is external to the project, they rely strongly on the project sponsor to identify stakeholders, arrange interviews, curate documents, and facilitate scheduling and logistics ahead of the on-site assessment. As was the case with Trung Son, the time and resources that a project sponsor (and the World Bank) will expend are much greater when translation and interpretation are necessary. The internal costs of carrying out a successful Protocol assessment were particularly onerous as the assessment took place around critical milestones in the construction schedule of the Trung Son Hydropower Project. The original date for the assessment had to be moved from November 2013 to January 2014 to allow EVN and TSHPCo staff to focus on the works related to diversion of the river and the subsequent river closing ceremony, which was held on December 1, 2013. The direct cost of conducting an official Protocol assessment was approximately US$130,000, not including the costs of translation, interpretation, travel, accommodations, and time for project staff. 16 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 3.7.3 Project Cycle and Timing of the Assessment The Preparation, Implementation, and Operation tools in the Protocol are designed to be used towards the end of the respective stages and prior to key decision points in the project cycle. Given that Trung Son was very early in its construction schedule at the time of the assessment, there were initial discussions about which stage tool would be the most appropriate for the assessment: Preparation or Implementation. Ultimately, it was decided to conduct the assessment using the Implementation tool in order to provide the most useful and actionable information to EVN and TSHPCo about the sustainability performance of Trung Son. Conducting an assessment early in a specific stage of the project forces the assessors to make judgments on how the current performance could affect the project’s outcomes. Because Trung Son was so early in its implementation, some aspects, most notably related to the reservoir, were not yet relevant, and certain gaps that were identified were not deemed to be significant as there was ample time to address them before they posed any substantial risk to the project. For example, the assessment identified some delays in the salvage of physical cultural resources, but since there was sufficient time to address the issue and the relevant project parties were aware of the need to do so, this non-conformance was not considered to be significant. In some cases, it may be advantageous to identify gaps while there is still a lot of time to take corrective action. On the other hand, it may be frustrating for the developer when known gaps, such as delays, are regarded as significant if, had the assessment been carried out at a slightly later date, they might already have been rectified. This is an important consideration in defining the objectives of a Protocol assessment and choosing when to conduct it. 3.7.4 Document Management The on-site assessment and subsequent requests for additional evidence occurred in a very condensed timeframe. As such, document management on the part of the EVN/TSHPCo and the Figure 3.3. Protocol Manuals According to Project Stage Early stage Preparation Implementation Operation 17 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients World Bank was perhaps the most critical factor, after commitment and buy-in, for the success of the Protocol assessment. In the end, the assessors reviewed more than 200 pieces of documentary evidence to support its conclusions. Without the provision of necessary documents in a timely manner, it is likely that a project would receive poor scores from an assessment and/or have topics not scored due to insufficient evidence. 3.7.5 Accredited Assessors The Trung Son assessment highlighted the importance of assessor experience and judgment. There were many instances in which topics overlapped, and the same pieces of evidence were relevant to multiple criteria. For example, the scope of work for the Independent Environmental Monitoring Consultant for Trung Son spanned many issues and several Protocol topics: Cultural Heritage; Biodiversity and Invasive Species; Environmental and Social Issues Management; Water Quality; Waste, Noise, and Air Quality; and Reservoir Preparation and Filling. Thus, the management and performance of this consultancy would influence the scoring on many topics. Even though the assessors were each responsible for different topics, they had to manage these overlaps to avoid duplicating gaps. It was also observed that, despite following the Protocol methodology, the assessors had different interviewing techniques, which influenced the flow of conversations with stakeholders. Overall, the assessors were received as experienced professionals, but it was clear that, in general, producing representative results with the Protocol depends on the training of assessors and their sectoral background. 3.7.6 Complementarity with World Bank Policies and Procedures Despite the added complexity of having the World Bank involved as an additional stakeholder in the assessment, the assessors stated that the availability of documentation was better in this case than is typical of non-Bank-financed projects in developing countries. Moreover, thanks to the World Bank’s involvement, much more documentation was available in English and did not need to be translated for the assessment. The assessors also viewed the involvement of the World Bank or other multilateral development banks as favorable, given that financing for capacity building using the Protocol could increase the uptake of the tool in developing countries, where very few assessments have so far been conducted. Given that the Protocol is not a standard but is meant to serve as a measurement tool, the assessors did not see any conflict between it and World Bank policies and procedures. Rather, the assessors saw value in the Protocol as a supplement to Bank supervision. Because of the procedures followed during preparation of Bank investments, the assessors estimated that if a preparation-stage assessment had been carried out for Trung Son, the project would likely have scored very highly. From the perspective of the World Bank task team, there is strong overlap between the Protocol and the policies and procedures of the World Bank. There are, however, some areas where the requirements of both regimes differ in their approach. 18 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Most notably, differences in how “project benefits” are defined were a source of disagreement. The I-7 topic, as defined by the Protocol, “addresses the additional benefits that can arise from a hydropower project, and the sharing of benefits beyond one-time compensation payments or resettlement support for project-affected communities” (IHA 2014). Specifically, there was disagreement surrounding what constitutes an “additional benefit,” “benefit sharing” in the Vietnamese context, and use of the broad term “project benefits” to refer to the narrowly defined concepts. “Additional benefits” are defined by the Protocol as “benefits that can be leveraged from the project; examples include: capacity building, training, and local employment; infrastructure such as bridges, access roads, and boat ramps; improved services, such as health and education; support for other water usages, such as irrigation, navigation, flood/drought control, aquaculture, and leisure; increased water availability for industrial and municipal water supply; benefits through integrated water resource management, etc.” (IHA 2014). While the World Bank team was confident that the project would deliver additional benefits as defined above, the assessor team had difficulty distinguishing between commitments that were intended to deliver additional benefits and those that were measures to mitigate social impacts. On benefit-sharing, the legislation in Vietnam that will enable and regulate profit-sharing from projects was not yet effective. From the perspective of the TSHPCo, this should be a consideration in the assessment of the Project Benefits topic. There was also feedback from the World Bank team to the effect that the topic name “Project Benefits” was misleading, as it only considered benefits for project-affected people. Related to this, the absence of a topic on economic viability in the Implementation stage tool is a major departure from the World Bank’s approach, as the generation of benefits to the local community, region, and wider society are of great importance beyond the Preparation stage. In the current iteration of the Protocol, a project such as Trung Son would be seen to get no “credit” for wider benefits such as electricity generation during project implementation and operation, or for the construction of a 25 km access road that brought communities together in a remote mountainous area, fostering economic activities and trade. 19 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 4 REVIEW OF OTHER EXPERIENCES WITH THE PROTOCOL 4.1 Overview of Previous Assessments The developers and operators that have used the Protocol so far range from large, diversified global and regional businesses such as EDF and E.ON, with installed capacity of 134 GW and 68 GW, respectively, to hydropower-dominated companies largely operating in one jurisdiction, such as Manitoba Hydro (Canada) or ISAGEN (Colombia). There are also single-project companies such as Shardara (Kazakhstan) and ESBR (Brazil). In total, five Implementation-stage, five Operation- stage, and two Preparation-stage assessments have been conducted. There have been seven assessments from high-income economies, four from upper-middle-income economies, and one from a lower-middle income economy. No low-income country has yet produced an assessment. The full text of the published assessments can be found online at http://www.hydrosustainability.org/ Protocol-Assessments.aspx. 4.2 Feedback from the Developers Based on interviews with several adopters of the Protocol as well as on their feedback to the IHA, some common lessons can be drawn about the practical purpose of the Protocol, the effort required Table 4.1. Main Protocol Assessments to Date (IHA, 2014) Owner Country Project Capacity Stage Date Published Shardara JSC Kazakhstan Shardara 100 MW Operation 2010 Hydro Tasmania Australia Trevallyn 97 MW Operation 2011 2012 E.ON Germany Walchensee 124 MW Operation 2012 2013 Sarawak Energy Malaysia Murum 944 MW Implementation 2012 Landsvirkjun Iceland Hvammur 84 MW Preparation 2012 2013 Statkraft Norway Jostedal 290 MW Operation 2012 2013 ESBR Brazil Jirau 3,750 MW Implementation 2012 2013 Manitoba Hydro Canada Keeyask 695 MW Preparation 2012 2013 EDF France Gavet 92 MW Implementation 2013 2013 Landsvirkjun Iceland Blanda 150 MW Operation 2013 2013 Isagen Colombia Sogamoso 820 MW Implementation 2013 EVN Vietnam Trung Son 260 MW Implementation 2014 20 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients to carry out an assessment, the quality of the assessment experience, and its overall usefulness. The lessons are consistent with those from the Trung Son case study described above. 4.2.1 Objectives For the majority of developers that have carried out a Protocol assessment, the primary purpose was to have the project assessed by third-party professionals in order to identify opportunities for improvement. Other common objectives were to benchmark the performance of their projects against internationally applicable criteria and to facilitate learning within their organizations on sustainability. Other objectives by particular developers included giving visibility to the positive aspects of the project, and improving communications and relationships with stakeholders with the Protocol assessment as a transparent process. For example, the assessment for the Keeyask Project by Manitoba Hydro was carried out during an intensive period of public review, and several stakeholders, such as local non-governmental organizations (NGOs) and Aboriginal communities, were aware of the tool and expressed interest in its use. In the case of the Jirau project, ESBR envisioned that the assessment would strengthen the project’s case for participation in carbon markets. ISAGEN used the Protocol assessment to frame discussions with the local environmental licensing agency and to determine whether Colombia’s legal framework guides a project to meet the requirements of basic good practice. For all of these objectives, the Protocol was chosen because it is the first hydropower-specific measurement tool, and it has an emphasis on continuous improvement. 4.2.2 Motivations for Adoption For almost all of the assessments to date, the developers, as members of the IHA, were closely following the development of the Protocol for several years prior to its official launch. These companies became Sustainability Partners with the IHA, which entailed capacity building around the Protocol and a number of Protocol Assessments. This was part of the early rollout of the Protocol in order to test the tool’s practicality and produce assessments that would be available to the public. Even though these companies were engaged with the formulation of the Protocol, they acted prudently, electing to carry out their first assessments on projects that they expected to perform well. Some companies are global in nature and have investments in many countries, but all of the early assessments were conducted in their respective home markets. In addition to the Sustainability Partners, the other two early adopters came on board with encouragement and financing from international development partners—i.e., GIZ and the World Bank. 4.2.3 Resource Requirements Developers have consistently reported that assessments required substantial resource commitments and the (internal) time and financial contributions turned out to be higher than expected. Costs included direct costs for the assessment team; travel and translation; opportunity costs for developer staff (and third-party interviewees); and, depending on the intended use and outcomes of the assessment, follow-up processes, such as investments to close identified gaps, and changed designs of future projects. For example, E.ON estimated that the assessment cost approximately €50,000 for external services and travel, and two man-months for preparation, execution, and post- 21 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Photo 4.1. Assessor with Sheep Farmers, Photo 4.2. Generating Dialogue in the Blanda Project Jirau Project evaluation, and noted that costs of future assessments would have to be managed carefully. Hydro Tasmania also estimated two man-months. Sarawak Energy estimated internal costs of US$50,000, and mentioned reduced costs as an incentive for future assessments. ESBR estimated direct costs for assessors and travel at about US$150,000 and staff costs and overheads at about US$320,000. Companies estimated that resource commitments for subsequent assessments should be 20-50 percent lower thanks to the initial investments in capacity building and familiarity with the tool. Hydro Tasmania agreed that reductions by 30 percent would be possible in the future, noting however that there would be costs associated with having new staff on subsequent assessments. Manitoba Hydro estimated the cost of the first assessment at nearly Can$1 million. While the cost of an assessment is low compared to the cost of developing a hydropower project and conducting the studies on which it is based, these costs are not insignificant for a developer that plans to do repeated assessments or use the Protocol throughout its entire portfolio of projects. 4.2.4 Expectations and Outcomes The developers have expressed that the Protocol assessments largely fulfilled their stated objectives, and that the results were generally as expected and fair. There were, however, instances in which the developers disagreed, sometimes strongly, with a particular topic score. For example, Hydro Tasmania did not agree with the score awarded on the topic of Biodiversity for the Trevallyn Project, and for the Keeyask Project, Manitoba Hydro was not satisfied with the outcome on Infrastructure Safety, which ultimately led to the creation of an expert panel and a partial re-assessment of that topic. Disagreements on topic scoring can stem from differences between what the developer views as important or good practice and what is described in the Protocol. Disagreements can also arise from misalignment between the Protocol criteria and what is commonplace or, possibly, legally required in a specific locality. 4.2.5 Value Added and Uses of the Protocol In general, the developers placed value on having a third-party assessor for transparency and objectivity. They also assigned value to the multi-stakeholder aspect of the Protocol, which provided 22 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients an entry point for constructive, science-based discussions with stakeholders. The assessments were found to be educational and valuable for engagement with non-affected and more distant stakeholders, but perhaps less so for local stakeholders and project-affected people due to the Protocol’s language and complexity. With the exception of one project, where it was felt that the performance was good enough, all of the other published assessments have led to management action to close the gaps identified. The types of actions vary with the performance of the project and the importance given to the topics by the developers. For example, one project simply extended a monitoring program while another developed a comprehensive action plan to tackle the areas of poor performance and to fully embed the Protocol in its project model. In one instance, the results led the developer to take action on gaps in a project different from the one assessed. In addition to closing gaps, the Protocol results were useful in many other ways. The high-scoring companies used the report to build their reputation, give visibility to the project or company, and educate company staff about the projects’ technical, social, and environmental aspects. For one of the lower-scoring projects, the assessment catalyzed the public disclosure of documents and resulted in better recordkeeping and information management in the company that would benefit future projects. The Protocol, in general, is also being used for internal review and screening for early stage investments and acquisitions of existing facilities. Some developers plan to use it for internal benchmarking within the company and drive competition among project teams towards operational excellence. Despite the heavy resource requirements, it was felt that the assessments delivered value for money, and virtually all of the project owners had the intention of carrying out assessments on other projects. At the time of writing this report, there were multiple assessments underway by developers that had previously carried one out. Developers would be further incentivized to continue using the Protocol if the total costs of conducting assessments were lower, if assessment results would be accepted in lieu of other audits and compliance requirements for banks, and if the developers could better align their own corporate processes with the Protocol to increase resource efficiency. 4.3 Feedback from the Accredited Assessors The following points represent a consolidated perspective from assessors and are based on structured interviews with four of the six assessors that were accredited at the time of writing. 4.3.1 Requirements of a Good Assessment Critical to the success of a Protocol assessment is client buy-in. Because the assessors rely on efforts by the project owner to furnish documentary evidence and arrange interviews, any lack of interest on the part of the owners limits the ability of the assessors to triangulate their conclusions. This could potentially result in some topics being “not scored” for dearth of evidence. Also essential is good document management and scheduling discipline by the project owner. On-site assessments take place over a short, intense timeframe, and thus, if key interviews do not take place, this could have a detrimental impact on the scoring of the project. A good understanding of sustainability concepts is also important insofar as these are borne out in the design, implementation, and 23 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients operation of the project. It also makes the assessment process less cumbersome if the project staff is versed in the relevant terminology. 4.3.2 The Protocol in Developing Countries Based on the few assessments in middle-income countries to date, the assessors noted some differences in the capacity of clients on key issues, such as document management. In general, document management and scheduling certainty has been better in higher-income countries and more advanced hydropower sectors. The assessors were not able to share the specific results of assessments that have not been published, but they indicated that projects in countries with more mature hydropower sectors have tended to score high and projects in countries with less experience and capacity have scored lower. The process of carrying out assessments is the same regardless of a country’s income level, but the prevailing political system and culture could affect the relative ease of the undertaking. A certain degree of openness on key issues, such as indigenous rights and the compensation of affected peoples, is necessary. The planning and implementation of an assessment is also more complicated with the involvement of a financier or international development partner. 4.3.3 Training and Capacity Building The assessors generally saw the Protocol as a good guidance and reference document, organized in a logical way, which comprehensively addresses the various dimensions of sustainability. It is designed in a way that is conducive to repeat assessments, which can measure improvements in an individual project over time. It is also the only hydropower-specific tool that distinguishes among assessment, management, and outcomes. Thus, it was felt that the Protocol documents would present a good overview of the sustainability issues associated with hydropower, but as the methodology is complex, it would not be suitable as the only training material if technical depth is the desired outcome. 24 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 5 THE VIEWS OF WORLD BANK STAFF ON THE PROTOCOL As an important basis for recommendations, several Bank staff members were interviewed, including senior operational energy, water, and safeguards specialists with knowledge of and experience with the Protocol. Their experience with the Protocol is mainly through their participation in its development and testing. Similar to the interviews with the Accredited Assessors, the views on the benefits and usefulness of the Protocol vary with different backgrounds and sector specialties. Below is a summary of the common perspectives on the Protocol among Bank operational staff working in hydropower. • The Protocol is mainly a tool for developers to ensure they do hydropower well. It can be used both as a reference document to check that all aspects are considered and as a tool to convince local stakeholders of the necessary steps involved in developing hydropower at the level of international good practice. • The Protocol must be considered both an assessment tool and a set of criteria, against which the projects are scored. The main strength of the Protocol is that it is a comparison against international good practice as defined by a multi-stakeholder group, including industry and civil society. • The use of Protocol for World Bank clients must be seen against the background of the limited capacity and understanding of what is required to develop complex projects in developing countries. The Protocol is a source of knowledge and experience that goes beyond an auditing tool. Its value proposition is a structured list of crucial topics—all compatible with crucial topics under World Bank Safeguards—that need to be considered to raise the performance of utilities and enable the development of sustainable hydropower in a complex reality. • The application of the Protocol by Bank clients in the early stage or preparation stage of hydropower projects would be an additional piece of information to determine whether projects are being prepared well. It cannot, however, replace the appraisal procedures used by the World Bank. Even though the principles supporting the Protocol are very similar to those behind the Safeguards, the use of these in parallel could create the risk of conflicting interpretation and comparison, except in cases where the Protocol framework informed the preparation of the project from an early stage. • Clients should use the Protocol on a voluntary basis, without any indication from the World Bank of minimum scores that must be achieved. Any recommendations by World Bank staff to clients of World Bank-supported projects about using the Protocol should avoid placing an additional burden on the client during times when project preparation or implementation is already intense. It is essential that the use of the Protocol is conducted with the understanding of and full commitment from the developer, as well as with joint understanding among all involved on the purpose of its use. • It is essential that the World Bank, through its role on the Council, is involved in the future development and improvement of the Protocol to ensure that quality is maintained in its evolution and that the World Bank can, with confidence, recommend its use to clients. 25 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 6 CONCLUSIONS: OPPORTUNITIES AND CHALLENGES These conclusions are based on the lessons from previous assessments, as expressed by hydropower developers, assessors, and World Bank staff. The initial findings further benefited from a thorough peer review and feedback process, which contributed to the final conclusions. They do not, however, necessarily represent a common perspective from all involved stakeholders. The Protocol is a good tool and offers balanced guidance for the development of responsible and sustainable hydropower in developing countries. The Protocol has benefitted from more than 100 years of lessons learned in the global development and operation of hydropower and is the product of a multi-stakeholder forum. At the same time, the Protocol has benefitted from the input of experienced hydropower practitioners. It encompasses all aspects of sustainability: technical, economic/financial, social, and environmental. It is, thus, a balanced instrument that acknowledges the high degree of site-specificity in hydropower and does not go to the extreme in its requirements for any specific topic. The feedback from hydropower developers that have applied the Protocol, including the Trung Son Hydropower Company, show unanimously that the results of the assessment helped to identify areas for improvement. Client ownership is essential to a successful assessment, and a Protocol assessment should not be conducted unless there is full commitment from the developer. The experience to date shows that the assessment process and scoring are based heavily on documentary evidence, some of which is not public. This must be supplied by the developer. In addition, the assessment relies on a large number of interviews with a broad range of stakeholders, including contractors, supervision consultants, project-affected people, government agencies, and NGOs. Lack of buy-in from the client would, therefore, hinder a proper assessment, and this would likely result in lower scores or incomplete assessments. The scores from a Protocol assessment are not normalized for country-specific contexts. The Protocol is mainly based on the experience and norms for hydropower development and operation in developed countries, which benefit from strong institutional capacity and legal frameworks. Basic good practice as defined by the Protocol may, therefore, not reflect normal practice in a given client country, especially in countries with weak institutional environments. There may be expectations of high scores for projects that are perceived to be very good in comparison to others in the same country. If a project is regarded as exceeding national standards, clients will be disappointed and possibly reject the Protocol as a reasonable tool if the expected results are not achieved. This may be one of the reasons that only assessment reports from developed and emerging economies have been made public so far. Managing expectations on project scoring is, thus, fundamental to the perception of the Protocol as a useful tool in developing countries. For an assessment to deliver “value for money,” other benefits beyond the bolstering of a company’s reputation with high scores must be emphasized when setting objectives. Building a constituency for the Protocol in developing countries may require emphasizing a focus on continuous improvement and avoiding it as an absolute measure of performance, comparable across countries. There is a risk that overemphasis on the Protocol assessment as 26 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients an audit with scoring will discourage its use in many of the countries that are less developed. Given the experiences with the tool so far, it is reasonable to expect that achieving basic good practice as defined by the Protocol on all topics will be challenging in most World Bank client countries in the short term. The scoring mechanism is less sensitive at the lower end of the spectrum; for a particular topic, a project could have two gaps or five gaps against basic good practice and in both scenarios receive a score of 1. Thus, if clients perceived the primary purpose of the Protocol as finding the weakest links in the chain, this would guide developers or countries to focus on the areas needed to raise their standards to meet international good practice of sustainable hydropower over the long term. A challenge in using official Protocol assessments in low-income countries will be to mobilize the human and financial resources. Although the resources needed for a Protocol assessment are small compared to the cost of developing a hydropower project, the experience of its use so far has shown that it is laborious for the developer. A significant investment of time is needed for planning, logistics, and document management ahead of an assessment. Furthermore, Protocol assessments, whether official or unofficial, require training in the purpose and application of the tool in the language of the target country. Careful planning, realistic timeframes and budgets, coordination with other development activities, and capacity building are essential for creating the buy-in needed in order for the experience to be well-received and not seen as an added burden during project development. The first full Protocol assessment is more resource-intensive for a developer. In addition to the time and resources expended by the developer during the preparation and implementation of a Protocol assessment, an official assessment requires time and travel by a full team of international assessors, who often need to visit remote areas and interview many external stakeholders during the on-site assessment. The intention would be to ensure a very thorough report that can be publicly disclosed and can stand up to external criticism. These costs, however, are small compared to the overall investment in a hydropower project, and official assessments, because their results are backed by the Governance Committee of the Protocol, can be valuable to a developer that wants to make public claims based on the outcome of an assessment. The choice of conducting official assessments should, therefore, consider the purpose and expected uses of the findings. Once a developer has gained experience from carrying out a full Protocol assessment and has built up some internal knowledge of the Protocol, it may be possible to do unofficial assessments with a mixture of Accredited Assessors and in-house resources, which could reduce the costs. These unofficial assessments may fulfill the purpose of identifying gaps and internal benchmarking, but the reputational benefits would be limited without the ability to make any public claims on the results of the assessment. Especially in low-income countries, where much of the unharnessed hydropower potential is, more diverse ways of using the Protocol need to be encouraged. It is likely that, because of the costs of an assessment and the existing burdens of monitoring and reporting for donors and financiers, uptake of the protocol in developing countries will be low if the Protocol is only offered in the form of discrete assessments providing a snapshot of project performance. The uses of the Protocol should, instead, be diversified to include step-wise processes, whereby intermediate goals can be reached and highlighted to show progress that the developer can be proud of. These approaches could include less detailed and partial assessments to guide targeted actions and improvements in management. As previously indicated, a full official assessment, made publicly 27 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients Figure 6.1. Types of Applications according to Client Objectives Assessment Type Conditions Client Objective Published Reputation building and reports increased visibility Management oversight Official Assessments and quality assurance PROTOCOL Over time with Signal of lower risk market to investors development Internal processes strengthened Capacity building Official/Unofficial/ with developing Self-Assessments country clients Incremental improvement of project components available, may be more beneficial as an end goal for the use of the Protocol in developing countries, rather than the first entry to its use. The Protocol documents themselves are complex and insufficient for capacity building; a program must be built around them for training on sustainable hydropower. Although the Protocol documents are extensive in addressing all aspects of sustainable hydropower development, they are designed, first and foremost, to support the structured process of an assessment. They are not specifically designed as pedagogic material for learning, especially when it comes to non- experts. Furthermore, the Protocol assessment report, which follows a strict template for describing and justifying gaps, is not easily translatable into actions for addressing these gaps. Identifying the areas that must be prioritized for a management response is a straightforward process, but the precise action that must be taken thereafter necessitates deeper analysis and interpretation. After the Trung Son assessment, the developers specifically commented on the lack of concrete recommendations coming out of the exercise. Especially for low-income countries with low institutional capacity, it is essential to develop further training and capacity-building modules linked to the Protocol, which can assist the developers in taking the findings from theory to practice. Assessor experience and training are essential for maintaining the quality assessments and capacity building. Because of the complexity of the Protocol documents and the extreme site- specificity of hydropower, the leadership of Accredited Assessors is essential for maintaining high quality. A full understanding of the structure and principles of the Protocol as well as experience from previous assessments are prerequisites for becoming an Accredited Assessor and are fundamental to conducting a fair and accurate assessment. The Trung Son assessment showed very clearly how important the assessors’ judgment and discretion can be for the scoring of topics, especially when dealing with issues related to topic overlap and double counting of gaps. Also the pre-assessment training experience in Trung Son showed the importance of having trainers with 28 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients a wealth of understanding and experience of actual assessments to utilize the full potential of the Protocol for institutional and individual capacity building. The Protocol needs to be further refined before it can be applied at scale. While the Protocol is acknowledged as a good tool, there are aspects that need to be improved. The Protocol is very sensitive at the top of the scoring scale but lacks sensitivity at the lower end. The presence of one gap will determine whether a project receives a score of 4 instead of 5; meanwhile, a project could have two gaps or four gaps against basic good practice on a particular topic and it would score a 1 in both scenarios. There are also criteria and statements in the Protocol that require further definition, particularly the scoring statements at level 5. In addition, the Protocol deliberately assesses the performance of the project based on criteria that may be out of the hands of the developer to control. While this may be appropriate on some issues, certain issues simply cannot be addressed or rectified at the individual project level. 29 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 7 RECOMMENDATIONS Based on the above conclusions and the operational practice of the World Bank, the following recommendations are offered for use of the Protocol. The recommendations are the joint results of the discussions among the reference group of World Bank staff created for this study. 7.1 On the Use of the Protocol by Public Sector Clients of the World Bank The Protocol is not a suitable replacement for World Bank policies and procedures; it may be complementary. The Protocol offers a snapshot of a project at a particular moment in time. While this provides a good summary of project performance, it cannot replace the extensive body of policies and procedures that were formulated specifically to inform World Bank investment decisions. The Protocol does, however, complement the World Bank’s general Safeguards with content that is hydropower-specific. The World Bank should, however, explore the possibilities and encourage the use of the Protocol by its clients in other forms. The application of the Protocol followed by a management plan to address identified gaps are likely the most powerful existing tools to improve the sustainability performance of hydropower schemes. Its explicit strength is that it is exclusively focused on hydropower and has been developed by stakeholders with first-hand experience in hydropower development. Given its potential use to assess performance, build institutional capacity, and foster stakeholder dialogue, the World Bank should contribute to building awareness and understanding of the benefits of the Protocol in developing countries. The World Bank should, where there is interest, provide the means to help developers and countries incorporate the Protocol into their processes of preparing, developing, and operating hydropower projects. The World Bank should also encourage its clients to use Accredited Assessors in order to ensure high-quality advisory services. The World Bank should support the Protocol as a tool for its clients to improve the process of developing sustainable hydropower in their countries. The Protocol has its strength, in that it forces the developer or operator to provide evidence of good procedures and internationally recognized practices, which lead to sustainable hydropower projects. Providing a consolidated framework for this with an emphasis on continuous improvement will be the main value proposition for the Protocol with World Bank clients. Using the Protocol as a catalyst for improved procedures and documentation is more appropriate than using it for a pass/fail evaluation. The World Bank’s support should be based on voluntary use of the Protocol and full ownership by its clients. The contribution of the Protocol to sustainable hydropower is only effective if it results in changed management and measures to deal with identified gaps and in preventing new gaps from emerging. These measures may require long-term commitment from the developer. Any obligatory use of Protocol assessments, in addition to the World Bank policies and procedures, should therefore be avoided, as lack of ownership could generate results and benefits that are only temporary and/or limited. If a country client, however, sought to require the use of 30 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients the Protocol in its projects by private developers, the World Bank should support the client in the formulation of such conditions. The World Bank supports the use of both official and unofficial assessments without any requirement for public disclosure. The World Bank acknowledges that the end goal should be towards official and public Protocol assessments but recognizes that the up-front requirement of such may deter adoption of the tool and developer action based on its findings. The World Bank believes that, with its assistance, increasing the number of developing countries interested in using the Protocol, even in an unofficial capacity, would build a strong constituency and long-term pipeline of official assessments. The World Bank promotes transparency but will respect its clients’ rights not to publish Protocol results if this will create an easier path for addressing gaps and improving hydropower practice and development. The World Bank should provide regular training in the Protocol to its hydropower practice staff to make them knowledgeable about the tool and ensure balanced recommendations on its use to clients. Prior to the effective use of the Protocol by clients, World Bank hydropower practitioners must first become aware of and knowledgeable about the Protocol in order to provide correct advice. It is not envisaged that there is a need for Bank staff to become Accredited Assessors, but specialists in relevant fields may find it useful to participate in the assessor training. 7.2 On the Use of the Protocol in the World Bank (IBRD/IDA) Project Cycle There is no blueprint for how and when the World Bank would promote the use of the Protocol to its clients. The applicability and expected benefits of its use are country-specific and project-specific. It is recommended that World Bank staff follow the structured reasoning below for evaluating and justifying how and when the Protocol could provide added value. Unofficial or Official Assessments Protocol In-House with Accredited User Documents Assessments Assessors International Developer / Operator Criteria: Developing Country Developer / 1. Value proposition Operator 2. Cost and ease of use 3. Applicability to local contexts Development Country Government 4. Impact on project/company performance (Planning Agency, Regulator) 5. Commitment from Client The following guidance on potential applications of the Protocol is based on the instruments that the World Bank has available and on stages of the project cycle: Knowledge and Advisory Services: The World Bank will likely have the broadest use of the Protocol in its knowledge and advisory services. A long-term capacity-building program, for example, with a governmental utility could include training in sustainable hydropower supported by the Protocol documents, an unofficial assessment of planned and existing hydropower schemes led by an Accredited Assessor with in-house resources, plans developed for addressing the gaps, and official assessments of key schemes as an end goal. Similarly, the World Bank could support 31 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients a capacity development program for a governmental planning department or regulator, potentially informing improvements in national policy, based on the Protocol documents. The use of the Protocol documents and assessment experiences can also be a possible means for facilitating dialogue among stakeholders on national and trans-boundary river basin management. It is envisaged that the use of the Protocol for knowledge and advisory services will mostly be conducted as Recipient- or Bank-executed support by external consultants (including Accredited Assessors). Country Assistance Frameworks: During the formulation of the Country Assistance Frameworks, the Country Management Unit may find the Early Stage manual of the Protocol useful as a screening or risk assessment tool when considering how the World Bank will support that country’s hydropower sector. Investment Projects (Greenfield or Rehabilitation): Prior to PCN (Preparation or Operation Stage Tools). The Protocol will be useful as a reference document during the preparation of Terms of Reference for studies and consultants for project preparation. The list of topics provided by the Protocol for the Preparation stage is a checklist for which studies, or components of studies, are missing and need to be addressed before commencing the appraisal process. An evaluation of strengths and weaknesses during this period, based on the topics of the Protocol, can provide guidance for the World Bank on the preparation time and budget needed for the project. Such evaluation of strengths and weaknesses could later be enclosed as an annex to the PCN. It is envisaged that the use of the Protocol in the preparation of investment projects will mostly be conducted as short advisory services by Bank staff well acquainted with the Protocol, or by individual consultants. Preparation and Appraisal (Preparation or Operation Stage Tools). Given that this is a short and intensive period, where the focus is on compliance with policies and procedures, this is when the Protocol will be of least interest from the perspective of the World Bank. Nevertheless, the Protocol topics may be a consultative tool (checklist) for the World Bank team and the client in the time leading up to the appraisal of a hydropower project. It may also be useful for informing implementation plans; Environmental and Social Management Frameworks (ESMFs), and monitoring and evaluation plans. If the client, of its own accord, is interested in a Protocol assessment, the World Bank should be supportive of this while ensuring that the client is aware that the results of the assessment will only be informative during appraisal. It is envisaged that the use of the Protocol during the Appraisal stage of investment projects will mostly be in providing guidance for Bank teams. Supervision (Implementation Stage Tool). The Protocol has potential as a tool for monitoring project performance during implementation. An assessment by independent, Accredited Assessors, both halfway and near the end of the implementation schedule, can provide a broad snapshot, beyond the scope of normal project supervision. This would provide input to the Implementation Status Report and Implementation Completion Report, respectively. Alternatively, a project may have a built-in capacity-building component, which could involve regular in-house assessments by the developer in order to build capacity on sustainable hydropower. Introducing the Protocol into project implementation, however, must be accompanied by careful planning during the preparation phase, in liaison with the developer, to assure full ownership and that it does not represent an additional 32 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients requirement from the World Bank. It is envisaged that the use of the Protocol in the implementation stage of investment projects will mostly be conducted as recipient-executed unofficial or official assessments. 7.3 On the Governance of the Protocol The World Bank should continue its membership in the Financial Institutions Chamber and should strive towards the position of Chair or Alternate in order to have a seat on the Governance Committee of the Protocol.6 Through this participation, the World Bank will have the opportunity to work for the Protocol’s development in a way that favors its use in client countries and to exercise the World Bank’s support for the development of responsible hydropower. As a member in the Financial Institutions Chamber, World Bank institutions should specifically advocate that the Governance Committee and Management Entity: • Make significant efforts to garner the interest and support of developing countries for the Protocol; • Enable diversification of the way the Protocol can be used, making it more attractive and cost- effective for use in developing countries; • Explore adaptation of the Protocol for use in small-scale hydropower programs and multipurpose projects; and • Develop training modules, not just on the use of the Protocol, but in sustainable hydropower principles, helping to build a wider constituency for the Protocol. The World Bank should actively work toward finding financial support for the governance of the Protocol to ensure that the quality of Accredited Assessors and official assessments is upheld, and that sufficient resources are available for the continuing development and promotion of the Protocol. The World Bank should utilize its contacts with industry and the donor community to seek funds to support the Governance Committee and/or Management Entity, either through direct financial support or through the financing of targeted activities. It is expected that the Protocol would achieve wider acceptance as an objective tool if it was financially independent of the IHA. 6As of April 2014, the World Bank and IFC were both members of the Financial Institutions Chamber, with the World Bank holding the position of Alternate. 33 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients 8 REFERENCES Hydropower Sustainability Assessment Protocol. (2011, June). Hydropower Sustainability Assessment Council Charter. Retrieved April 8, 2014, from http://www.hydrosustainability.org/ getattachment/Governance/Charter/HydropowerSustainabilityAssessmentCouncilCharter_ FINAL.pdf.aspx IHA. (2014). Governance. Retrieved April 8, 2014, from Hydropower Sustainability Assessment Protocol: http://www.hydrosustainability.org/Governance.aspx IHA. (2014). Hydropower Sustainability Assessment Protocol. Retrieved April 8, 2014, from Hydropower Sustainability Assessment Protocol: http://www.hydrosustainability.org/ Protocol.aspx IHA. (2014, March 31). List of Sustainability Partners. Retrieved from Hydropower Sustainability Assessment Protocol: http://www.hydrosustainability.org/Programmes/Sustainability-Partners/ List-of-Sustainability-Partners.aspx IHA. (2014). Protocol Assessments. Retrieved April 9, 2014, from Hydropower Sustainability Assessment Protocol: http://www.hydrosustainability.org/Protocol-Assessments.aspx J. Potts, M. L. (2014). The State of Sustainability Initiatives Review. Manitoba: International Institute for Sustainable Development. Retrieved from http://www.sustainablecommodities.org/node/215 UN Water. (2014). Case study: Hydropower Sustainability Assessment Protocol. Retrieved March 31, 2014, from Water for Life Decade: https://www.un.org/waterforlifedecade/water_and_ energy_2014/hydropower_sustainability_assessment_protocol.shtml Wenban-Smith, M. (2010). Comparing the Hydropower Sustainability Assessment Protocol with Other Selected Approaches to Improving the Sustainability of Hydropower Projects. Retrieved May 15, 2014, from http://www.hydrosustainability.org/IHAHydro4Life/media/PDFs/ Protocol%20phase%201/MWS_HSAP-Comparative-Analysis_MainReport_2010-10-31.pdf World Bank. (2011, March 23). Trung Son Hydropower Project: Facts. Retrieved May 12, 2014, from World Bank: http://go.worldbank.org/DJ7FDO9AQ0 World Bank. (2011). Vietnam—Trung Son Hydropower Project. Project Appraisal Document. Washington, DC: World Bank. 34 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients ANNEX 1: QUESTIONNAIRE FOR ACCREDITED ASSESSORS 1. How have you, in your professional experience, used the Protocol, including official and unofficial assessments? 2. What do you see as the best use(s) of the protocol in developing country environments? Is the protocol a suitable tool for training on sustainable hydropower? 3. What differences have you noted in conducting protocol assessments in developed versus developing country environments? 4. What differences have you noted in conducting protocol assessments with developers that were Sustainability partners compared with those that were not? 5. In reference to the Trung Son assessment, what differences have there been compared to assessments with no financier (World Bank) involvement? 6. In assessing a World Bank-financed project, what is the interaction between the protocol and World Bank policies and procedures? 7. How do you think training on sustainability in hydropower (on the part of the client) would influence the outcome of an assessment? 8. What would be the impact of limited client buy-in on the quality and outcome of a protocol assessment? 9. In your opinion, do you think it is necessary to have a full team of Accredited Assessors, or do you think a mixed team of accredited and unaccredited Assessors could carry out a protocol assessment of similar quality? 35 The Hydropower Sustainability Assessment Protocol for Use by World Bank Clients ANNEX 2: QUESTIONNAIRE FOR SUSTAINABILITY PARTNERS 1. Basic information on stage, size, and location of the project. 2. How did your company/organization learn about the Protocol? 3. What were your objectives for the assessment? 4. Why was the Protocol chosen to fulfill these objectives? Did the assessment fulfill the aforementioned objectives? 5. Would you have been able to achieve these objectives with your company/organization’s internal resources or by hiring a consultant? 6. What was the extent of the time and resource commitments needed to make the assessment successful? Was the required input from your company more or less than expected? 7. Can you estimate the value, in terms of time and money, of the resources committed to the assessment from your company? 8. Would you expect the time and resource commitments to be reduced with your second or subsequent assessments? If so, by how much? 9. Were the findings of the assessment expected or not? Did you agree with the findings? In terms of the Protocol criteria, were the findings fair? 10. Were the scores of the assessment expected or not? Did you agree with the scoring? 11. Have you used the results of the assessment? If so, how? 12. What amount of resources did you or are you going to invest to address the gaps identified, and how does that compare to the assessment cost? 13. What resources have you applied to addressing the gaps identified (e.g., internal specialists, additional employees, external experts, investment costs, etc.)? 14. Have the gaps identified on this project assessment led to broader improvements: i) identifying improvements to corporate systems and practice, ii) in regulatory influence? 15. Did your external stakeholders perceive any value in the assessment, and if so how (e.g., financers, shareholders, regulators, equipment providers)? 16. Was the assessment useful as a tool for the process of stakeholder dialogue itself, including community-level and civil-society consultation and engagement? 17. (If applicable) How do you consider the publication of the assessment report? Was it useful and valuable, or not? 18. Do you perceive the assessment as having good value for the resources committed? 19. What would incentivize your continuing use of the Protocol? 20. Have you considered using the protocol in any way other than for official assessments? If so, how? 36