73488 2012 DEVELOPMENT POLICY LENDING RETROSPECTIVE: RESULTS, RISKS, AND REFORMS OPERATIONS POLICY AND COUNTRY SERVICES THE WORLD BANK October 31, 2012 ABBREVIATIONS AND ACRONYMS AAA Analytic and Advisory Activities ICRR Implementation Completion and AFR Africa Region Results Report BP Bank Procedure IDA International Development Cat DDO Catastrophe Deferred Drawdown Association Option IEG Independent Evaluation Group CCT Conditional Cast Transfer IFI International Financial Institutions CEM Country Economic Memorandum JBS Joint Budget Support CFAA Country Financial Accountability LCR Latin America and the Caribbean Assessment Region CIFA Comprehensive Integrated Fiduciary M&E Monitoring and Evaluation Assessment MICs Middle-income Countries CPAR Country Procurement Assessment MNA Middle East and North Africa Region Report MoU Memorandum of Understanding CPIA Country Policy and Institutional NGO Nongovernmental Organization Assessment NRM Natural Resource Management CRW Crisis Response Window OD Operational Directive DDO Deferred Drawdown Option OP Operational Policy DPL Development Policy Loan OPCS Operations Policy and Country DPO Development Policy Operation Services DSA Debt Sustainability Analysis PAF Performance Assessment Framework EAP East Asia and Pacific Region PBG Policy-Based Guarantees ECA Europe and Central Asia Region PD Program Document EFA-FTI Education for All – Fast Track PEFA Public Expenditure and Financial Initiative Accountability EGRG Economic Governance Reform Grant PER Public Expenditure Review EITI Extractive Industries Transparency PFM Public Financial Management Initiative PREM Poverty Reduction and Economic ESAL Emergency Structural Adjustment Management Loan PRSC Poverty Reduction Support Credit ESW Economic and Sector Work PRSG Poverty Reduction Support Grant FCS Fragile and Conflict-Affected PSIA Poverty and Social Impact Analysis Situations RDB Regional Development Bank FY Fiscal Year SAR South Asia Region GAC Governance and Anticorruption SDN Sustainable Development Network GFMRAP Government Financial Management SDPO Special Development Policy and Revenue Administration Project Operation GNI Gross National Income SEA Sectoral Environmental Assessment GPN Good Practice Note SIR Sector Implementation Review IAD Internal Audit Department SPF State-and-Peace-Building Fund IBRD International Bank for Reconstruction WDR World Development Report and Development 2012 DEVELOPMENT POLICY LENDING RETROSPECTIVE: RESULTS, RISKS, AND REFORMS CONTENTS Executive Summary .......................................................................................................................v  I.   Introduction ........................................................................................................................1  II.   Characteristics of DPOs ....................................................................................................4  A.   Overall Trends in Development Policy Lending ........................................................4  B.   Development Policy Lending in IBRD Countries ......................................................7  C.   Development Policy Lending in IDA Countries.........................................................9  III.   Results ...............................................................................................................................15  A.   Performance of DPOs ...............................................................................................15  B.   Country Results Influenced by DPOs .........................................................................20  C.   The Quality of Results Frameworks .........................................................................26  D.   Assessment of M&E Arrangements..........................................................................29  IV.   Risks and Opportunities ..................................................................................................31  A.   Assessing Macroeconomic Risks ..............................................................................31  B.  Weighing Governance Opportunities and Risks .......................................................33  C.  Weighing Fiduciary Opportunities and Risks ...........................................................35  D.   Assessing Poverty and Social Impacts ......................................................................41  E.   Assessing Environmental Impacts ............................................................................45  F.   Assessing and Addressing Implementation Capacity Risks .....................................46  G.  Managing Development Effectiveness Risks ...........................................................47  H.  Overall Risk Management ..........................................................................................49  V.   Reforms .............................................................................................................................50  A.   Reforms to the Policy Framework ............................................................................50  B.  Other Reforms with Implications for Development Policy Lending ........................59  C.   Reforms and Evolution of Practices .........................................................................60  VI.   Conclusions: From Retrospective to Prospective.........................................................66  References .....................................................................................................................................71  BOXES Box 1. 2009 Development Policy Lending Retrospective—Recommendations and Follow-up Actions ................................................................................................................................2  Box 2. IDA Crisis Response Window ..........................................................................................10  Box 3. The Long March from Inputs to Development Outcomes ................................................19  Box 4. Examples of DPOs that Fell Short of Achieving their Intended Results ..........................23  Box 5. Example of DPOs in FCS that Achieved the Intended Results.........................................24  Box 6. Examples of DPOs in FCS that Fell Short of Achieving the Intended Results ................25  Box 7. Examples of DPOs Across the Wide Fragility Spectrum .................................................26  ii Box 8. DPL Review Processes – Recognized by IAD as a Key Contributor to Quality and Effectiveness .....................................................................................................................37  Box 9. Taking Calculated Risks in Weak PFM Environments .....................................................38  Box 10. Approach of the Poverty and Social Impact Analysis.....................................................42  Box 11. Upstream Consideration of Poverty and Social Impacts .................................................45  Box 12. Addressing Implementation Capacity Risks through Technical Assistance ...................47  Box 13. PEFA-Informed DPO Design: West Bank & Gaza PRDP-2 .........................................48  Box 14. Indonesia: Support through Contingency Financing ......................................................53  Box 15. Events Triggering Drawdown under DPO with Cat DDO ..............................................55  Box 16. Support to Latvia during the Crisis: The Use of a SDPO ..............................................59  Box 17. High Transaction Costs in JBS Coordination Structures ................................................62  Box 18. Difficulties in Adjusting the Content of DPOs in the Context of JBS ............................63  FIGURES Figure 1. The Spectrum of Bank Lending Instruments...................................................................1  Figure 2. Regional Distribution of Commitments and Operations, FY09 Q4-FY12 Q3................4  Figure 3. Types of DPOs by Region, FY09 Q4-FY12 Q3..............................................................5  Figure 4. Total Commitments, FY95-12.........................................................................................6  Figure 5. Total Disbursements, FY95-12 .......................................................................................6  Figure 6. Average Number of Prior Actions in Policy Based Lending, FY95-12 ..........................7  Figure 7. IBRD Commitments and Operations, FY09 Q4-FY12 Q3 .............................................8  Figure 8. Share of Policy Based Lending in Total IBRD and IDA Lending, FY95-12..................8  Figure 9. IBRD Thematic Distribution of Prior Actions ................................................................9  Figure 10. IDA Commitments and Operations, FY09 Q4-FY12 Q3 ............................................11  Figure 11. IDA Thematic Distribution of Prior Actions ...............................................................11  Figure 12. Trends in Gender-Informed Lending Instruments, FY10-12 ......................................12  Figure 13. DPO Commitments and Operations to FCS, FY05-12 ...............................................14  Figure 14. Bank and IEG Evaluations of DPOs, FY06-10 ...........................................................15  Figure 15. IEG Evaluations of DPOs by Region, FY05-12..........................................................17  Figure 16. IEG Evaluations by Type of DPO and Region, FY05-12 ...........................................17  Figure 17. IEG Evaluations by Lending Instrument, FY00-10 ....................................................17  Figure 18. Achievement of Targeted Results by Indicator ...........................................................18  Figure 19. Achievement of Targeted Results by Theme ..............................................................19  Figure 20. Quality of Results Frameworks ...................................................................................26  Figure 21. Quality of Results Frameworks and DPO Size ...........................................................27  Figure 22. Classification of Prior Actions ....................................................................................28  Figure 23. Average Number of Results Indicators per Operation ................................................28  Figure 24. Average Number of Indicators by Region...................................................................28  Figure 25. Assessment of M&E Arrangements ............................................................................29  Figure 26. Mitigation Measures Set Out in the DPO ....................................................................29  Figure 27. Commitments by CPIAD Rating Categories ................................................................34  Figure 28. Quality of Governance and the Average Number of Prior Actions in Governance ....34  Figure 29. Commitments by CPIA13 Rating Categories ...............................................................36  Figure 30. PFM Quality and the Average Number of PFM Prior Actions ...................................37  Figure 31. Ratings of the Fiduciary Risks ....................................................................................40  iii Figure 32. Potential Poverty and Social Effects of the Prior Actions ...........................................42  Figure 33. Poverty and Social Impact Analysis (PSIA) Completed, when needed, FY07-12Q3.43  Figure 34. Poverty and Social Impact Analysis (PSIA) Completed, when needed, by Year and Region .........................................................................................................................44  Figure 35. Potential Effects of Prior Actions on the Environment, Forests, and Other Natural Resources ....................................................................................................................45  TABLES Table 1. PEFA Scores and Commitments by Instrument .............................................................36  Table 2. Summary of Changes to Financial Features of DDOs ....................................................51  Table 3. DPOs with a DDO, April 2008-September 2012 ...........................................................52  Table 4. Summary of Changes to Financial Features of Cat-DDOs .............................................54  Table 5. DPOs with Cat DDO, April 2008-September 2012 ........................................................55  Table 6. DPOs to Political Subdivisions, March 2009-September 2012 ......................................56  Table 7. Pre-2010 Recipients of DPOs that did not Publish their Budgets ..................................64  Table 8. Public Availability of Key Budget Documents ..............................................................65  ANNEXES Annex A. Number of Operations and Commitments....................................................................73  Annex B. Methodology for Evaluating the Quality of Results Frameworks ................................75  Annex C. Methodology Protocol for Poverty and Social Impact Analysis ..................................77  Annex D. Methodology Protocol for Environmental Impact Analysis ........................................79  Annex E. External Views on Development Policy Lending.........................................................81  Annex F. Consultation Feedback ..................................................................................................83  Annex G. Detailed Thematic Breakdown of Prior Actions ..........................................................85  Annex H. DPOs ─ Retrospective Universe ..................................................................................87  Acknowledgments This report was prepared by Manuela Francisco with contributions by Guillermo Arenas, Lawrence Bouton, Francisco Carneiro, Patricia Geli, Habiba Gitay, Gabriela Inchauste, Alma Kanani, Nora Kaoues, Jason Mayfield, Peter Moll, Ambar Narayan, Nicola Pontara, Christine Richaud, Pablo Saavedra, Seynabou Sakho, Hoon Soh, Michael Stevens, Sanjay Vani, and Adriana Weisman. Mustafa Ulukan provided excellent research assistance. The authors are grateful for comments from Margaret Dijkerman, Egbert Gerken, the OPCS Leadership Team and participants in the 2012 Spring Meetings. Hiwot Demeke and Pansy Chintha provided diligent support and outstanding assistance. We are also grateful to Pauline Chin-Mori for providing back-up assistance. The work was conducted under the overall guidance of Joachim von Amsberg, Vice President (OPCS) and Aloysius Ordu, Director (OPSPQ), and under the supervision of Edward Mountfield, Manager (OPSPQ). 2012 DEVELOPMENT POLICY LENDING RETROSPECTIVE RESULTS, RISKS, AND REFORMS EXECUTIVE SUMMARY 1. This report reviews the World Bank’s use of its development policy lending instrument over the past three years. This is the third Development Policy Lending Retrospective since the Operational Policy (OP 8.60) was introduced in August 2004. It covers all 221 Development Policy Operations (DPOs) approved by the Board in the last three years1—since the end of the last Retrospective and through the third quarter of FY12.2 The objective of this exercise is to take stock of, and distill lessons from, the Bank’s experience with the use of DPOs. In particular, it aims to assess (i) the extent to which DPOs have contributed to country results; (ii) the management of risks associated with DPOs and how opportunities were balanced against risks; and (iii) the effectiveness of the reforms that have been introduced to the governing operational policy as well as to practices. 2. Over the past three years, development policy lending has remained a key financing and policy dialogue instrument for both IDA and IBRD. In the context of the global crisis and the volatile recovery, the flexibility of the instrument proved to be valuable in supporting needed reforms and pursuing important development results for countries with diverse needs – from upper middle income countries like Croatia to fragile states emerging from conflict such as Liberia. Although the financing provided through DPOs is a very small portion of the total financing needs, the convening framework for policy dialogue and structural reforms offered by the instrument continue to be highly valued by clients, other international financial organizations, development partners, and other stakeholders working at the country level. A. Trends in Development Policy Lending 3. DPO commitment and disbursement volumes increased during the period under review. Total development policy lending during the three year Retrospective period reached approximately $45 billion, a nominal increase of 53 percent compared to the previous Retrospective. Commitments in the form of DPOs included over $37 billion of IBRD funds and nearly $7.4 billion of IDA funds. There were also $243.5 million in DPO commitments through trust funds. The largest recipient regions of development policy lending were Latin America and the Caribbean (LCR) (32 percent) followed by Europe and Central Asia (ECA) (29 percent). The Africa region (AFR) received 11 percent, East Asia and Pacific (EAP) received 15 and South Asia (SAR) and Middle East and North Africa (MNA) received 6 and 7 percent, respectively. During FY09 and FY10, DPOs reached nearly 40 percent of Bank commitments, partly as a response to the global financial crisis. In terms of disbursements, DPOs reached 51 percent in FY10. DPOs returned to around 27 percent of total Bank commitments in FY11 and rose again to 35 percent in FY12. 1 One of the operations approved by the Board, in September 2009 to Hungary for a total of $1.4 billion, was never signed and is therefore not included in the retrospective. 2 An additional 14 operations totaling $5.4 billion ($5.1 billion of IBRD, $254 million of IDA, and $23 million of trust funds) in commitments were approved in the fourth quarter of FY12, but are excluded in the main analysis unless otherwise noted. vi 4. While IBRD-financed DPO volumes increased during this period (72 percent in nominal terms), IDA-financed development policy lending fell slightly (3 percent). By number, roughly half of the 221 DPOs were supported by IDA and half by IBRD. The share of IDA development policy lending in total IDA, however, has fallen significantly. While it represented around 25 percent of total IDA commitments in FY05-09, it dropped to 12 percent by end-FY11, increasing slightly to 13 percent in FY12. This is largely explained by the sharp fall in IDA financed development policy lending in SAR: DPOs fell from 25 percent of total SAR IDA commitments in the last Retrospective period to close to zero in the current one. It also reflects a declining share of DPOs in IDA commitments in Africa (AFR), where it fell from 24 to 21 percent. All other regions registered some increase in the share of DPOs in IDA commitments. Commitments to Fragile and Conflict-Affected Situations (FCS) through DPOs have been low, at around 3 percent of total lending and 15 percent of total IDA lending (FY10-12).3 5. DPOs have continued to be selective in their use of conditionality, supporting government-owned programs, and focusing on critical prior actions. The average number of prior actions remained at 10 per operation.4 Whereas most reforms were in the area of public sector governance (40 percent), the share of prior actions in the areas of social protection and economic management rose during the last three years, highlighting the adaptability of the instrument to changing development policy priorities in the context of the global financial crisis. A number of reforms focused on the IDA16 themes of climate change and gender. Moreover, the number of gender-informed DPOs has grown steadily over the last three years. B. Results 6. For the most part, DPOs have been relatively successful in achieving targeted development outcomes, and quality appears to be holding up well, despite increased lending volumes. According to the Implementation Completion and Results (ICR) reports5 available to date, approximately 91 percent of the DPOs approved since FY05 were moderately satisfactory or higher (and 63 percent were satisfactory or higher).6 The Independent Evaluation Group (IEG) also has a relatively positive assessment of DPOs, rating 81 percent of such operations as moderately satisfactory or higher and 49 percent as satisfactory or higher.7 However, both ICR and IEG datasets are based on an incomplete universe of evaluations, especially for the latter years. As more data becomes available for the later years, it is possible that the percentage of operations rated as moderately satisfactory and above may change. 7. While overall performance has remained relatively good, there are variations by region and client segment. According to IEG evaluations, ECA had the largest fraction of DPOs rated as moderately satisfactory or higher (94 percent). If the bar is raised to satisfactory or higher, results show that SAR and ECA did the best (both with 63 percent).8 The regions with the lowest shares of DPOs rated satisfactory or higher are MNA (32 percent) and AFR (40 percent). When comparing 3 Commitments to FCS through DPOs reached 15 percent in 2008 due to three large arrears clearances operation (Cote d’ Ivoire, Liberia and Togo). 4 A result consistent with the last retrospective. 5 To evaluate the extent to which a DPO achieved its intended development outcomes, the Bank prepares an ICR report. 6 Based on 332 ICR reports available (evaluation of DPOs approved since FY05). 7 Based on 268 IEG reports available (evaluation of DPOs approved since FY05). 8 It should be noted that in the case of SAR this result mostly reflects performance of DPOs approved before this retrospective (there were only 9 DPOs to SAR during this retrospective). vii IBRD and IDA, it emerges that 84 percent of the IBRD financed DPOs were rated as moderately satisfactory or higher compared to 78 percent of IDA financed DPOs. Achieving intended results proved to be more challenging in IDA and FCS countries, where approximately 6 and 9 percent of the operations, respectively, were rated as unsatisfactory. 8. Analysis suggests that programmatic DPOs perform better than stand-alone DPOs. Approximately 86 percent of the programmatic DPOs were rated as moderately satisfactory or higher compared to 76 percent of stand-alone DPOs. If we consider satisfactory or higher, the difference is even more notable (57 percent vs 39 percent). Multi-tranche DPOs (now quite rare) also underperform relative to programmatic DPOs. 9. IEG and ICRs evaluations suggest that policy-based lending has performed at least as well as investment lending. Using data from as early as FY00, IEG evaluations have rated the performance of policy based lending generally above that of investment lending. Since FY09 the ratings of DPOs seem to have held up, despite substantially increased volumes – or even possibly improved (although the ratings are based on a less than complete universe of IEG evaluations). This positive outcome may reflect progress in the design of the results frameworks. Approximately 85 percent of the results indicators of the DPOs in this Retrospective included baselines and end-of- series targets, whereas only 42 percent of the DPOs in the previous Retrospective included baselines. The number of indicators has also declined significantly (from an average of 28 in the last Retrospective to 14 in the current one), suggesting greater prioritization and focus. 10. Despite progress in improving results frameworks, there is room for further strengthening. Efforts could be devoted to avoiding excessive numbers of prior actions of a process or preparatory nature, focusing only on critical policy and institutional actions that drive measurable development results. It would also be useful to enhance the linkages of attribution between policies and results. Nonetheless, there are still important questions about attribution, criticality, and causality that would benefit from further examination. To help analyze these issues, staff will take steps to establish a program of more in-depth impact evaluation for DPOs with the objective of helping answer questions regarding long-term impact. 11. Monitoring of progress of countries’ results could also be enhanced by more systematic support to countries’ Monitoring and Evaluation (M&E) systems. While implementation and M&E are the borrower’s responsibility, the Bank is responsible for assessing and monitoring the adequacy of the country’s M&E arrangements. While most DPOs discussed the country’s M&E framework, the quality of this discussion varied. Only a limited sample of DPOs (less than one- third) included an assessment of the strengths and weaknesses of the country’s M&E arrangements. In most cases, however, remedial measures were adopted when the country’s M&E systems were found to have weaknesses (either through policy and institutional reform, technical assistance, or a combination of both). In the future, greater attention needs to be paid to countries’ M&E systems and support should be considered if weaknesses are found. viii C. Risks and Opportunities 12. Another important issue is whether DPOs that disburse into the general budget are embedded in a robust risk framework.9 Operational policy does not specify a particular “minimum� threshold for governance and fiduciary standards; instead, the policy prescribes a risk- based approach, where risks are identified, mitigated to the extent possible, and weighed against opportunities. Results suggest that the Bank has effectively made use of the flexibility embedded in OP 8.60 and has responded weighing both risks and opportunities. 13. Analysis suggests that the Bank has heavily weighted governance and fiduciary risks in the decision to extend a DPO. From the analysis of the distribution of DPOs according to the quality of PFM and governance10 using FY05-11 disbursement data and the Bank’s 2010 Country Policy and Institutional Assessment (CPIA) ratings, it emerges that: (i) a larger share of DPO commitments went to better fiduciary and governance performers; and (ii) countries with stronger fiduciary and governance environments received a larger share of Bank financing in development policy lending (compared to investment lending). While the former partly reflects the emphasis on governance and fiduciary matters in both the IDA Performance Based Allocation (PBA) formula and the IBRD creditworthiness assessments, the latter indicates that for any given IDA or IBRD allocation, a country with stronger fiduciary and governance performance will also typically have a significantly larger share of DPOs in its portfolio. 14. Besides taking governance and fiduciary risks into account when deciding to extend a DPO, the Bank has also seized opportunities to address such challenges in the design of the DPO. When governance and fiduciary environments were poorer, the policy content of the DPOs, measured by the number of prior actions, had a stronger focus on these areas. The policy content in these areas can be seen as a mitigating measure for the fiduciary and governance risks the DPO may entail. 15. Macroeconomic issues and risks are also key considerations in development policy lending. All DPOs are embedded in a medium-term macroeconomic policy program, including an assessment of financing needs and risks. Operational policy stipulates that the Bank undertake development policy lending only when it assesses the country’s macroeconomic policy framework to be adequate. Since the 2009 Retrospective, progress has been made in ensuring that Program Documents systematically include a forward-looking discussion of the macroeconomic outlook and include the required bottom-line assessment of the adequacy of the macroeconomic policy framework. Yet some weaknesses (more commonly found in sector-specific DPOs) are observed in the quality of the macroeconomic discussion in Program Documents, and in particular in monetary policy and external sustainability issues. A more detailed account of the expenditure composition and revenue structure would also enhance the quality of these assessments. To guide and support teams with macroeconomic assessments, OPCS will partner with the PREM Anchor to prepare a Good Practice Note by the end of FY13. 9 In very unusual circumstances the Bank can make alternative disbursement arrangements such as paying a third party. 10 Governance is measured using CPIA Cluster D (Public Sector Management and Institutions) and Public Finance Management measured using CPIA 13 (Quality of Budgetary and Financial Management). ix 16. Strong analytical underpinnings are an opportunity for the Bank and its clients to strengthen the effectiveness of DPOs and a key tool for managing risks to development outcomes. While all DPO Program Documents include a discussion of the analytical underpinnings, many could be strengthened by being more precise about the linkages between the Bank’s Analytical and Advisory Activities (AAA) and the design of the program. While the Bank has continued to base its DPOs on a solid platform of core Economic and Sector Work (ESW), investment in core ESW must be continued and could be strengthened to maintain and improve the quality of DPOs. Ongoing work to strengthen the results and client orientation of the Bank's knowledge work and the overall approach has been outlined in the Bank’s first report on its knowledge work.11 This ongoing work will help to strengthen the analytical underpinnings of DPOs. 17. Among the critical analytical underpinnings to inform the design of a DPO is poverty and social impact analysis. There has been a gradual improvement in the extent to which the Bank makes use of poverty and social impact analysis of policies where poverty/social impacts may be likely. However, the frequency of poverty and social impact analysis, when potentially needed (as per OP 8.60) dipped in FY09 and FY10 due to the pressures of the crisis (with more socially sensitive prior actions in the context of fiscal consolidation programs, and increased pressure to respond swiftly). While the trend has been positive overall, efforts need to be made to conduct more upstream analysis of poverty/social opportunities and risks, to inform the design of DPOs and to be ready in the event of a crisis. To that end, it will be important to ensure that there is adequate Bank budget for poverty and social impact analysis, especially when multi-donor trust funds end. 18. Overall, there is room to improve the clarity, consistency and rigor with which risks are assessed in DPOs. While most operations do identify and detail the risks to the operation, there is some unevenness in the consistency and depth of discussion and the potential mitigation factors. DPOs do not systematically and consistently rate risks, so it is hard to acquire a sense of the relative levels of risks across countries. At the same time, DPOs could be more explicit about how they could support the identification and enhancement of development opportunities and the trade-off between opportunities and risks so that both can inform decisions. To that end, a more systematic approach to discuss risks in DPOs will be established. D. Reforms 19. Since the adoption of OP 8.60 in August 2004, a number of reforms have been introduced to the operational policy in response to changing country priorities and circumstances. These include a streamlined disbursement verification protocol for DPOs with a Deferred Drawdown Option (DPO DDOs); the introduction of a DPO with a Catastrophic Risk DDO (Cat DDO) to provide liquidity in emergency situations caused by natural disasters; changes to allow for the provision of DPOs to political subdivisions below the level of states and provinces; and changes in the Special DPO option to clarify how and when the Bank participates in international rescue packages for countries in or approaching a crisis. The Bank has confirmed that in general it extends DPOs only to borrowers that publish their budgets. There have also been Bankwide reforms, in particular the Access to Information Policy, with important implications for DPOs. 20. Thus far experience with DPO DDOs and Cat DDOs has been positive. The Board has approved eight DPOs with a Cat-DDO for total commitments of $1.26 billion, of which 65 percent 11 State of the World Bank Knowledge Services: Knowledge for Development, 2011. x has so far been withdrawn supporting countries in emergency situations caused by natural disasters (including earthquakes, tropical storms, eruption of volcanoes, and flooding). The streamlined disbursement verification protocol for DDOs also appears to have been successful in reducing the barriers to use of the DDO (15 DPO DDOs were approved since the change compared to two before).12 Experience with Special DPOs confirms that it is expected to be used in very exceptional circumstances (the Board approved only two Special DPOs13 during the recent crisis). 21. DPOs to political subdivisions are an option with significant untapped potential. The operational policy was modified in 2011 to clarify the requirements for DPOs to political subdivisions and to allow DPOs at the municipal level (such as the recent DPO to the City of Rio de Janeiro). While only twelve subnational DPOs were undertaken during the last three and half years,14 there is potential for significantly more. At the same time, the risks typically found at the national level can be accentuated when working at the subnational level so care must be taken in assessing intergovernmental fiscal relations, performing debt sustainability analysis for the political subdivision, and evaluating capacity and political constraints that could hamper the expected objectives of the engagement. Another untapped opportunity is given by regional/multi-country DPOs, which are feasible under OP 8.60 and could be an excellent vehicle to deal with multi-country policy challenges. 22. An important ongoing reform to development policy lending is the modernization of the Bank’s operational policy on guarantees.15 The use of Policy Based Guarantees (PBGs) could be encouraged in several client niches. For example, in small countries with good policy performance but limited access to capital markets, countries moving out of IDA and into IBRD, and medium-size lower-middle-income countries with limitations in accessing or re-accessing capital markets. For IDA-only countries, Management has discussed the possibility of extending PBGs to such countries provided that they have a low risk of debt distress and adequate management capacity. Expanded training and deployment of required specialized staff could help further facilitate the use of PBGs. On the operational policy side, teams could discuss with governments the possibility of transforming one or more operations in a programmatic series of DPOs into PBGs without interrupting the programmatic engagement. This is one of several improvements being considered to existing operational policy on PBGs, as PBGs are incorporated into OP/BP 8.60. The Approach Paper on Modernizing the Bank Operational Policy on Guarantees (2012) provides a more detailed discussion of potential reform options. 23. An emerging practice with important implications for Bank DPOs has been the preparation of DPOs in the context of Joint Budget Support (JBS), which has contributed to greater consistency across development partners’ policy reform goals. Previously, a multitude of partners and related aid instruments often led to fragmentation of the policy dialogue among development partners and high transaction costs to governments—a problem that was especially critical for governments with limited capacity to own and manage their development agenda. The joint Performance Assessment Frameworks (PAFs) have served as important coordination vehicles for countries’ development goals. 12 The total of 15 includes two DPO DDOs approved in Q4 of FY12. 13 First and Second Safety Net and Social Sector Programs to Latvia. 14 From March 2009 to September 2012. 15 The Approach Paper on Modernizing the Bank Operational Policy on Guarantees provides a more detailed discussion of potential reform options. xi 24. JBS has also increased the predictability of aid flows. The increased transparency in disbursement conditions, clearly spelled out in the PAFs, has sometimes made aid flows and financing more predictable. However, in very rigid and formal partnerships this can also lead to increased risks of a financing gap if reforms do not materialize and many development partners disburse against the same conditions. This has also been perceived as tipping the balance of power in favor of development partners. 25. While JBS is the right approach to effectively support countries with multiple budget support partners, questions regarding the impact and effectiveness of some coordination modalities have been raised. While there are many good examples of coordination, at the country and operation level, there are also examples of time-consuming and burdensome processes. Experience suggests that certain JBS arrangements that entailed highly rigid prescriptions: (i) lessened the flexibility to adjust to changing circumstances and country priorities, especially in times of crisis; (ii) imposed time costs of their own; (iii) weakened policy dialogue by reducing the level to the lowest common denominator and, most notably, (iv) weakened Government ownership. 26. Pursuing the fundamental principle of partnership, OPCS is finalizing guidance on how the Bank can engage in effective JBS arrangements. The objective of the guidance is to strengthen the Bank’s ability to coordinate its DPOs within effective JBS partnerships that are led by the Government and avoid overly rigid and process-driven approaches. OPCS will be coordinating this guidance with the Legal Department and the Regions. 27. Finally, the Bank has pursued higher standards of transparency in DPOs in recent years. In April 2011, the Bank clarified that it does not provide DPOs to countries that do not publish their budgets, save in exceptional circumstances and provided that there is a commitment to start publishing the annual budget within the next 12 months. Publication of an annual budget plan is only the first step in fiscal transparency and there may be scope to raise this bar over time to include the publication of budget execution and audit reports. xii 2012 Development Policy Lending Retrospective Selected Future Directions: Results: Enhance the focus on sustainable results. OPCS will continue efforts with Regions to improve the quality of results frameworks (ongoing). OPCS will continue to maintain a DPL corporate support team, participating in concept and corporate reviews for all operations. Building on discussions with IEG, OPCS will work with other parts of the Bank to establish a DPO impact evaluation program (by mid-FY14). Risks and Opportunities: Establish a standardized risk assessment framework for DPOs. OPCS will work with other parts of the Bank to establish a simple, standardized risk assessment framework for DPOs, allowing for more systematic and objective comparison of risks across operations (by mid-FY14). Strengthen the quality of macroeconomic assessments. OPCS will work with PREM and regions to improve the quality and consistency of macroeconomic assessments in DPO Program Documents (including the analysis of expenditure and revenue composition in the budget). OPCS and PREM will work together to prepare a good practice note on macroeconomic assessments in DPOs (by end-FY13). Enhance analysis of poverty and social impacts. PREM, SDN and OPCS will work together to review options for more upstream analysis of poverty/social impacts of key policy reforms and their mitigation, so that this work can be drawn upon quickly in DPO Program Documents when reform opportunities and financing needs arise (ongoing). Reforms: Mainstream Policy Based Guarantees (PBGs) into OP 8.60. OPCS will present for the Board’s approval a modernization of the operational policy on guarantees, including the extension of Policy-Based Guarantees to IDA-only countries with a low risk of debt distress and adequate debt management capacity, and a complete mainstreaming of PBGs into OP 8.60 to facilitate their use (by end-FY13). Improve the effectiveness of DPOs under JBS. OPCS is finalizing guidance on how to engage in effective JBS partnerships, in particular the need to avoid overly rigid and process-driven approaches in the context of MoUs, to maximize benefits and minimize potential costs. OPCS will be coordinating this guidance with the Legal Department and the Regions (by end-FY13). 2012 DEVELOPMENT POLICY LENDING RETROSPECTIVE RESULTS, RISKS, AND REFORMS I. INTRODUCTION 1. World Bank1 development policy lending aims to help the borrower achieve sustainable growth and poverty reduction through a program of policy and institutional actions. Development Policy Operations (DPOs), which may be extended either to member countries or to political subdivisions of members, are provided in the form of non-earmarked loans, credits, or grants that support the country’s economic and sectoral policies and institutions—for example, through measures to improve public finance or the investment climate, diversify the economy, and create employment. The Bank bases its decision to extend a DPO on its assessment of the borrower’s institutional and policy framework, the adequacy of its macroeconomic policy framework, and the borrower’s commitment to and ownership of the reform program to be supported by the operation. Development policy lending complements the Bank’s other financing instruments: investment lending, which provides project financing; program-for-results lending, which finances a borrower’s program of expenditures and disburses against results; and guarantees, which help mobilize private financing for critical projects and programs (see Figure 1). Figure 1. The Spectrum of Bank Lending Instruments Country’s National and Sectoral Development Programs World Bank’s Country Assistance Strategy Development Policy Lending Program for Results Lending Investment Lending (General Budget Support (Expenditure Program Support (Project Support – Disburses against – Disburses against – Disburses against Policy/Institutional Actions) Measurable Program Results) Specific Investments) Policy and Project based Guarantees 2. Since policy-based lending was introduced in the 1980s, the Bank has systematically analyzed its experience with the instrument.2 This Retrospective is the third since the introduction of OP 8.60 in August 2004 (Box 1 takes stock of actions taken following the 2009 Retrospective). There have also been several reviews of experience with development policy lending by independent researchers, the Bank’s Independent Evaluation Group (IEG), other development partners, and civil society organizations (see Annex D). Over time, the Bank has used the analysis to adapt the instrument to a changing world and improve its development impact. 3. The most recent Retrospective, in 2009, confirmed the overall robustness of development policy lending as a useful instrument to provide financing and policy advice in support of a country’s medium-term development goals. It found that DPOs generally continued 1 Hereafter referred to as the Bank. 2 For earlier reviews, see Structural Adjustment Lending: A First Review of Experience, Operations Evaluation Report No. 6409, World Bank, September 24, 1986; Report on Adjustment Lending: Policies for the Recovery of Growth (R90-51, IDA/R90-49), March 26, 1990; and The Third Report on Adjustment Lending: Private and Public Resources for Growth (R92-47, IDA/R92-29), March 24, 1992. 2 to be prepared according to Bank operational policy requirements and in line with the Bank’s good practice principles on conditionality. In addition, it highlighted that DPOs (i) have supported country- owned reforms aimed at achieving specific development results in a broad range of countries with different needs, from middle-income countries to fragile states emerging from conflict; (ii) have supported borrowing countries with both the “what� and the “how� of development, by providing financing and policy advice in areas where country authorities required expertise and technical knowledge; and (iii) have remained focused on results and have been associated with positive outcomes in the delivery of social services and public sector reforms, among others. 4. The 2009 Retrospective also identified a number of weaknesses and follow-up actions (see Box 1). Areas identified in the 2009 Retrospective in need of strengthening, included: (a) assessment of the country’s macroeconomic policy stance and the underlying public expenditure framework for subnational DPOs; (b) discussion of the processes for consultation on and participation in the country’s development programs; (c) the use of environmental analysis and poverty and social impact analysis to inform DPOs; (d) assessment of the adequacy of fiduciary arrangements and residual fiduciary risks; and (e) the overall quality of results frameworks. The current Retrospective builds on the 2009 Retrospective and takes stock of the progress achieved in these areas in the past three years. Box 1. 2009 Development Policy Lending Retrospective—Recommendations and Follow-up Actions Update guidance to staff. Guidance to staff on how to design DPOs in line with operational policy to reflect the lessons of the experience with results-focused DPOs, the updated good practice note on using poverty and social impact analysis to support DPOs, the toolkit for assessing the environmental impacts of actions, and to address weaknesses in design aspects. Accordingly, Good Practice Notes on designing DPOs, DPOs in fragile situations, results in DPOs, and public financial management in DPOs were revised; and new training for staff on environmental, and poverty and social impact analysis in DPOs were introduced. Make better use of good practice. If the quality of DPOs is to be strengthened, regions would need to thoroughly implement operational guidance and follow the advice provided at the corporate review stage. Regions were expected to encourage DPO task team leaders to attend OPCS’s DPL Academy, offered twice a year. In consultation with OPCS, Networks and anchors were encouraged to collect and share good practice with staff involved with development policy lending, and continue to provide advice and support to teams preparing DPOs. OPCS staff continued to provide guidance to regions including through review of DPOs at regional and corporate reviews. OPCS continued to offer the DPL academy twice a year and also has developed DPL e-learning. Consolidate good practice on poverty and social impact analysis. Lessons suggested the need to focus more on outreach and learning efforts to build staff capacity and awareness within the Bank about poverty and social impact analysis. To help consolidate good practice, the Bank was expected to fund poverty and social impact analysis through a new multidonor trust fund that became effective early in FY10. The Multi-donor Trust Fund (MDTF) has contributed to the improvement in undertaking poverty and social impact analysis. The MDTF provides approximately $20 million over a three year period (and has been recently extended for two more years) to support the Bank’s work on distributional impacts of policy reforms. More than 100 poverty and social impact analyses have been approved under the MDTF as of June 2012. In addition, the Bank has rolled out an e-learning course, designed to inform task teams on the fundamentals of such analysis. Also, a major learning and training event took place in 2011, convening policy makers, academics, civil society, donors and Bank staff to share their experiences and knowledge about poverty and social impact analysis and how it can influence policy design. Develop guidance to staff on DPOs in the context of joint budget support. Based on the growing experience with joint budget support frameworks prepare guidance to reflect on the lessons of this collaborative engagement and to use these lessons to identify ways to perform better in the future. OPCS (in coordination with LEG) will finalize a guidance note on JBS partnerships, reflecting lessons learned. 3 5. The objective of this Retrospective is to assess whether DPOs have contributed to development results, whether they are embedded in a robust risk assessment framework, and whether reforms to the instrument’s operational policy framework have been effective. As the recent global financial crisis affected many of the industrialized countries that are also the main providers of aid, taxpayers in those countries have become increasingly concerned about “value for money� as regards the use of aid funds in recipient countries. This type of concern has generated demand for a fresh review of the evidence on the issues around results, risks, and reforms. Thus the Retrospective assesses (a) the extent to which DPOs have contributed to and influenced country results, and what can be done to enhance their results focus; (b) the risks associated with DPOs and how these have been balanced against opportunities—including governance risks, fiduciary risks, and macroeconomic risks—and what can be done to enhance the risk management framework for DPOs; and (c) the effectiveness of reforms introduced to the operational policy framework and of emerging practices, and what further reforms might be considered in the coming years. 6. This report reviews all DPOs approved by the Board between April 2009 (after the last Retrospective) and March 2012.3 Chapter II presents the facts about these operations—their numbers, coverage, and amounts; Chapters III, IV, and V analyzes these operations in more depth from the standpoint of results, risks and opportunities, and reforms, respectively; and Chapter VI presents conclusions and sets out future directions. 3 An additional 14 operations were approved in FYQ4 totaling $5.4 billion in commitments. These DPOs are not included in the main analysis unless otherwise noted. Details on these operations can be found in Annex G. 4 II. CHARACTERISTICS OF DPOS 7. This chapter summarizes the main trends of development policy lending over the past three years. It summarizes the key statistics of 221 operations and six supplementals which amounted to roughly $45 billion. Results are shown by region and source of financing (IBRD and IDA). A. Overall Trends in Development Policy Lending 8. During the period of this Retrospective, a total of 221 DPOs and 6 supplemental financing operations were approved.4 These operations represent total commitments of nearly $7.4 billion of IDA funds, over $37 billion of IBRD funds, and $243.5 million of trust funds (see Figure 2). The supplemental financing operations5 represent $466 million of this total.  Approval date. Of the 221 operations, 28 were approved in the last quarter of FY09, 81 operations and 4 supplemental financing operations in FY10, 70 operations and 1 supplemental financing operation in FY11, and 42 operations and one supplemental through the third quarter of FY12.6  Source of financing. Of the 221 operations, 103 (47 percent) were IDA-funded only, 2 operations (to Cambodia and Kosovo) were IDA-trust fund blends, 6 (3 in West Bank and Gaza, 2 in Burkina Faso, and 1 in Rwanda) were trust fund only; 101 operations (46 percent) were IBRD-only, and 9 were IBRD-IDA-blend ($484.8 million of IBRD and $663 million of IDA). Figure 2. Regional Distribution of Commitments and Operations, FY09 Q4-FY12 Q3 $ Billion IBRD IDA TF # of Operations 90 14 80 12 70 10 60 8 50 40 6 30 4 20 2 10 0 0 AFR EAP ECA LCR MNA SAR AFR EAP ECA LCR MNA SAR 4 One of the operations approved by the Board in September 2009 to Hungary (for a total of $1.4 billion) was never signed and is therefore not included in the retrospective. 5 The supplemental financing operations were for Benin, Niger, Philippines, São Tomé and Príncipe, Sierra Leone, and Tanzania. 6 There were 56 DPOs in total in FY12 (including the DPOs approved in the fourth quarter). 5 9. Of the 221 operations, 64 percent were part of a programmatic series of operations and 32 percent were stand-alone operations.7 In addition, 8 were multi-tranche operations8 (7 involved two tranches and 1, in Egypt, involved three). Since the introduction of the DPO policy in 2004, just over 30 percent of operations have been stand-alone operations. This share rose to nearly 40 percent in FY10 in response to the financial crisis, but it returned to 28 percent in FY11. As Figure 3 shows, programmatic operations have been the predominant modality in the East Asia and Pacific (EAP), Europe and Central Asia (ECA) and Africa (AFR) regions. In the Latin America and Caribbean Region (LCR), stand-alone operations represent 48 percent of the total. In the case of AFR, the prevalent nature of programmatic operations has been determined to a large extent by the Joint Budget Support (JBS) frameworks, through which the Bank often operates in the region. Figure 3. Types of DPOs by Region, FY09 Q4-FY12 Q3 Percentage Multitranche Stand alone Programmatic 100 90 80 70 60 50 40 30 20 10 0 AFR EAP ECA LCR MNA SAR 10. Since FY05, development policy lending has averaged about 30 percent of total Bank commitments, but the share rose to nearly 40 percent during the recent global financial crisis.9 More recently, the share of commitments returned to 27 percent in FY11, and rose again to 35 percent in FY12 (see Figure 4). The rebound in FY12 may reflect countries’ response to the lingering uncertainties about the weak global recovery and concerns about a possible “double-dip.�10 The recent increase in development policy lending was even more pronounced in the case of disbursements, reaching 51 percent in FY10, stabilizing at 40 percent in FY11-12 (see Figure 5). The average size of DPOs has increased from $177 million to $203 million during the Retrospective period. 7 Of the 14 approvals in Q4 FY12, 8 were programmatic and 6 were stand-alone operations. 8 One additional two-tranche operation was approved in Q4 of FY12 to the state of Bahia in Brazil. 9 As part of the operational policy change in 2004, the 25 percent ceiling for the Bankwide share of policy- based lending was removed and substituted by an annual report by Bank Management on the anticipated Bankwide share of development policy lending in total lending. On the basis of that report, Executive Directors would issue guidelines for the average annual Bankwide share of development policy lending on a rolling three-year basis. In the context of the IDA15 Replenishment discussions, it was agreed that Management would seek additional guidance from IDA’s Executive Directors if the projected share of DPO commitments exceeds 30 percent for any future year. See Additions to IDA Resources: Fifteenth Replenishment. IDA: The Platform for Achieving Results at the Country Level, Report from the Executive Directors of the International Development Association to the Board of Governors, February 28, 2008, page 11, footnote 41. 10 To ensure contingency financing three DPOs with DDO were approved (Indonesia US$ 2 billion, Romania US$ 1.3 billion and Uruguay US$ 260 million). 6 Figure 4. Total Commitments, FY95-12 $ Billion Investment Lending Policy Based Financing Policy Based Financing as Percentage of Total Percentage 60 60 50 50 40 40 30 30 20 20 10 10 0 0 FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 Figure 5. Total Disbursements, FY95-12 $ Billion Investment Lending Policy Based Financing Policy Based Financing as Percentage of Total Percentage 45 60 40 50 35 30 40 25 30 20 15 20 10 10 5 0 0 . FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 11. DPOs support reforms, typically drawn from the government’s reform program, that are deemed critical to achieving development results. All actions are implemented before the operation is presented to the Board for approval (therefore, they are known as prior actions). In a programmatic series, indicative actions (triggers) for subsequent operations in the series are included when the initial operation is presented for approval. These indicative actions are not binding and are meant to be flexible and to be adjusted to the country’s circumstances. 12. Overall, the 221 DPOs reviewed for this Retrospective contained a total of 2,186 prior actions, an average of 10 per operation. This result is consistent with the previous Retrospective.11 11 The previous retrospective also covered a three-year period but included only 165 DPOs with 1664 total prior actions, also an average of 10 prior actions per DPO. 7 The average, which has remained at about 10 since 2005, indicates consistently greater selectivity and focus in DPO programs than in the structural adjustment loans of the mid-1990s (see Figure 6). Figure 6. Average Number of Prior Actions in Policy Based Lending, FY95-12 IBRD IDA+TF 40 35 30 25 20 15 10 5 0 FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 a a First to third quarter  only. B. Development Policy Lending in IBRD Countries12 13. During the period covered by this Retrospective, IBRD extended 110 DPOs13 for a total of $37.3 billion to 42 different countries and to several political subdivisions.14 Several IBRD borrowers received four or more DPOs each: Brazil, Dominican Republic, Indonesia, Mexico, Morocco, Peru, and Turkey. 14. The share of IBRD DPOs increased significantly in FY09 and FY10, largely in response to the global financial crisis (see Figure 8). ECA and LCR saw the largest increases: ECA commitments, which had ranged between $500 million and $1.0 billion a year, increased to nearly $6.4 billion in FY10; and LCR DPO commitments, which had been approximately $2 to $3 billion a year, rose to nearly $7 billion a year in FY09 and FY10. In both regions the volumes were significantly lower in FY11 and FY12. The increase in volume during this Retrospective (73 percent higher in nominal terms) was due to an increase in the number of operations (from 69 to 110); and the size of operations (the average size increased from approximately $313 to $339 million) (see Figures 7 and 8, and Annex A).15 The average time to prepare an operation during FY09-11 fell to 6.1 months, from 8.2 months during FY01-08.16 12 Policy-based financing to IBRD countries is offered to middle-income countries and creditworthy lower-income countries, on nonconcessional terms. 13 Five additional IBRD DPOs were approved in Q4 of FY12 for a total of $5.1 billion in commitments. Details can be found in Annex H. 14 Including the first DPO to a municipality, the city of Rio de Janeiro, in Brazil. 15 In FY05 and FY06, the average IBRD DPO was only $187 million. 16 Time from concept note to Board approval. 8 Figure 7. IBRD Commitments and Operations, FY09 Q4-FY12 Q3 2012 DPL Retrospective 2009 DPL Retrospective 2012 DPL Retrospective 2009 DPL Retrospective $ Billion # of Operations 16 50 14 45 40 12 35 10 30 8 25 6 20 15 4 10 2 5 0 0 AFR EAP ECA LCR MNA SAR AFR EAP ECA LCR MNA SAR Figure 8. Share of Policy Based Lending in Total IBRD and IDA Lending, FY95-12 Percentage IBRD IDA 70 60 50 40 30 20 10 0 FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 15. The principal area of support was public sector governance, although its share of prior actions had decreased since the last Retrospective (see Figure 9). Prior actions in public sector governance focused primarily on public expenditure, financial management, and procurement. Reforms related to transparency and accountability17 have become more common. For example, the Uruguay Second Programmatic Public Sector DPO supported e-government reforms. Financial and private sector development is the second most important area, with 21 percent of the prior actions. 17 There were 30 prior actions specifically targeting accountability and anticorruption, and many other actions in the areas of public financial management and procurement had an accountability and anticorruption focus. In the previous retrospective, only 10 actions were primarily focused on accountability and anticorruption. 9 Figure 9. IBRD Thematic Distribution of Prior Actions FY06 Q4 - FY09 Q3 FY09 Q4 - FY12 Q3 Urban Urban Trade and Development Economic Trade and Development Economic Integration 4% 0% Management Integration 2% 2% Management 5% 5% Social Dev. & Environment Social Dev. & Environment Prot. 7% and Natural Prot. 17% and Natural Resource Resource Rural Management Management Development 11% 10% 2% Rural Financial and Financial and Development Private Sector Private Sector 1% Development Development 23% 21% Public Sector Public Sector Governance Governance 30% 41% Human Human Development Development 7% 12% 16. The area of social development and social protection received substantially increased attention, partly because of the global economic crisis. Many operations, including several sector- specific DPOs supporting social sectors, aimed at strengthening social protection and mitigating the impact of the crisis. Several countries—Albania, Colombia, Grenada, Latvia, Panama, and Saint Lucia—prepared DPOs with a focus on social development and social protection. In Colombia, for example, the DPO supported the preparation of a strategy to address extreme poverty; in Panama the DPO supported the increase in monetary transfers to the beneficiaries of the conditional cash transfers program to help extremely poor households to cope with the increase in food prices and the impact of the global slowdown; and in the Dominican Republic three operations focused on improving performance and accountability in the social sectors. 17. Support to reforms in the area of environment and natural resources remained approximately the same. However, while in the previous Retrospective the 11 percent of actions related to the environment were spread across many more operations, the current 10 percent are much more targeted to sector-specific DPOs addressing reforms related to environmental management and green growth. Notable examples of sector-specific DPOs include the Climate Change DPO in Indonesia, a Solid Waste Management DPO in Morocco, a series of three programmatic environmental DPOs in Peru, and three stand-alone DPOs in Mexico targeting green growth, climate change issues in the water sector, and low carbon growth. C. Development Policy Lending in IDA Countries18 18. Approximately 50 percent of the DPOs approved during this period were funded by IDA or trust funds, down from 58 percent in the last Retrospective. These 103 IDA-only operations19 were extended to 45 countries, including Fragile and Conflict-Affected Situations (FCS) and countries receiving IDA budget support under the small islands exception: Cape Verde, Grenada, 18 Policy-based financing offered to IDA countries is either on concessional or grant terms. Eligibility depends primarily on a country’s relative poverty, defined as GNI per capita below an established threshold and updated annually (in FY12: US$1,175). IDA also supports several small island economies, which are above the operational cutoff but fall into the small island exception category. Some countries, referred to as “blend countries� are IDA- eligible based on per capita income levels, but are also creditworthy for some IBRD borrowing. 19 Eight additional IDA-only operations were approved in Q4 of FY12 totaling $254 million in commitments. 10 Maldives, Saint Lucia, Samoa, and Tonga. Of the $7.4 billion in IDA-financed DPOs, $283 million was provided via the IDA15 Crisis Response Window (see Box 2). As of end FY12, IDA 16- financed DPOs reached over $1.8 billion, through 28 operations of which 21 were in AFR. Box 2. IDA Crisis Response Window As part of the Bank’s response to the global financial crisis in IDA countries, a Pilot Crisis Response Window (CRW) was approved by the Board on December 2009. The Pilot amounted to $1.4 billion. The main objectives of the Pilot CRW were to protect core spending on health, education, safety nets, infrastructure and agriculture, and build the resilience of IDA countries to cope with any future crises. Development policy lending represented 20 percent of total commitments from the CRW pilot: 21 DPOs for a total of $834 million (of which $283 million from the CRW Pilot). Of the 21 DPOs that were augmented by CRW resources, 14 were in AFR and the rest were in EAP, ECA and SAR. 17 operations were part of ongoing programmatic series, two were stand-alone (Moldova and Samoa), and the other two were the first operations in programmatic series (Kosovo and Maldives). 19. The volume of IDA DPOs has been stable at approximately $7 billion, albeit with great variation among regions (see Figure 10 and Annex A). IDA development policy lending increased threefold in EAP and more than twofold in ECA and LCR. But there was a notable decline in SAR, which had commitments for $2.7 billion in the last Retrospective and only $394 million during the current Retrospective period. The volume of development policy lending to AFR continued to increase, from $3.9 billion in the previous Retrospective period to $4.6 billion in the current review period (a change of approximately 19 percent). Most of the $2 billion decline in SAR can be attributed to fewer DPOs in Bangladesh and Pakistan. Bangladesh, which borrowed nearly $1 billion of IDA funds through six DPOs and two supplemental operations during the last Retrospective, borrowed none during the current period, owing to poor program performance. Additionally, Pakistan borrowed almost $1.4 billion through seven IDA-funded DPOs under the prior Retrospective, but only two for $300 million under the current period, due to deterioration of the macroeconomic policy stance. 20. IDA development policy lending, which represented around 25 percent of total IDA commitments in FY05-09, dropped steadily as a share to 12 percent by end-FY11, increasing only slightly in FY12 to 13 percent (see Figure 8). The share of development policy lending in total IDA lending also varied among regions. The region with the largest share of development policy lending in total IDA during this Retrospective was EAP (30 percent) followed by ECA (27 percent). SAR had the lowest share (0.02 percent) with almost $17 billion in investment lending and only $394 million in development policy lending. In the other regions, development policy lending represents about 20 percent of total IDA lending: AFR (21 percent), MNA (19 percent) and LCR (18 percent). Compared with the last Retrospective, the only two regions that registered a decline in the share of development policy lending in total IDA were AFR (from 24 to 21 percent) and SAR with a sharp decline (from 25 to 0.02 percent). In the case of AFR investment lending grew faster (43 percent) compared to development policy lending (19 percent). The decline in the share of development policy lending in total IDA in AFR may also reflect (i) an increased focus on governance and fiduciary risks; (ii) countries’ preferences for investment lending due to the focus on infrastructure investments (including in the energy sector); (iii) the lesser severity of the global financial crisis in the region; and (iv) the sometimes excessive bureaucracy and rigidity associated with Joint Budget Support arrangements, discussed in Chapter V, which are most prevalent in AFR. 11 21. The number of IDA-financed DPOs has remained stable at 32 DPOs in FY10 and 33 in FY11, compared to an average of 33 in FY05-09. However, the average size has decreased slightly from $79 million to $71 million. The average time to prepare an operation during FY09-11 fell to 5.9 months, from 7.6 months during FY01-08. Figure 10. IDA Commitments and Operations, FY09 Q4-FY12 Q3 $ Million 2012 Retrospective 2009 Retrospective # of Operations 2012 Retrospective 2009 Retrospective 5,000 80 4,500 70 4,000 60 3,500 3,000 50 2,500 40 2,000 30 1,500 20 1,000 500 10 0 0 AFR EAP ECA LCR MNA SAR AFR EAP ECA LCR MNA SAR 22. The thematic composition of DPOs in IDA countries continued to be heavily focused on public sector governance, which accounts for half of all prior actions (see Figure 11). The largest number of prior actions under this thematic group focused on strengthening public financial management systems and procurement. Additionally, there were an increased number of actions related to transparency, audit of public funds, accountability, and strengthening countries’ systems to ensure that resources deliver development results. The number of prior actions focused on accountability and transparency more than doubled since the previous Retrospective period, reaching 88 actions in 28 countries. The majority of DPOs in IDA countries (60 percent) continued to be multisectoral, contributing to policy reforms in several sectors. Figure 11. IDA Thematic Distribution of Prior Actions FY06 Q4 - FY09 Q3 FY09 Q4 – FY12 Q3 Urban Urban Trade and Development Economic Trade and Development Economic Integration 3% 0% Management Integration 2% 1% Management 2% 5% Social Dev. & Environment Social Dev. & Environment Prot. 5% and Natural Prot.9% and Natural Resource Resource Rural Management Management Rural Development 5% 5% Development 5% 4% Financial and Financial and Private Sector Private Sector Development Development 16% 15% Public Sector Public Sector Human Human Governance Governance Development Development 51% 51% 8% 13% 12 23. IDA support to financial and private sector development through DPOs continues to be strong, with 15 percent of all prior actions. There were 35 prior actions related to the private or financial sector development in seven FCS, including reforms in Burundi to support the creation of a new commercial code and improve the performance of commercial courts; in Côte d’Ivoire to support the restructuring of the microfinance sector; in Guinea-Bissau to adopt a draft investment code; and in Tajikistan to deliver a new aviation sector policy, update the banking regulations, and improve the private sector by codifying procedures for obtaining permits. 24. As in IBRD-funded DPOs, there has been an increase in the focus on social protection and development (from 5 to 9 percent). The reforms in this area focused on strengthening social safety nets, social inclusion, and the adoption of poverty strategies that enabled governments to target resources to the most vulnerable groups. Some operations focused on improving social protection— for example, the Pakistan Social Safety Nets DPO, the series of three Rwanda Community Living Standards DPOs, and the Moldova Economic Recovery DPO. However, the percentage of prior actions related to human development in IDA countries has decreased from 13 to 8 percent. This is because the previous Retrospective period included several operations that were a direct response to the global food crisis,20 and had prior actions related to food security and nutrition. 25. A number of DPOs have supported gender-related reforms. For example, the Vietnam Program 135 Phase II support DPO included a reform related to female participation in civic engagement, and the Vietnam PRSC-10 supported the creation of a set of gender- Figure 12. Trends in Gender-Informed Lending disaggregated development indicators. The Instruments, FY10-12 Rwanda Education for All - Fast Track Development Policy Lending Investment Lending Percentage Catalytic Fund - DPO supported reforms 90 related to education policy for girls. There 78 80 were also other reforms that did not have gender as the primary theme, but included 70 64 71 64 a gender dimension by supporting the 60 participation and civic engagement of traditionally excluded members of the 50 52 community, maternal health, among others. 40 47 Over time, DPOs have become more gender-informed (see Figure 12).21 30 FY10 FY11 FY12 26. A number of reforms focused on the IDA16 theme of climate change, through both mitigation and adaptation. The Ghana Natural Resources and Environmental Governance DPO supported the drafting of a national climate change strategy, and the Vietnam PRSC-8 supported a nationally targeted program on adaptation to climate change. In addition, a Climate Change DPO was approved for Vietnam in the third quarter of FY12. Operations in the Central African Republic, Mali, and Mongolia—although not specifically focused on climate change— contained some components that supported environmental policies and institutional strengthening. Overall, sector specific operations related to climate change have increased significantly over time. During FY04-07 20 For example, operations to Bangladesh, Burundi, Djibouti, Guinea, and Sierra Leone. 21 Whether an operation is gender-informed depends on a consideration of three dimensions: analysis, actions, and monitoring and evaluation. If at least one dimension considers gender, then the project is deemed gender- informed. 13 $700 million were committed through 3 DPOs, and more recently, during FY08-11, $ 6.1 billion were committed through 15 DPOs, in 6 countries. 1. Joint Budget Support Operations 27. Approximately half of IDA operations (50 percent by volume, 45 percent of commitments) were prepared in the context of Joint Budget Support (JBS). That is, they were guided by a Memorandum of Understanding (MoU)22 and/or included a joint Performance Assessment Framework (PAF).23 The modality was especially prevalent in AFR, where about 74 percent of IDA operations (71 percent of commitments) were prepared in partnership with others. Most of the JBS operations were prepared in the context of a programmatic framework (88 percent) and tended to be cross-sectoral (86 percent) budget support operations. 2. Use of DPOs in FCS24 28. DPO commitments to FCS have fluctuated over the years. Development policy lending to FCS as a share of total lending has been low at around 3 percent (FY10-12) (see Annex A). As a percentage of IDA lending25 it averaged 15 percent in FY10-12. The spike of over $1 billion in 2008 was due mainly to three large arrears clearances (Côte d’Ivoire, $308 million; Liberia, $430 million; and Togo, $175 million). 29. The average number of DPOs in FCS has increased significantly, from 6 operations per year in FY05-07 to approximately 11 in FY10-12 (see Figure 13). However, the average size of an FCS operation was in generally relatively small. For example, in FY12 the average size of an IDA-only FCS operation was $38 million, compared to the $60 million average of an IDA-only financed DPO. The Bank has increasingly engaged with FCS through programmatic series: the proportion of all programmatic DPOs increased from 38 percent in 2005 to 66 percent in 2012. Reforms supported during the Retrospective period focused predominantly on public sector governance (65 percent of prior actions) and finance and private sector development (12 percent). 22 MoUs describe (sometimes in great detail) the agreed rules on design and supervision of the Joint Budget Support, between the budget support group and the government, and among the development partners that comprise the budget support group. MoUs often create reputational obligations, although they do not constitute legally binding obligations on the Bank or any other signatory. 23 The PAF is a framework agreed between governments and development partners for evaluating the progress of key reforms in the country. Ideally, the PAF is drawn from the government’s National Development Strategy. 24 For the first time since the introduction of the DPO policy in 2004, this retrospective examines the use of the instrument in FCS. Therefore, this section covers operations since 2005. 25 Most FCS are IDA borrowers, except the West Bank and Gaza and the Blend countries of Kosovo, Georgia, and Bosnia and Herzegovina. Approximately 91 percent of the DPOs to FCS were financed with IDA funds. Two operations (Bosnia and Herzegovina, and Georgia) were IBRD-IDA blend funded, three DPOs in the West Bank and Gaza were TF-only funded, and one DPO in Kosovo was IDA-TF blend funded. 14 Figure 13. DPO Commitments and Operations to FCS, FY05-12 $ Million IDA TF IBRD # of Operations IDA TF IBRD 1000 14 900 12 800 700 10 600 8 500 400 6 300 4 200 2 100 0 0 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 3. Development Policy Lending Commitments to Small States 30. The number and volume of DPOs to small states26 also continued to increase during the period covered by this Retrospective. During the previous Retrospective, 11 DPOs were approved to six small states, in the amount of $72.6 million of IDA funds, and $160 million of IBRD funds and $5 million in special financing (Djibouti, food crisis operation). During this period, 19 DPOs were approved to 12 different small states for a total of $168 million of IDA funds and $26.5 million of IBRD funds.27 Two operations in Seychelles were the only operations funded solely from IBRD, while one operation each in Grenada and Saint Lucia received a combination of IDA and IBRD financing. The other small states receiving DPOs on IDA terms were two operations each in Bhutan, Cape Verde, São Tomé and Principe, and Tonga, and three operations in Guinea-Bissau. Comoros, Maldives, Samoa, and Gambia each had one IDA operation. 26 Countries with a population below 1.5 million people. 27 Additionally, 3 DPOs were approved in the fourth quarter of FY12, in the amount of $US 20 million (IDA financed). 15 III. RESULTS 31. This chapter reports on the results of Bank evaluations of DPOs based on Implementation Completion and Results (ICR) reports and their validations by the Independent Evaluation Group (IEG). It assesses the achievement of targeted results of the DPOs in the Retrospective and discusses countries’ results, and in particular those in Fragile and Conflict-Affected Situations. It discusses the assessments in the Program Documents of countries’ monitoring and evaluation arrangements. A. Performance of DPOs 1. Bank and IEG evaluations of DPO since FY05 32. To evaluate the extent to which a DPO achieved its intended development objectives and targeted results, the Bank prepares an Implementation Completion and Results (ICR) report for every DPO operation or programmatic series. The preparation of an ICR report is required within six months after the closure of the operation. For programmatic operations, an ICR report is due within six months after the closure of the last operation in the series, and it includes a separate assessment (but not a rating)28 of the contribution of each individual operation to the program. ICR reports are prepared with the participation of the borrower and other stakeholders. All ICRs are evaluated by IEG and disclosed to the public. 33. Evaluations suggest that DPOs have Figure 14. Bank and IEG Evaluations of DPOs, been generally successful in contributing to FY06-10 the intended results (see Figure 14). Percentage IEG ICR According to the available ICRs, 29 the Bank 100 Moderately satisfactory or higher has rated approximately 91 percent of the 90 operations approved since FY05 as moderately satisfactory or higher (and 63 percent as 80 satisfactory or higher). IEG also has a positive assessment,30 rating 81 percent of these 70 operations as moderately satisfactory or higher 60 and 49 percent as satisfactory or higher. 31 50 FY06 FY07 FY08 FY09 FY10 34. While overall performance has Exit Fiscal Year remained relatively high, there are variations by region and client segment (see Figure 15). According to IEG evaluations, ECA had the largest percentage of DPOs rated as moderately satisfactory or higher (94 percent). If the bar is raised to satisfactory or higher, results show that SAR and ECA did the best (both with 63 percent, 28 In May 2010, a simplification was introduced for ICRs of programmatic DPOs. Thereafter, ICRs provide ratings at the program level, based on the overall programmatic series, as opposed to ratings for each individual operation. 29 Based on 332 ICR reports available of operations coded as DPOs. 30 Based on 268 IEG evaluations available of operations coded as DPOs. 31 Figures reported for both ICR and IEG ratings were based on available ICR reports and IEG evaluations. Data for FY09 and FY10, in particular, are relatively limited. The proportion of DPO exits evaluated by IEG is approximately 63 percent for FY09 and 58 percent for FY10. It is possible that the proportion of DPOs rated as moderately satisfactory or higher may fall as more evaluations become available. 16 respectively).32 The regions with the lowest shares of DPOs rated satisfactory or higher are MNA (32 percent) and AFR (40 percent). This difference in regional performance may reflect stronger implementation capacity in the ECA region compared to MNA and AFR; and more government ownership, with well defined strategies and a greater drive for reform. It is, however, worth noting that 4 of the 9 operations rated as highly satisfactory were in the AFR region. Performance of DPOs in MNA and AFR has also been affected by political instability. In addition, many DPOs in AFR supported countries of acute fragility which is reflected in weaker implementation capacity and more frequent changes in government ownership.33 35. When comparing IBRD and IDA, it emerges that 84 percent of the IBRD financed DPOs were rated as moderately satisfactory or higher, compared to 78 percent in IDA financed DPOs. This result is consistent with the finding for regional performance, as most DPOs in AFR were IDA financed (69 versus 5 during the period covered by this Retrospective) and most ECA DPOs were IBRD financed (30 versus 7). Achieving intended results proved to be more challenging FCS where only 33 percent of the DPOs were rated as satisfactory and 9 percent were rated as unsatisfactory. The result of poorer performance for FCS is correlated with the regional and IDA financing findings, as over 90 percent of the DPOs to FCS were IDA financed and over 50 percent are in AFR. 36. Analysis suggests that programmatic DPOs perform better than stand-alone DPOs. Approximately 86 percent of the programmatic DPOs were rated moderately satisfactory or higher compared to 76 percent of stand-alone DPOs. If we consider those rated satisfactory or higher, the difference is marked. About 57 percent of the programmatic DPOs were rated satisfactory or higher compared to 39 percent of stand-alone DPOs. This result holds for every region with the exception of ECA (see Figure 16). It is also worth noting that 24 percent of stand-alone DPOs were rated as moderately unsatisfactory or unsatisfactory compared to 15 of programmatic DPOs. Multi-tranche DPOs appear to perform slightly poorly than stand-alone. Approximately 72 percent of the multi- tranche DPOs were rated moderately satisfactory and higher and 28 percent were rated moderately unsatisfactory or unsatisfactory.34 This suggests that DPOs that are embedded in a medium to long term engagement with the client, and which support medium term reforms, perform better than the one-off type of engagement. The lack of flexibility of multi-tranche operations may also help explain the lower success rate of this type of DPO. 32 It should be noted that in the case of SAR this result mostly reflects performance of DPOs approved before this retrospective (there were only 9 DPOs to SAR during this retrospective). 33 To find the determinants of poor and good performance a regression analysis is warranted. Further research on the determinants of performance will be undertaken following this retrospective. 34 This is, however, based on a limited sample. There are 18 IEG evaluations of multi-tranche operations. 17 Figure 15. IEG Evaluations of DPOs by Region, FY05-12 Percentage 100 90 80 70 U 60 MU 50 MS 40 S 30 HS 20 10 0 AFR EAP ECA LCR MNA SAR Grand  IBRD IDA FCS Total Figure 16. IEG Evaluations by Type of DPO and Region, FY05-12 Percentage 100% 90% 80% 70% U 60% MU 50% MS 40% S 30% HS 20% 10% 0% Programmatic Programmatic Programmatic Programmatic Programmatic Programmatic Standalone Standalone Standalone Standalone Standalone AFR EAP ECA LCR MNA SAR Standalone Figure 17. IEG Evaluations by Lending Instrument, FY00-10 18 37. IEG evaluations suggest that policy-based lending has performed at least as well as investment lending.35 Using data from as early as FY00, IEG evaluations have rated the performance of development policy lending above that of investment lending, with few exceptions (see Figure 17). Since FY09 the ratings of development outcomes of DPOs seem to have held up, or possibly even improved, despite substantially increased volumes. However, this is based on a still incomplete universe of IEG evaluations.36 2. Achievement of Intended Results of the DPOs in the Retrospective37 38. A review of DPO achievements against Figure 18. Achievement of Targeted Results targets by indicator was carried out for the 78 ICR by Indicator reports for which detailed data on targets and actual Partially outcomes were available. The assessment looked at Achieved 13% Achieved whether the targeted results set out in the results 60% Not  frameworks were achieved, based on actual values Observable reported in the ICR. The review found that 60 percent of 9% targets described in the policy matrix were fully achieved or exceeded (see Figure 18). Only 18 percent of targets were marginally achieved or not achieved at all.38 Not achieved 18% 39. Expected results were achieved to a larger extent in the areas of social development, finance and private sector development, and economic management (see Figure 19).39 The areas of public sector governance, human development and trade and integration, have the lowest share of fully achieved targets. This may reflect the higher quality of the results framework of the former group, in the sense that there was greater realism in setting targets and determining the availability of data, or the difficulty of achieving results in certain 35     According to the IEG report “Results and Performance of the World Bank Group 2012�, comparison of DPOs and Investment projects exiting in FY09–11 show that DPOs are significantly above investment projects in the following IEG indicators: (i) the share of DPOs with moderately satisfactory or higher for development outcome ratings (at 84 percent) is significantly higher than that of investment projects (by 13 percentage points); (ii) the share of DPOs with moderately satisfactory or higher quality at entry rating is 30 percentage points higher than that of investment projects, mainly due to high quality of entry ratings for Middle Income Countries; and (iii) the share of DPOs with moderately satisfactory or higher quality of supervision rating is 10 percentage points higher than that of investment projects, however, quality of supervision rating for operations in low-income countries has declined; and (iv) two out of every five DPOs are rated substantial or high on M&E compared to only one out of every four investment projects.  36 See Footnote 31. 37 Analysis based on 78 ICRs (covering 87 operations). 38 “Not achieved or marginally achieved� applies when less than 50 percent of the expected target was achieved. “Partially achieved� applies when the actual value achieved was above 50 percent but below 100 percent of the expected target. “Fully achieved or exceeded� applies when the actual value achieved is equal to or above the expected target. It is considered “Not observable� when no actual value is provided or it is impossible to confirm whether the target has been achieved. 39 To explain the different degrees of success in achieving the target results across sectors an empirical analysis is warranted. Further research on the determinants of success in DPOs will be conducted following the retrospective. 19 areas in the short term. As an example, the second GAC report40 cautions that the journey from inputs to development outcomes in the area of public sector governance may take five to six years. In many cases it may take even longer, even without reversals (for an example, see Box 3). Figure 19. Achievement of Targeted Results by Theme Percentage Box 3. The Long March from Inputs to Development Outcomes The long march from inputs to development outcomes can be illustrated using a diagram from the Governance and Anti-Corruption report, with reference to any of the typical prior actions in the thematic area of public sector governance that are supported by DPOs. For example, a reform in the area of accountability, such as the implementation of an access to information law, would have to pass through several stages before tangible improvements to governance would be detected. In the first stage (Inputs), the move toward more open government would be initiated through the cabinet approval of an access to information law (which could constitute a prior action). In the second stage (Outputs or RA 2), if the law was approved by the legislature, a prior action could support its implementation and the results framework could reflect the expected result. Changes in laws and behaviors of officials (RA 3), however, would require the preparation and dissemination of implementing regulations, training of staff, and creation of an internal oversight body. While all this was going on, little would be visible to the outside world, save for the new law. The results would be visible only after several years, through changes in institutional behavior (RA 4), as departments and agencies become more responsive to civil society’s demands for information and pressure for greater accountability and performance. 40 Strengthening Governance, Tackling Corruption. The World Bank’s Updated Strategy and Implementation Plan, January 2012. 20 Box 3. The Long March from Inputs to Development Outcomes (cont’d) Source: Strengthening Governance, Tackling Corruption. The World Bank’s Updated Strategy and Implementation Plan, January 2012. Page 18. B. Country Results Influenced by DPOs 40. Through development policy lending, the Bank supports countries’ programs of policy and institutional reforms. It does so by providing: (i) financial resources that help bridge budget financing gaps; (ii) expertise, policy advice, and analytic work that help shape the content of the reform programs; and (iii) a platform for dialogue and coordination between the government at the central and local levels, nongovernmental agencies, beneficiaries, and bilateral and multilateral partners. However, when discussing the contribution of DPOs to results, the Bank focuses on the impact of the reforms (prior actions) on the country’s development results.41 Clearly, many factors besides the prior actions supported by the Bank may contribute to development results. For example, results can also be influenced by complementary reforms that may or may not be supported by the Bank or other development partners, or external factors affecting the country. 41. This section presents country case studies of results in selected areas, as well as some cases where the intended results were not achieved, and their lessons learned. These include the IDA16 special themes of gender, fragile situations (discussed in the following section), climate change, and crisis response, and themes selected for IBRD countries: public finance management, delivery of basic services, and crisis response. Given the problems of aggregation and the lack of consistent and internationally comparable data, presenting a systematic, quantitative analysis of the results for the whole sample of countries is difficult. Therefore the Retrospective showcases end-of- program results in selected areas and countries. 1. IDA16 Special Themes 42. Gender. The Rwanda Education for all—Fast Track Initiative catalytic fund aimed to improve the quality of basic education by supporting reforms on teacher development and 41 Since the financial resources are not earmarked, it is not possible to discuss the impact of the financial support. 21 management, textbooks, and girls’ education. It supported a reform that included the development and approval of a Girls’ Education Policy, and a strategy for its implementation. The policy also included measures for improved gender-disaggregated data collection and analysis. Core gender- sensitive indicators are now available and were discussed during the annual Joint Review of the Education Sector. The Girls’ Education Policy was disseminated to every district, and awareness- raising workshops were held with NGOs, faith-based organizations, and school administrators. The Bank rated the operation as satisfactory.42 43. Climate Change. The objectives of the three Natural Resources and Environmental Governance operations in Ghana were to (i) ensure predictable and sustainable financing for the forest and wildlife sectors and effective forest law enforcement; (ii) improve mining and forestry sector revenue collection, management, and transparency; (iii) address social issues in forest and mining communities; and (iv) mainstream environment into economic growth through strategic environmental assessment (SEA), environmental impact assessment, and development of a climate change strategy. The SEA model has been extended to a number of sectors, including the nascent oil and gas sector, and to routine local government planning. SEA activities have resulted in more transparent and participatory planning, training of more than 100 central government and more than 440 district officials, more funding for environmental activities in sector plans, incorporation of SEA tools into the National Development Planning Commission planning guidelines, and mainstreaming of environment in government planning. Awareness of and activity in climate change has greatly increased. An overall climate change management framework has been developed as a step toward a comprehensive investment plan, and several key sectors—agriculture, forestry, transport, and energy—are moving ahead with their own climate strategies. The Bank rated the operation as moderately satisfactory. 44. Crisis Response. The Third Economic Governance and Recovery Grant to Togo, co-financed with CRW funds, supported the implementation of a manual of procedures for the General Finance Inspectorate and the operationalization of the Court of Accounts. Between 2009 and 2010, the number of audits carried out by the Inspectorate increased from 8 to 25, well above the target of 10 audits. Progress was also achieved in the area of budget preparation and monitoring, with the preparation of medium-term expenditure frameworks for the ministries of education, health, and agriculture, and the timely publication of quarterly budget execution reports in 2010. The objectives in the area of procurement—as measured by the percentage of procurement contracts reviewed by the General Procurement Department—were not achieved. The program also supported reforms in the cotton, phosphate, and energy sectors, with mixed, but on balance substantial, results. The Bank and IEG rated the operation as moderately satisfactory. 2. Results in IBRD Countries 45. Public Financial Management. Two programmatic Fiscal and Institutional DPOs to Guatemala supported a new government through fiscal reforms and actions to strengthen social safety nets and improve the transparency of public spending. However, a shift of political allegiance in the legislature stymied many of the reforms. In 2010 a series of natural disasters (landslides, a volcanic eruption, and the Tropical Storm Agatha) further diverted the government’s attention. Reacting to earlier corruption scandals, the public mood motivated the Congress to pass an Access to 42 The IEG ratings are presented whenever they are available. 22 Information Law, which was energetically implemented. The ICR concluded that the Access to Information Law and creation of information offices in most ministries could be transformative in the long run. The Bank and IEG rated the DPOs as moderately satisfactory. 46. Delivery of Basic Services. The Public Finance Management, Employment, and Private Sector Development Programmatic DPO to Poland supported reforms to expand early childhood education, increase equity in the provision of financial support to students enrolled in higher education, and increase efficiency in the health sector. As a result of the program, the primary school age was lowered from 6 to 5 years of age, while the enrollment rate for 3- to 5-year-olds increased from 45 percent in 2007 to 60 percent in 2010. The percentage of higher education students who submitted a loan application and met the eligibility criteria but failed to obtain a loan declined from 10.2 percent in 2007 to 9.4 percent in 2010, falling short of the target of 7.5 percent. In parallel, 24 hospitals were corporatized in 2010, compared to none in 2007, although the target of 60 was not achieved. According to IEG, strong analytic underpinnings and government commitment to the program played an important role in the success of the operation. The Bank and IEG rated the operation as satisfactory. 47. Financial Crisis Response. The Financial Sector DPO to Latvia was part of a financing package by bilateral and multilateral partners to help Latvia address the severe impact of the global crisis on its financial system. The DPO supported the government in its efforts to (i) ensure adequate capitalization of the banking sector and prepare liquidity contingency plans; (ii) strengthen the bank resolution framework by amending the Credit Institutional Law; (iii) improve the management of distressed assets, notably by simplifying mortgage foreclosure processes and promoting proper debt restructuring; and (iv) strengthen the supervisory and regulatory framework for the financial sector. The capital adequacy ratio of banks increased from 11.8 percent in 2008 to 15.2 percent in 2010. The trend of declining deposits was reversed, and deposit growth returned to positive levels by early 2010. The operation also effectively contributed to facilitating corporate rehabilitation and debt restructuring, with 40 cases handled by end-2009, compared to 22 cases in 2008. This operation was rated highly successful by the Bank and IEG. 48. Food crisis response. The objective of the Global Food Crisis Response DPO to Philippines was to support the government in addressing the challenges of high food prices by supporting measures to strengthen social protection and safety nets to protect poor and vulnerable households. Among other things, the policy actions helped (i) to lower domestic food prices and decrease volatility by reducing the opportunities for speculating in the global rice market; (ii) to scale up subsidies and transfers to poorer households in response to the food price crisis; (iii) to establish the National Social Welfare Protection Program; and (iv) to adopt and launch the CCT program. The lessons learned from this DPO (as per the ICR and IEG evaluation) are: (i) the chances of a successful outcome are enhanced by technically sound pre-existing and ongoing analytical work and the extensive and long-standing policy dialogue over the areas of reform; (ii) the goals of a stand- alone DPO, and the prior actions selected to achieve them, need to be realistic in terms of what can be achieved in a short time frame - this operation was carefully designed in this respect and avoided trying to address more complex long term issues which would likely have made it impossible to address the food and related social assistance emergencies in a timely manner; and (iii) specific investment loans and technical assistance in the policy areas covered by the DPO provided important synergies. Both the Bank and the IEG rated this DPO as highly satisfactory. 23 3. Falling Short of the Intended Results 49. The assessment shows that some DPOs fell short of their intended results. A review of DPOs that were rated moderately unsatisfactory or below by the Bank (ICRs) suggests that key elements that were missing in these cases included: (i) government ownership and leadership that remained strong throughout the reform process; (ii) reform momentum that was sustained through political cycles; (iii) reform programs that were attuned with the political economy of the country; (iv) intergovernmental coordination; (v) adequate implementation capacity; and (vi) strong analytical underpinnings for the reform program. Unsatisfactory DPO outcomes were often the result of country circumstances which changed and/or external shocks and other factors outside the control of the Bank (see Box 4). Box 4. Examples of DPOs that Fell Short of Achieving their Intended Results Senegal Energy Sector Recovery DPO. The primary objective of this multi-tranche operation was to ensure the sustained and sound long term development of electricity services and supply of petroleum products. The DPO supported reforms to (i) restore the financial health of the public utility; (ii) ensure that the electricity and the downstream hydrocarbon subsectors operate in an efficient and transparent way; and (iii) ensure the sustainable long term development of the energy sector. The primary objective of the DPO was not met. Only two out of the 14 expected results were achieved. Given the slow progress in the reforms the second tranche was cancelled. The Bank and IEG rated the operation unsatisfactory. According to the ICR it did not meet its objectives due to: (i) lack of realism of the program - the DPO was overambitious in expecting that the sector could be transformed in two years; (ii) lack of ownership - the Ministry of Energy and the public utility had little sense of ownership of the program as the official implementing agency was the Ministry of Economy and Finance; and (iii) lack of coordination among implementing agencies hampered the implementation of the program. Pakistan Poverty Reduction and Economic Support DPO. The development objectives of this stand-alone DPO were to (i) regain and maintain macroeconomic stability through measures that included increased tax revenue mobilization, adjustment of fuel prices and power tariffs; (ii) enhance competitiveness; and (iii) protecting the poor and vulnerable through improved targeting of safety nets and cash transfer programs. Overall performance of the operation was mixed. Reforms progressed in two areas, but stalled in the most important area of economic stabilization where outcomes were partial or could not be sustained. Program implementation was partly set back by lack of commitment to complete the reforms of the power sector on a timely basis and improve revenue mobilization on a sustainable basis. The Bank and IEG rated the operation moderately unsatisfactory. Key lessons include: (i) the Bank should help create a broad-based constituency for politically sensitive reforms; (ii) continuous dialogue and engagement with stakeholders beyond the central administration is necessary to facilitate passage of reforms; (iii) need to address capacity and institutional issues up front with continuous technical assistance and capacity building to ensure that the reforms are followed through; and (iv) reforms need to be underpinned by robust and timely analytical work based on reliable data. St. Lucia Economic and Social DPO. This stand-alone DPO supported reform to (i) improve the business environment and strengthen the financial sector; (ii) improve public sector governance and economic management; and (iii) improve the effectiveness and efficiency of social safety nets. There were significant shortcomings in the achievement of the objectives, partially due to Hurricane Tomas, which caused serious damage in the island and diverted government attention to recovery efforts. Despite the commitment of the Ministry of Finance, Planning and Economic Development to stay the course in the reform process, capacity limitations in the public sector, the backlog in processing legislation in Parliament, and the damages caused by Hurricane Tomas hampered progress. The Bank and IEG rated the operation moderately unsatisfactory. Key lessons include: (i) a programmatic operation could have had stronger development impact by setting the stage for a longer term reform program; (ii) in small states, policy based programs need to be relatively simple, realistic in scope and targets, given the often lack of implementation capacity in these countries; and (iii) close supervision, coupled with technical assistance, is instrumental for achieving results in a one-off DPO. Sources: Senegal ICRR for Energy Sector Recovery Development Policy Credit, Report ICR1833; Pakistan ICRR for Poverty Reduction and Economic Support Operation, Report ICR1782; and ST. Lucia ICRR for Economic and Social Development Policy Credit, Report ICR1978 24 4. The increased challenge of delivering results in Fragile and Conflict-Affected Situations 50. The challenges of delivering results in DPOs in the FCS are magnified due to the more difficult operating contexts. Factors that emerge as critical in determining the successes of DPOs in FCS include: (i) strong government commitment to the program; (ii) politically feasible programs; (iii) the undertaking of projects and technical assistance to address weaknesses in local capacity; (iv) realistic result frameworks; (v) informed risk taking; and (vi) strong analytical underpinnings (see Box 5). The case studies highlight that the factors that contribute to success in DPOs in FCS do not vary significantly from other DPOs. An important difference is that it is more difficult to ensure that the factors for success are present - for example, adequate capacity, strong analytical underpinnings when support is resumed after a long hiatus, a stable economy, the fine balance between supporting needed but politically difficult measures in an environment of political instability, and adequate country systems to monitor results. 51. These adverse initial conditions are compounded by situations of acute fragility, lack of security, post-crisis environments, and underlying social tensions, among others. There are also external circumstances that, in spite of government commitment and satisfactory Bank performance, make it difficult for DPOs to attain the expected results (see Box 6). In addition to the difficulty of attaining results it is also more challenging to sustain them. According to a recent World Bank study,43 the sustainability of PFM reforms, which represent the bulk of reforms in FCS, is challenged by “the heavy reliance on externally financed support, which combines policy conditions, extensive use of international consultants, and salary incentives for staff working in key areas.� On balance, DPOs in FCS carry high risks and potential high returns that need to be weighed up. Box 5. Example of DPOs in FCS that Achieved the Intended Results Sierra Leone Governance Reform and Growth DPOs – a series of three DPOs. The design of the program reflected several of the key factors highlighted subsequently in the 2011 World Development Report (WDR) regarding the transformation of institutions in FCS: a systematic program of phased capacity building and expanded accountability, anti-corruption measures (through actions/performance indicators for increased transparency of procurement) and decentralization for greater inclusion of the population, thus strengthening peace and security. Two additional features benefited the overall satisfactory rating of these DPOs: the undertaking of a programmatic approach, particularly as it included an assessment of progress before each successive operation; and the embedding of policy reforms in the government’s reform agenda, which instilled greater government ownership of the program, diminishing the risk of delays or shortcomings in achieving objectives. The Bank and IEG rated the series Satisfactory. Liberia Reengagement and Reform Support DPO. This stand-alone DPO supported the adoption of a new Public Financial Management Act as well as improvements in regulations and practices in the area of procurement. As a result, the value of noncompetitive public procurement declined from 80 percent in 2008 to 9.2 percent in the first three quarters of FY09/10, exceeding the program target of 20 percent. The DPO also supported improved budget preparation and execution, notably by giving the Comptroller General’s Office access to the Ministry of Finance’s system for accounting and reporting. As IEG emphasized, what was critical to the success of the operation was strong government commitment to a program that was designed to be politically feasible, helped by associated technical assistance to address weaknesses in local capacity. Both the Bank and IEG rated this DPO as Satisfactory. Sources: Sierra Leone ICRR for a series of Governance Reform and Growth Operations, Report ICR176; Liberia ICCR for Reengagement and Reform Support Program, Report ICR 1014 43 Public Financial Management Reforms in Post-Conflict Countries – Synthesis Report, World Bank. 25 52. There are also factors that, if not taken into account, can jeopardize the achievements of expected results in FCS. DPOs that did not achieve satisfactory results failed to recognize the length of time it takes to build institutions in FCS, and the potential for setbacks. Some operations overestimated the country’s capacity, as evidenced in the ‘lessons learned’ sections of many PDs. Others did not manage to formulate prior actions in FCS that were critical, realistic in scope and manageable. Moreover, when results frameworks were overly ambitious, DPOs did not achieve the expected results. Box 6. Examples of DPOs in FCS that Fell Short of Achieving the Intended Results Yemen Private Sector Growth and Social Protection DPO. The development objectives of this stand-alone operation were to: (i) foster private sector growth in the non-hydrocarbon sector; (ii) improve public financial management; and (iii) mitigate the impact of the fuel subsidy reduction on the poor. Implementation of the DPO was severely affected by the social and political unrest that broke out in Yemen in early 2011, within weeks after approval of the DPO. According to the ICR, the DPO could have acknowledged more explicitly the high political risk environment and that much of this risk could not be effectively mitigated. Also, the ICR considers that the DPO was ambitious in terms of its objectives, given the nature of its design as a stand-alone DPO. It envisioned the promotion of growth in the non-oil sector which necessarily involved a series of actions that would need to take place over time. In the context of a single tranche operation, only initial measures could be envisioned, leaving open the question of how these reforms would be sustained upon completion of the operation. By contrast, the measures in the areas of public finance management and reform of the cash transfer system were both reasonably well contained in the context of the program period. The Bank rated the operation unsatisfactory. Haiti Third Economic Governance Reform DPO, first of a series. The operation supported policy measures in three areas deemed critical for continued progress in economic governance and public financial management. In January, 2010, an earthquake hit Haiti, causing widespread damage and massive loss of life, and resulting in the disruption of economic, financial and governance activities. This represented a major setback for the overall country poverty reduction and economic management strategy. As a result, and despite the government’s commitment and satisfactory performance of the Bank, the development objectives were not attained. The ICR retains the following lessons: (i) the need for adopting a simple approach in FCS, particularly in a country with a high risk of occurrence of major natural disasters; (ii) countries that have experienced major natural disasters need strong institutional support to back up critical reforms aimed at restoring the state functions quickly, and (iii) close coordination among development partners within a simple, commonly agreed framework of reforms is important. The Bank rated the operation as moderately unsatisfactory. Sources: Yemen ICRR for the Private Sector Growth and Social Protection Policy DPO, Report ICR225; Haiti ICRR for the Third Economic Governance Reform Operation, Report ICR1990 53. DPOs in fragile contexts have had a spectrum of goals – from those targeting “quick wins� to those supporting longer-term institution building. The design of DPOs has been informed by the varying nature of fragility in FCS. Some countries experienced a post-conflict transition following a civil war, others have been affected by sudden but short-lived episodes of violence (for example, Kyrgyz Republic in 2010, Cote D’Ivoire in 2010-11). DPOs have assisted post-conflict countries to deliver quick wins with the potential for breaking cycles of violence. They have also helped FCS to support long term policy reforms. DPOs in FCS have tackled areas that are singled out as key by the WDR, that is, security, justice, and jobs. 26 Box 7. Examples of DPOs Across the Wide Fragility Spectrum Kyrgyz Republic Economic Recovery Support Operation. It supported the authorities’ aspiration to redress grievances in the areas affected by the 2010 conflict. Measures were put in place to protect benefits for the poor and provide social assistance to the conflict-affected population, including: (i) safeguarding social spending by ensuring essential social protection assistance; (ii) supporting the re-establishment of livelihoods through tax exemptions; and (iii) providing new shelters and housing. Clearly, these measures were geared at compensating the effects of the conflict on the population and restoring stability. No evaluation is available. Burundi Second and Third Economic Reform Support DPOs. These supported the country to tackle complex, long term policy reforms, and politically sensitive actions in the coffee sector. This was done through a series of prior actions aimed at introducing new regulation to enable the liberalization of the sector, including the abolition of the system of guaranteed prices and the contracting out of the marketing of fully washed coffee to a single foreign buyer; and the undertaking of a diagnostic of the sector. The Bank and IEG rated the series as satisfactory. C. The Quality of Results Frameworks 1. From Prior Actions to results 54. Most DPOs covered by the Retrospective had results frameworks of satisfactory quality (see Figure 20).44 Overall, 78 percent of operations had satisfactory results frameworks as measured by the standard that at least 70 percent of the prior actions had a results indicator in which (i) there was a clear causal link between the prior action and the result; (ii) the result was distinct from the prior action; (iii) the result had a results indicator; and (iv) the results indicator was precise.45 Figure 20. Quality of Results Frameworks Percentage 100 90 80 70 60 50 40 30 20 10 0 44 Analysis of the quality of the results framework is a complex exercise. Annex B provides details on the methodology used. 45 If the bar is raised so that at least 80 (90) percent fulfill the conditions described above the percentage of operations with satisfactory results frameworks falls to 62 (40) percent. 27 55. The quality of results frameworks Figure 21. Quality of Results Frameworks and varied by region and amount, especially for DPO Size very large operations. LCR had the most Percentage 90 operations with satisfactory results frameworks followed by ECA. IBRD results frameworks were more satisfactory than IDA 80 ones. The size of the operation may have had some relation with the quality of the results 70 framework, especially at the upper end, as suggested by Figure 21. This may reflect the 60 fact that countries recipient of larger DPOs (usually IBRD countries) may have better 50 All >$10m, >$50m, >$100m, >$500m monitoring and evaluation systems and that operations <=$50m <=$100m <$500m those operations may be subject to greater scrutiny. 56. Results frameworks set up in the context of JBS operations, with PAFs, were typically somewhat weaker. Approximately, over 80 percent of non-PAF-linked operations in AFR had satisfactory results frameworks, while about 72 percent of PAF-linked operations did (Figure 20). This seems to be the case despite the fact that several partners have a results focus rather than a focus on policy or institutional actions. In addition, stand-alone operations seem to have slightly better results frameworks. This is in spite of the advantage that when running programmatic series, more effort can be put in over a two- to five-year period to improve the logical links between the prior actions and the results, and to identify the baselines and targets. 2. Robustness of the Program 57. Just over half of the prior actions were policy and institutional reform actions.46 Among the regions, ECA had the largest proportion of policy and institutional actions (see Figure 22). FCS tended to have more preparation, process and implementation prior actions than non-FCS—which is understandable, given the nature of the reform process in some of these countries. The prior actions of operations prepared in the context of JBS (joint PAFs) tended to be more in the nature of implementation and processes than of policy and institutional actions compared to non-joint PAF operations. Many of these JBS operations were prepared in highly rigid contexts, including the requirement that all development partners endorse the same reform program. Thus development partners tended to cluster around the “lowest common denominator� or a minimal common agenda in terms of policy content (see Chapter V). 46 Policy actions refer to the adoption or enactment of laws, decrees, directives, amendments or regulations by parliament or congress; approval or establishment of rules, procedures, frameworks or draft laws and amendments by government; implementation actions refer to the implementation or operationalization of government programs, initiatives, strategies, pilots, procedures, rules, laws, codes, regulations, decrees, functions, units, capacity-building programs, and staffing and system requirements; process or design actions refer to the preparation of action plans, strategies, studies, surveys, reviews, timetables, methodologies and guidelines or operational manuals; and output-related actions refer to the achievement of measurable targets as the consequence of an action or a set of actions under the program. 28 Figure 22. Classification of Prior Actions Percentage Process and outcome Implementation Policy and institutional 100 80 60 40 20 0 AFR EAP ECA LCR MNA SAR FCS Non-FCS PAF, AFR Non-PAF, AFR 3. Quality of Results Indicators 58. The number of results indicators Figure 23. Average Number of Results Indicators per per operation has consistently declined Operation over time, reflecting greater strategic outcome indicators with baselines focus. In the previous Retrospective, the 35 average number of indicators per operation 30 33 was 28, with considerable variation (from 5 25 to 121). In the DPOs reviewed for this 20 Retrospective, the number of results 15 14 12 indicators varied somewhat less—from 4 10 to 60—with an average of 14 per operation 12 (see Figure 23). The reduction in the 5 number of indicators seems to reflect Bank 0 staff’s more realistic approach to FY06* FY07 FY08 FY09* FY10 FY11 FY12** *Fourth quarter data only. monitoring and evaluation (M&E) by ** Only data up to third quarter included. decreasing the burden on countries to collect and analyze data to monitor the Figure 24. Average Number of Indicators by Region progress of the program, and evaluate results indicators with baselines with targets results on completion. 25 20.7 20 59. Moreover, an important change 15.3 since the last Retrospective is the larger 15 13.4 13.7 13.4 11.3 number of results indicators that include 10 baseline and end-of-series targets. About 85 percent of indicators in DPOs in this 5 Retrospective had baselines and end-of- 0 series targets. In comparison, only 42 AFR EAP ECA LCR MNA SAR percent of the results indicators in the previous Retrospective had even a baseline. The reduction in the number of indicators reflects the more selective approach to choosing measurable results. 60. There was no significant variation in the number of indicators across several DPO classifications. The differences in the average number of indicators among regions are relatively small (see Figure 24), with the exception of SAR. Compared to stand-alone DPOs, programmatic 29 series tended to have more indicators, and a higher percentage of them had baselines and end-of- series targets. Operations in FCS had on average 12 results indicators. Given the weak M&E systems in FCS, this finding indicates that there is room to reduce the number of indicators in these countries and prepare more focused and realistic results frameworks. D. Assessment of M&E Arrangements 61. Nearly all PDs reviewed included a description of the country’s M&E systems and of the arrangements and responsibilities for M&E. While implementation and M&E are the borrower’s responsibility, Bank staff is responsible for assessing and monitoring the adequacy of the implementation and the country’s M&E arrangements, taking into consideration the borrower’s capacity.47 While most DPOs discussed the country’s M&E frameworks, the quality of this discussion varied. Some operations merely indicated the country’s implementing agency that would be responsible for M&E without providing details about arrangements, data sources, and frequencies. The First Community Living Standards DPO to Rwanda is a good practice example. It clearly indicates the country’s agency responsible for monitoring every indicator and provides details on the data sources for monitoring progress and for evaluation upon completion. 62. However, less than one-third of the DPOs included an assessment of the Figure 25. Assessment of M&E Arrangements strengths and weaknesses of the Percentage positive negative No Assessment 100% country’s M&E arrangements. In ECA, MNA, and LCR, in particular, few PDs 80% included an assessment of the country’s 60% M&E systems (see Figure 25). It is unclear whether the absence of such an assessment 40% means that there are no weaknesses in the 20% country’s M&E systems or that staff simply 0% omitted their assessments in the PD. There AFR EAP ECA LCR MNA SAR Total are, however, examples of DPOs that discuss the weaknesses of the M&E systems, in a more or less explicit fashion. Figure 26. Mitigation Measures Set Out in the DPO These include the Fourth Reengagement None and Reform Support Program to Liberia, the 11% Third DPO on Performance and Accountability of Social Sectors to the TA & Prior Dominican Republic, and the Second Action Restoring Equitable Growth and 21% Employment DPO to Turkey. 63. Measures were adopted to Prior TA only address weaknesses in countries’ M&E Action 64% only 4% systems when these were found. Approximately 90 percent of the operations 47 OP 8.60 states: “The borrower implements the development policy operation, monitors progress during implementation, and evaluates results on completion. Bank staff assess and monitor the adequacy of the arrangements by which the borrower will carry out these responsibilities, with due regard to the country’s capacity.� 30 set out measures to be taken by the Bank or other partners to address those weaknesses (see Figure 26): technical assistance, a prior action that explicitly dealt with the M&E issue at hand, or a combination of technical assistance and the use of a prior action. For example, the Third DPO on Performance and Accountability of Social Sectors to the Dominican Republic includes a prior action that strengthens M&E: “The Government has designed and mandated the establishment of an integral monitoring and evaluation system for purposes of sharing technical information.� 64. While there has been good progress in the quality of results frameworks and in the delivery of intended results, there are still problems of attribution, causality, and criticality that are hard to address. There are significant limitations in determining a DPO’s impact on the country’s development results as it is practically impossible to disentangle all the various elements that affect countries’ results. Moreover, it may be possible to make progress towards targeted results as set out in the results framework without addressing the most critical development issues in the country. These issues will be analyzed in the context of more in-depth impact evaluation for DPOs with the objective of answering questions on the long-term impact of DPOs. 31 IV. RISKS AND OPPORTUNITIES 66. This chapter reviews how the Bank has addressed risks and opportunities in the provision of DPOs, including macroeconomic, governance, and fiduciary risks. It also analyses how poverty and social impacts, and the environmental impact of the reforms supported have been assessed; and how implementation capacity and development effectiveness risks have been assessed and addressed in DPOs. A. Assessing Macroeconomic Risks 67. In providing development policy lending, the Bank requires that the member country have an adequate macroeconomic framework. The Bank’s assessment of the country’s macroeconomic framework focuses on the soundness of the policies pursued by the authorities, thus emphasizing the right program of macroeconomic policies rather than current outcomes or imbalances. The PD also identifies the downside risks to the economy and provides the Bank’s bottom-line assessment of whether the government’s macroeconomic policies are an appropriate basis for a DPO to proceed. In this assessment, the Bank takes into consideration the views of the IMF. DPOs to political subdivisions have the additional requirements that the subdivisions have an appropriate expenditure program, sustainable debt, and appropriate fiscal arrangements with the central government or with the applicable political subdivision. 1. Quality of the Discussion of Macroeconomic Risks 68. DPOs’ assessment of the macroeconomic policy framework has improved since the 2009 Development Policy Lending Retrospective. In line with the recommendations of that Retrospective, the macroeconomic assessments in the PDs reviewed gave greater attention to the outlook/forward-looking discussion of macroeconomic policies. PDs also consistently included three-year projections of key macroeconomic indicators. In addition, they typically included a clear statement regarding the Bank’s assessment of the adequacy of macroeconomic policy framework. 69. The fiscal analysis generally focused on fiscal aggregates; PDs generally contain insufficient analysis of expenditure composition and revenue composition. PDs consistently had an adequate discussion of aggregate fiscal accounts. On fiscal imbalances, the discussion tended to focus on headline fiscal balances without much consideration of structural fiscal balances. Only a few PDs contained a detailed discussion of public expenditure composition in its economic and functional classifications, and of the revenue structure, of the budget being supported. 70. The robustness of the fiscal outlook was often assessed through a formal debt sustainability analysis. In IDA-only countries, the debt sustainability analysis was typically based on the Bank-Fund Debt Sustainability Framework for low-income countries. In few cases, the analysis was based simply on the nominal debt stock, whereas a more pertinent indicator would have been the present value of future debt service obligations. The robustness of the fiscal analysis and the sensitivity analysis of public debt sustainability also depended on whether the definition of the public sector was sufficiently comprehensive to include local governments and public enterprises as appropriate, and on whether contingent liabilities were quantified. 32 71. Risks associated with external imbalances, monetary and exchange rate policies, and financial sector vulnerabilities received focused attention in operations in countries heavily affected by the global financial crisis. This was the case, in particular in ECA, where global developments exposed preexisting vulnerabilities. In these operations, the PDs adequately discussed the elevated risks associated with the pre-crisis demand boom, particularly in non- tradable sectors, fueled by unprecedented capital inflows and credit expansion. The analysis consistently examined the rise of external imbalances, in some cases also exacerbated by pro- cyclical fiscal policies. PDs discussed the problems posed by fixed exchange rate regimes in the context of absorbing shocks and rebalancing the economy during the crisis. Operations in ECA also featured a detailed assessment of the external financing requirements and, where relevant, included an analysis of the debt composition and external debt rollover risks. DPOs from other regions and sector-specific DPOs across the Bank did not contain the same depth of analysis of monetary and exchange rate policies, even when such a thorough discussion was warranted. 72. Stabilizing the financial sector was a major priority in the countries hardest hit by the global financial crisis. A focused analysis of the financial sector became a fairly standard feature of the macroeconomic assessments in operations under this Retrospective. As commercial banks dominate financial intermediation in most member countries, the analysis generally focused on assessing the stability of the banking sector, using the key indicators of liquidity, solvency, asset quality, and profitability. In some countries that experienced a deeper crisis, DPOs provided more in-depth assessments of financial sector risks. The analysis highlighted the vulnerabilities that had built up during the preceding boom years of exceptionally rapid credit growth financed by external borrowing. The PDs also linked the analysis to the vulnerabilities of households and businesses saddled with high indebtedness and large unhedged exposures. 73. Some unevenness remains in the comprehensiveness of the analysis and projections. In line with the recommendations of the last Retrospective, the tables with key macroeconomic indicators in PDs contained three-year projections. However, the detail on the macroeconomic indicators showcased in the tables varied significantly, with some operations providing too little detail for reviewers. The assessment of external debt sustainability was more prevalent in countries that suffered deeper recessions and presented higher macroeconomic vulnerabilities. However, it was, to some extent, disregarded in regions and countries that had not suffered large shocks in terms of GDP but had significant pressures on their external debt. In spite of the larger external imbalances and risks seen globally in recent years, only a few PDs contained tables highlighting the financing requirements and sources of the balance of payments, vulnerability ratios on foreign exchange reserves and external debt, and vulnerability ratios in the financial sector. 74. In sector-specific DPOs, the discussion of macroeconomic developments and outlook and its associated risks could have been more comprehensive. All standard operational provisions for development policy lending apply to sector-specific DPOs, including the requirement for an adequate macroeconomic policy framework.48 While PDs for sector-specific 48 Sector-specific describes DPOs that focus on one sector only. The Bank’s operational policy on DPOs neither refers to sector-specific operations nor makes separate provisions for such operations. Hence, sector-specific DPOs must adhere to all the standard operational provisions for development policy lending. 33 DPOs generally provided the required macroeconomic assessment, in several cases the analysis of the macroeconomic outlook could have been more detailed and comprehensive. 75. The macroeconomic discussion with regard to political subdivisions needs to be more systematic across countries. Sub-national DPOs provided the required review of the economic performance and outlook both at the national and sub-national levels.49 However, some operations did not present the expenditure and revenue programs in detail, or did not clearly indicate the appropriateness of the political subdivision’s expenditure program. A more thorough discussion and assessment of the intergovernmental fiscal arrangements in the country, including those affecting the political subdivision, or a reference to a separate study assessing these issues, was also missing in some cases. Sub-national DPOs would also need to incorporate consistently a debt sustainability analysis of the political subdivision.50 B. Weighing Governance Opportunities and Risks 76. While governance is a broad term, many operations—over 30 percent—identified weak governance as a risk. Countries with better governance are more able to foster economic opportunity, deliver services to the poor, and regulate factor and product markets—all things that are critical for the sustainability of the reforms supported by a DPO. Some of the DPOs were more specific and described the lack of accountability and corruption as risks. Governance risks were identified most commonly in PDs in AFR (35 percent of all AFR DPOs), followed by EAP and LAC (with 26 and 23 percent of the operations in the region, respectively). 77. This section looks at whether the Bank has factored in governance risks when deciding to extend a DPO. Using disbursement data from FY05-1151 and from the World Bank’s governance ratings in Cluster D of the 2010 Country Policy and Institutional Assessment (CPIAD),52 the review analyzes (i) whether better performers are associated with larger shares of development policy financing in their portfolio; and (ii) whether better performers receive a larger share of the total of development policy lending. It also assesses whether measures to address governance risks were incorporated in the design of the operation. The robustness of the results was tested with other indicators such as World Governance Indicators. 78. No absolute threshold is established as minimum for governance standards for DPOs. Instead, the policy prescribes a risk-based approach. OP 8.60 requires that teams take the country’s policy and institutional framework, including governance, into account in deciding 49 In Brazil, the PDs highlighted the Fiscal Responsibility Law, which institutionalized fiscal discipline at all levels of government by incorporating hard budget and borrowing constraints at the subnational level. 50 The debt sustainability requirement was added as part of the revisions, approved by the Board in February 2011, of OP 8.60 with regards to development policy operations for political subdivisions. 51 To provide sufficient historical perspective, this, as well as the fiduciary section, look at all investment loans and DPOs committed between FY05-11 in countries with a CPIA assessment. Arrears clearance operations were excluded. 52 The 2010 CPIA is the latest available. Cluster D, “Public Sector Management and Institutions,� comprises 5 questions: question 12 “Property Rights and Rule-based Governance,� question 13 “Quality of Budgetary and Financial Management,� question 14 “Efficiency of Revenue Mobilization Quality,� question 15 “Quality of Public Administration,� and question 16 “Transparency, Accountability, and Corruption in the Public Sector.� CPIAD is the average of the cluster, and therefore the rating can be a non-integer. 34 whether to proceed with a DPO, in determining the volume of the operation, and in designing the program focus and content.53 79. The analysis shows that the Bank is heavily weighting governance considerations in its decisions. Countries with stronger governance environments receive a larger share of Bank finance in development policy lending (Figure 27, right-hand panel). In countries with a CPIAD below 3.5,54 more than two-thirds of the portfolio is in the form of investment lending. As the CPIA increases, the share of development policy lending in the portfolio also increases. Countries with a CPIAD of 4-6 have on average 59 percent of their financing in the form of development policy lending. Figure 27. Commitments by CPIAD Rating Categories by Percentage 0.8 70 4 60 12.8 Investment Lending 50 21.2 2 - <2.5 40 2.5 - <3 30 3 - <3.5 3.5 - <4 20 Developmnent   Policy Lending 4-<6 10 61.2 0 2 - <2.5 2.5 - <3 3 - <3.5 3.5 - <4 4- 6 CPIAD 80. A result of this governance selectivity of DPOs is that a much larger amount of DPO commitments went to countries with stronger governance environments. Between 2005 and 2011 only 4.8 percent of total development policy lending was provided to countries Figure 28. Quality of Governance and the Average with a CPIAD score of less than 3.0, while Number of Prior Actions in Governance 74 percent went to countries with a CPIAD 8 greater than 3.5 (see Figure 27, left-hand 7 6 panel). Using an alternative measure of 5 country governance, the proportion of total 4 development policy lending provided in the 3 same period to countries in the lower range 2 1 of the Worldwide Governance Indicators 55 0 (countries with governance scores from -2.5 2.0-<2.5 2.5-<3.0 3.0-<3.5 3.5-<4.0 4.0-6.0 56 to -1.0) was 8.2 percent. CPIAD 53 OP 8.60 states: “The Bank’s decision to extend development policy lending is based on an assessment of the country’s policy and institutional framework—including the country’s economic situation, governance, environmental/natural resource management, and poverty and social aspects.� 54 CPIA rating ranges from 1 to 6. Rating 1 is the lowest and 6 is the highest. 55 http://www.govindicators.org 56 Control of Corruption was the indicator used (1 out of 6). 35 81. When governance was poorer, the policy content of the DPOs had a notably stronger focus on governance reforms. Countries with low governance scores that did obtain DPOs had more governance-related prior actions than better-governed countries (see Figure 28). Thus, besides taking governance risks into account when deciding to extend DPOs, the Bank has also considered them in the design of the operation. Overall, DPOs are increasingly supporting prior actions that directly address more open government. Noticeable in recent years is the appearance of new types of prior actions that directly address government transparency, with the aim of creating more space for non-state actors to hold the executive accountable for performance—for example, measures like the enactment of access to information laws, and, for countries endowed with natural resources, seeking membership in such international processes as the Extractive Industries Transparency Initiative (EITI). DPOs have supported such measures in, for example, Ghana, Mali, Mongolia, and Togo. C. Weighing Fiduciary Opportunities and Risks 82. This section deepens the analysis by focusing on fiduciary risks, a subcomponent of the governance cluster. Just as with governance, there are no thresholds for “minimum� PFM standards for DPOs prescribed in the policy. Instead, once again, the policy prescribes a risk- based approach. Using CPIA indicator 13 (CPIA13 – “Quality of Budgetary and Financial Management�) as a measure of fiduciary risks, this section assesses whether the Bank took into account the fiduciary risks of providing fungible budget support, in the form of DPOs, to countries with widely different public financial management (PFM) capacities. To check the robustness of the results, the data were also compared against the Public Expenditure and Financial Accountability (PEFA) assessment scores for selected countries. 83. Countries with better fiduciary arrangements receive more Bank finance in development policy lending. Countries with a CPIA13 of 2 or less get practically no development policy lending (Figure 29, right-hand panel) but receive almost all their assistance via investment loans. Countries with a CPIA13 of 2.5 get 17 percent of their Bank financing in the form of development policy lending, and so on up the spectrum to the best performers (CPIA13 of 4 or better) which receive 39 percent of their Bank support in the form of development policy loans. This indicates that the Bank takes into account the fiduciary environment by limiting the number of DPOs extended to countries with high fiduciary risks, curtailing their size, and ensuring that the bulk of Bank resources made available to those countries comes in the form of investment operations. A corollary of this selectivity is that only 9 percent of all development policy lending from 2005 to 2011 went to countries with a CPIA13 score of 3.0 or lower, and 90 percent went to countries with a CPIA13 score of 3.0 or above (Figure 29, left-hand panel).57 57 In IDA countries this is partly explained by the Performance-based Allocation formula, which factors in countries’ CPIA ratings. 36 Figure 29. Commitments by CPIA13 Rating Categories by Percentage 1.3 16 7.8 2 - <2.5 2.5 - <3 26.2 3 - <3.5 3.5 - <4 4-<6 48.7 84. Similar results are found when the Public Expenditure and Financial Accountability (PEFA)58 scoring approach is used instead of the CPIA. The share of development policy commitments rises steadily as the PEFA score improves. Development policy lending represents 29 percent of commitments when the average PEFA is D or D+, 31 percent when the average PEFA is C, 33 percent when the average PEFA is C+, and 48 percent when the average PEFA is B or better. In dollar terms, only a small part of the Bank’s development policy lending goes to countries with low average PEFA scores (see Table 1). Only 6.9 percent of all development policy lending went to countries with average PEFA scores of D+ or less. Table 1. PEFA Scores and Commitments by Instrument Quality of public finance management, as proxied by the country’s total score on the PEFA Amounts committed D to D+ C C+ B to B+ Total Investment lending (US$ billions) 6.2 17.3 34.2 15.7 73.3 a 2.7 12.1 11.7 12.4 38.9 Development policy lending (US$ billions) Shares of total commitments b 71.0 69.0 66.6 51.7 Investment lending (%) b 29.0 31.0 33.4 48.3 Development policy lending (%) Total (%) 100 100 100 100 D to D+ C C+ B to B+ Total DPO commitments by PEFA scores (%) 6.9 31.2 30.0 31.9 100 a Excluding arrears clearance operations; b Means of country-specific ratios of DPO (or Investment Lending) commitments to total commitments. 85. The extent of the emphasis on PFM reforms is inversely associated with the quality of the PFM systems. Operations focus more intensely on PFM when countries score poorly on question 13 of the CPIA: the average number of prior actions dealing with PFM falls as the CPIA13 score improves (see Figure 30). In weaker fiduciary environments, most of these prior actions address the basics of PFM, such as generating timely fiscal reports during the year, providing 58 www.pefa.org. 37 accurate accounts and financial Figure 30. PFM Quality and the Average Number of PFM statements within a few months after the Prior Actions end of the year, and auditing by an 4 independent supreme audit institution. 3.5 Prior actions in such countries typically 3 include reforms to the public 2.5 2 procurement system, such as the 1.5 enactment of a public procurement law, 1 the issuance of implementation 0.5 regulations and standard bidding 0 documents, and the establishment of an 2.5-3.0 3.5 4 4.5-6.0 CPIA13 oversight/appeals body. 86. Between 2005 and 2010, only nine countries with particularly poor fiduciary systems (CPIA13 of 2.5 or lower) and associated high risks received DPOs as returns were expected to outweigh the risks. Some of these operations were designed to clear arrears and/or support strong economic reforms. It should be noted that arrears clearance operations while being prepared following all the requirements under OP 8.60, are substantially ring-fenced, as the financial arrangements for such operations effectively earmark the funds for arrears clearance. The Bank may extend DPOs to such countries as a key element in renewing relations with a country (in the case of arrears clearance), or to support the country in very special circumstances, such as the end of a political conflict (see Box 9, the examples of Central African Republic, Côte d’Ivoire and Liberia). In either case, a strong program of reforms is required, and the risks are clearly spelled out and are met with enhanced internal scrutiny, frequently a dedicated Operations Committee meeting (for an overview of the development policy review process, see Box 8). Box 8. DPL Review Processes – Recognized by IAD as a Key Contributor to Quality and Effectiveness All DPOs are subject to mandatory Bank wide concept note review and corporate review by the Regional Operations Committee (usually chaired by the Regional Vice President) or the Operations Committee (chaired by a Managing Director). The Operations Committee meets to review operations that: i) pose a high risk for the institution due to their characteristics or context; ii) involve an unprecedented or potentially controversial waiver to Bank policy; or iii) are considered to have exceptionally innovative features in their design or content. On conclusion of the review, the Chair of the Regional Operations Committee or Operations Committee specifies further actions, and, as appropriate, authorizes appraisal and negotiations. An audit in June 2010 of Bank processes for managing DPOs, conducted by the Bank’s Internal Audit Department (IAD), found that governance, risk management and control processes over the Bank’s management of DPOs are “satisfactory,� the highest rating category used by IAD. Furthermore, IAD indicated that the Country Economics Team in OPCS served an important corporate role by framing operational policy, which is approved by the Executive Board; encouraging a focus on measurable results; providing training and customized support; encouraging consistency in the application of policies and procedures; providing guidance to regions on program packages; suggesting factors that could impact the achievement of program objectives; and providing other support· as requested. Its unique position as a reviewer of all DPOs processed by the Bank provides OPCS with an opportunity to cross fertilize good practices and to continuously improve the relevant policies, procedures, and practices. 87. In six of the nine cases—Central African Republic, Comoros, Côte d’Ivoire, Guinea- Bissau, Liberia, and Togo—the DPOs satisfactorily achieved their intended objectives. Relations with the country were normalized, paving the way for a resumption of investment lending, and/or substantive improvements were made in PFM, notwithstanding the continued political 38 turmoil. In the remaining three cases—Democratic Republic of Congo, Iraq, and Madagascar (see Box 9)— the Bank decided to discontinue development policy lending as it recognized that its fast disbursing instrument was not the best suited to continue supporting the countries. This demonstrates that the Bank has an ability to take risks while keeping the flexibility to adapt to changing circumstances. Box 9. Taking Calculated Risks in Weak PFM Environments Central African Republic. Central African Republic went through a period of sustained conflict from mid-1990s to 2003. A new constitution was adopted in 2004, and parliamentary and presidential elections were held in 2005. In 2006 the Bank extended an arrears clearance grant to the Central African Republic of $82 million in order to reengage and build institutions. A key focus was public finance management – limiting and monitoring the use of cash advances, reducing off-budget revenues, and improving controls. The operation classified as high risk was rated “satisfactory� by IEG and the Bank. Another grant followed in 2008 focusing on budget execution procedures, a new accounting classification, steps towards a single treasury account, and other governance-related items. This operation was also rated satisfactory by IEG. A grant in 2009 stressed competitive procurement, tax and customs reforms, and external audit. The operation was rated moderately satisfactory by IEG and the Bank. Finally, a grant in 2010 sought to limit tax exemptions, clear arrears to domestic suppliers, and introduce commitment plans and procurement plans for priority ministries. Côte d’Ivoire. Following a period of political turmoil and civil war, a DPO was granted in 2008 ($308m) for arrears clearance. The operation was subjected to Operations Committee scrutiny. It supported reforms to improve governance and efficiency in public expenditure management and in the cocoa, energy and financial sectors. The Bank and IEG rated the DPO as satisfactory. Two more DPOs (2009, 2010) included PFM reforms (budgeting, budget transparency, budget execution reporting, treasury improvements, extension of the electronic financial management system, and procurement). The PD indicated that were significant fiduciary risk. The 2010 operation signaled “high risk, high-profile and potentially high-gain.� A PEMFAR produced in 2008 supported the design of the program. Technical assistance was provided by the LICUS Trust Fund, and a Bank-financed technical assistance operation focused on PFM. Dem. Rep. of the Congo. The 2005 DPO followed a period of political turmoil, and was the third DPO since 2002. The PD indicated that was a “high risk� operation. The program content included PFM reforms. IEG and the Bank rated the outcomes of this operation as “unsatisfactory.� No further DPOs were granted. The Bank continued supporting the country with investment loans, technical assistance, and advisory products. Iraq. In 2010 the Bank approved a programmatic series of which the first operation was $250 million. The operation was subject to Operations Committee scrutiny. Fiduciary risks were recognized as high and difficult to mitigate because of associated political, economic, and security risks. Program content included procurement, chart of accounts, budget planning and auditing. IEG and the Bank rated the operation as moderately satisfactory. The DPO was planned as a series of two operations, with the second contingent on government commitment and financing need. With the steep rise in international oil prices since 2010, and a large budget surplus in 2011, there was no financing need, so the second DPO did not proceed. Liberia. In 2007 the Bank extended a ($430 million) DPO to Liberia, a post-conflict country, for arrears clearance, to normalize relations with the Bank and pave the way for access to debt relief. Three more DPOs followed in 2009, 2010 and 2011. All DPOs supported PFM reforms informed by a PEMFAR conducted in 2008: cash management, budgeting, and taxation. The arrears clearance operation noted that there were significant country and fiduciary risks. IEG and the Bank rated the outcomes of the 2007 and 2009 operations as satisfactory. Among the outcomes was a reduction of non-competitive procurement from 80 percent in 2008 to 9 percent in 2009. Madagascar. Five DPOs (PRSC1-5 and a supplemental) from 2004 to 2008 supported important reforms in the area of PFM, including internal controls and procurement. Analytic underpinnings were particularly strong: PEFA assessments (2006 and 2008), public expenditure reviews (2005 and 2007), and government-led procurement audits (2007-2008). The series of DPOs was discontinued in 2009 due to political turmoil. IEG rated the PRSC1-3 series as moderately unsatisfactory and the Bank rated it as moderately satisfactory. 39 1. Managing Risk and Opportunities in Fragile and Conflict-Affected Situations 88. Risks are heightened in FCS. There is usually a higher risk that the macroeconomic framework could go off track, that the political commitment to reform and commitment to tackle weak economic governance could wane, and that low technical capacity could jeopardize the sustainability of reforms. There are also other risks stemming from the weak nature of institutions and the difficulty of engaging in sustained policy dialogue, including: (i) the low capacity of the administration to follow up on the technical aspects of reforms; (ii) the reluctance of civil society to engage in dialogue due to the unstable political climate; (iii) shifting political alliances, leading to changes in the ‘champions for reforms’; and (iv) the hesitation of governments to agree to a set of policies if support from key stakeholders is suddenly withdrawn. In some cases technical assistance accompanying a DPO can improve capacity and mitigate some of these risks. 89. While it is critical to be cognizant of the risks and candidly discuss them in the PD, the rationale for engagement in FCS is compelling. First, taking calculated risks in DPOs has been essential to the Bank’s development effectiveness in FCS. Second, the risks of inaction (or slow action) can have significantly adverse consequences. While the concern of protecting the use of resources channeled through DPOs continues to be of paramount importance from an integrity standpoint, the true spectrum of risk also includes failure in human and institutional outcomes and risks to peace and stability if decisions are taken not to provide budget support or suspend operations. The Guinea DPO presents a frank discussion of such risks. It indicates that there are substantial risks to the operation, including macroeconomic, security, fiduciary, implementation and political economy risks. It maintains, however, that the potentially high benefits of engaging would outweigh the very high costs of inaction. 90. Improvement of the macroeconomic situation over the medium term should be achieved in tandem with addressing the underlying cause of fragility in the short run, so as to avoid a relapse into conflict. In the short term, there may be an urgent need to meet peace building- related expenditures (including, demobilization and reintegration of soldiers, labor work programs) or to deliver tangible benefits to the population, particularly when it has been affected by violence and conflict. For example, the 2011 First Economic Recovery Support Operation in Kyrgyz Republic supported measures to redress grievances in the areas affected by conflict in 2010, despite fiscal pressure. Those measures included benefits for the poor and social assistance to the conflict-affected population, including safeguarding social spending, and distributing social compensation. Overall, public expenditure increased from 36 to 38 percent of GDP in 2010 as a result of crisis-related expenditures. 91. There are measures that to some extent can mitigate fiduciary risks. These include depositing the proceeds of Bank support into a dedicated account and auditing that account, and using external auditors, among others. In the West Bank and Gaza, the second, third and fourth Palestinian Reform and Development Plan DPOs highlighted the high fiduciary risk in the PFM system. The Palestinian Authority was asked to hire an independent external auditor to perform one- time audits of the grants’ deposit account at the Ministry of Finance. The audits would comply with international standards and be submitted to the Bank within six months from the release of the single tranche payment. 40 2. Review of the Fiduciary Risks section of DPOs 92. OP 8.60 provides guidance for reviewing fiduciary arrangements in DPOs. The Bank focuses on the borrower’s overall use of foreign exchange and budget resources; it typically disburses the DPO proceeds into an account that forms part of the country’s official foreign exchange reserves, normally held by the central bank; and it reviews the IMF’s most recent assessment of the borrower’s central bank and its analysis of the supporting PFM system. When the assessment suggests that the control environment of the central bank is satisfactory, or reveals issues for which the borrower has agreed to take remedial actions that are monitored by the IMF, the Bank takes no further action. Overall, when the general PFM system is weak, or when there are material weaknesses in the central bank’s operations, additional measures are required. 93. All DPOs in this cohort59 provided sufficient information to assess the Figure 31. Ratings of the Fiduciary Risks fiduciary risk. Many were explicit about # Operations the risk rating, although that is not required 140 by OP 8.60 (see Figure 31).60 While the 120 depth of the analysis varied significantly, 100 all PDs included a discussion of the three 80 components that constitute fiduciary risk: 60 the foreign exchange control environment, 40 the disbursement and audit procedures, and 20 the health of the country’s financial systems. Overall, the analysis of fiduciary 0 Low Moderate Signif icant High risk has improved since the last Retrospective.  Foreign exchange control environment. PDs reported on IMF foreign exchange control environment assessments61 as one component of the fiduciary risk assessment. While only one country in three had had such assessments in the last three years, the IMF regularly updates these assessments as a regular part of country monitoring activities. When the IMF’s assessments62 were not available because the countries did not have an active IMF program, the Bank confirmed the adequacy of the foreign 59 DPOs approved between FY09 Q3 and FY12 Q2. 60 Approximately 75 percent of all DPOs specifically identified the level of risk. For example, the second Economic Recovery and Governance DPO to Togo indicated that the operation presented significant country and fiduciary risks; the second Economic Governance and Reform DPO to Guinea-Bissau reported that the weaknesses in PFM were pervasive, affecting virtually all the system’s components; and the third Economic Governance Reform Operation to Haiti observed that, despite the advances in PFM, the overall fiduciary environment remained risky. 61 The IMF assessments are based on five pillars: (i) an independent external audit mechanism to audit the central bank’s financial statements; (ii) an appropriate legal structure for the central bank that ensures its autonomy, transparency, and accountability; (iii) a sound financial reporting framework that ensures the provision of financial information to central bank management and external parties; (iv) a modern internal audit function; and (v) a sound system of internal controls governing the central bank’s operations. 62 The IMF shares the Safeguards Assessment report with the Bank subject to certain confidentiality criteria and the consent of the central bank. To date three central banks have withheld consent for such sharing. 41 exchange control environment in other ways—for example, by examining the annual audit reports and financial statements of the central bank and the Management Letter issued by the auditors.  Disbursement and audit arrangements. The majority of DPOs fully met all of the requirements for sound disbursement and audit arrangements. Additional measures may be needed when the Bank has inadequate knowledge of the control environment in the recipient’s central bank or the PFM system, when audit reports indicate weaknesses in central bank or PFM, when there is a significant risk that the proceeds of Bank operations could be used for ineligible expenditures, and when there is a strong development rationale for tracking the proceeds of Bank support. A few DPOs included such measures (for example, the West Bank and Gaza DPOs, as discussed above, and the fourth and fifth DPOs for Tajikistan).  Health of PFM systems. While all fiduciary risk sections discussed PFM systems, the coverage varied considerably. DPOs with PFM prior actions were generally more thorough and comprehensive than DPOs that did not support PFM reforms. A good practice example is the first Financial Sector DPO for Montenegro which rated the PFM system risk as significant; identified accounting, internal audit capacity, external audit, and oversight as areas of weakness; and described the government’s planned mitigation actions. D. Assessing Poverty and Social Impacts 94. OP 8.60 requires the Bank to determine whether specific country policies supported by a DPO are likely to have significant poverty and social consequences, especially for poor people and vulnerable groups. When such effects are likely, the PD is expected to summarize the analytic knowledge of these effects, identify who will be affected and how they will be affected, and discuss the borrower’s systems for reducing adverse effects and enhancing positive ones.63 It is expected to discuss how the benefits and opportunities the reforms would deliver in the medium term and/or to a large segment of the population would outweigh the risks of adverse social impacts in the short-term. For the later, the DPO is also expected to discuss mitigation measures to minimize the adverse effects. 95. All prior actions supported by development policy lending are expected to contribute to poverty reduction in the medium and long term. Nevertheless, in the short term most prior actions have limited direct distributional impacts. This is because a large number of prior actions relate to reforms in the areas of public sector governance, finance, private sector development, and human development, and they typically have gestation periods of a few years as laws are passed, regulations are developed, institutions are changed, and implementation is 63 Since the implementation of OP 8.60, poverty and social impact analyses have not only been linked to DPOs but also occur as stand-alone products, inputs into larger pieces of analytic work, or technical assistance projects. In many cases, such analyses play a key role as “upstream� inputs into the design of a project, the Bank’s policy dialogue, or sector strategies, even though they are not linked to a specific DPO. Thus the retrospective takes into account an important but narrow slice of all poverty and social impact analysis. 42 rolled out.64 When considering the direct and Figure 32. Potential Poverty and Social Effects of the short-terms effects only, most prior actions Prior Actions (75 percent) were assessed likely to have Possibly neutral distributional impacts.65 negative 8% Positive Approximately 17 percent were considered to 17% have direct positive distributional effects in Neutral 75% the short term, while 8 percent were found to potentially have direct negative poverty and social consequences, in the short term or on certain groups (see Figure 32). 96. Preparation of some form of poverty and social impact analysis to inform DPOs of the potential adverse short-term effects has increased significantly since FY07, despite a fall in FY09. Approximately 71 percent of the prior actions of the Retrospective that could possibly have direct adverse poverty and social consequences were assessed to be underpinned by some form of poverty and social impact analysis.66 This represents a significant improvement compared to previous Retrospectives (29 percent in the 2006 Retrospective and 64 percent in the 2009 Retrospective). The PDs identified the vulnerable groups that were likely to be affected and discussed the borrower’s systems to mitigate the negative effects and enhance positive ones as required by OP 8.60. Of the PDs containing poverty and social impact analysis, approximately 70 percent were found to have prior actions with likely positive or neutral distributional effects, and 37 percent included mitigating measures in the design of reforms. Box 10. Approach of the Poverty and Social Impact Analysis Identifying the potential distributional consequences of each prior action is complex because the distributional impact of an action is shaped not only by its design, but also by its implementation and the economic, social and political context in which the action is carried out. This context and the ultimate distributional impact are difficult to discern with certainty from the information provided in the PD. As a result, and in line with previous Retrospectives, this review considers only the direct and short-term effects of each prior action. When prior actions were considered to have potential distributional effects, the PD was reviewed to verify whether it (a) identified the vulnerable groups that could potentially be adversely affected by the reforms being supported by the operation; (b) summarized analytic work by the Bank, the borrower, or others on the possible adverse effects of the reforms being supported; and (c) discussed the borrower's systems to reduce such effects and to enhance positive ones. 97. The dip in the percentage of completion of poverty and social impact analysis in FY09 was short lived and a steady and positive trend has emerged since then (see Figure 64 For example, the preparation of annual financial reports and audits for higher education institutions (second Vietnam Higher Education DPO) or the creation of an Interagency Committee to coordinate policy actions in the health, education, and nutrition sectors (Dominican Republic second Performance and Accountability Social Sectors DPO). 65 From a total of 2,186 prior actions. For the methodology please see Box 10 and Annex C. 66 Although a formal stand-alone piece was not required for meeting the requirement of containing a PSIA, objective evidence to support the distributional analysis was provided. In line with previous retrospectives, when PDs claimed potential distributional impacts without offering evidence to support the claim, the associated prior actions were not counted as having an underlying PSIA. See Annex C. 43 33). Measures to support fiscal consolidation in the context of the global economic crisis led to a large increase in the number of prior actions with likely negative effects and thus a corresponding increase in the number of poverty and social impact analyses required. Because of the urgency of the crisis and the constraints of the Bank’s budget, it was difficult to carry out all the analysis that was needed.67 In the period since the crisis, DPOs have increasingly been informed by poverty and social impact analysis. The percentage of prior actions with poverty and social impact analysis when prior actions were anticipated to have potentially adverse distributional impacts, increased to 74 and 83 percent in FY11 and FY12 (up to Q3), suggesting that greater attention is being paid to the analysis of the likely effects of the prior actions. Figure 33. Poverty and Social Impact Analysis (PSIA) Completed, when needed, FY07-12Q3 Prior actions Percentage 1000 90 900 80 800 70 700 60 600 50 500 40 400 30 300 200 20 100 10 0 0 FY07 FY08 FY09 FY10 FY11 FY12 (up to Q3) Prior actions with likely negative effects Prior Actions PSIA PSIA undertaking completed when needed (right axis) 98. There were noticeable differences across regions in terms of the likely poverty and social effects of the reforms supported. Policy reforms with likely significant negative effects were most frequent in AFR, ECA, and LCR, partly reflecting structural and fiscal consolidation reforms during the period under review (Figure 34). For instance, a number of DPOs aimed at fiscal consolidation at the national (Poland – Third Development Policy Loan) and subnational levels (Brazil- Rio de Janeiro State, Municipal Fiscal Consolidation DPO). 99. On average, AFR produced the lowest share of needed poverty and social impact analysis (see Figure 34). However, there was a remarkable improvement in 2011 (and through 2012), when AFR undertook 20 poverty and social impact analyses (77 percent of those needed that year), surpassing the other regions. In FCS, only 64 percent of the needed poverty and social impact analysis was undertaken, slightly below the overall average. This may reflect greater capacity constraints and less relevant analytic knowledge of the potential distributional effects. 67 The average time to process DPOs during FY01-08 was 8.2 months and 7.6 months for IBRD and IDA operations, respectively. During FY09-10, the average fell to 6.1 and 5.9 months for IBRD and IDA operations, respectively. 44 Figure 34. Poverty and Social Impact Analysis (PSIA) Completed, when needed, by Year and Region Prior actions PSIA done No PSIA Total PSIA needed 30 26 25 21 21 20 19 15 11 11 10 20 9 9 8 8 7 17 7 6 14 6 13 5 5 9 3 7 7 6 6 4 1 1 4 5 5 3 3 3 0 1 1 FY09 FY10 FY11 FY12 FY09 FY10 FY11 FY12 FY09 FY10 FY11 FY12 FY09 FY10 FY11 FY12 FY09 FY10 FY11 FY12 FY09 FY10 FY11 AFR EAP ECA LCR MNA SAR Note: Data for 2009 refer to the last quarter of the FY. Data for 2012 refer to the first 3 quarters of the FY. 100. In future, it will be important to conduct more upstream analysis of poverty/social impacts, to enhance readiness, when opportunities present themselves, to move forward quickly when needed. The 2006 and 2009 Retrospectives stressed the importance of conducting upstream poverty and social impact analysis, and many measures have been put in place to promote such analysis—for example, new training and tools for staff on how to use poverty and social impact analysis to inform DPOs, and the establishment of a Multi-Donor Trust Fund (MDTF)68 to support poverty and social impact analysis in the Bank and build countries’ capacity to conduct their own analysis. As a result, there are more examples of upstream consideration of poverty and social impacts among the operations reviewed for this Retrospective compared to previous ones (see Box 11). However, greater efforts are needed to more consistently assess the distributional impacts of prior actions supported by DPOs. Although financing for PSIA work has been supplemented by the MDTF, it is critical to ensure that there is adequate Bank budget for poverty and social impact analysis, to sustain the increasing trend, especially when the MDTF ends. 68 The MDTF, supported by Germany, the Netherlands, Norway, Switzerland, and UK, provides approximately $20 million over a three-year period (recently extended for two more years) to support the Bank’s work on the distributional analysis of policy impacts. The activities being supported reflect the diversity of analysis in terms of type of products, sectors, policy focus, and analytic methods adopted, consistent with the mainstreaming objective of the MDTF. More than 100 poverty and social impact analyses have been approved under the MDTF as of June 2012, spanning a range of themes, analytic designs, and outputs. 45 Box 11. Upstream Consideration of Poverty and Social Impacts Dominican Republic (Second Development Policy Loan on the Performance and Accountability of Social Sectors). The PSIA evaluated the expected impacts and potential risks of overhauling the Solidarity Program, and substantiated the expected positive impacts of the proposed redesign. This client-driven mixed-method PSIAa was a key input for the triggers of the third DPO in the series. Vietnam. A two-phase PSIA was conducted in support of a DPO series on climate change. The first phase analyzed the impacts of climate change on the poor in water resource management and the institutional structures being used to implement climate change policies. The second phase examined the potential poverty and distributional impacts of different structures and funding mechanisms for climate change policy implementation, to help the government of Vietnam consider which structures would allow the most pro-poor spending that ultimately would lead to the most pro-poor outcomes. ________ a Client-driven mixed-method PSIA refers to PSIA that uses a mix of quantitative and qualitative techniques and is requested by the country to help design policies that best address its needs. E. Assessing Environmental Impacts 101. OP 8.60 requires Bank staff to determine whether specific policy actions to be supported by the operation are likely to cause significant effects on the country’s environment, forests, and other natural resources. For policies with likely significant effects, the Bank discusses in the PD the borrower’s systems for reducing such adverse effects and enhancing positive ones, drawing on relevant country-level or sectoral environment analysis. If there are significant gaps in the analysis or shortcomings in the country’s system or capacity, the PD should describe how these shortcomings would be addressed before or during program implementation. The Retrospective examined the extent to which the PDs have addressed the environmental consequences of the reforms supported by DPOs (see Annex D for the methodology). The Retrospective reviewed all prior actions and assessed whether they could potentially have positive, negative, or neutral effects. Figure 35. Potential Effects of Prior Actions on the Environment, Forests, and Other Natural Resources 102. Of the prior actions reviewed, Possibly positive Possibly negative approximately 90 percent69 were assessed to significan Significant t positive positive 3.1% 0.3% be environment-neutral.70 Approximately 7.6 effect, effect 7.6% Neutral percent would be likely to have significant 7.6% 89% positive effects on the environment, and a further 3.1 percent would possibly be positive. About 0.3 percent of the prior actions were considered to have possible, but not substantial, negative effects (see Figure 35). Prior actions with likely positive impacts were mostly found in DPOs focused on the thematic area of Environment and Natural Resource Management. 69 This was a high level assessment conducted by staff not involved in the preparation and implementation of the operations. This exercise was not intended to reach definitive assessments on individual operations or prior actions. Based on the information contained in the Program Document only, it intended to provide an overall gauge of the Bank’s performance in this aspect of DPO preparation. For the methodology please see Annex D. 70 The 2009 Development Policy Lending Retrospective reported a similar finding. 46 103. However, the depth of the discussion of the analytical knowledge of the potential adverse effects varies across DPOs. Nearly 30 percent of the DPOs with prior actions that could potentially have negative consequences discuss the relevant analytical knowledge of such effects and the country’s capacity to with mitigate them. Another nearly 30 percent include a limited discussion of the analytical work, although in some cases there is a discussion on investment projects that are expected to mitigate potential negative effects. In some of these DPOs, there is also planned or ongoing support for institutional capacity building to minimize potential negative effects. In the remaining (approximately 40 percent) there was no discussion of the analytical work or of the countries’ capacity to mitigate potential effects. F. Assessing and Addressing Implementation Capacity Risks 104. Most reforms rely heavily on technical capacity for implementation. Even where there is commitment to reform at the highest levels, limited capacity in program implementation and monitoring and evaluation poses risks to the implementation of the reforms supported by a DPO and to the sustainability of the reforms and the results they achieve. 105. Over 60 percent of the operations reviewed identified implementation capacity as a risk.71 This was the case in the majority of operations in AFR (80 percent), SAR (88 percent), and 35- 40 percent in the other regions. As expected, most operations in FCS (75 percent) identified implementation capacity as a risk. Among the risks most commonly identified are inadequate institutional and technical capacity at the decentralized level and weak human resource capacity in the public service. 106. As public services are increasingly decentralized, challenges arise from the paucity of resources and the weaker capacity of local administration. The Burkina Faso Education FTI Program was designed to support the devolution of responsibilities from the central government to local governments and communities. The school-based management approach included establishing a framework for increased collaboration between the units and local government authorities, increased involvement of communities in managing schools, and increased attention to capacity building at the commune level and below. In the DPO to the state of Alagoas, Brazil, the state was cognizant of the limited technical capacity of its administration and undertook capacity-building activities covering critical areas such as human resource management, social security, and investment management. 107. Weak human capacity in the public service is also often cited as an important risk. A shortage of skilled labor may be a sign of a small population base or competition from the private sector to attract and retain qualified workers. Some mitigation efforts include aiming for more realistic targets and focusing on a limited number of reform areas, mobilizing existing capacities, and depending more heavily on those administration units with stronger capabilities. In that spirit, the PRSC-9 for Burkina Faso limited the coverage in terms of policy areas and number of reform measures. It emphasized building on ongoing initiatives rather than introducing new ones, and it relied extensively on the Bank’s and other development partners’ instruments for alleviating capacity constraints. In some cases, for example in Rwanda, a Bank-financed Public Sector Capacity Building Project supported skills development and staff retention in the civil service. 71 It should be noted that OP 8.60 does not require discussion of implementation capacity risks. 47 108. Approximately 72 percent of the DPOs that identified implementation capacity as a risk indicated technical assistance as a mitigation measure (see Box 12). Technical assistance provided by the Bank or other development partners was found mostly in LCR (approximately 80 percent) and AFR (over 70 percent). However, only six operations had prior actions directly linked to capacity building. An example is the Seventh Poverty Reduction Support DPO to Rwanda, which had the prior action: “Adoption by the Recipient's Cabinet, and operationalization, of a comprehensive five-year capacity building strategy for Local Government.� 109. While technical assistance may contribute to mitigate risks stemming from limited capacity, it can also be critical in less capacity constrained environments. For example, in Mongolia, sustained technical assistance and policy support by development partners was considered key to prevent a return to policy uncertainty. The Second Development Policy Credit supported reforms in mining, banking, social protection and the fiscal framework. These reforms were supported by an ongoing Mining TA and a new Multi-Sectoral Technical Assistance Project (MTAP). The Mining TA helped with complex tax issues and prepared for the introduction of a progressive royalty. The MTAP supported the social protection agenda, in particular, by setting up a national beneficiary database, and assisting with poverty and social impact analysis. Box 12. Addressing Implementation Capacity Risks through Technical Assistance Guinea-Bissau Third Economic Governance Reform DPO. It focused on the computerization of the accounting system, external controls, and budget execution. The DPO signaled that the low capacity of government staff was a risk to the program’s objectives, but noted that a State-and Peace-building Fund (SPF) grant of $1.7 million was assisting the Ministry of Finance with the public finance management plank of the operation. It noted also that several development partners were involved with public finance management, notably the Public Administration Reform Assistance Project of the European Union. The Bank rated the DPO as satisfactory and noted that the assistance provided by the SPF was a key factor for the success of the DPO. Samoa Economic Crisis Recovery Support DPO. This operation helped the country to recover from the tsunami and also tackled reforms in public finance management, education and telecommunications. The program document signaled weak public sector capacity as a risk, which is typical in small, remote islands. In Samoa’s case the risk had been aggravated by the tsunami and the global economic crisis. The Bank provided TA to strengthen capacity in cash management and to support the School Fees Grant Scheme, which had a positive impact on the financial management and accounting practices of primary schools. The DPO was rated as satisfactory by the Bank and moderately satisfactory by IEG. Seychelles First Development Policy Operation. It supported reforms in procurement, public financial administration, public sector, and social assistance. To support them, the Bank provided an Institutional Development Fund grant to build capacity. The assistance included simulating fiscal and social impacts of policy changes. TA for these reforms was provided by the Bank through a fee based services agreement. The Bank rated the DPO as highly satisfactory, while IEG rated it satisfactory. G. Managing Development Effectiveness Risks72 110. An important risk to the achievement of the intended development goals is the poor identification, selection or design of reforms, which can lead to results that are undesired or that fall short of the objectives. Analytic work (prepared by the Bank, the government, or other parties) is critical in supporting policy dialogue with the client and adding value to the country 72 Many factors affect development effectiveness. The review focuses on analytical underpinnings and country ownership. 48 knowledge. Thus OP 8.60 requires that a DPO draw on relevant analytic work on the country, and that the PDs describe the main pieces of analytic work used in the preparation of the operation and show how they are linked to the proposed development policy program. 111. While all PDs contained a discussion of the analytic underpinnings, some do not adequately articulate the criticality of the selected reforms in light of the analytical underpinnings. In most PDs there is an explicit section on analytic underpinnings, and the few remaining discuss them in the “Other Bank Operations� section. Whereas many PDs discussed the analytic findings and explained how they informed the preparation of the operation, others summarized the analytic activities without clarifying how their findings and recommendations helped to shape the operation. Additionally, some PDs only provided a list of analytic underpinnings on themes related to the operation without describing their main findings or linking them to the formulation of the operation. 112. DPOs with a strong focus on PFM appear to be particularly well underpinned analytically, with most PDs showing that the reforms are clearly grounded in analysis. In a sample of 21 operations with strong PFM content (more than five prior actions on PFM), 18 (86 percent) referenced four or more analytic studies.73 Four main analytic documents underlay the PFM policy dialogue and the design of the programs: the CFAA and its equivalents (PEMFAR, GFMRAP and the CIFA); the CPAR; the PER; and, increasingly, the PEFA studies. Overall, 96 percent of the prior actions were linked to weaknesses identified in previous PEFA studies (see Box 13). Box 13. PEFA-Informed DPO Design: West Bank & Gaza PRDP-2 2007 PEFA Assessment 2009 PFM Prior Actions 1. Weak linkages between investment projects and future 1. Approve the 2009 annual budget law with a combined recurrent expenditures. presentation of recurrent and development budgets for line ministries. 2. Redevelop existing accounting system to include 2. Issue resolution to amend the financial regulations to commitment controls and meet ministry MIS needs. support the new computerized accounting system. 3. No systematic cash forecasting system. No cash 3. Submit an annual cash plan to guide budget execution management committee for daily cash management in 2009, with monthly actuals and updated forecasts. decisions. 4. The prior action has no link to PEFA findings. 4. Complete the pilot phase of the new financial management information system in eight municipalities. 5. Poor transparency of transfers to subnational 5. Approve regulation to integrate all cash assistance governments. Poor communication of transfers to be programs into the Ministry of Social Affairs made and weak accounting and consolidation of fiscal 6. Adopt a clear criterion for performance-based transfers data. of monetary grants to municipalities. 6. No integration of personnel records and payroll 7. Implement a hiring control system to ensure that public records; reconciliation problems. sector hiring aligns with the approved budget. 113. Development effectiveness also requires continuity and sustainability74 of the reform process for which ownership and country-wide support of the reform program are vital. To help ensure that a DPO supports a reform program that has adequate ownership, the DPO program is typically drawn from the country’s national or sectoral development strategies. In addition, the preparation of the policy matrix of relevant actions and results (which basically spells out the means to achieve the country’s development goals) generates a dialogue among various parts of the 73 The review does not infer that the number of analytical underpinnings is a measure of quality and relevance. 74 Reform reversal or reform dilution represent important development effectiveness risks. 49 government—and often with nongovernmental stakeholders—that can improve the design of the reform program and build broader support for the reforms. 114. The Bank advises borrowing countries to consult with key stakeholders and engage their participation in the process of formulating the country’s development strategy. A DPO is expected to draw on this process of strategy formulation to determine the form and extent of consultations and participation in preparing, implementing, and monitoring and evaluating the operation. The PD is expected to describe the country’s arrangements for consultations and the outcomes of the participatory process that was used to formulate the operation. 115. Virtually all DPOs in the Retrospective discussed the country’s consultations and the participatory process used in the formulation of the operation. However, the robustness of the discussion varied. Over 80 percent of the documents discussed consultations and the participatory process, while the remainder noted that consultations took place but did not provide a detailed description of the participatory process. H. Overall Risk Management 116. Overall, risks could be more thoroughly and consistently assessed in DPOs and opportunities more clearly spelled out. While most operations discuss macroeconomic risks (95 percent), fiduciary (75 percent), implementation (67 percent), development effectiveness (4 percent), and other risks to the operation, there are important differences across DPOs. There is some unevenness in the identification of risks and the quality of the discussion of the risks and the potential mitigation factors. This makes it hard to compare the risks sections of PDs across DPOs, and acquire a sense of the different levels of risks across countries. On the other hand, DPOs could be more explicit about how they could support the identification and enhancement of development opportunities and the trade-off between opportunities and risks so that both can inform management decisions. 117. While systematic management of risks is imperative, some degree of risk needs to be tolerated in view of the potential development benefits. Whilst DPOs have mitigation measures built in to the extent possible, there are always risks that cannot be mitigated. The right balance between risks and opportunities needs to be struck so that excessive risk aversion does not reduce our net development impact. One risk often discussed in DPOs that is almost impossible to mitigate is political instability. For example, the implementation of the DPO in Yemen was severely affected by the social and political unrest that broke out in early 2011, within weeks of the approval of the DPO (see Box 6). On the other hand, capacity implementation risks have been successfully mitigated in a number of cases (see Box 12). 50 V. REFORMS 118. This chapter assesses how the Bank has reformed the operational policy framework for DPOs since its adoption in August 2004, in response to changing country priorities. More specifically, it examines the strengthening of the Deferred Drawdown Option (DDO) and the introduction of a specialized DDO feature, the Catastrophic Risk DDO; changes in the DPO policy to allow for the provision of DPOs to political subdivisions in a borrowing country below the level of states and provinces; and changes in the Special DPO option. It also examines developments in the areas of partnership and transparency. A. Reforms to the Policy Framework 1. The Enhanced Deferred Drawdown Option in DPOs 119. The Deferred Drawdown Option (DDO) for policy based lending instruments was introduced in 2001 but was severely underutilized until 2008.75 The DDO feature gives IBRD- eligible countries the option to defer disbursement up to three years with the possibility of renewing, with Board approval, for an additional period of up to three years. Disbursements are subject to the adequacy of the country’s overall program implementation and macroeconomic policy framework. By 2008, only two DPOs with DDOs had been approved.76 A review found that borrowers were hesitant to use this feature because of its harder financing terms and the perception that funds might not be available when required, given that withdrawal required the Bank’s reconfirmation that the macroeconomic policy framework and overall program implementation were adequate. As a result, the Bank introduced a streamlined verification protocol in 2008,77 under which the Bank continues to monitor the macroeconomic policy framework and adherence to the overall program, and advises the borrower of the need for a review if at any time the conditions are not satisfied. Once both drawdown conditions are satisfied, the Bank confirms that the eligibility to submit disbursement requests has been restored. As part of the 2008 revisions, pricing changes were also made to increase the DDO’s attractiveness78 (see Table 2). 120. The Bank revised DDO pricing further in 2009 and 2012. In the midst of the global financial crisis and reflecting the implications for Bank capital of a sharp increase in the demand for the DDO feature, a new pricing schedule was approved resulting in an increased front-end fee on undrawn DDOs and the introduction of a fee at renewal.79 In February 2012, the Board approved a further change to DPO DDO pricing.80 The DPO DDO front-end fee was reduced by 50 basis points, and the 50 basis points fee charged at renewal was eliminated. These fees were replaced by a new “standby� fee, equivalent to the contractual lending spread accruing from the date of effectiveness and applied to the actual undrawn balance. The new pricing structure aims to eliminate the potential for mispricing by better aligning the charges for undrawn balances to the actual 75 See Proposal to Introduce a Deferred Drawdown Option (DDO) for Use with IBRD Adjustment Loans (R2001- 0174), September 26, 2001. 76 These were the Latvia Private Sector Adjustment Loan and the Chile Social Protection Sector Adjustment Loan. 77 See Recommendations for Simplifying and Improving the Competitiveness of IBRD New Loan Pricing (R2007- 0196), September 19, 2007. 78 See Proposal to Enhance the IBRD Deferred Drawdown Option (DDO) and to Introduce a DDO Option for Catastrophic Risk (R2008-0018), February 4, 2008. 79 See 2009 Review of New Loan Pricing, Revised (R2009-0182/1), July 30, 2009. 80 See Review of DPL-DDO Pricing Structure (R2012-0005), January 10, 2012. 51 undrawn period, thereby better protecting the Bank from the consequences of underpricing as well as providing greater fairness to borrowers.81 121. Management intends to devise procedures to limit the deadline for signing of regular DPOs, in order to ensure all deferences of drawdown are priced fairly. Most DPOs become effective shortly after Board approval.82 92 percent of the operations in the Retrospective were signed within 3 months and 97 percent within 6 months. However, there was one operation that took 14 months and another that took 18 months to be signed. This raised concerns as to whether these operations bypassed DDO standby fees. Table 2. Summary of Changes to Financial Features of DDOs Before 2008 2008 2009-2011 Since 2012 Contractual spread at time Contractual spread at time Contractual spread at time Contractual spread at time of signing of each drawdown of each drawdown of each drawdown Standard FSLa pricing IFLb variable or fixed IFL variable or fixed IFL variable or fixed except for the following: spread in effect at time of spread in effect at time of spread in effect at time of  0.25% front-end fee withdrawal: withdrawal: withdrawal: applied to approved  0.25% front-end fee  0.75% front-end fee  0.25% front end fee loan amount payable applied to approved applied to approved applied to the approved upon drawdown; loan amount payable at loan amount payable at loan amount payable at  0.25% commitment fee effectiveness. effectiveness; date of effectiveness; surcharge;  Renewal fee of 0.50%.  Standby fee of 0.50% of  0.25% per annum undisbursed balance surcharge for DDOs accruing from date of with extended maturity. effectiveness. a Fixed spread loan; b IBRD flexible loan 2. Experience with DPO DDOs 122. Since the 2008 revisions to the DDO, 15 DPO DDOs have been approved for eleven member countries, totaling $ 9.2 billion in commitments (see Table 3). The increased use of DPO DDOs was part of the overall Bank response to greater demand for financial support in the context of the global financial crisis. With an increase in uncertainty surrounding borrowers’ access to financing, DPO DDOs proved to be a useful signal to markets that the Bank, often in partnership with other international financial institutions, stood ready to provide contingent financing if a borrower’s market access should become constrained. Some countries, such as Indonesia, signaled in advance that drawdown would occur only under exceptional circumstances, while others explicitly noted that drawdown would occur soon after effectiveness. Of the 15 operations approved by the Board, 10 have been fully or partially disbursed, with a total of $2.5 billion drawn down (27 percent). 81 The new pricing structure decreases the charges payable by borrowers whose drawdown period is less than 1 year, and increases the charges payable by borrowers whose drawdown period is more than 1 year. For DPO with DDOs for which the drawdown period equals 1 year, the charges remain unchanged. 82 Currently the deadline for signing is 18 months. BP 13.00 states that if the borrower does not meet the requirements for signing before the 18-month deadline from the approval date, the RVP decides whether to (i) give the borrower additional time to sign (see paragraph. 19), or (ii) withdraw the DPO (see OP 13.00, paragraph 4). 52 Table 3. DPOs with a DDO, April 2008-September 2012 Disbursed Amount (as of (US$ September Project ID Country DPO Title Board Date million) 2012) P105029 Colombia Third Business Productivity and Efficiency* 04/08/2008 550 550 P110849 Mexico Climate Change* 04/08/2008 501 501 P101590 Peru Second Programmatic Fiscal Management and 08/05/2008 370 220 Competitiveness* P102160 Bulgaria Second Social Sectors Institutional Reform* 11/04/2008 150 150 P115120 Peru Supplemental Financing for Second 12/18/2008 330 - Programmatic Fiscal Management and Competitiveness* P106724 Uruguay Second Programmatic Reform 02/03/2009 400 400 Implementation* P101471 Peru Environmental* 02/17/2009 330 20 P115199 Indonesia Public Expenditure Support Facility* 03/03/2009 2,000 - P096711 Serbia Programmatic Private and Financial* 03/05/2009 50 50 P112369 Mauritius Third Trade and Competitiveness* 03/31/2009 100 100 P101177 Peru Second Results and Accountability 04/09/2009 330 20 P115173 Costa Rica Public Finance and Competitiveness 04/30/2009 500 500 P123242 Uruguay Second Programmatic Public Sector, 10/25/2011 260 - Competitiveness and Social Inclusion P130051 Romania Development Policy Operation* 06/12/2012 1,333 - P130048 Indonesia Program for Economic Resilience, Investment 05/12/2012 2,000 - and Social Assistance* *Operations outside the Retrospective Period. 123. Across IBRD-eligible borrowers, LCR countries have taken 9 of the 15 DPO DDO operations ($3.6 billion in commitments, of which $2.2 billion have been drawn down). By number of operations, Peru has been the most active borrower using the DPO DDO. Reflecting its risk management strategy, it has drawn down only $260 million of the total $1.4 billion in commitments available under the DPOs DDO. By volume, Indonesia has been the biggest borrower using the DPO DDO – although the first US$2.0 billion option closed without drawdown and the second US$2.0 billion option has not been drawn to date. This operation was part of a broader multi- partner package of contingent financing designed to signal to international and domestic markets the confidence that Indonesia would have access to back-up financing arrangements should the need arise. Colombia, Mexico and Costa Rica have fully drawn down the financing available under their DPO DDOs. Of the three operations in ECA (Bulgaria, Serbia and Romania) only Romania has not drawn down while the only DPO DDO in AFR (Mauritius) has been fully drawn down. While contingency financing can be valuable, the existence of such an open line of access to financing can potentially create budgetary pressures, thus undermining the effort to bring confidence to domestic and international markets. As a result, some DPOs have opted to include a drawdown trigger, thus containing these pressures while keeping the flexibility embedded in the DDO. 53 Box 14. Indonesia: Support through Contingency Financing While at the start of the global financial crisis in 2008 Indonesia had strong macroeconomic fundamentals, by late 2008 the ripple effects of the global turmoil were adversely affecting the country’s financial markets. The rupiah came under pressure, the stock market declined, and the government debt market was hit by the increase in global risk aversion. There were sizable risks that the spillover effects of the global crisis could negatively affect the economy and Indonesia’s development progress. Because the government would face financing constraints if global liquidity conditions did not ease, the government requested support from the Bank in the form of the $2 billion Public Expenditure Support Facility DPO DDO, part of a broader financing package, with other development partners (Australia, Japan, and the Asian Development Bank) providing support on similar terms. In addition to providing contingent financing, the facility supported a set of confidence-boosting policy measures that contributed to improving market sentiment. During this period, the Bank also provided support to Indonesia through two other DPOs and additional financing to existing investment loans. In the end, the Indonesia DPO DDO, which was never disbursed, helped ensure that the Indonesian government had access to resources if markets failed to provide the required financing at a reasonable cost. Source: Implementation Completion and Results Report on a Public Expenditure Support Facility DPO DDO, Report No ICR1952. 3. Development Policy Loans with Catastrophe DDO 124. Along with the other 2008 revisions to the DDO feature, the Bank introduced a specialized DDO feature to provide immediate liquidity in emergency situations caused by natural disasters and catastrophes. The main purpose of this product is to support countries’ efforts to enhance their capacity to manage natural hazard risk and provide a source of bridge financing while other resources are being mobilized. Like regular DDOs, a DPO with a Catastrophic Risk DDO (Cat DDO) is available only to IBRD-eligible countries. In addition, because the Cat DDO emphasizes disaster prevention rather than disaster response, the country must have in place, or under preparation, a hazard risk management program. DPOs with Cat DDO can be drawn down only after the occurrence of a natural disaster that results in the borrower’s declaration of a state of emergency.83 When the Cat DDO was introduced, its financial features were similar to those of the DPO DDO. However, subsequent revisions made DPOs with Cat DDO less expensive than regular DDOs (see Table 4).84 125. A DPO with Cat DDO may be renewed four times for up to three years each time, for a total deferment of 15 years. The extended renewal feature is intended to provide borrowers with a reasonable assurance that support will be available over a sufficiently long period of time. Renewals require the reconfirmation of the adequacy of the macroeconomic framework and hazard risk management framework. Like regular DDOs, DPOs with Cat DDO can be disbursed partially or in full, but they also have a revolving feature that allows repayments to be again available for drawdown. The total deferment of 15 years raises a question about the value and relevance of the results framework and of an ICR completed 15 years after Board approval. While there is no presumption that the Cat DDO will be renewed, there may be a need to have more frequent reporting, possibly at the time of each renewal, to monitor the program’s implementation and results. 83 Given that the procedures for declaring a state of emergency vary from country to country, these drawdown triggers are defined on a case-by-case basis, taking into consideration each country’s natural hazard risk management legal and policy framework. 84 See Review of DPO-DDO Pricing Structure (R2012-0005), January 10, 2012. 54 Table 4. Summary of Changes to Financial Features of Cat-DDOs Feature 2008 2009-2011 Since 2012 Contractual Contractual spread at time of Contractual spread at time of Contractual spread at time of lending spread each drawdown each drawdown each drawdown Pricing IFLa variable or fixed spread in IFLa variable or fixed spread in No change structure effect at time of withdrawal: effect at time of withdrawal:  0.25% front-end fee applied to  0.50% front-end fee applied to approved loan amount payable approved loan amount payable at effectiveness. at effectiveness;  Renewal fee of 0.25%. a IBRD Flexible Loan 126. Given the risk of disbursing in an adverse macroeconomic environment that may follow a catastrophe, the volume of DPOs with Cat DDO to any country is limited to 0.25 percent of a country’s GDP or US$500 million, whichever is smaller. Unlike the requirement for other DPOs, the Bank does not reconfirm the adequacy of the macroeconomic framework before disbursement, as this could hamper the objective of providing quick liquidity injection after a catastrophic event. However, the adequacy and relevance of the volume limit can be questioned, given that the instrument has a possible duration of 15 years. Moreover, there is currently no mechanism by which the Cat DDO can be augmented to account for the economic growth of the country. Some consideration might be given to allow for it to be “topped up,� accounting for space in the country exposure limits that comes from economic growth and initial “underutilization,� without necessitating a completely new operation. 4. Experience with DPO Cat DDO 127. Since the introduction of the Cat DDO in 2008, the Board has approved eight DPOs with Cat DDO, for total commitments of $1.26 billion. These operations were to Colombia, Costa Rica, El Salvador, Guatemala, Panama, Peru, and the Philippines. Following natural disasters and declarations of emergencies, a few countries have withdrawn some or all of the amount (see Table 5 and Box 15)—a total of $819 million, or approximately 65 percent of total commitments. None of these countries has yet made use of the revolving feature of these credit lines by repaying amounts that have been drawn down. The development policy objective of all of these operations was to enhance the governments’ capacity to implement disaster risk management programs for natural disasters. All countries supported by DPOs with Cat DDO had already begun to take a proactive approach to disaster risk management focusing on disaster prevention, preparedness, and mitigation. The disaster risk management programs and institutional arrangements were adequately described in all the PDs. Given that these countries have been subject to numerous natural disasters in the past, the PDs also highlighted how existing arrangements have dealt with previous natural disasters. 55 Table 5. DPOs with Cat DDO, April 2008-September 2012 Loan Disbursed amount (as of Board (US$ September Project ID Country DPO Title date million) 2012) P111926 Costa Rica Costa Rica DPO with Cat DDO* 09/16/08 65 34 P113084 Colombia Disaster Risk Management DPO with Cat DDO* 12/18/08 150 150 P112544 Guatemala Disaster Risk Management DPO with Cat DDO 04/14/09 85 85 P120860 Peru Disaster Risk Management DPO with Cat DDO 12/09/10 100 - P122640 El Salvador Disaster Risk Management DPO with Cat DDO 02/01/11 50 50 P125943 Philippines Disaster Risk Management DPO with Cat DDO 09/13/11 500 500 P122738 Panama Disaster Risk Management DPO with Cat DDO 10/18/11 66 - Second Disaster Risk Management DPO with Cat P126583 Colombia DDO* 07/10/12 250 - *Operations outside the Retrospective Period. Box 15. Events Triggering Drawdown under DPO with Cat DDO As of September 2012, five countries have secured access to funds after natural disasters by drawing down the outstanding balance on DPO with Cat DDOs: (i) Costa Rica drew down $34 million after a 6.2 magnitude earthquake in January 2009; (ii) Guatemala accessed $85 million—the full value of its DPO Cat DDO—after tropical storm Agatha and the eruption of the Pacaya volcano in 2010; (iii) Colombia also disbursed the full balance of its $150 million DPO Cat DDO following the widespread flooding associated with La Niña at the end of 2010; (iv) the DPO Cat DDO for El Salvador was also fully drawn down at the end of 2011 to address the impacts of unprecedented precipitation levels from a tropical depression; and (v) the Philippines Cat DDO was fully disbursed to assist the country’s recovery and reconstruction efforts in the wake of tropical storm Washi. 128. Prior actions have typically focused on elevating disaster risk management to a national priority. During the period under review, DPOs averaged 10 prior actions each. However, DPOs with Cat DDO averaged only 3 prior actions per operation. Broadly, one or more prior actions focused on mainstreaming disaster risk management into the country’s national development plans as well as incorporating disaster risk analysis in the screening of investment projects. Operations also focused on improving financing mechanisms to facilitate resource mobilization to respond to natural disasters—for example, establishing and funding national emergency funds in Costa Rica. Others focused on budgetary appropriations to support disaster risk management programs (Guatemala) or to ensure that public assets have financial protection (Panama). A number of prior actions focused on risk management related to specific types of disasters (earthquakes and flooding in El Salvador) or to a specific type of infrastructure (hospitals in Peru). 5. Development Policy Lending to Political Subdivisions 129. In February 2011, the policy framework for lending to subnational divisions was amended to include political subdivisions other than states and provinces.85 Before the revision, lending to such a subnational division required a waiver from the Board. A political subdivision is an 85 See Development Policy Lending to Political Subdivision, February, 2011. 56 entity that, in accordance with the country’s constitutional and legislative framework, has budgetary and legislative authority for the actions supported by the DPO. The political subdivision must have the legal ability to borrow from, or enter into a contractual relationship with, the Bank. In addition to the standard requirements for development policy lending, the political subdivision must have an appropriate expenditure program, sustainable debt, and appropriate fiscal arrangements with the central government and/or the applicable political subdivisions, in accordance with the country’s constitutional and legislative framework. The country also needs to fully guarantee the repayment of the principal and the payment of interest and charges of the loan. The Bank reviews the financial management and procurement arrangements of the country, of the political subdivision, and of the applicable subdivisions in accordance with the country’s constitutional and legislative framework. 6. Experience with DPOs to Political Subdivisions 130. Before February 2011, the Bank had approved 18 subnational DPOs,86 including the first loan to a municipality—the Rio de Janeiro Fiscal Consolidation DPO.87 Since March 2009 the Bank extended twelve DPOs to political subdivisions: 9 to Brazilian subdivisions, 2 to Nigerian States and one to an Indian State (see Table 6). The increase in the share of policy-based lending to political subdivisions in Brazil reflects mainly a shift of interest by the federal government of Brazil and the Bank to support: (i) further fiscal consolidation of political subdivisions; (ii) inclusive growth and social inclusion through increased coverage and quality of public service delivery to the poor (political subdivisions are the main providers of public services); and (iii) public sector management to increase the efficiency of political subdivisions’ expenditures. In Nigeria, the engagement is more recent and has been limited to the State of Lagos and the State of Edo. Table 6. DPOs to Political Subdivisions, March 2009-September 2012 Loan Amount Project ID Country DPO Title Board Date (US$ million) P103770 Brazil Alagoas State Fiscal and Public Sector Reform 12/17/2009 195 P117244 Brazil Rio de Janeiro State Fiscal Sustainability Human 02/02/2010 485 Development and Competitiveness P111665 Brazil Municipality of Rio de Janeiro Fiscal Consolidation for 07/01/2010 1045 Efficiency and Growth P122391 Brazil Rio de Janeiro Metropolitan Urban and Housing 03/15/2011 485 P117237 Nigeria First Lagos State 03/17/2011 200 P126449 Brazil Piaui State Green Growth and Inclusion 03/06/2012 350 P106753 Brazil Pernambuco State Expanding Opportunities, Enhancing 3/22/2012 500 Equity P123353 Nigeria First Edo State 3/29/2012 75 P126351 Brazil Bahia Inclusion and Economic Development* 06/28/12 700 P121590 Brazil Third Minas Gerais Development Partnership* 07/26/12 450 P126465 Brazil Third Rio State* 08/30/12 300 P124041 India Promote Inclusive Green Growth and Sustainable 09/06/12 100 Development in Himachal Pradesh* * Operations outside the Retrospective Period. 86 Eight in Pakistan, five in Brazil, and five in India. 87 This operation was the first loan to a municipality. It required a waiver from the Board as it was approved before the policy framework for lending to subnational divisions was amended. 57 131. Experience with DPOs to political subdivisions indicates their significant potential for Bank support for policy reform, but also highlights several risks to design and implementation. Bank support has often focused on fiscal issues related to fiscal decentralization, weak fiscal management, and fiscal transfers related to public service delivery. Other reforms have aimed at increasing the quality and coverage of public services delivery (especially in poor areas), increasing the efficiency of public sector management, and addressing policy issues regarding social inclusion and inclusive growth. Bank experience has shown, however, that DPOs to political subdivisions have specific risks that need to be taken into account. Among them is the limited implementation capacity, the high vulnerability to political cycles because the alignment with central and other subdivision- level policies is more acute, the high risk of policy reversals, and the high vulnerability to macro shocks and fiduciary risks at the national and subnational levels. Also see Section IV part A on macroeconomic issues that need to be carefully considered and discussed in the context of DPOs to political subdivisions. 132. At the other end of the spectrum, the potential for regional/multi-country DPOs remains untapped, despite being potentially an excellent vehicle to deal with regional policy and integration processes. Although regional IDA funds are not available to DPLs, regional DPOs are entirely feasible under OP 8.60 and could play a key role in enabling the policy framework to support regional integration. Recent Bank analytical work has concluded that regional infrastructure interventions may not lead to the expected results in terms of additional trade flows unless the regulations and norms governing the movement of goods and people across countries are also taken into consideration.88 In addition, regional DPOs could also be a conveyer belt to deal with many other multi-country policy challenges such as regional integration, resource management, and investment climate. DPOs could foster mutually beneficial reforms with positive regional externalities. Nonetheless, no regional/multi-country DPO has been approved to date. 7. Special Development Policy Lending 133. Special development policy lending is the policy-based lending option the Bank uses on an exceptional basis for IBRD-eligible countries that are approaching or are in a crisis. The crisis must have substantial structural and social dimensions, and the country must have urgent and extraordinary financing needs. The magnitude of such financial support is subject to the availability of adequate IBRD financial and risk-bearing capacity. This option was first introduced in October 1998 as emergency structural adjustment lending (ESAL).89 With the transition from adjustment lending to development policy lending in 2004, the Bank retained its ability to provide DPOs with ESAL features by introducing the “option� of special development policy lending in OP8.60. 88 World Bank 2012. “De-fragmenting Africa: Deepening Regional Trade Integration on Goods and Services�, The World Bank. 89 See Programmatic and Emergency Adjustment Lending: World Bank Guidelines (R98-249), October 22, 1998. 58 134. In late 2009, in response to the global financial crisis and economic recession, the Bank reviewed the use of special development policy lending and introduced minor revisions to the policy framework and financial terms.90 The review confirmed that special development policy lending would continue to be a rarely used instrument for Bank participation in IMF-led international rescue packages for countries experiencing a financial crisis. To align special development policy operations (SDPOs) with the new CAS architecture, the linkage between SDPOs and the country’s CAS lending envelope was deemed unnecessary and was eliminated from the Bank’s operational policy framework. While SDPOs would still have a shorter maturity and higher price than normal DPOs, the changes introduced greater flexibility into SDPO repayment terms by changing the 3-year grace period to a 3- to 5-year grace period, and changing the 5-year maturity to a 5- to 10-year final maturity. To maintain broad comparability with the pricing of IMF emergency assistance, the minimum fixed spread (over LIBOR) was reduced from 400 bps to 200 bps, while the front-end fee of 100 bps was retained. 135. OP 8.60 provides that, to be eligible for special development policy lending, a country must have a disbursing IMF-supported program in place, and the SDPO must be part of an international support package. The international support package may include multilaterals, bilateral partners, and private lenders and investors, and may address structural, social, and macroeconomic policy, with conditionality embedded in a strong policy program. With these limitations on SDPOs, the Bank cannot extend the instrument to countries that encounter financial or economic difficulties because of the global financial crisis but are not themselves in or approaching a crisis that necessitates an IMF-led international rescue package. Other instruments, such as regular DPOs or DPO DDOs, are considered more suitable for responding to such demand. 136. The use of this instrument has been quite limited. Since 1998, the Bank has participated in five international support packages—for Argentina in 1998, Brazil in 1999, Turkey in 2001, Uruguay in 2002, and Latvia in 2010-2011.91 In Latvia, for example, the Board approved a programmatic series of two single-tranche €100 million SDPOs (the Safety Net and Social Sector Reform Program), part of a broader €7.5 billion package of emergency assistance from the IMF, the EC, and the Nordic and other European countries in response to the severe financial and balance of payments crisis and economic recession that had hit Latvia at the end of 2008. The Bank’s support focused on measures to mitigate the impact of the required fiscal consolidation and supported the implementation of the government’s Emergency Social Safety Net Strategy (see Box 16). 90 See World Bank Response to Financial Crises: The Special Development Policy Lending Option (R2009- 0187), July 31, 2009. 91 The Board approved a Financial Sector and Macro Stability DPO for Hungary in September 2009 as part of an international support package requested by the government. The package included the IMF, the European Commission, and the World Bank, notwithstanding the fact that Hungary had “graduated� from IBRD in 2007. The DPO was never signed. While not formally designed as an SDPO, it was on SDPO terms. 59 Box 16. Support to Latvia during the Crisis: The Use of a SDPO In late 2008, a number of factors—credit and housing bubbles, acute banking pressures, the global financial crisis, and concerns over the sustainability of the exchange rate peg—contributed to a severe economic crisis in Latvia. By 2009, unemployment had increased to almost 23 percent, the economy had contracted by 18 percent, and further declines were expected. The shrinking economy resulted in a 40 percent increase in the share of the population living below the poverty line. While Latvia had graduated from Bank financing in 2007, the crisis led to an exceptional request by the government for renewed access to Bank lending resources. Under a stabilization program led by the IMF and the European Commission, the Bank undertook a programmatic series of two SDPOs to help local governments protect vulnerable households and support social sector reforms. The first operation, approved on March 4, 2010, focused on supporting efforts to protect vulnerable groups with emergency safety net support and to mitigate the cost of fiscal consolidation. It provided cash for work to unemployed people who were ineligible for unemployment benefits and ensured that municipalities had sufficient resources to pay social assistance benefits to the poorest people. The second operation focused on structural reforms to lay the foundation for medium-term improvements in the education and health sectors. In education, this entailed financing reforms so that “funds follow the student.� In health, the reforms aimed at improving administrative capacity and transparency and rationalizing the provision of care by substituting expensive in-patient services with increased use of outpatient surgery. Source: First Safety Net and Social Sector Reform Program; and Second Safety Net and Social Sector Reform Program. B. Other Reforms with Implications for Development Policy Lending 1. Guarantees 137. Management is nearing completion of a comprehensive reform of the Bank’s operational policy on guarantees. Guarantees are one of the Bank’s four development finance instruments, distinct from and complementary to investment lending, program-for-results financing, and development policy lending. Unlike the other instruments that provide financing directly to member countries, Bank guarantees facilitate the mobilization of private financing for development purposes by providing partial guarantees of debt service payments for borrowing by the government, or by guaranteeing government contractual undertakings (such as payment obligations) for a private sector project. There are three guarantee options that vary by types of coverage and beneficiaries. Among the three, policy-based guarantees (PBGs) provide partial credit guarantees to help borrowers access external financing for general budgetary borrowing associated with policy and institutional reforms. Thus, PBGs are similar to DPOs in that they support the country’s reform program, but they differ in that they have the additional objective of improving access to financial markets. PBGs are currently available only for IBRD-eligible countries. 138. As part of the overall policy reforms, Management intends to take two proposals to the Board:  Incorporate PBGs into OP 8.60. In doing this, the Bank would streamline eligibility criteria and restrictions that apply only to PBGs to align the two instruments: it would remove policy provisions that limit PBGs to external financing and require a strong track record of country performance; and it would broaden the requirement that PBGs must result in improved market access (greater volume of financing or longer maturities) to include significant financial leverage, 60 improvements in the financial terms, access to new sources of financing or a combination thereof.  Extend the availability of PBGs to IDA-only countries that have low risks of debt distress and adequate debt management capacity. The purpose is to ensure that the Bank facilitates commercial borrowing by IDA-only countries, but in a sustainable manner. 2. Access to Information Policy 139. The Bank allows access to any information in its possession that is not on the list of exceptions.92 The new Bank Policy on Access to Information effective July 1, 2010,93 set forth a ground-breaking change in how the Bank makes information available to the public. Under the new policy, the Bank will disclose any information in its possession that is not considered restricted information. In addition, over time the Bank declassifies and makes publicly available certain information that falls under the exceptions. Notwithstanding the broad intent of this policy, the Bank reserves the right, under exceptional circumstances, to disclose certain information covered by the list of exceptions, or to restrict access to information that it normally discloses. 140. Selected Board papers, including program documents of DPOs, are eligible for simultaneous disclosure. This means that documents can be made publicly available before the Board has considered them, provided that the client has given written consent. This allows the stakeholders to access the document’s content before it is approved and potentially influence how the Board votes on the document. Since the introduction of the policy, 29 percent of the DPOs approved have been simultaneously disclosed. Examples of publicly disclosed documents include the Second Sustainable Employment Development Policy Operation to Kosovo, Expanding Opportunities, Enhancing Equity in the State of Pernambuco, in Brazil, and the Second Growth Policy Reform Credit to Niger. C. Reforms and Evolution of Practices 1. Joint Budget Support DPOs 141. A large number of operations in IDA countries covered by this Retrospective were prepared jointly with other development partners, including both bilateral agencies and Regional Development Banks (RDBs). Half of all IDA operations and 74 percent of IDA operations in AFR were prepared jointly with other partners, including sharing a Performance Assessment Framework (PAF) and/or a Memorandum of Understanding (MoU). 142. The preparation of operations in the context of JBS has contributed to greater consistency across development partners’ policy reform goals. A survey conducted by OPCS with staff working on 52 IDA-eligible countries suggests that budget support coordination has 92 The 10 types of information considered restricted are: personal information, communications of ED’s offices, Board Ethics Committee, attorney-client privilege, security and safety information, confidential client/third party information, corporative administrative, deliberative, and financial. 93 The World Bank Policy on Access to Information, July 2010. 61 contributed to significant progress.94 Over 80 percent indicate that there is increasing coherence of development partners’ policies. Previously, a profusion of partners and related aid instruments often led to fragmentation of the policy dialogue among development partners and high transaction costs to governments—a problem that was especially critical for governments with limited capacity to own and manage their development agenda. 143. JBS has reduced transaction costs for the countries, allowing for a more coordinated dialogue. Without such a framework, the country can be confronted with a throng of development partners, each with its own policy goals and conditionality, and overseen by different missions. The joint PAFs can serve as a coordination vehicle for countries’ and multiple development partners’ development goals. They can also help to enhance government ownership by serving as a tool that lays out the implementation of countries’ priorities, which is especially useful when a country does not have a workable prioritized development program, by clearly identifying key policy measures and indicators of progress. 144. Another positive result of JBS may have been some increase in the predictability of aid flows. The increased transparency, with the disbursing conditions of the budget support clearly spelled out in the joint PAFs, has increased the predictability of aid flows and financing. However, as discussed below, in very rigid and formal partnerships this can also lead to increased risks of a financing gap if reforms do not materialize and many development partners disburse against the same conditions. This has also been perceived as giving excessive influence to development partners at the expense of the authorities. 145. While JBS is the right approach to support countries effectively given the above- mentioned benefits, some modalities of JBS, usually emerging from MoUs, have limited countries’ ability to fully benefit from it. These modalities are typically characterized by rigid and formal partnerships which have (i) limited the ability of countries to respond to changing circumstances; (ii) weakened policy dialogue by reducing the level to the lowest common denominator; (iii) imposed time costs of their own; and (iv) contributed to a disproportionate focus on development partners’ processes. 146. The excessive emphasis on the processes of coordination led, in some cases, to the creation of dozens of committees, working groups, and other fora with such processes sometimes getting in the way of country ownership and meaningful dialogue (see Box 17). There are also transaction costs associated with long and sometimes difficult negotiations, fragmented policy dialogue across different foras, and increased difficulty in reaching consensus and establishing priorities. In Mozambique, for example, the number of working groups and task forces in 2009 reached a total of 71, of which 29 groups were joint with the government, and 42 were inter- partners only.95 Thus in many cases the purpose of reducing transaction costs by dealing with numerous development partners through the coordinated approach of JBS was defeated by the increased transactions costs associated with the proliferation of coordination structures to a point that neither development partners nor client countries could have a meaningful engagement. As a result, 94 Aid Coordination in IDA countries: A Roadmap to More Effective Aid Coordination, Operations Policy and Country Services (OPCS), April 2011. Surveyed staff working on 52 IDA-eligible countries. 95 Mozambique Implementation and Completion Results Report for PRSC-3-4-5, Report ICR1039. 62 some governments have found that joint missions and joint assessment frameworks did not reduce transaction costs by as much as they had hoped.96 147. Development partners also voice concerns about the sometimes unwarranted efforts to regulate and formalize development partner’s relations, at the expense of focus on the countries’ development goals. There can be difficulties in reconciling different development partners’ mandates, and operational policies. Reaching agreements on the MoUs can entail lengthy and difficult negotiations, due to their quasi-legalistic language, often taking literally years of development partner and government dialogue to finalize, while diverting development partners attention from the key development goals. Box 17. High Transaction Costs in JBS Coordination Structures An OPCS study (Aid Coordination in IDA Countries: A Roadmap to More Effective Aid Coordination) reported the existence of 638 coordination structures in 52 IDA countries, of which only about 60 percent were found to be performing well. Further follow-up indicates that the total number of coordination structures across IDA countries is more likely to be in the range of 1,000 to 1,200. According to another OPCS study (Aid Coordination in IDA Countries—Role of the World Bank) a substantial amount of staff time, especially in country offices, is devoted to aid coordination. More than 20 percent of staff time in 38 countries is reportedly spent on aid coordination, including 40 percent or more in 6 countries and 30 percent in 14 countries. An IEG evaluation (Evaluation of Poverty Reduction Support Credits: An Evaluation of World Bank Support) reports similar findings. Over 80 percent of DPO task team leaders interviewed said that transaction costs have increased substantially, sometimes at the cost of substantive work and interaction with governments. 148. JBS characterized by overly rigid harmonization contexts may have limited the flexibility and, at times, the ability to provide support for the most critical reforms. The lack of flexibility in some of the MoUs and PAFs was particularly evident at the time of the financial crisis and ensuing global recession, when PAFs designed and agreed before the onset of the crisis did not allow for adjustment in policies to respond to the changing circumstances. Similarly, governments have expressed the view that the harmonization process can imply a loss of flexibility for including new areas of importance in the joint agenda, tying them to reforms that sometimes are no longer deemed appropriate or important (Box 18 provides examples).97A good example of flexibility is the Joint Matrix (PAF) in Cape Verde. PRSC-5 and PRSC-6 included reforms in the transport and energy sectors that are not included in the PAF, but were discussed with and supported by the other partners. This flexibility allowed the Bank to address critical issues in the energy and transport sectors. In Liberia, flexibility was embedded in the PAF to include or revise reforms as needed, and partners regarded the PAF as a resource to draw upon. The lack of a MoU may account for the evident flexibility. 96 See Aid Coordination in IDA Countries: The Role of the World Bank, Operations Policy and Country Services, April 2011. 97 For example, in Benin, these concerns were expressed during the consultations associated with the 2009 Development Policy Lending Retrospective. 63 Box 18. Difficulties in Adjusting the Content of DPOs in the Context of JBS Mozambique. The government adjusted its program to respond to the food crisis in 2008. The Mozambique PRSC-5 was then adjusted to support the government’s Food Production Action Plan. As the MoU stipulated that prior actions had to be drawn from the PAF, the adjustments in PRSC-5 led to intense discussions and tensions between the Bank and development partners. Ghana. The Bank prepared an operation outside the JBS framework to address and support the country’s challenges during the global economic crisis in 2009. The Economic Governance and Poverty Reduction Credit was a focused operation aiming to support the immediate reforms needed to address the severity of the situation. These measures had not been anticipated when the PAF was designed three years before. Development partners reacted negatively to the Bank’s approach, strongly criticizing the decision to move out of the harmonized JBS framework. The need for flexibility was later recognized and that awareness led to the strengthening of the PAF for the follow-on operation (which was fully aligned with the JBS framework). Sources: Mozambique Implementation and Completion Results Report for PRSC-3-4-5, Report ICR1039; Ghana Economic Governance and Poverty Reduction Credit, Report No. 47723-GH 149. Finally, it has often been recognized that very rigid and formalized partner relationships have contributed to a tendency for development partners to cluster around the “lowest common denominator,� or a minimal common agenda of policy content. This has limited the ability of the government, the Bank, or other development partners to bring contentious issues (or new areas of importance) to the PAF, so that such issues have to be addressed outside the context of JBS. The minimal common agenda has been reinforced by some governments’ preference to peg disbursement conditions to what is most likely to be achieved—especially when delays in achievement could limit or postpone disbursements.98 While it is widely accepted that JBS increases the predictability of aid flows, there is also the perception that in highly structured contexts the dialogue is dominated by the development partners and that, in case of disagreement, the development partners could jointly withdraw their support, leaving the country with a large financing gap. As a result, some governments may have preferred separate arrangements to spread the risk.99 150. It is imperative that the Bank strengthen its ability to coordinate with other partners, among others with regional development banks, by engaging in productive partnerships. Continued efforts to improve cooperation must be pursued. According to the results of a survey (MOPAN – Common Approach, 2009), “twenty-four percent of all respondents consider that the Bank’s performance in relation to donor coordination is an area for improvement. They particularly point to a lack of cooperation, coordination, and alignment with other donors.�100 151. At the same time partnerships need to be reformed to focus on development results and country ownership rather than on donor-driven process. Efforts must be made to reduce the bureaucracy associated with JBS. It is time to consider whether formal MoUs associated with JBS have become more of a hindrance than a help to JBS partnerships. A robust division of labor among development partners, the principle of sharing relevant documents and conducting joint missions can be achieved through good understanding among partners, based on accepted principles, without the difficulties and costs associated with MoUs. OPCS is finalizing guidance on how to engage in effective JBS partnerships, in particular the need to avoid overly rigid and process-driven approaches 98 IEG’s Evaluation of Poverty Reduction Support Credits: An Evaluation of World Bank Support. 99 IEG (op. cit.). 100 MOPAN Common Approach – World Bank 2009, page 22. 64 in the context of MoUs, to maximize benefits and minimize potential costs. OPCS will be coordinating this guidance with the Legal Department and the Regions. 2. Focus on Budget Transparency in DPOs 152. In April 2011, the Bank clarified its focus on budget transparency in DPOs, confirming that it generally extends DPOs only to countries that publish their budgets, or are committed to publishing them within 12 months. Operational policy already states that the decision to extend a DPO is guided by the Bank’s review of the country’s fiduciary arrangements, which informs Bank decisions on the amounts of the DPO, program content and risk mitigation measures. Specifically, this clarification provided that: (i) the pre-appraisal package would confirm that either the executive budget proposal or the enacted budget is published, that is, made available to the public in printed form or on a website; and (ii) in certain exigent circumstances, when the budget is not being regularly published, the DPO would be approved only if the government gave assurances that the budget would be published within 12 months. 153. As of 2010, the Bank had extended DPOs to only six countries that published neither the executive budget nor the enacted budget. These countries were: Comoros, Djibouti, Eritrea, The Gambia, Iraq, and Togo. These DPOs were granted in exigent circumstances (see Table 7). The DPO to The Gambia, the 2010 Public Sector Reform and Growth Grant, supported the government’s poverty reduction strategy by, among other things, requiring submission of government accounts to the audit office and, in due course, to the National Assembly. Togo has received four DPOs since the Bank’s reengagement in 2008, starting with an arrears clearance operation, and all operations have had a strong focus on public financial management. While quarterly budget execution reports have been published since 2009 (a reform supported by the Third Economic Recovery and Governance Grant), the enacted budget became publicly available at the Ministry of Finance Website only in 2011, before the negotiations of the Fourth Economic Recovery and Governance Grant. Table 7. Pre-2010 Recipients of DPOs that did not Publish their Budgets Date of Country DPO Summarized comments Comoros 2010 Fragile state. The Economic Governance Reform Support Grant of 2010 was part of a broader reengagement effort on the part of the international community. Among the key prior actions were to produce a consolidated budget for FY10, using a participatory process among all concerned stakeholders, and to produce quarterly budget execution reports. The participatory process of elaborating the budget was akin to, if not tantamount to, publication of the budget. The PD signaled that there were both “significant systemic risks� and “significant fiduciary risks.� Djibouti 2008 The Food Crisis Response Development Policy Grant ($3 million) responded to a sharp rise in food prices. The prior actions were to exempt basic food items from the domestic consumption tax and to prepare a plan for direct support for vulnerable households. Eritrea 2003 Post-conflict country. The Supplemental Financing to the Emergency Reconstruction Project ($15 million) mitigated the consequences of a devastating drought. Iraq 2010 Post-conflict country. Government had requested $1 billion, and the Bank granted a DPO of $250 million. Fiduciary risks were recognized as “high� and impossible to mitigate. Program content included procurement, chart of accounts, budget strategy, and auditing. 65 154. Since the April 2011 clarification, all countries that received a DPO published the budget and all PDs stated that the budget was published. The countries involved are assessed as publishing their budgets regularly. The program content of several operations approved recently included additional elements of transparency. For instance, in 2011 the Fifth Economic Reform Support Grant for Burundi supported the publication of budget execution reports on the government website. In the Dominican Republic, in the context of the 2011 Third Development Policy Loan on Performance and Accountability of the Social Sectors, the authorities launched an online budget transparency tool that facilitates public consultation on government revenues and expenditures, including data on execution. 155. While there is a positive trend worldwide for increasing transparency,101 publishing the budget is just one of the several essential elements needed for transparency and accountability. Public availability of key budget documents varies widely (see Table 8). The document most likely to be available in the public domain is the enacted budget, and the least likely to be available is the audit report. Table 8. Public Availability of Key Budget Documents Number of countries Percentage of the 97 countries Item that publish participating in the survey Executive’s budget proposal 72 74.2 Enacted budget 82 84.5 In-year budget execution reports 71 73.2 End of year budget execution report 73 75.3 Audit report 61 62.9 Source: International Budget Partnership (2010), Table 1. Note that the countries participating include some developed countries. 156. There are additional opportunities for increasing transparency. Auditors’ reports on the executed budget or the financial statements are less well represented in the sample. Consolidated financial statements are becoming more common, but their incidence is still low. The use of international public sector reporting standards by national governments in preparing their financial statements is infrequent, thus diminishing the transparency of the governments’ operations and their accountability for financial performance. This is now becoming part of many reforms supported by DPOs. Procurement is another area in which transparency is of critical importance. Since public procurement is the largest part of a government’s operating budget after payroll and it offers many opportunities for fraud and corruption, it is an area of material fiduciary risk. Transparency helps address this risk—full transparency on the annual procurement plans of individual ministries and the government as a whole, all upcoming bidding opportunities, and full details on bid awards, appeals launched and resolved, and other summary statistics on procurement. 101 International Budget Partnership, 2010 Open Budget Survey, p. 4. 66 VI. CONCLUSIONS: FROM RETROSPECTIVE TO PROSPECTIVE 157. Development policy lending has continued to be a flexible instrument to respond to client country needs. In the context of a global economic crisis, once more the nimbleness of the instrument proved to be valuable in supporting critical reforms and pursuing key development outcomes across a broad range of countries – from upper middle- income countries such as Croatia to fragile and post-conflict states such as Liberia. Although development policy lending provides only a very small portion of client countries’ increased financing needs, client countries, international financing institutions, and other development partners continue to value the convening framework for policy dialogue and structural reforms it offers. Public sector governance reforms have continued to be among the core targets of DPO supported reforms. At the same time, the share of social development and protection, and economic management in prior actions has risen, highlighting the pressures and needs arising from the global economic context. 158. DPOs have been generally successful in achieving the intended development results, and program results frameworks have improved. According to available ICRs, the Bank has rated approximately 91 percent of the DPOs approved since FY05 as moderately satisfactory or higher (and 63 percent as satisfactory or higher). IEG also has a positive assessment, rating 81 percent of the operations as moderately satisfactory or higher and 49 percent as satisfactory or higher.102 However, this is based on a less than complete universe of evaluations, particularly in FY09 and FY10 (65 and 58 percent of the exits have been evaluated by IEG, respectively). Thus, as more IEG reports become available, the percentage of operations rated as moderately satisfactory or better may change. According to the limited data, the level of achievement of results has been generally higher in DPOs than in investment lending operations between FY09 and FY11. Furthermore, as evidenced by the analysis in this Retrospective, there has been significant progress in the design of results frameworks, and particularly in measuring results and the linkages between supported reforms and development outcomes. 159. A larger share of DPOs commitments went to better fiduciary and governance performers, and countries with stronger fiduciary and governance environments received a larger share of Bank financing in development policy lending (as compared to investment lending). Conversely, countries with weaker fiduciary and governance tended to obtain the bulk of their Bank financing through investment lending. Overall, the analysis suggests that the Bank’s regional and corporate decisions on the allocation of funds is informed and influenced by the risks arising from the fiduciary and governance environments in client countries. 160. Despite significant progress made in several areas that the 2009 Development Policy Lending Retrospective highlighted as needing strengthening, there is room for further improvement. With regard to participation and consultations, Program Documents now do a better job of describing the country’s arrangements for consultation relevant to the program supported, but further improvement would enhance the DPOs. There are also significant improvements in highlighting the fiduciary risks. On the macroeconomic analysis 102 Based on 268 IEG evaluations and 332 ICRs available. Figures reported for both ICR and IEG ratings will be updated as more evaluations become available. 67 in PDs, progress has been achieved in emphasizing the forward-looking discussion of macro policies and presenting the bottom-line assessment of the adequacy of the macroeconomic policy framework. Nevertheless, even in these specific areas of progress, continued efforts are needed to achieve full consistency across the Bank. Below, in more detail, are some of the key areas that require further attention and work. Areas for Strengthening and Future Directions 161. This section highlights areas that could be further strengthened and discusses areas for future work. 162. Despite progress achieved in improving policy and results frameworks, there is room for further improvement. Efforts could be placed on avoiding prior actions of process or preparatory nature, focusing only on critical policy and institutional actions that would drive the expected outcomes. This is particularly the case of sector-specific operations and operations in the context of JBS. Further attention could also be given to enhance the linkages of attribution between policies and outcomes. There is also a need to improve the relevance of the results and measurement by thinking more consistently ex-ante about the monitoring and evaluation systems available at the country level to generate the data needed to feed baselines and measure targets. A pilot program of impact evaluation for DPOs will be established to assess in more depth the relevance and longer-term criticality of results delivered, particularly in countries that have received consecutive series of DPOs over the years. 163. To improve the monitoring of results it is critical to assess the M&E systems of countries more systematically and support their strengthening if needed. A limited proportion of DPOs – less than one-third – included an assessment of the strengths and weaknesses of the countries’ M&E arrangements. Nonetheless, most DPOs that identified weakness also discussed measures to overcome these weaknesses. In some cases, a prior action that explicitly dealt with the M&E issue at hand was included, while in others a combination of technical assistance and prior actions were used. Technical assistance was provided by the Bank or other parties. Overall, more effort needs to be undertaken to consistently assess countries’ M&E systems and support measures to overcome their shortcomings. 164. DPOs could more thoroughly and consistently assess risks and opportunities. While most operations do describe the macroeconomic, governance, implementation, and other risks to the operation, there is some unevenness in the quality of the discussion of these risks and the potential mitigation factors. It is also hard to compare risk sections across DPOs, and thus to acquire a sense of the different levels of risks across countries. To that end, a more systematic approach to discuss risks in DPOs will be established. 165. Despite improvements over the past years, there are still some weaknesses in macroeconomic assessments as regards the coverage of key macroeconomic areas necessary to assess the adequacy of the macro framework. These include, for example, the frequent absence of analysis of the expenditure composition and revenue structure in the budget being supported by a DPO. A more thorough discussion of monetary policy and external sustainability issues would also be welcome. There seems to be some inconsistency 68 across the Bank in the approach to macroeconomic monitoring and modeling that is reflected in the quality and comprehensiveness of the discussion in the macro sections. Shortcomings in macroeconomic assessments in DPOs are more commonly found in sector-specific DPOs. Given that OP 8.60 mandates the Bank’s own assessment of the macroeconomic policy framework, whilst taking views from the IMF, the regions and the PREM Network could work to improve the tools to analyze and monitor macroeconomic developments. For sector- specific DPOs, teams could act earlier to enlist the more active involvement of their regional PREM colleagues in preparing this section of the PDs. OPCS will partner with the PREM Anchor to prepare a good practice note on assessing macroeconomic policy frameworks in DPOs and carrying out cross-sector collaboration for this purpose. 166. While the analytical underpinnings of the programs supported by DPOs seem strong, they could sometimes be better highlighted. The Bank has continued to produce core pieces of relevant analytical work (i.e., Country Economic Memoranda, Public Expenditure Reviews, and Poverty Assessments). While all PDs reviewed showcased a list of AAA that informed DPOs, they did so without much clarity as to the relevance of each AAA to the specific program of prior actions. This area could be strengthened by being more precise about the links between the specific recommendations/advice of AAA work and the selection of the reforms to be supported. This would highlight that the Bank is leveraging the knowledge provided to its clients through AAA work, and transforming advice into reforms and development results. It would also highlight the intrinsic value-added of the Bank in informing client countries’ reform agendas, particularly in MICs. 167. A broader and ongoing effort to strengthen the results and client orientation of the Bank's knowledge work has been outlined in the Bank’s first report on its knowledge work. The emphasis is on tools and systems to enable clear articulation of the reform to which the knowledge work will contribute and the results expected to be supported. Results are defined in terms of how well the knowledge work has informed policy dialogue and capacity support to client countries. This ongoing work will help to strengthen the analytical underpinnings of DPOs. 168. It is critical to continue to make efforts to assess consistently the poverty and social impacts of prior actions supported by DPOs. Overall, there is progress in the extent to which the Bank undertakes poverty and social impact analysis where required. Nonetheless, in FY09 and FY10 there was assessed to be a decline in the percentage of prior actions with poverty/social impact analysis where this was needed. This is partly explained by the increase in the number of operations and prior actions with likely adverse poverty and social effects, given the nature of many reforms supported (for example, fiscal consolidation). In the future, more emphasis should be placed on doing upstream work on poverty and social impact analysis that can inform the design of policy reforms supported by DPOs rather than analysis of the likely distributional impacts of reforms when they are close to being implemented. Gender aspects in poverty and social impact analysis could also be more thoroughly highlighted. 169. The clarification on expecting DPO recipient countries to publish their budgets was a positive step, and further attention to budget transparency could be pursued. For 69 example, the publication of budget execution reports could also become an area of focus for DPOs, although in circumstances of constrained capacity, this can take time to achieve. 170. Coordination with development partners, as well as Regional Development Banks, is a key objective for the Bank and every effort needs to be made to enhance a collaborative engagement. External feedback suggests that the Bank may have room to improve its partnerships at the country level, and particularly in the area of budget support. At the same time, greater flexibility is needed to (i) enhance government ownership and support countries evolving priorities; (ii) avoid undue bureaucracy and disproportionate focus on the coordination of partners’ processes; and (iii) enable Bank teams to engage effectively in policy dialogue that is consistent with country goals and challenges, the Bank’s Articles of Agreement and Board-approved operational policy. In overly rigid Joint Budget Support (JBS) frameworks, usually governed by MoUs, this flexibility has been undermined. 171. Pursuing the fundamental principle of coordinating with other partners, OPCS is finalizing guidance on how to engage in effective JBS partnerships. The objective of the guidance is to strengthen the Bank’s ability to coordinate its DPOs within effective JBS partnerships that are led by the Governments and avoid overly rigid and process-driven approaches. OPCS will be coordinating this guidance with the Legal Department and with regional teams. 172. DPOs in political subdivisions—supporting reform at the level of the government that provides most services to the public—have significant potential. Increasing urbanization across MICs and the related fiscal, social, infrastructure, and service delivery issues that political subdivisions face indicate a strong potential interest and demand for DPOs. But there are also risks, which are common at the national level and are accentuated at the sub-national level, that need to be taken into account when preparing DPOs for political subdivisions. They include: fiscal pressures, high vulnerability to macro shocks, high vulnerability to political cycles, fiduciary issues, and implementation capacity. In this context, Bank staff would need to perform thorough ex-ante assessments of the adequacy of the intergovernmental fiscal relations in the country (i.e., expenditure responsibilities, revenue assignments, intergovernmental fiscal transfers, and sub-national borrowing) and consistently perform DSAs at the level of the political subdivision. The context of political and administrative decentralization issues needs to be taken into account as well as the risks arising from limited implementation capacity. OPCS will update existing guidance based on the experience over recent years and foreseen challenges. On the fiduciary side, a good understanding of the sub-national budgeting and fiscal management environment would also be needed. OPCS has developed guidance in this area. 173. There is scope to use Policy-Based Guarantees more. The use of PBGs could be encouraged in several country niches: for example, in small countries with good policy performance but limited access to capital markets, countries moving out of IDA and into IBRD, and medium-size lower-middle-income countries with limitations in accessing or re- accessing capital markets. For IDA-only countries, Management has explored in an Approach Paper the possibility of extending PBGs to such countries provided that they have a low risk of debt distress and adequate management capacity. Expanded training and deployment of required specialized staff could help further facilitate the use of PBGs. On the 70 operational policy side, teams could discuss with governments the possibility of transforming one or more operations in a programmatic series of DPOs into PBGs without interrupting the programmatic engagement.103 174. Regional/multi-country DPOs are feasible under OP 8.60 and could bring significant positive benefits to client countries where regional and cross-border issues need to be addressed. DPOs could be an excellent vehicle to deal with multi-country policy challenges such as regional integration, resource management, infrastructure policy, and investment climate. DPOs could foster mutually beneficial reforms with positive regional externalities. No regional/multi-country DPO has been approved to date. Teams in AFR are developing proposals for operations involving such innovative methods. Such novel approaches deserve support given the potential regional benefits. 103 The Approach Paper on Modernizing the Bank Operational Policy on Guarantees provides a more detailed discussion of potential reform options. REFERENCES Action Aid International, 2006. What Progress? A Shadow Review of the World Bank Conditionality. Beynon, Jonathan and Andra Dusu, 2010. Budget Support and MDG Performance, Development Paper No. 2010/01, European Commission. de Renzio, Paolo, 2009. Taking stock: What to PEFA Assessments Tell us about PFM Systems Across Countries? Working Paper 302. Overseas Development Institute. Christian Aid, 2006. Challenging conditions: a new strategy for reform at the World Bank and IMF. Eurodad, 2008. Turning the Tables: Aid and Accountability under the Paris Declaration – A Civil Society Report. Faust, J. and Svea Koch, Nadia Molenaers, Heidi Tavakoli & Jan Vanheukelom, 2012. The Future of EU Budget Support: Political Conditions, Differentiation and Coordination. European Think-Tanks Group in collaboration with the Institute of Development Policy and Management, University of Antwerp International Budget Partnership, 2010. Open budgets. Transform lives. The Open Budget Survey 2010: Data tables. www.openbudgetindex.org. Independent Evaluation Group (IEG). 2009. The World Bank Group’s Response to the Global Economic Crisis – Phase I. World Bank. ___ .2010a. World Bank Engagement at the State Level. The Cases of Brazil, India, Nigeria, and Russia. World Bank. ___, 2010b. Poverty Reduction Support Credits – An Evaluation of World Bank Support. World Bank. ___, 2010c. Analyzing the Effects of Policy Reforms on the Poor. An Evaluation of the Effectiveness of World Bank Support to Poverty and Social Impact Analyses. World Bank. ___, 2011a. Results and Performance of the World Bank Group, World Bank. ___, 2011b. World Bank Country-Level Engagement on Governance and Anti-Corruption: An Evaluation of the 2007 Strategy and Implementation Plan. World Bank. ___, 2012a. The World Bank Group’s Response to the Global Economic Crisis – Phase II. World Bank. ___, 2012b. Results and Performance of the World Bank Group. World Bank. ___, 2012c. Strengthening Governance, Tackling Corruption. The World Bank’s Updated Strategy and Implementation Plan. Kauffmann, D., Aart kray & Massimo Mastruzzi, 2010. The Worldwide Governance Indicators – Methodology and Analytical Issues. Policy Research Working Paper 5430. World Bank. Krishnamoorthy, K. and Jessica Thomson, 2002. Hypothesis Testing about Proportions in Two Finite Populations. The American Statistician, 56/2 (August), pp. 215-222. 72 McElhinny, Vince, 2011. The World Bank and DPLs: What Middle Income Countries Want. Bank Information Center, February. MOPAN, 2009. MOPAN Common Approach – World Bank 2009. OECD-DAC, 2012. Evaluation of the Implementation of the Paris Declaration – Synthesis Report. Overseas Development Institute, 2010. Sector Budget Support in Practice, Synthesis Report. Oxfam, 2010. Oxfam Contribution to the EC Green Paper on the Future of Budget Support: Ensuring EC Budget Support is Long-Term, Predictable and Accountable. Oxford, UK. Tavakoli, H. and Gregory Smith, 2011. Insights from Recent Evidence on Some Critical Issues for Budget Support Design, Background Note, Overseas Development Institute. Tavakoli, H. and Edward Hedger. 2011. Pieces of the Puzzle: Evidence, Dilemmas and the Emerging Agenda for Budget Support. Overseas Development Institute. World Bank, 2006. Development Policy Lending Retrospective, Operations Policy and Country Services. ___, 2010a. 2009 Development Policy Lending Retrospective: Flexibility, Customization and Results. Operations Policy and Country Services. ___, 2010b. 2010 IDA16 Second Replenishment Meeting Summary Note – Supporting Development Results. IDA Resource Mobilization Department (CFPIR). ___, 2011a. National Aid Coordination Fora: What, Where and Why? Operations Policy and Country Services. ___, 2011b. Aid Coordination in IDA countries: The Role of the World Bank. Operations Policy and Country Services. ___, 2011c. Aid Coordination in IDA countries: A Roadmap to more Effective Aid Coordination. Operations Policy and Country Services. ___, 2011d. Providing Budget Aid in Situations of Fragility: A World Bank- African Development Bank Common Approach Paper. Operations Policy and Country Services. ___, 2011 e. Public Financial Management Reforms in Post-Conflict Countries – Synthesis Report. Global Center on Conflict Security and Development and Poverty Reduction and Economic Management Network. ___, 2011 f. State of the World Bank Knowledge Services: Knowledge for Development. ___, 2012a. De-fragmenting Africa: Deepening Regional Trade Integration on Goods and Services. ___, 2012b. Strengthening Governance, Tackling Corruption. The World Bank’s Updated Strategy and Implementation Plan. ANNEX A. NUMBER OF OPERATIONS AND COMMITMENTS Table 1. Regional Distribution of Operations and Commitments ($ millions), FY09 Q4 – FY12 Q3 Commitments Region Operations IBRD IDA TF Total AFR 77 DPOs, 5 supplementals 103 4,630 102 4,835 EAP 30 DPOs, 1 supplemental 5,287 1,603 8 6,898 ECAa 38 12,618 459 14 13,090 LCR 50 14,048 215 0 14,262 MNA 17 2,990 70 120 3,180 SAR 9 2,257 394 0 2,651 Total 221 DPOs, 6 supplementals 37,303 7,370 244 44,916 a The operation to Hungary (P114991) is not included in any table of this annex or this report Table 2. IBRD Commitments ($ millions) and Number of Operations, FY09 Q4 – FY12 Q3 IBRD operations IBRD commitmentsc (includes IBRD/IDA Blends) Fiscal year AFR EAP ECA LCR MNA SAR Total AFR EAP ECA LCR MNA SAR Total FY09a 0 0 2,300 1,095 0 0 3,395 0 0 3 5 0 0 8 FY10 59 2,150 6,353 6,796 1,710 2,107 19,175 2 5 14 17 7 2 47 FY11 9 2,137 2,689 3,508 1,030 150 9,524 1 5 9 13 5 1 34 FY12b 35 1,000 1,276 2,648 250 0 5,209 2 3 4 11 1 0 21 2012 Retrosp. 103 5,287 12,618 14,048 2,990 2,257 37,303 5 13 30 46 13 3 110 2009 Retrosp. 175 4,730 4,552 9,585 1,783 785 21,610 5 8 13 29 8 6 69 a Fourth quarter only; b First to third quarter only; c Including supplementals Table 3. IDA Commitments ($ millions) and Number of Operations, FY09 Q4 – FY12 Q3 Operations IDA commitmentsc (does not includes IDA/trust fund Blends) Fiscal year AFR EAP ECA LCR MNA SAR Total AFR EAP ECA LCR MNA SAR Total FY09a 800 540 0 0 0 55 1,395 12 4 0 0 0 2 18 FY10 1,444 327 261 24 0 314 2,370 20 4 4 1 0 3 32 FY11 1,348 407 77 105 70 25 2,032 23 5 1 2 1 1 33 FY12b 1,038 329 120 86 0 0 1,573 14 3 2 1 0 0 20 2012 Retrosp. 4,630 1,603 459 215 70 394 7,370 69 16 7 4 1 6 103 2009 Retrosp. 3,878 588 219 88 51 2,732 7,556 51 8 11 4 1 16 91 a Fourth quarter only; b First to third quarter only; c Including supplementals Table 4. Trust Fund Commitments ($ millions) and Number of Operations, FY09 Q4 – FY12 Q3 Operations Trust Fund commitmentsc (includes IDA/trust fund Blends) Fiscal year AFR EAP ECA LCR MNA SAR Total AFR EAP ECA LCR MNA SAR Total FY09a 22 0 0 0 40 0 62 1 0 0 0 1 0 2 FY10 35 8 0 0 0 0 43 1 1 0 0 0 0 2 FY11 45 0 14 0 40 0 99 1 0 1 0 1 0 3 FY12b 0 0 0 0 40 0 40 0 0 0 0 1 0 1 2012 Retrosp. 102 8 14 0 120 0 244 3 1 1 0 3 0 8 2009 Retrosp. 41 0 0 10 45 0 96 3 0 0 0 2 0 5 a Fourth quarter only; b First to third quarter only; c Including supplementals 74 Annex A Table 4. Development Policy Lending Commitments ($ millions) and Number of Operations to FCS, FY05-12 Fiscal FCS (commitmentsa) FCS % FCS % FCS (number of operationsb) FCS % FCS % year IDA TF IBRD Total total IDA IDAb TF IBRDc Total Total IDA FY05 266 0 0 266 4 11 8 0 0 8 14 24 FY06 179 0 0 179 2 7 4 0 0 4 8 13 FY07 265 0 0 265 4 10 6 0 0 6 11 17 FY08 952 55 0 1,007 15 36 7 2 0 9 19 24 FY09 269 56 0 325 2 12 9 4 0 13 18 27 FY10 348 0 45 393 2 15 9 0 1 10 12 30 FY11 328 54 10 391 3 16 12 1 1 14 20 37 FY12 270 63 0 333 3 15 7 2 0 9 16 25 Total 2,876 227 55 3158 4 15 62 9 2 73 15 25 a Including supplementals; b Including IDA/TF blends; c Corresponds to IBRD/IDA blends Table 5. Development Policy Lending Commitmentsa ($ millions), FY05-12 Fiscal Year AFR EAP ECA LCR MNA SAR Total FY05 1,115 415 538 3,022 400 1,105 6,595 FY06 1,352 509 1,061 2,614 820 985 7,340 FY07 970 1,085 975 1,388 200 1,662 6,280 FY08 1,780 975 786 1,437 796 920 6,694 FY09 1,675 3,690 4,810 7,172 423 685 18,455 FY10 1,538 2,485 6,614 6,820 1,710 2,421 21,587 FY11 1,402 2,545 2,779 3,613 1,140 175 11,654 FY12 1,325 3,331 3,743 3,434 390 0 12,223 Total 11,156 15,034 21,308 29,500 5,879 7,953 90,829 a Including supplementals ANNEX B. METHODOLOGY FOR EVALUATING THE QUALITY OF RESULTS FRAMEWORKS 1. The construction of the universe of DPOs for evaluating the quality of the results framework was done as follows. First, supplemental DPOs were omitted because these have no prior actions and no results framework. Second, double-counting in the context of programmatic series was avoided. That is, since the results frameworks in the various operations constituting a series are essentially the same, the earlier operations were omitted and only the last in the series, or the most recent operation in an ongoing series, was analyzed. This yielded 170 operations from the universe of 221. A stringent standard was used to determine the quality of the results frameworks. Each prior action was examined to check whether all the following elements are satisfied simultaneously:  The prior action is associated with a result.  There is a clear causal link between the prior action and the result.  The result is distinct from the prior action; it is not a restatement of it. An exception is made for certain prior actions in public finance management for which outcomes cannot readily be found. For instance, a prior action of completing an audit may be appropriate, but a measure of the desired outcome—in the form of less corruption, more effective use of government funds, etc.—is unlikely to be available. In these cases it is acceptable to list the output as a result, e.g. “Timing of audit submission in line with the national law, viz. no more than x months after the end of the financial year�.  The result has a results indicator.  The results indicator is precise, not vague. If a prior action is associated with more than one result, the linkage is deemed satisfactory if at least one of the results satisfies all the conditions above. 2. If all of these conditions are satisfied then the prior action and the result are deemed satisfactory. A further stringent standard is used to gauge the results framework as a whole, setting an arbitrary bar of 70 percent. Thus if at least 70 percent of the prior actions in the results matrix have at least one result that satisfies all the criteria above, then the results framework is deemed satisfactory. ANNEX C. METHODOLOGY PROTOCOL FOR POVERTY AND SOCIAL IMPACT ANALYSIS 1. The Retrospective assessed adherence to OP 8.60 with regard to poverty and social impact of the prior actions for all operations in the Retrospective period (FY09 Q4 - FY12 Q3). This was a high level assessment conducted by staff not involved in the preparation and implementation of the operation. This exercise was not intended to reach definitive assessments on individual operations or prior actions, but was a high-level desk review, based on the information contained in the Program Document, and only intended to provide an overall gauge of the Bank’s performance in this aspect of DPO preparation. 2. It classified each prior action according to potential direct and short-term distributional impact on the poor and vulnerable into one (or more) of three categories: (i) no clear direct and short- term distributional impact was apparent, (ii) a direct and short-term positive distributional impact is possible, (iii) a direct and short-term negative distributional impact is possible. 3. Examples of prior actions considered to have possible short-term and direct positive impacts1 include: increased coverage of a social development or human development program; policies that result in greater access by poor people to social or human development programs or to a social safety net program, or expansion of coverage of such a program; and increased spending for human development programs (for example, increase in budgetary allocation for education or health care). 4. Whenever a prior action was considered to have potentially direct and negative distributional impacts, it was classified as requiring a PSIA. Close attention was paid to three types of actions, in particular: (i) those that marked a change in an existing policy, and hence were likely to have winners and losers; (ii) policies with possible distributional effects, such as a new tax or tariff policy that would increase the burden on certain groups; and (iii) politically sensitive policies for which evidence on the distributional impacts are critical to avoid policy reversals. 5. Given that reforms often involve a series of steps, a prior action is counted as possibly “needing� PSIA if it refers to any stage of a process where policy decision-making is taking place beyond the creation of a committee for a policy. For instance, in a series of prior actions aimed at reducing untargeted subsidies, PSIA was counted as not being required if a committee was being formed; however, it would be needed if policies were being reviewed for implementation. Double- counting was avoided when PSIA was needed in cases where multiple prior actions were aimed at the same reform. 1 Those with positive indirect impacts—for example, audits of public educational institutions, public financial management reforms, and employment surveys—were not counted here. ANNEX D. METHODOLOGY PROTOCOL FOR ENVIRONMENTAL IMPACT ANALYSIS 1. The Retrospective assessed adherence to OP 8.60 with regard to the environmental impact of the prior actions for all operations in the Retrospective period (FY09 Q4 - FY12 Q3). This was a high level assessment conducted by staff not involved in the preparation and implementation of the operations. This exercise was not intended to reach definitive assessments on individual operations or prior actions, but was a high-level desk review, based on the information contained in the Program Document, and only intended to provide an overall gauge of the Bank’s performance in this aspect of DPO preparation. 2. Identifying the potential consequences for the environment from each prior action was complex. In some cases, such as supporting development of legislation to increase energy efficiency in the transport sector, the consequences were relatively easy to determine; in others, such as increased private sector involvement in the distribution of fertilizer, they were difficult to discern, especially as the assessment was based only on the information provided by the Program Document. 3. The exercise classified each prior action according to the potential effects on the environment. Prior actions were classified according to four categories: (i) no clear effect was apparent, (ii) a positive effect is possible, (iii) a significant positive effect is expected; and (iv) a negative effect is possible. 4. When it was considered that prior actions could potentially have effects on the environment, the Program Document was reviewed: (i) to assess whether it included a discussion on the borrower’s systems for reducing adverse effects and enhancing positive effects; and (ii) to determine if there were significant gaps in the analysis or shortcomings in the borrower’s systems, and whether the Program Document discusses how such gaps or shortcomings would be addressed before or during program implementation, as appropriate. ANNEX E. EXTERNAL VIEWS ON DEVELOPMENT POLICY LENDING 1. In previous Retrospectives, civil society organizations raised concerns about weak country ownership and donor-driven conditionality.1,2 Coming into this Retrospective, a consensus has emerged that strong domestic leadership and political support is vital to produce lasting change —a view reflected in Bank policy and practice. In a recent note, Oxfam suggested further areas that need to be strengthened including, for example, the impact of budget support on governance and mutual accountability. It has concluded that budget support is an important instrument with considerable advantages in terms of delivering value for money, pro-poor results, and building and strengthening strong national public systems.3 A recent report from the Overseas Development Institute (ODI) highlights the increasing pressure to show the value for money of aid and the risk of corruption, particularly in fragile countries.4 Recent research results also show that countries receiving large amounts of budget support perform better in terms of the MDGs than those receiving little or no budget support.5 The provision of budget support has also been associated with improved performance in public financial management systems.6 2. Three recent IEG evaluations looked at the Bank’s experience with DPOs. The Evaluation of Poverty Reduction Support Credits7 concluded that there has been a reduction in the number of conditions per operation and an increase in the flexibility with which they have been applied. The IEG evaluation of the Bank’s country-level engagement on governance and anti-corruption (GAC),8 found that, for example, DPOs supported governance reforms and were positively associated with the achievement of Public Sector Management (PSM) objectives. The crisis response evaluation9 raised questions about the way DPOs were used during the crisis. It questioned why the Bank did not make more use of Special Development Policy Loans during the crisis and challenged the adequacy of some stand-alone DPOs in crisis response situations. IEG also recognized that most DPOs during the crisis aimed to improve the cost efficiency of public expenditures and that PFM reforms supported were typically part of longstanding policy dialogue and were underpinned by adequate diagnostic work. 1 Including Christian Aid, Eurodad, Oxfam, and Action Aid International. 2 See example, the following reports: Turning the Tables: Aid and Accountability under the Paris Declaration – A Civil Society Report, Eurodad, April 2008; Challenging Conditions: A New Strategy for Reform of the World Bank and IMF, Christian Aid, July 2006; and What Progress? A Shadow Review of World Bank Conditionality, Action Aid International, September 2006. 3 Oxfam (2010), Oxfam Contribution to the EC Green Paper on the Future of Budget Support: Ensuring EC Budget Support is Long-Term, Predictable and Accountable, Oxford, UK. 4 Pieces of the Puzzle: Evidence, Dilemmas and the Emerging Agenda for Budget Support, Overseas Development Institute, January 2012. 5 See, for example, Beynon, Jonathan and Andra Dusu (2010), Budget Support and MDG Performance, Development Paper No. 2010/01, European Commission, March 2010. 6 See Tavakoli, Heidi and Gregory Smith (2011), Insights from Recent Evidence on Some Critical Issues for Budget Support Design, Background Note, Overseas Development Institute, March 2011. 7 IEG (2010), Poverty Reduction Support Credit—An Evaluation of World Bank Support, Independent Evaluation Group, The World Bank, Washington, DC. 8 IEG (2011), World Bank Country-Level Engagement on Governance and Anti-Corruption: An Evaluation of the 2007 Strategy and Implementation Plan, Independent Evaluation Group, The World Bank, Washington, DC. 9 IEG (2011), The World Bank Group’s Response to the Global Economic Crisis: Phase II, Independent Evaluation Group, The World Bank, Washington, DC. ANNEX F. CONSULTATION FEEDBACK 1. In preparing the Development Policy Lending Retrospective, the Bank has sought feedback from stakeholders since April 2012, through a series of external and internal meetings; a multi- stakeholder roundtable discussion in Washington, DC; multi-stakeholder discussions in Croatia, Tanzania and Uganda; and a dedicated website with an online survey. In addition, there was a consultation event with FCS countries, hosted by the Bank’s Fragile States Hub in Kenya. The engagement included a presentation of emerging issues and recommendations and the discussion was structured around a series of guiding questions to seek input on: (i) the extent to which DPOs have contributed to country results and what can be done to further enhance their results focus; (ii) what can be done to further enhance the risk management framework for DPOs; (iii) the extent to which recent Bank reforms to the operational policy framework have increased the effectiveness of DPOs; (iv) how can the Bank engage in productive partnerships that support country-led development results; and (v) the extent to which DPOs can contribute to further progress in transparency, accountability and participation. Summary of Stakeholder Comments 1. Design of DPOs / Prior Actions / Conditionality / Joint Budget Support  Welcome the decrease in the number of prior actions in Bank DPOs, but would like to see still fewer. It is important that the overall number of prior actions in multi-donor operations be reduced.  DPOs now have prior actions, not conditionality, but they seem to be similar.  Concerns about the quality of the prior actions. Sometimes it seems that there is a “one size fits all� approach; DPOs support “low-hanging fruit� reforms.  Propose to use the word “standard� instead of the word “conditionality.�  It would be useful to have a more detailed breakdown of the prior actions by themes.  Would like to see a change to three-year budgetary tranches as a default option, while doing an analysis every year to ensure accountability.  PSIA should be used ex-ante for the design of policy reforms.  Joint budget support has become an industry of its own. There are immense challenges in coordinating joint budget support. 2. The Impact of DPOs  Surprised by the fact that most reforms supported by the Bank are neutral in terms of short-term poverty and social effects.  While the Retrospective is a very useful instrument, it may not be the right process for evaluating the impact of DPOs. A comprehensive and independent review should be conducted.  Appreciate the Bank’s continuing use of DPOs as they give governments the necessary predictability and flexibility.  Results are difficult to assess because M&E frameworks are seen to be weak, and lack clear baselines.  DPOs have been very helpful in supporting countries’ reforms. Governments said that reforms under their Structural Adjustment Lending programs helped prepare them for the 2008 financial crisis. 3. Transparency and Access to Information  Impressed by the transparency of the roundtable consultation exercise. Appreciate the frankness of the discussion.  Questioning the implementation of the Access to Information Policy in DPOs. Most documents are in English, with no translations to other languages. All transparency and accountability mechanisms should be fortified.  Establish a forum for the Bank and CSOs on the ground to promote freedom of information, rather than conditionality, to establish long-lasting dialogue.  Appreciate the fact that Implementation Completion Reports are now public, but the content was seen as rather diluted and not made available in time.  A statement in the presentation that “all documentation is in public domain� would find strong disagreement within civil society. 84 Annex F  Would like to see commitment to transparency translated into a requirement. Require budget transparency, not only of budget proposals, but also reports of audited accounts.  The Bank should strengthen the standard on budget transparency in broad consultation with stakeholders, including CSOs. 4. Participation  Include as many stakeholders as possible at the design stage of the DPO.  To improve the impact of DPOs, clear guidance on the consultation process during the DPO design stage is needed. CSOs could play an important role in this process.  DPOs could better support country-led results by requiring a more consultative and transparent process, particularly in countries with weak governance and accountability.  Would like to understand how the policies are discussed with the country.  Country ownership is good at the top level but that does not necessarily apply to all levels of the administration.  It would be useful to promote dialogue between civil society, the government and private sector.  Include adequate mechanisms for stakeholder engagement and consultation in the definition of country systems. 5. Accountability and M&E  Suggest using INT whenever needed to investigate financial transactions. The Bank must ensure that anti- corruption officers are able to track down the information.  Lack of information on costs and benefits of DPOs represents an accountability gap. Countries need to strengthen monitoring and evaluation.  Would like to suggest a list of outstanding problems with transparency associated with DPOs, including, among others, timely disclosure of CPIAs and relevant economic and sector work; past, relevant project evaluations and evidence of learning from them, disclosure of the draft results framework, and the overall risk assessment and categorization.  The Bank should be focusing on the quality of governments’ audit reports  DPO funds are not disaggregated in the national budgets. This makes it difficult to evaluate how the funds are being used.  Witnessed weaknesses in the DPO design which are due to weak monitoring and evaluation systems. For example, one participant noted an example where a legal framework on environmental impact is a prior action in a DPO, but the participant averred that the framework is not in place.  Suggest not viewing accountability as an internal Bank process only, but ensuring that external accountability mechanisms are available.  Require a more robust and systematic M&E framework. Consistently engage stakeholders in key-decision making processes consistent with the recommendations in the GAC Updated Strategy and Implementation Plan. 6. Assessment of Risks  Would like to see more analytical assessments of safeguards in DPOs.  Participants noted reforms in the area of natural resource management which have impacts on the environment, but which lack Environmental Assessments, or are considered to be of poor quality.  There seem to be no verifiable criteria for the identification and classification of environmental and social risks. Would like to address this issue during the review process for the Bank's Safeguards’ Policies.  Include a categorization of risk into high and low risk, with differentiated obligations for assessment and mitigation.  The current operational policy is seen as weak in ensuring proper risk mitigation.  To improve risk assessment and management, the Bank should ensure the availability of proper analytical tools to assess both significant impacts including indirect and long-term, and the capacity of countries to mitigate such impacts.  Risk assessments must be inclusive and should include local communities and CSOs. ANNEX G. DETAILED THEMATIC BREAKDOWN OF PRIOR ACTIONS Economic Management 5% Social Safety Nets 8% Debt Management and Fiscal Sustainability 2% Vulnerability Assessment and Monitoring 0% Economic Statistics, Modeling and Forecasting 1% Other Social Protection and Risk Management 0% Macroeconomic Management 2% Social Risk Mitigation 0% Other Economic Management 0% Social Development, Gender and Inclusion 1% Public Sector Governance 40% Participation and Civic Engagement 1% Administrative and Civil Service Reform 6% Conflict Prevention and Post-Conflict Reconstruction Decentralization 0% 0% Public Expenditure, Financial Management and Gender 0% Procurement 20% Other Social Development 0% Tax Policy and Administration 5% Social Inclusion 0% Other Accountability/Anti-Corruption 5% Other Public Sector Governance 1% Human Development 10% Managing for Development Results 1% Child Health 0% E-Government 0% Education for All 4% Education for the Knowledge Economy 1% Rule of Law 1% Health System Performance 4% Judicial and Other Dispute Resolution Mechanisms 0% Nutrition and Food Security 1% Legal Institutions for a Market Economy 1% Population and Reproductive Health 0% Personal and Property Rights 0% Other Human Development 0% HIV/AIDS 0% Financial and Private Sector Development 18% Non-Communicable Diseases and Injuries 0% Corporate Governance 1% Infrastructure Services for Private Sector Urban Development 2% Development 2% Urban Services and Housing for the Poor 1% Regulation and Competition Policy 6% Municipal Finance 0% Micro, Small and Medium Enterprise Support 1% Municipal Governance and Institution Building 1% International Financial Standards and Systems 1% Other Urban Development 0% State-Owned Enterprise Restructuring and Urban Planning and Housing Policy 0% Privatization 3% E-Services 0% Rural Development 3% Financial Consumer Protection and Financial Rural Markets 0% Literacy 0% Rural Policies and Institutions 1% Other Financial Sector Development 1% Rural Services and Infrastructure 1% Other Private Sector Development 0% Other Rural Development 0% Other Fin/Pvt Sector Development 3% Global Food Crisis Response 0% Trade and Integration 2% Environment and Natural Resources Export Development and Competitiveness 1% Management1 7% International Financial Architecture 0% Biodiversity 0% Regional Integration 0% Climate Change 1% Technology Diffusion 0% Environmental Policies and Institutions 2% Trade Facilitation and Market Access 1% Land Administration and Management 1% Other Trade and Integration 0% Pollution Management and Environmental Health 1% Social Protection and Management 12% Water Resources Management 1% Improving Labor Markets 1% Other Environment and Natural Resources Natural Disaster Management 1% Management 1% Poverty Strategy, Analysis and Monitoring 1% 1 There are 14 prior actions in the area of Forestry. ANNEX H. DPOS ─ RETROSPECTIVE UNIVERSE Country Proj ID Name Board Date $ IBRD $ IDA $ TF Africa Region Benin P117287 PRSC 6 04/29/10 0.0 30.0 0.0 Benin P125114 PRSC 6 Supplemental 04/21/11 0.0 22.0 0.0 Benin P122803 PRSC 7 12/19/11 0.0 30.0 0.0 Burkina Faso P099033 PRSC 9 06/25/09 0.0 100.0 0.0 Burkina Faso P115264 EFA-FTI for Basic Education DPO 1 06/29/09 0.0 0.0 22.0 Burkina Faso P117278 PRSC 10 06/29/10 0.0 90.0 0.0 Burkina Faso P119413 EFA-FTI for Basic Education DPO 2 03/24/11 0.0 0.0 45.0 Burkina Faso P122805 PRSC 11 07/26/11 0.0 125.0 0.0 Burkina Faso P126207 Growth and Competitiveness DPO 1* 06/26/12 0.0 90.0 0.0 Burundi P113235 Economic Reform Support DPO 3 10/29/09 0.0 25.0 0.0 Burundi P117510 Economic Reform Support DPO 4 12/09/10 0.0 25.0 0.0 Burundi P119324 Economic Reform Support DPO 5 12/15/11 0.0 35.0 0.0 Cape Verde P113306 PRSC 5 12/17/09 0.0 15.0 0.0 Cape Verde P121812 PRSC 6 12/16/10 0.0 10.0 0.0 Cape Verde P122669 PRSC 7* 06/26/12 0.0 12.0 0.0 Central African Rep. P120534 Economic Management and Gov. Ref. DPO 3 09/17/10 0.0 8.8 0.0 Comoros P117229 Economic Governance Reform DPO 06/01/10 0.0 3.0 0.0 Cote d'Ivoire P117281 Economic Governance & Recovery DPO 3 05/04/10 0.0 90.0 0.0 Cote d'Ivoire P122800 Post-conflict Reconstruction & Recovery DPO 09/15/11 0.0 150.0 0.0 Gambia, The P107398 Public Sector Reform and Growth DPO 05/12/09 0.0 7.0 0.0 Gambia, The P123679 Economic Governance Reform DPO 1* 05/31/12 0.0 6.0 0.0 Ghana P113172 Natural Resources & Environ. Gov.DPO 2 06/30/09 0.0 10.0 0.0 Ghana P113301 Economic Gov. & Poverty Reduction DPO 06/30/09 0.0 300.0 0.0 Ghana P110147 Agriculture DPO 2 06/03/10 0.0 25.0 0.0 Ghana P118188 Natural Resources & Environ. Gov. DPO 3 06/03/10 0.0 10.0 0.0 Ghana P117924 PRSC 7 01/20/11 0.0 215.0 0.0 Ghana P122796 Agriculture DPO 3 05/26/11 0.0 57.0 0.0 Ghana P127314 PRSG 8 01/26/12 0.0 100.0 0.0 Ghana P122808 Agriculture DPO 4* 05/15/12 0.0 50.0 0.0 Guinea P122807 Reengagement and Reform Support DPO 04/21/11 0.0 78.0 0.0 Guinea-Bissau P107493 Economic Governance Reform DPO 1 06/16/09 0.0 8.0 0.0 Guinea-Bissau P114937 Economic Governance Reform DPO 2 06/29/10 0.0 6.0 0.0 Guinea-Bissau P123685 Economic Governance Reform DPO 3 06/21/11 0.0 6.4 0.0 Lesotho P112817 PRSC 2 03/30/10 0.0 25.0 0.0 Lesotho P122783 PRSC 3 06/28/11 0.0 18.0 0.0 Liberia P113450 Reengagement & Reform Support DPO 2 05/21/09 0.0 4.0 0.0 Liberia P117279 Reengagement & Reform Support DPO 3 09/30/10 0.0 11.0 0.0 Liberia P123196 Reengagement & Reform Support DPO 4 10/18/11 0.0 5.0 0.0 Malawi P107303 PRSG 2 05/28/09 0.0 30.0 0.0 Malawi P117238 PRSG 3 06/08/10 0.0 54.0 0.0 Mali P113451 PRSC 3 05/19/09 0.0 65.0 0.0 Mali P117270 PRSC 4 06/03/10 0.0 70.5 0.0 Mali P122483 PRSC 5 04/28/11 0.0 70.0 0.0 Mauritius P116608 Trade and Competitiveness DPL 4 11/12/09 50.0 0.0 0.0 Mauritius P125694 Public Sector Performance DPL 1 03/27/12 20.0 0.0 0.0 Mauritius P126903 Private Sector Competitiveness DPO 1 03/27/12 15.0 0.0 0.0 Mozambique P117234 PRSC 6 11/12/09 0.0 110.0 0.0 Mozambique P122470 PRSC 7 12/16/10 0.0 85.0 0.0 Note: *Operations approved in the fourth quarter of FY12, thus not included in the Retrospective analysis. 88 ANNEX H Country Proj ID Name Board Date $ IBRD $ IDA $ TF Mozambique P126226 PRSC 8 03/15/12 0.0 110.0 0.0 Niger P117286 Growth Policy Reform DPO 2 06/23/11 0.0 52.0 0.0 Niger P129793 Growth Policy Reform DPO 2 Supplemental 02/23/12 0.0 15.0 0.0 Niger P125272 Shared Growth DPO 1 06/26/12 0.0 50.0 0.0 Nigeria P117088 Financial Sect. & Publ. Fin. Management DPO* 07/28/09 0.0 500.0 0.0 Nigeria (Lagos State) P117237 Lagos State DPO 1 03/17/11 0.0 200.0 0.0 Nigeria (Edo State) P123353 Edo State DPO 1 03/29/12 0.0 75.0 0.0 Rwanda P106834 Community Living Standards DPO 1 04/02/09 0.0 6.0 0.0 Rwanda P115816 EFA-FTI DPO 10/09/09 0.0 0.0 35.0 Rwanda P117758 Community Living Standards DPO 2 03/16/10 0.0 6.0 0.0 Rwanda P113241 PRSG 6 03/30/10 0.0 115.9 0.0 Rwanda P117495 Poverty Reduction Support Financing 7 02/24/11 0.0 104.4 0.0 Rwanda P122157 Community Living Standards DPO 3 03/22/11 0.0 6.0 0.0 Rwanda P122247 Poverty Reduction Support Financing 8 11/29/11 0.0 125.0 0.0 Rwanda P126877 Support to Social Protection System DPO 03/20/12 0.0 40.0 0.0 Sao Tome and P116178 Public & Nat. Res. Manag. Supplemental DPO 08/07/09 0.0 2.0 0.0 Principe Sao Tome and P123374 Public Resource Management & Gov. DPO 05/19/11 0.0 4.2 0.0 Principe Sao Tome and P128023 Governance And Competitiveness DPO 1 03/29/12 0.0 4.2 0.0 Principe Senegal P107288 Public Finance Support DPO 06/29/09 0.0 60.0 0.0 Senegal P117273 PRSC 4 06/01/10 0.0 43.0 0.0 Senegal P121178 PRSC 5 05/26/11 0.0 42.0 0.0 Seychelles P114822 DPL 11/05/09 9.0 0.0 0.0 Seychelles P120947 DPL 2 11/23/10 9.0 0.0 0.0 Sierra Leone P107335 Gov. Reform & Growth DPO 3 11/24/09 0.0 10.0 0.0 Sierra Leone P121056 Gov. Reform & Growth DPO 3 Supplemental 06/11/10 0.0 7.0 0.0 Sierra Leone P117822 Gov. Reform & Growth DPO 4 12/20/10 0.0 10.0 0.0 Sierra Leone P126355 Gov. Reform & Growth DPO 5 01/26/12 0.0 24.0 0.0 Tanzania P101230 PRSC 7 06/09/09 0.0 190.0 0.0 Tanzania P117345 PRSC 7 Supplemental 12/22/09 0.0 170.0 0.0 Tanzania P116666 PRSC 8 09/28/10 0.0 115.0 0.0 Tanzania P112762 PRSC 9 03/15/12 0.0 100.0 0.0 Togo P113456 Economic Recovery & Governance DPO 2 04/21/09 0.0 20.0 0.0 Togo P117282 Economic Recovery & Governance DPO 3 05/20/10 0.0 16.3 0.0 Togo P122806 Economic Recovery & Governance DPO 4 04/05/11 0.0 28.0 0.0 Togo P126897 Economic Recovery & Governance DPO 5* 05/24/12 0.0 14.0 0.0 Uganda P101232 PRSC 8 09/30/10 0.0 100.0 0.0 Uganda P117979 Financial Sector DPO 06/30/11 0.0 50.0 0.0 Uganda P097325 PRSC 9 02/28/12 0.0 100.0 0.0 Zambia P107218 PRSC 1 03/30/10 0.0 20.0 0.0 Zambia P117370 PRSC 2 03/31/11 0.0 30.0 0.0 Zambia P126349 PRSC 3* 05/03/12 0.0 30.0 0.0 East Asia/Pacific Region Cambodia P117203 Smallholder Agric. & Social Prot. Sup DPO 07/09/09 0.0 5.0 8.0 Indonesia P113638 DPL 6 09/24/09 750.0 0.0 0.0 Indonesia P115102 Infrastructure DPL 3 09/24/09 250.0 0.0 0.0 Indonesia P120313 Climate Change DPL 05/25/10 200.0 0.0 0.0 Indonesia P117874 DPL 7 11/18/10 600.0 0.0 0.0 Indonesia P118531 Infrastructure DPL 4 11/18/10 200.0 0.0 0.0 Indonesia P122982 DPL 8 11/22/11 400.0 0.0 0.0 Note: *Operations approved in the fourth quarter of FY12, thus not included in the Retrospective analysis. 89 ANNEX H Country Proj ID Name Board Date $ IBRD $ IDA $ TF Indonesia P130048 Prog. for Econ Resilience, Invest. & Soc. Ass. DPL* 05/15/12 2000.0 0.0 0.0 Lao PDR P110109 Poverty Reduction Support Operation 5 08/26/09 0.0 20.0 0.0 Lao PDR P118814 Poverty Reduction Support Operation 6 06/14/10 0.0 20.0 0.0 Lao PDR P122847 Poverty Reduction Support Operation 7 05/26/11 0.0 10.0 0.0 Mongolia P115737 Development Policy Credit 06/25/09 0.0 40.0 0.0 Mongolia P117421 Development Policy Credit 2 10/18/10 0.0 29.7 0.0 Philippines P120564 Food Crisis Response Supplemental 05/20/10 250.0 0.0 0.0 Philippines P118931 To Foster More Inclusive Growth DPL 1 05/19/11 250.0 0.0 0.0 Philippines P125943 Disaster Risk Manag. DPL Cat DDO 09/13/11 500.0 0.0 0.0 Samoa P118636 Economic Crisis Recovery Support DPO 05/12/10 0.0 20.0 0.0 Solomon Islands P126740 DPO 1* 04/26/12 0.0 2.0 0.0 Thailand P114154 Public Sector Reform DPL 11/18/10 1000.0 0.0 0.0 Tonga P121877 Energy Sector DPO 10/19/10 0.0 5.0 0.0 Tonga P126453 Economic Recovery DPO 11/22/11 0.0 9.0 0.0 Vietnam P107062 135 Phase - 2 Support DPO 2 05/21/09 0.0 100.0 0.0 Vietnam P104694 Higher Education DPO 1 06/23/09 0.0 50.0 0.0 Vietnam P111164 PRSC 8 06/25/09 0.0 350.0 0.0 Vietnam P117723 Public Investment Reform DPL 1 12/22/09 500.0 0.0 0.0 Vietnam P115874 Power Sector Reform DPO 1 04/06/10 200.0 111.8 0.0 Vietnam P111182 PRSC 9 06/24/10 0.0 150.0 0.0 Vietnam P116353 Higher Education DPO 2 11/30/10 0.0 50.0 0.0 Vietnam P117610 135 Phase - 2 Support DPO 3 04/26/11 0.0 50.0 0.0 Vietnam P120946 Public Investment Reform DPO 2 05/24/11 87.3 262.7 0.0 Vietnam P111183 PRSC 10 12/15/11 0.0 150.0 0.0 Vietnam P122667 Climate Change DPO 02/02/12 0.0 70.0 0.0 Vietnam P124174 Power Sector Reform DPO 2 03/22/12 100.0 100.0 0.0 ECA Region Albania P116937 Social Sector Reform DPL 04/28/11 25.0 0.0 0.0 Armenia P115626 DPO 1 07/02/09 0.0 60.0 0.0 Armenia P116451 DPO 2 01/11/11 4.0 21.0 0.0 Armenia P122195 DPO 3 02/14/12 30.0 50.0 0.0 Belarus P115700 DPL 12/01/09 200.0 0.0 0.0 Bosnia and P116951 Public Expenditure DPO 1 04/08/10 45.0 66.0 0.0 Herzegovina Bulgaria P115400 Social Sectors Institutional Reform DPL 3 05/14/09 200.0 0.0 0.0 Croatia P117665 Fiscal, Social and Financial Sector DPL 01/12/10 296.8 0.0 0.0 Croatia P122221 Economic Recovery DPL 05/05/11 213.0 0.0 0.0 Georgia P112700 DPO 1 07/02/09 0.0 85.0 0.0 Georgia P117698 DPO 2 07/29/10 10.0 40.0 0.0 Georgia P122202 DPO 3 07/21/11 0.0 40.0 0.0 Kazakhstan P119856 DPL 05/25/10 1000.0 0.0 0.0 Kosovo P112227 Sustainable Employment DPO 1 09/30/10 0.0 6.3 13.5 Kosovo P129327 Sustainable Employment DPO 2* 05/03/12 0.0 0.0 23.0 Kyrgyz Republic P125425 Economic Recovery Support DPO 08/02/11 0.0 30.0 0.0 Latvia P115709 Financial Sector DPL 09/22/09 282.7 0.0 0.0 Latvia P115732 Safety Net And Social Sector Reform S-DPL 1 03/04/10 143.9 0.0 0.0 Latvia P121796 Safety Net And Social Sector Reform S-DPL 2 05/26/11 142.1 0.0 0.0 Macedonia, fYR of P116984 DPL 1 12/15/09 30.0 0.0 0.0 Moldova P112625 Economic Recovery DPO 06/24/10 0.0 25.0 0.0 Montenegro P116787 Financial Sector DPL 1 09/01/11 85.0 0.0 0.0 Poland P116125 Empl., Entrep. & Human Capital DPL 2 06/30/09 1,300.2 0.0 0.0 Poland P117666 Empl., Entrep. & Human Capital DPL 3 06/17/10 1331.3 0.0 0.0 Note: *Operations approved in the fourth quarter of FY12, thus not included in the Retrospective analysis. 90 ANNEX H Country Proj ID Name Board Date $ IBRD $ IDA $ TF Poland P115426 Energy Efficiency & Renewable Energy DPL 06/07/11 1114.5 0.0 0.0 Poland P127433 Public Finance DPL 1* 06/19/12 991.4 0.0 0.0 Public Financial Management, Social Prot. & Romania P102018 07/16/09 423.0 0.0 0.0 Financial Sect. Strengthening DPL 1 Public Financial Management, Social Prot. & Romania P117667 01/20/11 380.5 0.0 0.0 Financial Sect. Strengthening DPL 2 Romania P122222 DPL 3 12/19/11 560.6 0.0 0.0 Romania P130051 DPO – DDO* 06/12/12 1333.0 0.0 0.0 Serbia P108759 Public Expenditure DPL 1 11/17/09 100.0 0.0 0.0 Serbia P115958 Private & Financial DPL 2 11/17/09 100.0 0.0 0.0 Serbia P120399 Public Expenditure DPL 2 04/28/11 100.0 0.0 0.0 Tajikistan P117692 Development Policy Grant 4 06/23/10 0.0 25.4 0.0 Tajikistan P120445 Development Policy Grant 5 06/07/11 0.0 10.0 0.0 Turkey P110643 Electricity Sector DPL 1 06/11/09 800.0 0.0 0.0 Turkey P112495 Restoring Equitable Growth & Empl. DPL 03/23/10 1300.0 0.0 0.0 Turkey P117651 Env. Sustainability & Energy Sect. DPL 2 06/15/10 700.0 0.0 0.0 Turkey P123073 Restoring Equitable Growth & Empl. DPL 2 05/05/11 700.0 0.0 0.0 Turkey P121651 Env. Sustainability & Energy Sect. DPL 3 03/27/12 600.0 0.0 0.0 Ukraine P115143 Financial Rehabilitation DPL 1 09/17/09 400.0 0.0 0.0 Latin America/Caribbean Region Brazil (Alagoas) P103770 Alagoas Fiscal & Public Sector Reform DPL 12/17/09 195.5 0.0 0.0 Brazil (RJ State) P117244 RJ State Fiscal Sust, Human Dev. & Competitiv. DPL 02/02/10 485.0 0.0 0.0 Brazil (RJ Municip.) P111665 RJ Munici. Fisc. Consol. For Efficiency & Growth DPL 07/01/10 1045.0 0.0 0.0 Brazil (RJ State) P122391 RJ Metropolitan Urban & Housing DPL 03/15/11 485.0 0.0 0.0 Brazil (Piauí) P126449 Piauí Green Growth & Inclusion DPL 03/06/12 350.0 0.0 0.0 Brazil (Pernambuco) P106753 Pernambuco Expand. Opport. & Enhanc. Equity DPL 03/22/12 500.0 0.0 0.0 Brazil P126351 Bahia Inclusion & Economic Development DPL* 06/28/12 700.0 0.0 0.0 Colombia P116088 Financial Sector DPL 08/04/09 300.0 0.0 0.0 Promoting an Inclusive, Equitable & Colombia P106708 02/23/10 500.0 0.0 0.0 Efficient Social Protection System DPL Colombia P123267 Fiscal Sustainability & Growth Resilience DPL 1 07/21/11 300.0 0.0 0.0 Costa Rica P115173 Public Finance & Competitiveness DPL DDO 04/30/09 500.0 0.0 0.0 Dominican Republic P115145 Public Finance & Social Sector DPL 11/17/09 150.0 0.0 0.0 Dominican Republic P116972 Perfor. & Account. Social Sectors DPL 1 11/17/09 150.0 0.0 0.0 Dominican Republic P121778 Perfor. & Account. Social Sectors DPL 2 11/23/10 150.0 0.0 0.0 Dominican Republic P125806 Perfor. & Account. Social Sectors DPL 3 11/17/11 70.0 0.0 0.0 El Salvador P118036 Sustain. Social Gains For Econ. Recov. DPL 11/24/09 100.0 0.0 0.0 El Salvador P122640 Disaster Risk Management DPL Cat DDO 02/01/11 50.0 0.0 0.0 El Salvador P122699 Public Finance and Social Progress DPL 06/02/11 100.0 0.0 0.0 Grenada P117000 Economic and Social DPO 06/08/10 4.5 3.5 0.0 Guatemala P112544 Disaster Risk Management DPL Cat DDO 04/14/09 85.0 0.0 0.0 Guatemala P114373 Fiscal and Institutional DPL 2 07/28/09 350.0 0.0 0.0 Haiti P117944 Economic Governance Reform DPO 3 12/08/09 0.0 12.5 0.0 Haiti P118239 Emergency DPO 08/05/10 0.0 30.0 0.0 Honduras P121220 Fiscal Emergency Recovery DPO 11/09/10 0.0 74.7 0.0 Honduras P127331 Reducing Vulnerabilities For Growth DPO 1 12/06/11 0.0 86.0 0.0 Jamaica P113893 Fiscal Sustainability DPL 1 02/23/10 200.0 0.0 0.0 Jamaica P123241 Fiscal Sustainability DPL 2 09/08/11 100.0 0.0 0.0 Mexico P115608 Framework for Green Growth DPL 10/20/09 1,503.8 0.0 0.0 Mexico P118070 Econ. Pol. In Response to the Global Crisis DPL 11/24/09 1,503.8 0.0 0.0 Mexico P112262 Upper Secondary Education DPL 1 05/11/10 700.0 0.0 0.0 Mexico P120134 Framework Adapt. Climate Change Water Sect. DPL 06/10/10 450.0 0.0 0.0 Note: *Operations approved in the fourth quarter of FY12, thus not included in the Retrospective analysis. 91 ANNEX H Country Proj ID Name Board Date $ IBRD $ IDA $ TF Mexico P121800 Low Carbon DPL 11/23/10 401.0 0.0 0.0 Mexico P112264 Strength. Business Env. for Eco. Growth DPL 01/18/11 751.9 0.0 0.0 Mexico P120170 Strength. Social Resilience Climate Change DPL 03/01/12 300.8 0.0 0.0 Mexico P126297 Upper Secondary Education DPL 2 03/13/12 300.8 0.0 0.0 Mexico P123505 Fiscal Risk Management DPL 03/22/12 300.8 0.0 0.0 Panama P115177 Protect Poor Under Global Uncertainty DPL 04/21/09 80.0 0.0 0.0 Panama P123255 Fiscal Manag.& Efficiency of Expenditures DPL 1 05/03/11 100.0 0.0 0.0 Panama P122738 Disaster Risk Management DPL Cat DDO 10/18/11 66.0 0.0 0.0 Paraguay P113457 Public Sector DPL 1 05/05/09 100.0 0.0 0.0 Paraguay P117043 Public Sector DPL 12/13/11 100.0 0.0 0.0 Peru P101177 Results & Account. DPL 2 DDO 04/09/09 330.0 0.0 0.0 Peru P106720 Fiscal Management And Competitiveness DPL 3 11/12/09 150.0 0.0 0.0 Peru P116152 Environmental DPL 2 12/08/09 50.0 0.0 0.0 Peru P118713 Environmental DPL 3 08/05/10 75.0 0.0 0.0 Peru P116214 Fiscal Management & Competitiveness DPL 4 08/26/10 100.0 0.0 0.0 Peru P116264 Results & Account. DPL 3 12/09/10 50.0 0.0 0.0 Peru P120860 Disaster Risk Management DPL Cat DDO 12/09/10 100.0 0.0 0.0 St. Lucia P117016 Economic and Social DPO 06/08/10 4.0 8.0 0.0 Uruguay P116215 Public Sector, Competit. & Soc.Inclusion DPL 1 10/14/10 100.0 0.0 0.0 Uruguay P123242 Public Sector, Competit. & Soc.Inclusion DPL 2 DDO 10/25/11 260.0 0.0 0.0 MNA Region Egypt, Arab Rep. of P112346 Affordable Mortgage Finance DPL 09/24/09 300.0 0.0 0.0 Egypt, Arab Rep of P120470 Financial Sector DPL 3 05/25/10 500.0 0.0 0.0 Iraq P119214 Fiscal Sustainability DPL 1 02/25/10 250.0 0.0 0.0 Jordan P117023 Recovery under Global Uncertainty DPL 11/19/09 300.0 0.0 0.0 Jordan P124441 DPL 1 01/24/12 250.0 0.0 0.0 Morocco P117201 Sustainable Access to Finance DPL 01/26/10 200.0 0.0 0.0 Morocco P112612 Public Administration Reform DPL 4 04/29/10 100.0 0.0 0.0 Morocco P117838 Education DPL 1 06/08/10 60.0 0.0 0.0 Morocco P119781 Municipal Solid Waste Sector DPL 2 12/20/10 138.6 0.0 0.0 Morocco P115659 Urban Transport Sector DPL 03/15/11 136.7 0.0 0.0 Morocco P116557 In Support Of The Plan Maroc Vert DPL 1 03/15/11 205.0 0.0 0.0 Morocco P120566 Skills and Employment DPL 1* 06/12/12 100.0 0.0 0.0 Tunisia P117161 Employment DPL 07/01/10 50.0 0.0 0.0 Tunisia P126094 Governance and Opportunity DPL 06/21/11 500.0 0.0 0.0 West Bank and Gaza P113621 Palestinian Ref. & Develop. Plan DPO 2 05/28/09 0.0 0.0 40.0 West Bank and Gaza P118593 Palestinian Ref. & Develop. Plan DPO 3 09/16/10 0.0 0.0 40.0 West Bank and Gaza P123437 Palestinian Ref. & Develop. Plan DPO 4 03/06/12 0.0 0.0 40.0 Yemen, Republic of P122414 Private Sector Growth & Social Protection DPO 12/14/10 0.0 70.0 0.0 South Asia Region Afghanistan P107921 Strengthening Institutions DPO 06/04/09 0.0 35.0 0.0 Bhutan P111222 Institutional Strengthening DPO 05/26/09 0.0 20.2 0.0 Bhutan P113069 Development Policy Credit 1 11/30/10 0.0 24.8 0.0 India P116020 Banking Sector Support DPL 09/22/09 2,000.0 0.0 0.0 India P108489 Statistical Strengthening DPL 06/01/10 107.0 0.0 0.0 India P108258 E-Delivery of Public Services DPL 03/31/11 150.0 0.0 0.0 Maldives P114463 Economic Stabilization & Recovery Program DPO 03/04/10 0.0 13.7 0.0 Pakistan P102607 Higher Education DPO 09/10/09 0.0 100.0 0.0 Pakistan P115638 Social Safety Nets DPO 09/10/09 0.0 200.0 0.0 Note: *Operations approved in the fourth quarter of FY12, thus not included in the Retrospective analysis.