Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) Report Number : ICRR0021315 1. Project Data Operation ID Operation Name P153243 AO-Fiscal Management Programmatic DPL Country Practice Area(Lead) Angola Macroeconomics, Trade and Investment L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IBRD-85260 31-Dec-2016 450,000,000.00 Bank Approval Date Closing Date (Actual) 30-Jun-2015 31-Dec-2016 IBRD/IDA (USD) Co-financing (USD) Original Commitment 450,000,000.00 0.00 Revised Commitment 450,000,000.00 0.00 Actual 450,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Paul Holden Judyth L. Twigg Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Policy Areas a. Objectives The Project Development Objective (PDO) appears in the Program Document (PD p. 1) and was "to strengthen the country’s fiscal management to create the fiscal space needed to better protect the poor and vulnerable." The program objectives were not revised during implementation. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) However, at the commencement of the program, the World Bank envisaged a two-loan series, with phased reform that would evolve from the first Development Policy Operation (DPO) into a second DPO. In addition, a policy-based guarantee of US$200 million was planned to cover a commercial loan. 　 During DPO1, it became apparent that falling oil prices were impacting the macroeconomic situation far beyond what had been anticipated in the PD. Both internal fiscal disequilibrium and the balance of payments situation were deteriorating rapidly. These unsustainable macro imbalances led to the cancellation of the second operation. b. Were the program objectives/key associated outcome targets revised during implementation of the series? --- c. Pillars/Policy Areas There were 3 pillars under the first operation (PD1). They were: Pillar 1: Introducing fiscal rules for the utilization of the Oil Funds and modernizing tax administration and tax policy to increase non-oil revenue collection. This reform was to improve non-oil tax revenue collection in order to reduce the dependence of budget revenue on income from oil. The program aimed to consolidate three tax agencies into one in order to improve efficiency through the creation of a single tax authority, the General Tax Administration (AGT) and the revision of key laws governing taxation. However, DPO1 did not include the implementation of the AGT, which was to have occurred under DPO2 that was cancelled. 　 When the program was being designed, it anticipated that in the second DPO, there would be reforms to the Oil Price Differential account, a fiscal stabilization fund, to make it more transparent and less discretionary. 　 Pillar 2: Increasing value for money by enhancing the efficiency and efficacy of public investment management. The efficiency of public investment (through the Public Investment Program, or PIP) was low, so that the large public investment program, required for the development and maintenance of oil resources and to rebuild the country following the civil war, was obtaining a poor rate of return on investment. The need to improve efficiency was magnified by the need to cut the public investment program as a result of falling oil prices. 　 There were three initiatives under Pillar 2, to be supported by DPO1 (ICR p. 3). • Rationalizing the PIP by eliminating projects that were not ready for approval, while maintaining all social projects; Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) • Introducing indicators to monitor the PIP more closely and also creating a manual for high-risk and high- profile projects; • Commencing integration of the information system of the budget and the PIP by establishing a joint committee and "implementing specific measures" (ICR p. 3). 　 Pillar 3: Moving from untargeted fuel price subsidies to targeted cash transfers and evidence-based policy-making. 　 This pillar supported three key policy initiatives that had already commenced: 　 • Phasing out the subsidies for fuel, which had consumed the equivalent of 6% of GDP between 2010 and 2014 (ICR p. 4). Under DPO1 the subsidies were to be reduced, and it was envisaged that they would be eliminated by the second, subsequently cancelled DPO; • Expanding social protection through the provision of cash transfers to 50,000 families; • Improving the statistical information system, to support evidence-based policy making. DPO1 envisaged the approval by the Council of Ministers of a National Statistical Strategy. it was envisaged that the second subsequently cancelled DPO would obtain statistics on poverty. d. Comments on Program Cost, Financing, and Dates DPO1 was approved and became effective on June 30, 2015 in the amount of US$450 million. A Policy Based Guarantee (PBG) of US$200 million was approved at the same time. DPO1 closed on schedule on December 31, 2016. Of the original loan commitment of US$450 million, all was disbursed. The PBG was extended to June 2017 to allow the Government to negotiate a Eurobond loan. Both DPO2 and the PBG were cancelled on November 13, 2017 due to the deteriorating macroeconomic situation and the poor policy response by the Government. 3. Relevance of Objectives & Design a. Relevance of Objectives At appraisal, the objective of DPO1, to create fiscal space to protect the poor and the vulnerable, was relevant. The poverty rate had declined from 77 per cent to 38 per cent between 2001 and 2014. However, poverty was still widespread, and social indicators such as life expectancy, maternal mortality and child malnutrition remained among the worst in Sub-Saharan Africa (PD p. 3). Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) However, the PDO was only tangentially related to the Government’s long-term strategic development objectives, as set out in the National Development Plan (NDP), 2013-2017. These included the pursuit of development, improving the quality of life, promoting employment of youth, enhancing private sector development, preserving national unity and cohesion, and making Angola internationally competitive (the Republic of Angola Letter of Development Policy p. 3). The NDP did contain macroeconomic objectives related to sound fiscal accounts, and the Letter of Development Policy stated that the decline in oil prices resulted in challenges to achieving this. The PD (p. 32) stated that the DPO "fits under the third pillar of the Country Partnership Strategy (CPS), aiming to build resilience to potential shocks from the global economy", although this is not directly related to the PDO. During the preparation of the DPO, its mandate was expanded to include social protection measures, in order to protect the poor and vulnerable from any adverse effects of the macroeconomic adjustment (ICR p. 11). However, this does not map directly into the objectives of either the NDP or the CPS, both of which focused on much broader development objectives. While the decline in oil prices did impose significant costs on the economy and required substantial adjustments to macroeconomic policy, it is difficult to associate the objectives of the NDP or the CPS with the objectives of the DPO. Nevertheless, the high poverty rate and poor social indicators existing in the country made the objectives relevant. At closing, the objectives remained relevant as little progress had been made in improving either poverty or social indicators. Although the poverty rate was a relevant target, the mismatch between the PDO and both the NDP and the CPS leads IEG to rate the relevance of objectives as Modest. Rating Modest b. Relevance of Design The PD stated that the design of the program was based on extensive analytical work (p. 19), undertaken by the World Bank, the International Monetary Fund (IMF), the African Development Bank, and private sector consultants. Many of the results indicators in the PD were based on the expectation that there would be a second DPO and pertain to outcomes that were expected to be achieved in the second operation, which was cancelled. The theory of change underlying the DPO was that by improving the structure, administration, and implementation of tax and other revenue policies, more resources would be available to assist the poor through the provision of social services and transfers. Relating this to the pillars, higher revenue would come from better management of the oil stabilization fund (OPDA) and improvement in tax administration through the consolidation of the three tax agencies under pillar 1. Under pillar 3, the removal of oil subsidies would increase resources to devote to social transfers. Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) A design issue from the outset was that for the program’s objectives to be realized, significant administrative capacity was required to upgrade and modernize both the institutional framework for public investment and the capacity of the agencies themselves. Under Pillar 1, the PD (p. 22) described the problems with the management of oil revenues through the OPDA, which had resources equivalent to only 1% of total government revenue. The PD stated that supporting fiscal stabilization through the OPDA was a central objective of government policy (p. 22). However, the PD (p. 22) also pointed out that the OPDA had no formal legal basis for performing any formal stabilization function, although it does state that the World Bank was working with the Ministry of Finance to develop fiscal rules for the OPDA, in order for a Presidential Decree to be enacted. The PD stated that the results indicator would be that the fiscal rules linking the OPDA to the budget cycle would be in place by the end of 2017, which related to the cancelled second DPO. For DPO1, pillar 1 supported integrating the tax agencies and customs administration into a single agency (the AGT) and providing it with a governance structure. This was the first prior action under pillar 1. The second prior action was to streamline tax policy to support the modernization of tax administration. The target indicator was for the share of non-oil tax revenue to increase from 13.4% of GDP in 2013, to 15.6% of GDP in 2017. Given the urgency of improving the management of the Oil Funds, the primary focus in the DPO on non-oil tax revenue did not address the fundamental issue identified in the PD and the CPS. Under pillar 2, the efficiency of public investment management was to be improved, with the PD stating (p. 25) that "the quality of public investment management (PIM) is a key determinant of economic growth and poverty reduction." The decline in oil prices necessitated a sharp decline in public investment, so that the PIP needed to become more efficient in order to achieve its objectives. The performance indicator for this pillar was the development and use of manuals and templates for the preparation of projects, particularly those deemed high risk or high profile, with the expected outcome being to ensure better value for money from the PIP. The PD stated that the Ministry of Planning and Territorial Development would proactively monitor the 2016 PIP through a new system of indicators (nearly 400 output indicators and 100 results indicators had been defined). The results indicator for the program was that all projects in the 2017 PIP would be subjected to the new PIM cycle, systems, and regulations (p. 27). 　 A shortcoming in the design of the DPO was the failure to included policy directed at reducing the drain of state-owned enterprises (SOEs) on the budget. The most recent IMF Article IV report (May 2018, p 10) pointed out that the public corporate sector is a burden on the Treasury, that insolvent SOEs should be closed, and that economically viable but inefficient SOEs should be restructured and/or privatized. It is therefore surprising that the DPO did not even mention this issue, especially since public investment efficiency was identified as a key problem, and the IMF Article IV report for 2015 stated that "the authorities remain committed to …strengthen the oversight of state owned enterprises" (p. 3). 　 The elimination of fuel price subsidies and transferring the realized savings to targeted cash transfers and "evidence-based policy-making" was a focus of design (pillar 3). This was to be achieved by raising retail fuel prices in order to start phasing out fuel subsidies, expanding social protection by finishing the registration of Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) 50,000 families in a pilot cash transfer program, approving a national strategy for upgrading statistical information, and preparing to launch a survey on "expenditure, revenue and employment of the population" (PD, p. 31). The indicator to validate reduced fuel price subsidies was that the subsidies would decline from 5.2% of GDP in 2013 to 2.3% by December 2017. It is difficult to see how this could be a meaningful results indicator in practice, since subsidy and GDP data only become available with a significant lag. The full rollout of the cash transfer program was only expected to have occurred by 2018 during the second DPO. Although the reduction in fuel subsidies would negatively affect the poor, the design of the program was based on the assumption that this impact would be attenuated by the regressive nature of the subsidies. Together, the two policy actions were expected to reduce poverty by 2.5 percentage points by 2018. The data collected by the statistical survey would be the basis for measurement of the poverty rate. The results framework required that the percentage of households protected from negative income shocks rise from 3% of the population to 7% of the population by December 2017. The results framework also required that results from the 2015 household survey be published by December 2017. 　 While the design of the program targeted relevant development objectives, its relationship with the PDO was only indirect. The ICR (p. 11) criticized the design of the program for a combination of limited scope of some policy actions, and over-ambition of others. Under the latter, it includes the registry of program beneficiaries and the household survey and poverty map. While the ICR did not relate the PDO to the program design, the criticisms were valid. Limiting the beneficiaries of the program to 50,000 when the total population of Angola is over 30 million suggests that the goal was close to irrelevant. A further question related to design was the extent of the macroeconomic risks at the outset of the program. The PAD (p. 3) described these risks, as did the IMF Article IV report for both 2014 and 2015. They all flagged the possibility of a decline in the oil price, poor implementation of expenditure reduction, delays in structural reform, and failure to maintain exchange rate flexibility, as well as the need to be firmly committed to the program of public financial management reforms and economic diversification. Certainly, with the benefit of hindsight, it is possible to maintain that the World Bank program was overly optimistic in the face of these risks. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To strengthen the country’s fiscal management to create the fiscal space to better protect the poor and vulnerable. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) Rationale The problems of Angola’s heavy dependence on oil revenue became apparent as the oil price declined between 2013 and 2017. It became clear that oil funds needed to be administered more efficiently and that non-oil revenue needed to increase, a goal that was hampered by complex institutional arrangements for the collection of taxes and inefficient tax policies. 　 During the first DPO, three tax agencies were merged, and the framework for a governance structure was published in a series of Presidential Decrees. In addition, tax policies were streamlined to improve compliance, simplify procedures and reduce distortions. A series of Laws and Decrees applied to this objective. However, the implementation of many of the policy measures was envisaged to occur as part of the second DPO, which was cancelled. As a result, the prior actions under the DPO that did occur were primarily inputs. 　 The results indicator requiring fiscal rules that linked the oil price differential account to the budget cycle be in place by end 2017 was achieved, although there is no information on what this meant for better budgetary control. 　 The second results indicator specified that the ratio of non-oil tax revenue to GDP would increase from 13.4% in 2013 to 15.6% by 2017. Initially this target was met (by 2015), but an ensuing recession led to the ratio declining to 10.8% of GDP. This implies that non-oil tax revenue declined proportionality more than did GDP. The substantial decline in oil prices necessitated a significant reduction in the PIP, the negative effects of which could only be attenuated by improving the efficiency of the program. To do this, three actions were undertaken: eliminating PIP projects that were not ready for approval; improving the monitoring of the PIP; and integrating budget information with the PIP. The indicator to measure the success of the readjustment to the PIP was that all (100%) new projects would be subject to the new PIM system by 2017, which would have been part of the discontinued DPO2. This results indicator was not achieved, which the ICR (p. 14) attributed to "the inability of the Bank or other development partners to provide more intensive technical assistance." Although the government took some steps to improve the functioning of the PIM system, regulations that had been drafted to improve it were never put in place, and as a result the PIP was still being formulated and implemented under the previous rules. 　 Fuel subsidies constituted a substantial drain on the budget, and analytical work demonstrated (PD p. 29) that the subsidies were regressive in that they benefited the well-off more than the poor, although their removal would still have negative effects on poor households. In order to mitigate negative impact, the DPO proposed that the subsidies be replaced by the expansion of the unconditional cash transfer program, with targeted enrollment being increased in a phased way. Furthermore, this objective also required the upgrading of statistical information, particularly through the development of a national household survey that would provide information on the extent of poverty and allow the targeting of social programs. Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) There were three results indicators: • Fuel price subsidies decline from 5.2% of GDP in 2013, to 2.3% by the end of 2017. This was achieved, with the actual outcome being 1.3% of GDP. • The percentage of households receiving cash transfers increase from less than 3% of the population, to 7% by the end of 2017 (PD p. 31). This was not achieved, with the actual outcome being 1.4%. • By the end of 2017, the household survey would be completed and the results appear in government documents proposing policy actions. This was not achieved because of lack of funding and capacity to undertake the survey, which was postponed by the Government to the end of 2018. (The survey has commenced at the time of this Review, with funding being provided by a World Bank Technical Assistance Loan.)　 　 Because of the failure to achieve most of the program targets, and the reversal of macroeconomic policies that would have provided fiscal space for assisting the poor and the vulnerable, IEG rates efficacy as Negligible. Rating Negligible PHREVDELTBL PHREVISEDTBL 5. Outcome The objectives of the Angola DPO were not closely related to either Angola’s National Development Plan or the World Bank Country Partnership Strategy. The original DPO series was directed at assisting Angola with adjustment to lower oil prices. While the aim of the adjustment was timely, it is difficult to discern how it fit with longer term goals. Furthermore, the operation depended heavily on appropriate macroeconomic adjustment, which in the event did not occur and resulted in the second DPO being cancelled. 　 The design of the operation did not focus on rapid adjustment to the decline in oil prices and improved uses of oil revenues. At the early stages of the operation the focus was on non-oil tax revenue, which accounted for less than 15% of GDP and over the course of the operation declined by over 20%. While improving the efficiency of non-oil tax collection was undoubtedly important in the medium term, the urgency of adjustment necessitated by the sharp decline in the oil price suggested a focus on the better and more efficient use of oil funds rather than a focus on non-oil tax revenue. There was also an urgent need to improve the efficiency of PIM, but severe delays in implementing a new system for investment meant that the old inefficient system continued. While the fuel Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) subsidies were substantially reduced, the retargeting of savings to poor and vulnerable households was delayed, with only 1.4% of all households receiving cash transfers by the end of 2017. 　 As a result of poorly focused objectives (a rating of Modest), design deficiencies (a rating of Modest) and a rating of negligible for efficacy, the Outcome of the DPO is rated Unsatisfactory, indicating major shortcomings. a. Outcome Rating Unsatisfactory 6. Rationale for Risk to Development Outcome Rating Achieving the desired development outcomes required substantial administrative capacity as well as strong political support. The ICR (p. 8) pointed out that support for the program waned as the fiscal position deteriorated, so that reform was delayed both by the need to address current crises and declining political buy- in for the reforms, which led to the dismissal of the Minister of Finance, who had been a reform champion. 　 Outside the Ministry of Finance, weak institutional capacity continues to hamper the implementation of reforms, particularly in the area of the public investment program. There was Insufficient technical assistance from development partners. This issue impacted the social reform program as well as public investment. At this point, these issues have not been addressed. 　 Presidential decrees have yet to be implemented for the PIP and for the OPDA, both of which are prerequisites for sustainable reform to continue. Substantial uncertainty continues, and there is no obvious evidence that the protection of the poor and vulnerable will increase. A mitigating factor in the risk to the development outcome is the stated commitment of the Government that assumed office in late 2017 to address macroeconomic imbalances. In addition, many of the reforms in the development policy financing (DPF) have been resumed (ICR p. 10). The World Bank is also resuming policy dialogue, and there is the possibility of a new DPF series. The ICR also stated (p. 17) that a contribution of the DPF was promoting institutional change and improved capacity. The ICR expressed optimism that tax reform, better fiscal policy, and the encouragement of evidence-based policy making would lead to improved social policies. a. Risk to Development Outcome Rating Substantial 7. Assessment of Bank Performance Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) a. Quality-at-Entry The ICR (p. 19) noted that this was the first opportunity for policy-based lending in Angola and that there was a critical need for international support during a time of crisis for the country as a result of the sharp decline in oil prices in 2014. World Bank design of the DPO was informed by analysis and a focus on the need to protect the poor and vulnerable from the worst consequences of the crisis. Care was taken to ensure fiduciary compliance under circumstances where public financial management systems were weak and non-transparent. Nevertheless, overly optimistic targets for development outcomes were adopted. In particular, limited domestic capacity was a significant constraint to achieving targets. The necessary reforms of the public sector investment program required major systemic changes, which will require many years to be put in place. Furthermore, the ICR noted that some contemplated policies lacked full political support, which subsequently became a barrier to effective implementation, particularly given the weak institutions that existed in Angola at entry. These weaknesses were compounded by the geographic challenges arising from the remoteness of many communities that were to be helped. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision In the early stages of the program, there was close supervision and engagement with the Angolan authorities. However, as performance under the program deteriorated because of the increasingly problematic macroeconomic conditions, supervision intensity waned (ICR p 19). World Bank recommendations regarding strengthening macroeconomic management were increasingly ignored. Initially consultations had included preparation for the second DPO in the series, but as the macro conditions deteriorated, these were put on hold, and policy discussion were reduced. The ICR (p. 20) noted that this affected not only preparation for the second DPO, but also the intensity of supervision of the first DPO. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory 8. Assessment of Borrower Performance a. Government Performance Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) The Ministry of Finance (MoF) played a central role and was responsible for the execution of the DPO. The ICR reported that the MoF failed to ensure effective implementation of the program. It had responsibility for coordinating the various agencies involved in the operation. The agreed monitoring committee that was to be responsible for tracking the progress of implementation was never formed. The MoF did not develop the automatic fuel price adjustment mechanism. The MoF was also responsible for appropriating funding for the components of the program, but the ICR (p. 21) reported that these were not fully funded and as a result, implementation suffered from limited resources. Furthermore, several decrees that were critical in achieving the success of the program were never approved. Because of major shortcomings in the performance of the MoF, IEG rates Government Performance Unsatisfactory, compared with the Moderately Unsatisfactory rating in the ICR. Government Performance Rating Unsatisfactory b. Implementing Agency Performance The ICR (p. 21) stated that many of the implementing agencies failed to implement the reforms critical to achieving the development objective. This was partly due to lack of sufficient resources and/or capacity. For example, the National Statistical Institute was unable to undertake the required poverty mapping and household survey, although the Bank provided "handholding" (ICR p. 21). The National Directorate for Public Investment contracted with consultants to draft policy reforms required under the program, but these were delayed and never implemented. The General Tax Administration, however, did undertake reforms and implement them in a timely fashion. Implementing Agency Performance Rating Moderately Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Half of the six indicators selected to monitor progress under each pillar of the program were specifically quantitative (ICR p. 9), while the other three were related to documentation that measures required under the program had been implemented. The qualitative results indicators were: Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) • Results indicator 1: The fiscal rules linking the operation of the oil equalization account would be in place by end 2017. This was a trigger for the cancelled DPO2; • Results indicator 3: 100% of all new projects in the 2017 Public Investment Plan would be subject to the new Public Investment Management cycle, systems and regulations. This results indicator was very similar to prior action 4 regarding strengthening the Public Investment Plan; • Results indicator 6: The results from the household survey would be published in at least one official government document by end 2017. This indicator was similar to prior action 9 and trigger 10 for the cancelled DPO2. If the indicators, prior actions and triggers are either the same, or similar, then the monitoring and evaluation framework, by definition, does not provide an independent measure of progress. The MoF was charged with monitoring and evaluating progress of the DPO. Although the PD (p. 41) stated that "to facilitate the coordination of the operation, the government will appoint, as soon as possible, a technical counterpart team, with a main DPF coordinator...(who) will be in charge of the monitoring and evaluation of the operation", the ICR (p. 10) stated that this did not occur. As a result, the various bodies involved in the program became responsible for their own monitoring of progress under the operation. b. M&E Implementation The limited capacity and dispersion of the groups monitoring their own progress resulted in poor implementation of the M&E system, although World Bank teams became involved in monitoring developments and making recommendations for what was required in order to ensure the successful completion of the DPO. c. M&E Utilization The ICR (p. 10) stated that, although numerical indicators became available and were reported in Government documents, and the World Bank team monitored the qualitative indicators during supervision missions, most of the indicators were not used for taking decisions, with the exception of progress on subsidies and non-oil revenue. M&E Quality Rating Modest 10. Other Issues a. Environmental and Social Effects Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) There were no environmental effects. The focusing on social protection was expected to have positive social effects. The overall objective of the program was to increase resources to protect the poor and the vulnerable. b. Fiduciary Compliance There were no reported instances of fiduciary non-compliance. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Unsatisfactory Unsatisfactory --- Risk to Development Substantial Substantial --- Outcome Moderately Moderately Bank Performance --- Unsatisfactory Unsatisfactory Political commitment waned and support for programs Moderately other than those related to the Borrower Performance Unsatisfactory Unsatisfactory budget was weak, to the extent that these programs were inadequately resourced. Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) 12. Lessons The ICR (pp. 21-22) offered a number of general lessons and recommendations, although some of them pertaining to political commitment overlapped. The most important conclusions from the ICR were: Targets for development outcomes should not be overly optimistic, and the time frame for achieving results should be realistic. Many parts of the program were based on the assumption that results that were unachievable, could be achieved. This arose from pressure to design a high-profile program that would garner strong international buy-in. The program did not sufficiently take into account the political and institutional realities in Angola. In a complex operation with multiple pillars, the project team needs to be satisfied that political support is strong and will be maintained through the life of the program. This manifested itself in insufficient resources being provided to fulfill many parts of the program, in particular the cash transfer program. In preparing and supervising a program such as this, strong buy-in and commitment from counterpart agencies is essential. The commitment of the MoF was strong in the areas of its direct interest, such as the subsidy and tax portions. It was much less so in other areas. IEG adds two additional lessons. IEG evaluations have revealed that a common issue related to DPOs is the over-estimation of the capacity of counterparts to implement the program. Design should always take this into account. It is better to have relatively modest goals that can be achieved, rather than an overly ambitious program that fails. At the first indication that Government commitment to the program appears to be waning, intensive consultations are necessary to decide on the program’s future. The ICR noted that as the macroeconomic situation deteriorated, and the Government’s policy measures departed from World Bank recommendations, the intensity of supervision declined. IEG suggests that this should have triggered more intensive consultations and coordination with other development partners in an attempt either to reverse unsuitable policies or cancel the operation at an earlier date. 13. Assessment Recommended? No Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AO-Fiscal Management Programmatic DPL(P153243) 14. Comments on Quality of ICR The ICR was clear and concise. It described and explained the challenges in the design and implementation of the program, providing a clear story line regarding how these evolved and the reasons for the cancellation of the second operation. While some of the lessons it drew overlap and apply generally to DPOs, they were relevant and should be heeded. a. Quality of ICR Rating Substantial Page 15 of 15