ICRR 14245 Report Number : ICRR14245 IEG ICR Review Independent Evaluation Group 1. Project Data : Date Posted : 10/22/2013 Country : Vietnam Is this Review for a Programmatic Series? Yes No How many operations were planned for the 5 series? How many were approved? 5 Series ID : S111183 First Project ID : P101724 Appraisal Actual Project Name : Vietnam Poverty US$M ): Project Costs (US$M): 175 179.4 Reduction Support Credit 6 L/C Number : C4332 Loan/ Loan US$M): /Credit (US$M ): 175 179.4 Sector Board : Poverty Reduction US$M): Cofinancing (US$M ): 89.5 89.5 Cofinanciers : European Board Approval Date : 06/21/2007 Commission, Canada, Closing Date : 12/31/2007 12/31/2007 Australia, Denmark, Ireland, Netherlands, Germany, Spain Sector (s): General public administration sector (46%); Other social services (18%); Banking (12%); Forestry (12%); Primary education (12%) Theme (s): Environmental policies and institutions (25% - P); Public expenditure; financial management and procurement (25% - P); Other accountability/anti-corruption (24% - P); Education for all (13% - S); State-owned enterprise restructuring and privatization (13% - S) Second Project ID :P105287 Appraisal Actual Project Name : Vietnam Poverty Project Costs ( US$M): US$M ): 150 137.4 Reduction Support Credit 7 L/C Number : C4487 Loan/ Loan US$M): /Credit (US$M ): 150 137.4 Sector Board : Public Sector US$M ): Cofinancing (US$M): 42.9 43.7 Governance Board Approval Date : 06/28/2008 Cofinancers :Denmark, MDTF Closing Date : 12/31/2008 12/31/2008 (Australia, Canada, EC, Ireland, Netherlands, New Zealand, Spain) Sector (s): Central government administration (34%), General agriculture fishing and forestry sector (22%), General finance sector (22%), Compulsory health finance (11%), General education sector (11%) Theme (s): Environmental policies and institutions (25% - P), Public expenditure, financial management and procurement (25% - P), Other accountability/anti-corruption (24% - P), Education for all (13% - S), State-owned enterprise restructuring and privatization (13% - S) Third Project ID :P111164 Appraisal Actual Project Name : Vietnam Poverty US$M ): Project Costs (US$M): 350 350 Reduction Support Credit 8 L/C Number : C4610 Loan/ Loan US$M): /Credit (US$M ): 350 350 Sector Board : Public Sector US$M ): Cofinancing (US$M): 11.2 9.8 Governance Board Approval Date : 06/25/2009 Cofinancers :Denmark Closing Date : 12/31/2009 12/31/2009 Sector (s): Banking (20%), Other social services (20%), General education sector (20%), General public administration sector (20%), General industry and trade sector (20%) Theme (s): International financial standards and systems (25% - P), Other public sector governance (25% - P), Debt management and fiscal sustainability (25% - P), Social safety nets (15% - P), Education for all (10% - S) Fourth Project ID :P111182 Appraisal Actual Project Name : Vietnam Poverty Project Costs ( US$M): US$M ): 150 150.1 Reduction Support Credit 9 L/C Number : Loan/ US$M): Loan /Credit (US$M): 150 150.1 Sector Board : Public Sector US$M ): Cofinancing (US$M): 17 17.5 Governance Board Approval Date : 06/24/2010 Cofinancers :Denmark, MDTF Closing Date : 12/31/2010 12/31/2010 (Canada, EC, Spain, Switzerland) Sector (s): Central government administration (34%), General agriculture fishing and forestry sector (22%), General finance sector (22%), Primary education (11%), Health (11%) Theme (s): Public expenditure, financial management and procurement (34% - P), Rural services and infrastructure (22% - P), Macroeconomic management (22% - P), Health system performance (11% - P), Education for all (11% - S) Fifth Project ID :P111183 Appraisal Actual Project Name : Vietnam Poverty Project Costs ( US$M): US$M ): 150 143.2 Reduction Support Credit 10 L/C Number : Loan/ Loan US$M): /Credit (US$M ): 150 143.2 Sector Board : Public Sector US$M ): Cofinancing (US$M): 17.3 17.6 Governance Board Approval Date : 12/15/2011 Cofinancers :Denmark, MDTF Closing Date : 06/15/2012 06/15/2012 (Canada, Spain, Switzerland) Sector (s): General education sector (20%), General public administration sector (20%), General finance sector (20%), Health (20%), General agriculture fishing and forestry sector (20%) Theme (s): Macroeconomic management (23% - P), Environmental policies and institutions (22% - P), Other accountability/anti-corruption (22% - P), Public expenditure, financial management and procurement (22% - P), Education for all (11% - S) Evaluator : Panel Reviewer : ICR Review Group : Coordinator : Clay Wescott Iradj A. Alikhani Christopher D. Gerrard IEGPS2 2. Project Objectives and Components: a. Objectives: The main objective of the second Poverty Reduction Strategy Credit (PRSC) series is to help Vietnam develop the economic policies and institutions needed to become a middle -income country (PRSC6 Program Document [PD], p. 17). All five programs shared this main objective . The objective is not stated in the Finance Agreement . Each PRSC is designed to support Vietnam ’s Socio-Economic Development Plan (SEDP). 2006 -2010 SEDP 2006- General Goals: Boost the economic growth rate, achieving rapid and sustainable development, quickly bringing our country out of the low development state . Significantly improve people’s material, cultural, and spiritual life . Boost the industrialization and modernization process and gradually develop a knowledge -based economy, creating foundations to make the country basically become a modern industrial country by 2020. Stabilize politics, orders, and social security . Firmly protect independence, sovereignty, territory, and national security . Continue strengthening and expanding foreign relations and improve Vietnam ’s status in the region and the world. (SEDP 2006, p. 54) 2011 -2015 SEDP 2011- Economy will be developed in a rapid, sustainable manner coupling with innovating growth model and restructuring the economy towards better quality and higher competitiveness efficiency . Social welfare and social security will be ensured while material and spiritual life of the people continue to be improved . Diplomatic activities will be enhanced to promote the efficiency of international integration . Independence, sovereignty, unification and territorial integrity will be firmly defended and political security and social order and safety be maintained, thus creating a foundation for the country to basically become a modernity -oriented industrial nation by 2020. During the first two or three years, the plan focuses on realizing the objectives of stabilizing the macro-economy, ensuring social security, achieving a proper growth rate and strongly expediting economic restructuring and growth model shifting . Taking into account the role that the PRSC had in supporting SEDP objectives, the four sub -objectives supported from the SEDP were as follows (PD, PRSC6, p. 6; ICR, p. 5): Promotion of growth and transition to a market economy that will foster the development of businesses in a business environment crucial to the sustained growth of the country ’s economy; Poverty reduction and social inclusion, enabling all players and stakeholders to participate and to enjoy the benefits of development and growth; Sustainable management of the environment and natural resources as the country seeks to harness the benefits of natural resources and the environment for development and growth while ensuring their sustainability; and Building institutions that can support the country ’s development strategy through improving governance . b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: All five operations operated within the same four SEDP sub -objectives. The following lists the policy areas within each sub-objective, and gives indicative examples from PRSC 6. The other four operations provided additional support for these policy areas . Business Development Includes global integration (reducing barriers to trade), state sector reform (divestiture and better monitoring of performance), financial sector reform (commercial orientation of state owned commercial banks ), private sector development (removing administrative barriers to entry ) and infrastructure (improving financing efficiency and funding levels). Social Inclusion Includes education (expanded coverage of high -quality schooling at lower levels with competition and incentives for excellence at higher levels ), health (improving budgetary balance between health infrastructure, human resources, subsidies and preventative care ), social protection (expanding pension programs ), and gender issues (unifying legal framework to address gender disparities and increases women’s participation in decision making) to support social inclusion. Natural Resources Includes land and forests (strengthening linkages between protective and economic functions of forests, and to promoting ownership by local households and communities), water & sanitation, and environment (improved consultation with communities and reporting back to relevant authorities ). Modern Governance Includes the planning process (transforming from direct allocation of resources to strategic steering of the economy), public financial management (PFM) (strengthening systems and increasing transparency ), legal development (increasing oversight of government actions at national and local levels ), fighting corruption (operationalizing legal and structural framework ), and public administration reform (PAR). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: PRSC6 was approved on June 21, 2007 for US$175 million, disbursed US$179.41 million upon effectiveness (December 3, 2007), and closed on schedule December 31, 2007. PRSC7 was approved on May 12, 2008 for US$150 million, disbursed US$137.24 million upon effectiveness (October 24, 2008), and closed on schedule December 31, 2008. PRSC8 was approved on June 25, 2009 for US$350 million, became effective on October 5, 2009, was fully disbursed, and closed on schedule December 31, 2009. It included US$200 million in supplemental funding (above planned allocation) in light of the global financial crisis . PRSC9 was approved on June 24, 2010 for US$150 million, effective on October 27, 2010, disbursed US$152.06 million, and closed on schedule December 31, 2010. PRSC10 was approved on December 15, 2011 for US$150 million, disbursed US$141.2 million upon effectiveness (April 11, 2012), and closed on schedule June 15, 2012. Each operation also had co-financing as indicated above . The Program Documents don’t distinguish between co-financing and parallel financing, so the above estimates of co -financing by IEG come from internal Bank systems, and consultation with the team. There was parallel financing from Asian Development Bank, UK, and Japan . For PRSC8, there was parallel financing from a multi -donor trust fund (MDTF) comprising funds from Canada, European Commission (EC), Ireland, New Zealand, Spain and Switzerland . A total of US$2 billion was contributed to the series from IDA, co -financing and parallel financing . The PRSCs were denominated in Special Drawing Rights (SDR). According to the team, the differences between appraised and actual amounts are due to fluctuations in SDR/US$ conversion rates. Approval of all operations was aligned with the Government ’s budget preparation cycle except PRSC 10, which was delayed for six months so that key policy actions involving state-owned enterprises (SOEs) could be completed. This alignment allowed most operations to disburse just before the beginning of the Governments fiscal year on January 1, and thus to support the annual budget process. 3. Relevance of Objectives & Design: a. Relevance of Objectives: The PRSC series was initially designed to support the 2006-2010 Five Year SEDP. The first three policy areas are directly relevant to goals set out there, dealing with improving the economy, social sector, natural resources and environment. The fourth policy area also supports the SEDP, which states the goal of developing and improving institutions within the socialist-oriented market economy. (SEDP 2006, p. 126) The objectives are still relevant in the context of the new SEDP. The series is also aligned with the Country Partnership Strategy for Vietnam for FY2012–2016, which continues the Bank's support of the SEDP under three pillars supporting competitiveness, environmental sustainability, and ensuring the opportunity that more citizens can benefit from development as a middle income country. Rating: High b. Relevance of Design: There is a clear logical chain between the sub -objectives under each pillar, the policy areas, and the most of the actions supported by the series and other related government actions taken under the SEDP . Each PRSC operation operated under the same four pillars but had prior actions in varying policy areas . The following table shows the prior actions of each operation by policy area . Prior Actions by Policy Area and Operation PRSC 6 PRSC 7 PRSC 8 PRSC 9 PRSC 10 Global Integration x State Sector Reform x x x x x Financial Sector Reform x x x x x Private Sector Development x Infrastructure x x x x Education x x x x x Health x x x x Social Protection x x Gender x x x Land and Forests x x x Water & Sanitation x x Environment x x x Planning Process x x PFM x x x x x Legal Development x x PAR x x Fighting Corruption x x x x Although some policy areas have only one or two series actions supporting them, they all have related government actions as well. There are some cases where it is hard to attribute the outcomes achieved to the actions supported by the PRSC and the Government . The most important exogenous factor affecting the series was the global financial crisis. This is foreshadowed in the risks section of PRSC 6 (PD, p. 32): “In the short term, the biggest vulnerability stems from possible turbulence in the financial sector �. An important endogenous factor was the government response to the crisis and perhaps persisting with the fiscal stimulus beyond what was needed (ICR, para. 8). Rating : High 4. Achievement of Objectives (Efficacy): Efficacy will be evaluated in terms of achievement of four sub -objectives. All five operations focused on the same four sub-objectives, and therefore efficacy will be discussed at the series level while highlighting contributions from individual operations . Achievement of the four sub-objectives will imply some level of achievement of the overall project objective . It is important to note that the PRSC supported the development goals of the government as set out in the SEDP . Therefore, the Bank was not solely responsible for outcomes in these policy areas, as other programs were occurring simultaneously supporting the same policy areas (cf. PRSC10, PD, Annex 2). The depth and breadth of the reforms, which were more ambitious than that found in most PRSCs, also warrants consideration . Sub- Sub -objective 1: Promotion of growth and transition to a market economy that will foster the development of businesses in a business environment crucial to the sustained growth of the country ’s economy . There were significant achievements in the policy areas within this sub -objective, supported by each operation . Global Integration PRSC7 was the only operation that supported this policy area with a prior action : identifying the social and environmental impact of World Trade Organization accession . Although this is not causally linked to the outcome targets related to global integration (re. manufactured exports, international alignment of standards, client satisfaction with customs, and foreign direct investment inflows ), there were also a number of complementary actions supported by the Government that are causally linked to these outcomes . State Sector Reform All five operations in the series had prior actions to encourage state sector reform that are closely linked to the outcome targets. As noted in the ICR, the Government was able to decrease the share of the state sector in GDP from 38 percent to 33 percent, although some of this was achieved because of poor performance of SOEs (ICR, p. 24). There was also a reduction in the share of state investments in total investments : from 47 percent in 2005 to 39 percent in 2011 (but short of the target of below 1/3 possibly due to slow-down of growth in the aftermath of the financial crisis). PRSC6 can be causally linked to a decrease in the relative size of the state sector, because it required a reduction in the number of SOEs . The regulations for equitizing SOEs called for by PRSC10 may help to reduce the size of the state sector in the future . Other prior actions supported improved management and governance of SOEs, although these were not directly linked to an outcome target . Financial Sector Reform All five operations in the series had prior actions to reform the financial sector . Outcome targets included the proportion of non-performing loans and the number of banks using qualitative criteria for loan qualification . As noted in the ICR, the non-performing loan (NPL) ratio has proven to be quite difficult to measure and the corresponding indicator has not been met – rising NPLs may have been linked to the financial crisis . Seven banks now use qualitative data when deciding loan qualification . In addition to the outcome indicators, significant qualitative achievements were made in the policy area . Equitization plans were made for two state -owned commercial banks under PRSC7 and regulations were issued to enhance public disclosure and communication of State Bank of Vietnam’s (SBV’s) policies and banking sector statistics under PRSC 10. Private Sector Development PRSC10 was the only operation in the series with a prior action in the private sector development policy area . Both private sector development outcome targets were either met or exceeded, with a 50% increase in paid employment since 2005, and a 17 percent increase in private investment as a proportion of total investment . The prior action (a plan to improve corporate and government bond markets ) combined with the many actions supported by the government contributed to these achievements . Infrastructure All operations except for PRSC9 contained prior actions to improve infrastructure . Targets for lowering the number of traffic fatalities and for electricity generated by non -Electricity of Vietnam entities were significantly exceeded. PRSC8 may have played a role in improving competition in the energy sector by improving pricing policies. It is difficult to link the outcome indicators with the prior actions of the series alone, but important achievements were made that support the objective by improving infrastructure that is crucial to sustained growth. For example, steps were taken by PRSC 6 to decentralize infrastructure investment and to clarify the contracts of Build-Operate-Transfer projects. Rating : Substantial Sub- Sub -objective 2: Poverty reduction and social inclusion, enabling all players and stakeholders to participate and to enjoy the benefits of development and growth . Some improvements were made in the policy areas under this pillar to support the objective, but the causal chain between prior actions and output indicators is not consistently strong . Still, there were quality institutional improvements made supported by the PRSC that contributed to long -run social inclusion and poverty reduction . Education All of the operations in the series had prior actions in this policy area . The target for net enrollment was met with minor shortcomings. This outcome indicator was supported by the prior action of PRSC 7 that revised education tuition fees to better reflect market conditions . Other prior actions reflected the long term focus to improve education quality. The outcome indicator on improved education quality has an end date of 2020. Health As noted in the ICR, 100 percent of the population below the poverty line has access to free health insurance, but the average annualized contacts with hospitals for the poorest quintiles fell from a baseline of 0.15 to 0.12. The PRSC series can be linked to improving the proportion of poor people with health insurance through the prior action of PRSC7. Steps were taken to improve the quality of healthcare in PRSC 9 and PRSC10 by working towards the establishment of a unified licensing system for healthcare professionals . The outcome indicator of the proportion of poor people with free health insurance was achieved, contributing to the sub -objective of social inclusion and poverty reduction . Social Protection The target for the proportion of ethnic minorities below the new poverty line was achieved, but there is no evidence that the PRSC series played a role in its achievement . PRSC7 and PRSC8 had prior actions related to improving old-age pensions, but the target in this area was not met . However, the Government adopted a Social Insurance Law making the system financially viable and extending its coverage, which in turn contributed to the sub-objective. Gender While the asset ownership target among women was 80 percent met (rising from 30 percent to 38 percent against a target of 40 percent), the proportion of female delegates in the National Assembly fell from 27 percent in 2006 to 24 percent in 2011. (ICR, p. 24) The prior actions of PRSCs 6 & 7 supported these outcome indicators by improving the legal frameworks regarding gender disparities and by ensuring enforcement of gender equality law. While these prior actions do support the partial achievement of the target for asset ownership, it is not reasonable to believe that the PRSC was solely responsible for this achievement . The prior action of PRSC10 ensured that data on gender disparity will be available in the future . Vietnam has also been successful in increasing education for girls, with close to 50% participation at primary and secondary levels . Rating : Substantial Sub- Sub -objective 3: Sustainable management of the environment and natural resources as the country seeks to harness the benefits of natural resources and the environment for development and growth while ensuring their sustainability . Land and Forests The operation built on Vietnam’s strong forestry record, managing to increase forest coverage from 28 percent in 1990 to 40 percent in 2010. Sixty percent of forest land had been allocated to local communities by 2012, up from 25 percent in 2005, and exceeding the target of 30 percent. However, there were no data on two other outcome targets: forest land allocated to local communities in the highlands, and the issuance of land -use certificates. Forest development guidelines were developed under PRSC 7 and PRSC6 called for a strategy to strengthen the linkages between protective and economics functions and to promote local ownership . Although these reforms seem to be substantial, there is no outcome indicator to demonstrate the quantitative effect that they had towards achieving the objective of sustainable management of the environment . Water and Sanitation PRSC 7 had a prior action which involved allocating institutional responsibilities for integrated river basin management. PRSC9 had two prior actions, one on rural sanitation, and the other on irrigation . Outcome indicators show that the population with access to improved or clean water decreased in urban areas, while it improved in rural areas, exceeding the target . Measures supported by PRSC prior actions and by Government were in the areas of disaster mitigation, irrigation, and sanitation, and there is no direct causal link with the measurable outcomes achieved . However, it is important to recognize that Vietnam has already achieved its Millennium Development Goal target 7c of halving the size of the population without access to safe drinking water and basic sanitation (World Health Organization/UNICEF Joint Monitoring Report, 2010). Environment The ICR notes that donors decided in a clear and open way that based on the level of depth needed on climate change policy dialogue, the Support Program to Respond to Climate Change provided a better vehicle for policy action on climate change. A prior action to approve a national target program on climate change was fully met . Three target outcomes on air and water quality, and on biodiversity, were not monitored . Therefore, it isn’t possible to tell whether the action supported by the PRSC, and other actions by the government, helped to achieve results in this policy area . Rating : Modest Sub- Sub -objective 4: Building institutions that can support the country ’s development strategy through improving governance . The PRSC series stressed the importance of improved governance . While some outcome indicators are too broad to be causally linked to the series, the operations all made important contributions to the achievement of this sub-objective. Significant progress was made in the planning process, public financial management, and fighting corruption policy areas . Planning Process 100 percent of provinces have adopted modern planning principles, achieving an outcome target . The prior action for PRSC6 of allocating state capital expenditure transparently is causally linked to the outcome of 100 percent of provinces having adopted modern planning principles . No progress was made in the other target in this area of provinces issuing guidelines for screening public projects at the pre -approval stage. Public Financial Management As noted in the ICR, key PFM contributions from the PRSC series were made in the areas of external audit, budget transparency and enhanced management of public debt . PRSC series prior actions on audit and debt management can be causally linked to the outcome indicator of ratio of public debt to GDP . Overall, quality improvements were made to support the sub -objective of improving governance in this policy area . One result is a modest improvement in Vietnam’s rating on the Open Budget Index, from 2 out of 100 in 2006 to 19 in 2012. However, this measure of fiscal transparency is still well below the rating of most South -east Asian countries. To address this and other concerns, a Strategy for Finance Development of Vietnam was recently approved for the period 2011-2020. There has been recent progress in the roll -out of the Treasury and Budget Management Information System. Legal Development PRSC6 contributed to this policy area by establishing separate legal, judicial, economic, and budget committees in the National Assembly, helping to support the outcome of 100 percent of draft laws posted on the National Assembly website, which was fully achieved . PRSC8 established a National Bar Association . Both of these prior actions are steps towards improving the legal system and thus to improving governance . The other target outcome in this area – proportion of civil judgments enforced – could not be measured. The International Country Risk Group (ICRG) rating on law and order was unchanged during the PRSC period at 4 out of 6. However, the rating on legislative strength declined from 4 out of 4 to 3. Public Administration Reform (PAR) PRSCs 9 & 10 supported prior actions to bring about more competitive personnel practices and adoption of a PAR Master Plan. Over the course of the series, the proportion of provincial departments that had adopted the one stop shop model (meaning that all government services needed by businesses are available at one place ) increased from 67 percent in 2006 to 100 percent in 2012.The PRSC likely did not have an impact on the achievement of this goal but none the less, important steps were taken towards improving governance through improved public service. The ICRG rating on bureaucratic quality was unchanged during the PRSC period at 2 out of 6. Fighting Corruption The outcome target about perception of the level of constraint that corruption places on business was met . The target on asset declaration by civil servants was partially met by 2009 with 24 out of 43 provinces in compliance, 32 out of 63 central ministries, and 788 verified declarations (against end of series target of 27, 42 and 1000 respectively). Prior actions included modernizing procurement practices, and implementing a monitoring framework for combating corruption . The target on revising the Press Law to encourage better corruption reporting was not met. At a least a portion of the achievement of the sub -objective in this policy area can be attributed to the PRSC series. However, corruption is difficult to measure, and it should be noted that the ICRG measure on corruption worsened during the PRSC period, from 3.5 out of 6 to 2.5. Rating : Substantial 5. Efficiency (not applicable to DPLs): 6. Outcome: The relevance of the objectives and design was high . The program as whole was substantial and ambitious, and evolved over time with each PRSC in light of experience, emerging opportunities and priorities . Three of four sub-objectives are rated substantial, and one is rated modest . Actions supported by the PRSC led to important outcomes in improving the business environment, social inclusion and public governance . There was less support offered in the environment and natural resources, and less achievement . The ICR notes that the majority of the outcome indicators were achieved . In some cases it is difficult to link actions supported by the PRSC with these outcomes, although related actions supported by the Government and the overall program within which the PRSC series was embedded had a role . An IEG evaluation looking at PRSCs across the world in 2010 found the Vietnam PRSC series to be exemplary in its predictability, continuity, and development partner coordination. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risks identified in the ICR closely reflect those identified by the Bank in its most recent country partnership strategy. There are concerns that Vietnam is vulnerable to external economic shocks, making sustained development outcomes contingent upon continued global economic recovery . Loose credit requirements have led to Vietnamese banks holding poor quality portfolios . The CPS highlights that there will be dialogue between the IMF and Vietnamese Government to monitor economic indicators and to maintain a sustainable level of public debt. (p. 34) There are concerns about implementation capacity, but given that the PRSC directly supported government development goals, it is likely that reforms that have been set in motion by the PRSC will be considered for implementation by government . A positive sign for the economy is that foreign investment in the first seven months of 2013 is 20% higher than in the same period last year, and much of it is in non-traditional locations for foreign investors . In addition, for the first time exports of manufactured goods exceeded traditional exports such as oil and minerals . Overall, the government has demonstrated continued support to reforms in the policy areas supported by the PRSC . (SEDP 2011-2015). Another positive sign is that there has been a significant increase in proved oil reserves, up from 0.6 billion barrels two years ago to 4.4 billion as of January 2013. In addition, inflation has declined rapidly since peaking in August 2011, and international reserves are beginning to recover . On the other hand, Vietnam is vulnerable to a range of environmental risks such as increased flooding to rice producing deltas, and these risks need to be addressed with large-scale infrastructure investment, improved spatial planning, expanded research, and enhanced public understanding and behavioral changes . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The design of this series followed the best practice guidelines for PRSCs . Lessons learned were taken from the first PRSC series and applied in the design of the second . The series was aptly designed as the main platform for dialogue between government and 13 other development partners around the budget and other processes as tools to promote pro -poor policies. The series was built on extensive analytical work, including the unique Vietnam Development Report series that is led by the Bank, extensively supported by other development partners, and highly regarded by the Government . The Bank did a good job of aligning series objectives with those of the Government to ensure that there would be substantial support for the program . Important pillars were accurately identified and appropriate levels of support were assigned . The PRSC’s were also supported by sector operations, for example in health, education, water supply and sanitation . The results framework was well constructed, clearly laying out the link between outcome indicators and the PDOs. Although the Bank did establish a large number of outcome indicators that were relevant to the objectives of the series, many of the prior actions were not causally linked to these indicators . This discrepancy is generally for one of two reasons, either that the outcome indicators were too broad and the prior actions too narrow, or that there was simply no reason to believe that project outcome indicators would be reasonably affected by PRSC prior actions . Intermediate outcome indicators should have been included to complete the causal chain . Because of this break in the results chain, it is difficult in some cases to link the prior actions of the series to the achievement of outcomes and objectives, although this improved somewhat over time in later operations in the series, and government supported actions contributed to the outcomes in some cases. More attention paid to the results framework would have resulted in improved relevance of the operations. As noted in the ICR, environmental aspects and other safeguards were covered . Fiduciary aspects were well managed, and risk assessments were realistic . Bank inputs and processes were adhered to, including seeking advice from a wide circle of people (ICR, p. 30). at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: The PRSC series was comprehensive and required a high level of supervision . Supervision measures consisted of annual briefing documents on each policy, as well as an annual workshop to review the broad goals of the PRSC. The Bank did a good job taking the lead role in coordinating policy dialogue with external partners. There were joint annual evaluations of the country's Poverty Reduction Strategy in collaboration with other partners. Government officials appreciated the triangulation among the national plan, the Vietnam Development Review, and the PRSC series . Outcome indicators were changed to reflect changing external circumstances. A 2011 independent evaluation conducted by the government and interested development partners led by the European Union found that participants from both the authorities and the partners found the series improved in later operations, as both sides gained experience, and the program design and implementation, such as the size of the policy action matrix and timely sharing of relevant documents, were modified to adapt to the changing circumstances (PD10, 27). Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Because the PRSC supported reforms laid out under SEDP 2006-2010, the Government was largely supportive of the PRSC. The ICR reports a slowdown in the momentum for structural reforms towards the end of the series, citing a lack of support in dealing with SOEs . There was a lack of progress in resolving structural weaknesses in the banking sector . Serious environmental challenges also remained to be addressed. The government maintained regular dialogues with donors throughout the series . A large number of outcome indicators were established but many were unmonitored due to a lack of data . Government Performance Rating : Moderately Satisfactory b. Implementing Agency Performance: The SBV was the implementing agency for the PRSC series . Citing interviews with Bank staff, donors, and government groups, the ICR reports that SBV performance has been very positive (ICR, p. 32). Program Documents state that the SBV served as a coordinator for technical meetings with relevant government agencies and line ministries that provided an opportunity to go into the specifics of each action, to comment of draft policy documents, and to agree on a time frame for their completion (PD9, p. 18). Many Government actions not supported by the series, but included in the policy matrix, were successfully carried out (PD10, Annex 2). Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The objectives were clearly specified and were reflected by the chosen outcome indicators . The outcome indicators were all measurable in numbers given a specific timeframe . 51 outcome indicators, 3 for each of 17 policy areas under the four sub -objectives, were originally proposed to measure the effect that the series had on achieving results in these areas . An effort was made to limit the reliance on administrative self -reporting and leverage the expected implementation of enterprise and household surveys . Most indicators already had baselines, but some (e.g., quality of surface water and quality of air ) required that standards first be defined and that monitoring systems be put in place . To concur with the ICR, the series could have benefitted from identifying intermediate outcome indicators that linked operation ’s actions to the outcome indicators (ICR, p. 35). b. M&E Implementation: The Vietnam Household Living Standards Survey was conducted and provided baselines for many of the outcome indicators. The survey was conducted bi -annually so there was some lag in monitoring . A significant number of the initial outcome indicators had issues with measurement and were eventually dropped in PRSC 10, where the number of monitoring indicators was reduced to 34, drawing on the results of the mid -term review. The modification of the set of indicators reflects quality oversight by management to react to changing priorities . The final selection of indicators functioned effectively and efficiently . The system can be sustained and can provide good insight about the effect that policy reforms have on the aggregate level . c. M&E Utilization: Little evidence is presented to suggest that outcome indicators were used to tailor the operations . Prior actions seemed to have been influenced more by government will than by evidence from outcome indicators, reflecting that these actions better measured overall government reform success rather than the success of the PRSC . Changes in policy preferences changed the indicators rather than evidence from the indicators having changed policy. However, the discussions over the PRSC allowed donors to raise their concerns with a large number of ministries and agencies. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The effects of the series on the natural environment was judged to be mainly positive, although some potential negative effects need to be managed from private sector development, SOE reform, trade integration and planning processes. The approach was to adopt a comprehensive legal framework to address these issues, carry out analytical work to identify and address gaps, support relevant policy actions through the PRSC, and to support environmental capacity building . b. Fiduciary Compliance: The most recent Country Financial Accountability Assessment (2007) found that there are moderate risks to the proper use, control and reporting of funds managed through government systems , and made recommendations for addressing these risks that the Government has been following up on. A 2002 Country Procurement Assessment Review also provided analytical underpinnings for the operation . Although there was no IMF assessment of the control environment of the SBV, improvements in audit systems were seen as important in the overall fiduciary risk rating of moderate for the series . The Open Budget report on Vietnam confirms that audit reports are published, despite the overall low rating for fiscal transparency discussed above . However, the revision of the State Audit Law is still pending, the internal audit function is only at an early stage, and the quality of audit strategies and methodologies need to be further strengthened to align with international practice . c. Unintended Impacts (positive or negative): None. d. Other: None. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Satisfactory High relevance of objectives and Satisfactory design, and mainly substantial efficacy Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Lessons from the ICR are supported, in particular that donor coordination can strengthen the policy dialogue and deliver benefits to the client . A PRSC series can serve as an effective framework for policy dialogue and interactions between the government and donors . The benefits can be fundamental and manifold, as supported by a number of outside evaluations . PRSCs can be the “one-stop shop� of the policy dialogue, supplemented by sector/thematic DPLs as reform dialogue deepens, and hence raised to a more senior level of government attention than any individual sector effort would have . The role of co-financiers under the PRSC can evolve over time from providers of financing to partners engaged in the preparation of the operations and the policy dialogue with the government . Information flows from the government will expand as trust with the Bank and other partners deepens . This coordination also greatly reduces the transaction costs for the government ministries and agencies (ICR, p. 34). Another lesson is that an incomplete causal chain can lead to difficulty in evaluating the effectiveness of an operation or series in achieving its objectives . When prior actions and targets are directly linked to target outcomes from a PRSC series, there will be a clearer link between the series and the achievement of objectives. Other lessons include the importance of continuity in task team leader (TTL) -- PRSCs 7-10 had the same TTL, and the value of combining policy operations with parallel technical assistance to allow for necessary capacity building. When a government has a sound medium term framework and the program supported by a policy operation is ambitious and broad, a longer series may offer the opportunity to achieve results sought and the flexibility to redirect actions based on experience . In addition, countries facing particularly difficult sectoral challenges should consider a sectoral policy operation to supplement a broader policy operation , allowing more extensive consideration and support to the most difficult issues . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is presents an honest assessment of the PRSC series . It gives a granular picture of the ambition and achievements, as well as necessary next steps to continue the momentum of reforms . However there could be more evidence presented documenting the attribution of operations in contributing to the achievement of program objectives, and alternative outcome measures presented from independent sources . The ICR could have benefitted from a more thorough examination of the causal chain . The ICR should have explained the reason for the difference between appraised and actual US$ costs, and should have included details on the co-financing provided through trust funds . The “lessons� presented in the ICR are actually findings, or interpretations of events that took place in Vietnam . Lessons should show the broader significance of a finding . a.Quality of ICR Rating : Satisfactory