ICRR 12768 Report Number : ICRR12768 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/21/2007 PROJ ID : P086360 Appraisal Actual Project Name : Poverty Reduction Project Costs (US$M): US$M ): 180 Strategy Credit IV Country : Vietnam Loan /Credit (US$M Loan/ ): US$M): 100 100 Sector Board : EP Cofinancing (US$M): US$M ): 80 Sector (s): General public administration sector (50%) General finance sector (20%) General education sector (10%) Health (10%) General agriculture fishing and forestry sector (10%) Theme (s): Regulation and competition policy (25% - P) Public expenditure financial management and procurement (25% - P) State enterprise/bank restructuring and privatization (24% - P) Education for all (13% - S) Export development and competitiveness (13% - S) L/C Number : C4091 Board Approval Date : 06/30/2005 Partners involved : Asian Development Closing Date : 12/31/2005 12/31/2005 Bank, European Commission, Japan Bank for International Cooperation, Canadian International Development Agency, UK Department for International Development European Commission, Netherlands Ministry of Development Cooperation, DANIDA Evaluator : Panel Reviewer : Group Manager : Group : Monika Huppi Kris Hallberg Jaime Jaramillo IEGCR 2. Project Objectives and Components: a. Objectives: The overarching objective of the PRSC program was to support implementation of Vietnam's Comprehensive Poverty Reduction and Growth Strategy (CPRGS) which was anchored around three core pillars : (i) transition to a market economy, (ii) social inclusion and environmental sustainability and, (iii) building modern governance. PRSC-4 specific objectives were: (i) Supporting the transition to a market economy : Promoting competition and integration with the world economy by moving towards WTO accession, accelerating divestiture of SOEs in non -strategic sectors, adopting a more determined approach to banking reform and introducing regulatory bodies for infrastructure services . (ii) Social inclusion and environmental sustainability : Promoting human development through improvements of education quality in poor areas . Promoting environmental protection through improved forest management and adoption of a revised environmental regulatory framework . (iii) Building modern governance through the introduction of improved planning methods geared towards attaining development outcomes, increased transparency and accountability of public entities, increased effectiveness of anti-corruption actions and enhanced poverty monitoring . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Transition to a market economy : (i) Integration into the world economy : Progress towards WTO accession by completing a full draft working party report for WTO accession and bilateral negotiations with six partners . (ii) SOE reforms: Narrowing the list of sectors with exclusive state ownership . Strengthening equitization mechanisms for SOEs, including through auctioning of shares and external valuations. (iii) Financial sector reforms : Announce separation of State Bank of Vietnam's supervision functions from State Owned Commercial Bank management function. Issue regulations requiring SOCBs to classify assets and provide for loan losses based on international standards. (iv) Infrastructure: Adoption of an electricity law allowing for equitization of EVN subsidiaries and establishing an electricity regulatory agency. Social inclusion and environmental sustainability : (i) Human Development: Strengthen quality of education by completing evaluation of 7% of primary school teachers in accordance with new teacher profile assessing attitudes, knowledge and pedagogical skills . (ii) Environmental protection: Enactment of law on forest protection and development recognizing forest use rights of households and communities. Submission of amended law on environmental protection to National Assembly and implementing regulations to Government, clarifying line ministries' responsibilities for environmental protection, decentralizing responsibilities within Ministry of Natural Resource Management and Environment to regional /local units and enabling introduction of new tools for environmental management (improved financial mechanisms for environmental protection, public awareness raising, coverage of private sector ). Modern Governance : (i) Planning: Integration of the CPRGS principles (planning, consultation, outcome orientation and monitoring ) into preparation of Socio-Economic Development Plan 2006-2010. (ii) Public Financial Management: MOF issues regulations requiring improved financial disclosure of state budget entities and SOEs . Launch assessment of financial risks associated with SOEs, SCOBs and DAF . (iii) Financial Accountability Enactment of Law on State Audit stipulating establishment of State Audit as independent specialized entity reporting to National Assembly. (iv) Anticorruption: Formulation of a program of organizational and operational reforms for State Inspectorate to increase its effectiveness in corruption combatting and prevention . (v) Monitoring: Revision of criteria to measure poverty and poverty targeting in line with international standards . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: PRSC4 was cofinanced by seven other multilateral and bilateral partners for a total mount of US$ 80 million. 3. Relevance of Objectives & Design: The objectives of PRSC4 were highly relevant as they were closely aligned with the Government's CPRGS and supported actions in pursuit of the CPRGS's three overarching goals . They were also relevant to the FY 03-06 CAS which was designed to support the objectives of the CPRGS . They remain relevant to the current CAS which aims to support the objectives of Vietnam's Social and Economic Development Plan 2006-2010 (the successor of CPRGS), including improving the business environment, strengthening social inclusion, natural resource management and environmental protection and improving governance . Project design was also relevant as agreed upon measures appear to have been fully owned by Government and built logically on the achievements of previous operations in pursuit of the overall program objectives . However, since a significant share of the policy measures supported by the operation involved adoption of new legislation /regulations or action plans/programs, the ultimate contribution of these measures towards meeting the program's objectives will hinge on successful implementation of the frameworks established under this operation . The ICR fails to properly assess relevance of project design . The Borrower's comment on the ICR states that the PRSCs were a much valued platform that brought together the Government, the Bank and other development partners and provided much appreciated policy dialogue, advice and technical expertise on issues of high strategic relevance to the country . The ICR echoes this. An independent evaluation of the PRSC process in Vietnam under the auspices of OECD -DAC concluded that in view of the small budgetary contribution of the PRSC in Vietnam (less than 2% of government expenditures), its value lay in providing Government with the analytical tools and policy advice necessary to implement its national development strategy which it had overall done well. 4. Achievement of Objectives (Efficacy): Promoting competition and integration with the world economy by moving towards WTO accession, accelerating divestiture of SOEs in non -strategic sectors, adopting a more determined approach to banking reform and introducing regulatory bodies for infrastructure services . On balance outcomes under this component have been substantial , though with considerable variation across sectors . (i) Trade reforms: Substantial progress. The full draft working party for WTO accession and successful completion of bilateral negotiations with six countries were important milestones towards Vietnam's accession to WTO which occurred in January 2007. Exports as a share of GDP grew from 47% in 2001 to 61% in 2005 (end of PRSC4), already surpassing the PRSC end of program target for 2006 at the end of PRSC4. The share of non-oil private sector exports rose from 44% in 2001 to 77% in 2005, suggesting that the end of program target of 85% is likely to be met. (ii) SOE reforms: Substantial progress. PRSC4 supported measures on SOE reform helped provide renewed impetus to SOE reforms. In particular, the number of sectors where 100% state ownership was to be retained was further reduced, limiting coverage primarily to 19 areas with strategic activities . The ICR states that substantial improvements were made to the equitization process under the PRSC program, with SOE appraisals now being conducted by external evaluators and share sales taking place at market value through auctions at securities trading centers. Measures supported under PRSC4 helped accelerate SOE reforms as evidenced by the relatively sharp drop in the number of SOEs between 2005 and 2006.The number of SOEs dropped from 5,344 in 2001 to 3,000 in 2005, and then to 2,000 at the end of the PRSC program surpassing the program target of 2,100. However, the share of credit going to SOEs dropped by less than ten percentage points from 41% at the beginning of the program to 32% in 2005, against a 2006 target of 25%. The ICR does not provide any information on the evolution of share of value added, employment or debt of SOEs, making it difficult to assess how substantial the progress achieved through a reduction in the number of SOEs is. The Vietnam Development Report 2007 suggests that SOE divestiture was mainly concentrated among smaller SOEs, although the size of SOE undergoing equitization has risen since 2004. (iii) Banking sector reforms: Progress in this area was modest. Prudential standards were raised under PRSC 4, but compliance is only gradually enforced . The ICR reports that the share of non -performing loans may have dropped from 15% at the beginning of the period to somewhere around 8%-10%, but that this figure needs revision . While the government's announcement of the intention to separate the State Bank of Vietnam's supervisory functions from its commercial banks may have been an important first step towards a more comprehensive financial sector reform program, a statement of intent alone is little guarantee of successful implementation and the ICR does not report on actual implementation progress. (iv) Infrastructure: Regulatory reforms in the energy sector supported under PRSC 4 provide for separation of generation, transmission and distribution which in the medium to long run are expected to contribute to lower utility costs. However, the ICR does not provide any information on actual implementation progress of the new regulatory framework. In the absence of such information progress in this area is rated modest. (2) Social inclusion and environmental sustainability : Promoting human development and environmentally sustainable development . Education: Substantial: The PRSC supported strategy in education focussed on improving quality of education particularly in poor areas. National quality standards covering school management, teacher training and performance, availability of classroom materials, quality of infrastructure, among others, were introduced and the Government initiated the assessment of primary school teachers against the quality and performance standards during PRSC4 (and continued implementation under PRSC 5). An annual audit of primary schools launched in 2004 points to substantial gains in the quality standards index by 2005, with gains higher in poorer districts than elsewhere. To what extent these gains in quality standards have translated into improved learning outcomes is not reported in the ICR. PRSC4 did not support any core actions in health or social protection . Environmental sustainability: Modest: PRSC4 supported the adoption of legislation recognizing communities' access to forest land, but the ICR does not provide any information on implementation progress under this law, nor does it provide any quantitative information on the share of forest land allocated to communities or the share of communities which have been granted forest use rights after 2004. The revised legislative framework for environmental protection submitted to Parliament under PRSC4 appears to have been enacted subsequently and mandates amongst others environmental impact assessments which were piloted under PRSC 5. The Vietnam Development Report 2007 points out that enforcement of the new legal framework remains a challenge, particularly at the local level . (3) Modern Government : Enhance efficiency, transparency and accountability of public financial management, improve alignment between planning and budgeting and shift planning away from focussing on production targets to outcome based planning and combat corruption . On balance a substantial contribution towards meeting this objective was made under PRSC4, although several measures were procedural in nature and will only yield the aspired results if they are followed up by effective implementation (e.g formulation of organizational and institutional reforms to strengthen the State Inspectorate's effectiveness to combat corruption, establishment of the State Audit of Vietnam (SAV) as an independent institution . The ICR fails to report on subsequent implementation progress in both cases ). The government has fully integrated the CPRGS principles into its planning processes, with the recently adopted SEDP 2006-2010 having been developed with broad participation and taking a results based outlook rather than aiming at production targets. The approach has also been expanded to 20 provinces. State budget agencies and SOEs are now required to disclose their state budget allocations, thus enhancing financial transparency . For SOEs the mandatory disclosure also includes their financial position, operational performance and return on equity . The State Audit of Vietnam is now an independent institution reporting directly to the National Assembly, with the Auditor General directly appointed by the latter . SAV audits are being made public. However, SAV's audit capacity requires further strengthening according to the Vietnam Development Report 2007. 5. Efficiency (not applicable to DPLs): ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Based on substantial relevance and substantial achievements under most objectives the overall outcome is rated as satisfactory . The success of the PRSC program is also reflected in the overall strong performance of Vietnam's economy over the PRSC period. GDP per capita grew at over 6% per year over the PRSC program period, with substantial acceleration in growth during PRSC 4 (and beyond). Solid growth resulted in unprecedented poverty reduction. The headcount index of poverty fell by almost 10 percentage points (29% to 19.5%) between 2002 and 2004 (latest reported data). a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: PRSC4 was the second to last operation in the first PRSC series . It was followed by PRSC5 which further promoted reforms in the areas supported by PRSC 4. Over the course of the first PRSC series there was no evidence of back-tracking on reforms, although progress in certain areas was faster than in others . The Government, with the support of the National Assembly, has recently adopted the SEDP 2006-2010 which fully subscribes to the continuation of the policies initiated under the PRSC program . The Bank and more than ten other development partners are supporting SEDP implementation with the next PRSC series (PRSC6-10). An OECD-DAC evaluation of the PRSC found that the relatively small share of public spending contributed by the PRSC can not be the primary reason for the government's interest in the operations, rather it is the policy advice and technical support that come with it that provide value added . The borrower's comment on the ICR confirms this view . This suggests that the reforms thus far supported under the first PRSC series are fully owned by the Government and unlikely to be reversed. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: PRSC4 was not subject to a QAG quality at entry assessment . Overall, the Bank's performance was satisfactory both during the design and the supervision of PRSC 4. The operation was well designed and relevant to Vietnam's circumstances . Like the rest of the series, it was underpinned by strong analytical work which helped cement policy directions and actions in areas where the CRPGS needed further refinement. On the whole, choices were relevant and the sequencing of measures under PRSCs in various areas appropriate and well tailored to Vietnam's conditions . The Bank also took the lead in coordinating technical as well as financial inputs from all other donors providing general budget support . The Bank made good use of potential synergies between programmatic policy based operations and more traditional investment operations. The presence of the TTL and other team members in the field, allowed the Bank to be continuously engaged in the policy dialogue and carry out supporting analytical work jointly with other donors and counterparts. at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: The borrower showed strong commitment towards the PRSC program as demonstrated by the decision to place overall oversight over the program directly under the First Deputy PM . A PRSC secretariat, under the chairmanship of the State Bank of Vietnam ensured inter -governmental coordination of the program . The ICR states that SBV's effective coordination of a large number of line agencies and donors involved in the PRSC was key to successful and timely implementation of the program . Overall, the Government provided strong leadership and guided the harmonization of various donors' general budget support . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: At the outset of the PRSC series, no M&E system was designed and agreed upon with the Government and other PRSC financiers that would have allowed to systematically monitor performance against program objectives . By the time of PRSC3, the team, in coordination with the Government and other donors put together a monitoring framework drawing on the CPRGS and other Government development strategies . With a few exceptions (e.g. Infant Mortality Rates, Under 5 Mortality Rates, which appear to only be available when DHS is carried out ) the choice of indicators is adequate to measure progress towards development outcomes and generally based on available data . However, since the ICR presents an assessment of the entirety of the program, rather than the status upon completion of PRSC4, it only partially presents data which would reflect progress towards development outcomes upon closure of PRSC4, the performance of PRSC4 alone is difficult to assess (data at the time of PRSC4 closure are only presented to the extent that more recent data were not available and the ICR does not indicate from which year the data is). The monitoring framework was used on a regular basis to rate progress against each outcome indicator and objective. A four scale rating system of Highly Satisfactory, Satisfactory, Partially Satisfactory and Unsatisfactory was utilized by PRSC financiers and Government to assess progress made against each indicator . Drawing on the experience with the PRSC monitoring framework, the Government has now developed a monitoring system which accompanies the Socio -Economic Development Plan. a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: PRSCs, even if they only provide a small share of budgetary expenditures, can serve as an effective platform for policy discussions if they are aligned with a country's national strategy and priorities and analytical work is strategically used to help fill country specific knowledge gaps Country ownership and political will to follow through on reforms is critical to program success . The case of the Vietnam PRSC series shows clearly that reforms advanced rapidly and consistently where there was political will, such as trade reforms, but less so in other areas where there was less political support, such as financial sector reforms Anchoring reforms to external incentives such as WTO accession may help build the necessary domestic consensus to proceed. An effective coordination mechanism is essential to overcome logistical challenges posed by wide ranging and cross-cutting policy reforms. 14. Assessment Recommended? Yes No Why? Many of the measures supported under PRSC 4 will only bear fruit if they are followed -up (e.g. adoption of legislation or action plans). A field assessment would permit to see to what extent this has been the case . It is recommended that the first series of PRSCs in Vietnam be assessed as part of IEG's overall evaluation of PRSCs . Contrary to many other PRSC beneficiaries, the Vietnam PRSC contributed only a small amount of financing and its value to the country thus lies elsewhere . It would thus be insightful to assess what the value added and achievements of a PRSC in such a context are compared to countries where the PRSC accounts for a substantially larger share of budgetary outlays . A field based assessment would also permit to see to what extent the PRSC and investment operations complemented each other and achieved synergies . 15. Comments on Quality of ICR: While it provides a summary description of outcomes achieved and presents outcome indicators, the ICR fails to make a clear assessment of the achievement of individual objectives . The ICR fails to clearly justify the ratings of each individual operation in the programmatic series and it does not discuss the extent to which each operation's activities and inputs contributed to outputs, outcomes and impacts of the series . The ICR notes that there was slow progress in some areas (e.g. water), but does not elaborate on the impact of such delays on achievements of outcomes or sustainability. The ICR is largely cut and pasted from the PAD of PRSC 5 which raises questions about the extend of ex-post self-evaluation performed for purposes of the ICR . The ICR fails to assess relevance of program design and implementation, nor does it asses the Bank's performance during supervision . The ICR at times presents two different numbers for the same indicator in different parts of the report . The ICR does not show how the indicators have evolved over time, nor does it state from what year the "last available" data is. Given the large number of co-financiers, it would have been desirable to also have inputs from some cofinanciers into the ICR . a.Quality of ICR Rating : Unsatisfactory