ICRR 13261 Report Number : ICRR13261 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 08/20/2012 PROJ ID : P104097 Appraisal Actual Project Name : Vn - Program 135 US$M ): Project Costs (US$M): 50 50 Phase 2 Support Credit Country : Vietnam Loan/ US$M): Loan /Credit (US$M): 50 50 Sector Board : ARD Cofinancing (US$M): US$M ): Sector (s): Sub-national government administration (40%) General agriculture fishing and forestry sector (30%) Other social services (20%) Central government administration (10%) Theme (s): Participation and civic engagement (33% - P) Public expenditure financial management and procurement (17% - S) Managing for development results (17% - S) Rural policies and institutions (17% - S) Social safety nets (16% - S) L/C Number : C4274 Board Approval Date : 03/21/2007 Partners involved : Closing Date : 12/31/2007 12/31/2007 Evaluator : Panel Reviewer : Group Manager : Group : Gita Gopal Jorge Garcia-Garcia IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: The credit, the first in a proposed programmatic series of three Development Policy Credits (DPCs), aimed to promote informed participation and civic engagement at community level in the Government's poverty -targeted public investment programs as a primary means to improve the results of these programs . Specifically, it aimed to improve the results of a Government Program for Socioeconomic Development in Communes Facing Extreme Hardship in Ethnic Minority and Mountainous Areas (Program 135 or P135). The credit supported policy and institutional actions to improve the results of the second phase of P 135 (P135-2) in the following four main ‘meso-level’ policy areas: (i) improved poverty targeting of program resource allocation; (ii) deepening and scaling-up of community-driven approaches to planning, managing, operating and maintaining basic rural infrastructure; (iii) enhanced fiduciary transparency and accountability in program implementation through wider use of ‘social audit’ approaches; and (iv) process monitoring and impact evaluation (Program Document (PD), pars. 57 and 58). The financing agreement did not state the credit ’s objectives. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The credit supported policy and institutional actions to improve the results of Program 135-2 as follows (Para 64 of the Program Document ): Policy Area 1: Poverty Targeting This area dealt with four issues : (i) the selection of communes, villages and households eligible for inclusion under P135 2; (ii) the criteria used for targeting program resources to participating communes, villages and households; (iii) appropriate policies for financing operation and maintenance costs without imposing a heavy burden on poor communities; and (iv) the quality of the policy framework underpinning the production /livelihoods support component of P135-2. Policy Area 2: Decentralization, Participation, and Empowerment This area dealt with the deepening and scaling -up of community-driven development (CDD) approaches for providing basic infrastructure and supporting household livelihoods . It addressed three issues : (i) promoting informed public participation in P 135-2 planning and management; (ii) devolving investment ownership to commune level to the extent appropriate; and (iii) supporting local institutional capacity building as a critical means to achieve both of the first two objectives Policy 3: Fiduciary Transparency and Accountability This area focused on: (i) procurement for small scale infrastructure; (ii) public financial management and auditing; and (iii) community contributions. These complemented the previous two policy areas . Policy Area 4: Monitoring and Evaluation This area dealt with results measurement or impact evaluation under P 135-2. The actions in this area aimed to evaluate P135-2 program outcomes to high methodological standards and to use an effective and reliable Management Information System (MIS) to track progress towards agreed outcomes and resolve program management issues. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The credit was approved in March 2007. US$ 50 million was disbursed upon effectiveness on August 15, 2007, and the credit closed in December 2007. Overall, donors financed up to one third of the costs of the Program, according to the ICR, and the Bank was the largest donor . Program implementation was delayed due to late issuance of operational guidelines. The second operation was presented to the Board in May 2009, 26 months after the presentation of the first operation, Therefore, there was a lag of more than 24 months between two operations. This required the preparation of ICR at the end of first operation, rather than at the end of the planned programmatic series of three development policy operations. The second operation was a US$ 100 million DPO. The third operation of this programmatic DPO series was a US$50 loan, and it was presented to the Board in April 2011. 3. Relevance of Objectives & Design: The relevance of objectives was high . The project sought to reduce the high levels of poverty that existed in the targeted communes through tested participatory and CDD approaches . These objectives were consistent with the Government's priorities for poverty reduction and decentralized rural d evelopment. They were aligned with the Bank’s Country Partnership Strategy (CPS) 2007-2011, which was based on the Government’s Socioeconomic Development Plan (SEDP) 2006-2010. Program objectives were also developed in consultation with other donors including Australia, Finland, Ireland, Sweden, the UK and IFAD. The relevance of project design was substantial . The choice of direct budget support as the instrument for support helped to encourage policy reform, which would have been difficult to achieve through an investment project . The design of the DPL was identified in collaboration with development partners, and based on collective experience gained through small poverty reduction investment projects . Some donors supported parallel TA to achieve program objectives. The results framework was overall sound . The policy areas identified for support were appropriate and there was demonstrable evidence from other interventions that the planned activities were likely to lead to effective planning and management of development interventions at the local level, which in turn would increase development effectiveness and reduce poverty . A few aspects were perhaps too ambitious, given the remote areas of project intervention . For example, local government institutions were expected to shift from a planning approach to a market-oriented paradigm in a short period of time . There was inadequate demand for such a shift at the local levels in several of the remote areas where the project operated . 4. Achievement of Objectives (Efficacy): Objective 1: Poverty Targeting Improved selection of eligible communes , villages, and households for inclusion under poverty reduction programs : Actions in this area aimed to improve the selection of communes for participation in the program . Prior to the support, provinces were using varied methods to select eligible communes, which did not always result in the choice of poor communes . By the end of this phase, all 47 participating provinces were using verifiable, poverty-based criteria for selection of communes and villages included in P 135-2 developed by the Ministry of Labor, Invalids and Social Affairs (MOLISA). This has resulted in the selection of poorer communes as verified by the baseline and evaluation survey . Improved criteria utilized for targeting program resources to poorer communes, villages, and households : Actions in this area aimed to ensure that more resources were allocated for the poorer localities . Prior to the intervention, resources were allocated equally among communes irrespective of the levels of poverty . At the time of closing, 44 of 47 participating provinces were utilizing criteria that would favor poorer villages and communes in resource allocation. The Committee for Ethnic Minorities (CEM), the implementing agency, took active steps to disseminate guidance on commune, village and household selection and introduce an improved resource allocation methodology . Increased resources for maintenance of small -scale infrastructure : Actions in this area supported increased sustainability of small infrastructure through better operations and maintenance . A central budget line for O&M was created to ensure increased resources for maintenance of small -scale infrastructure as a first step, although there is some concern that the 6 percent of funds reserved under this budget line may not be sufficient to maintain the investments. All investments were subsequently expected to have “maintenance plans developed and implemented;" only ten percent, however, did so at the time of closing . Increased market -oriented agricultural production approach among poor households : Actions in this area supported increased diversification of agricultural production at the household level to generate sustainable income, and thus reduce poverty . According to the ICR, there is some change of attitude in the approach of local governments but this has not been put into practice as yet among the majority of households . This is partly because of district extension staff who continue to encourage communes to purchase machinery, cattle and other inputs such as hybrid seed varieties to improve existing modes of production , rather than respond to market demand. There was a substantial progress in improving the poverty targeting . Objective 2: Decentralization, Participation, and Empowerment 135 -2 planning and management : Action supported improved participation Informed public participation in P 135- rates in community meetings that prioritized community needs and increased satisfaction rates of participants with selection of projects. The benchmark was set at 90 percent for both. The Citizen's Report Card Survey found that the participation rate in community meetings was 70 percent and that 60 percent of the participants were satisfied with the selection of projects . The lack of resources, however, resulted in the vast majority of communication being undertaken in the Kinh language rather than in ethnic languages . This hindered the public information and awareness campaign . Gender issues were also not well addressed constraining gender-inclusive participation. Devolving investment ownership to commune level to the extent appropriate : Actions in this area aimed to ensure that at least 60 per cent of communes would be investment owners under the program by 2008 (i.e., the communes were to be responsible for operation and maintenance of the infrastructure or production activity ). By end-2007, 35 percent of communes became investment owners of infrastructure projects . By end-2008, 55 percent of communes became investment owners for at least one of the two activities . While many communes are investment owners for production investment, relatively fewer communes are yet acting as full investment owners for infrastructure investments, which continue to be the responsibility of the intermediate levels of government. Supporting local institutional capacity building as a critical means to achieve both of the first two objectives : Actions in this area supported strengthening capacity of local institutions to increase the take over of management responsibilities for investments and to improve the quality of participation . An Inter-Ministerial Circular 676 requirement that at least 7% of the total program cost be allocated for capacity building at the central, provincial, district and local levels . The large number of participating communes and local government functionaries increased the challenge . Although technical support was provided by UNDP to enhance capacity, this did not effectively address the issue . Lack of adequate tools to provide training at the community level such as specific manuals in the areas of participatory planning, procurement, construction, financial management and accounting was another constraint . Thus, capacity at the grassroots level was strengthened but not sufficiently . There was a modest progress in achieving the second objective of the program . Objective 3: Fiduciary, Transparency, and Accountability Improved satisfaction with procurement for smallscale infrastructure : Actions in this area supported the improvement of more efficient procurement at the local levels . Procurement regulations and guidelines for the Program have been improved to allow more flexible forms of procurement, although there is need to further maximize transparency and efficiency . Community contracting has been introduced through an inter -ministerial circular, but there is resistance to the more flexible modes of procurement at the provincial levels . Only 25% of the communes were utilizing community contracting or competitive bidding by 2008 (benchmark being 50%), thus not achieving the planned levels of efficient procurement . Improved public financial management and auditing : Actions in this area supported increasing oversight of public funds through transparent use of resources, as well as making audits publicly available at national and local levels. To achieve transparency, the MOF issued Guidance No . 2849 that governed financial management for P135-2, and as a result spending agencies are currently presenting quarterly financial reports and reconciling them with the State Treasury of Vietnam . Spending agencies and investment owners are also preparing quarterly statements of flow of funds . Regular and independent financial audits for P 135-2 are carried out indicating areas of weakness and making recommendations for further strengthening; this facilitates scrutiny and provides assurance. Audits are publicly available at national and local levels and follow -up actions taken are documented, thereby facilitating increased oversight of public funds . MOF, however, is yet to publish reports on the allocation, transfer and usage of P 135 funds to provincial level on its webpage, another action envisaged during this phase. Equitable community contributions : Actions in this area aimed to exempt the poorest households from compulsory community contribution in cash and also supported the use of local labor by contractors, with priority given to the poorest households in at least 50% of infrastructure investments under P 135-2. The value of unpaid labor contributions were also to be captured in the MIS and commune accounts . Community contracting occurred only in 25% of the communes, and arrangements to allow poor households to contribute through paid labor has been implemented in some cases . However, this practice is not yet "widespread" (ICR, Para 57). Unpaid community contribution is not yet recorded in the formal system . There was substantial progress in achieving the fiduciary, transparency, and accountability objective . Objective 4: Monitoring and Evaluation Use of high quality impact evaluation methods : Actions in this area supported enhanced evaluation of program outcomes through improved methodology including baseline and control group -based survey methods. A baseline survey was designed, which included a matched control group, although the survey was not carried out in 2007 as planned, but completed only in 2008 just before the MTR. Consistent and comprehensive monitoring system : Actions in this area supported a centralized coordination unit to improve implementation, as well as identify opportunities for sharing innovations and good practice amongst provinces. An MIS system has been put in place to monitor outputs and intermediate objectives on a regular basis, along with the use of community report cards to assess beneficiaries ’ satisfaction with program outcomes. Data has been generated on ethnic minority poverty . The establishment of the MIS was, however, delayed to October 2008 and it will take some time for it to become embedded within the implementation process and to improve development planning more broadly . There was substantial achievement in monitoring and evaluation objective . 5. Efficiency (not applicable to DPLs): ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Bank support has contributed to improving the framework for informed participation and civic engagement at community level in the Government's poverty -targeted public investment program . Although results have been somewhat mixed, there is progress in all four policy areas and four main objectives : In the area of poverty targeting there has been substantial improvement in selecting poor communities and targeting resources to them, but there has been slow progress in the maintenance of infrastructure and in shifting households to a market oriented production approach . In the area of decentralization, participation and empowerment, there has been significant progress in enhancing informed public participation in P 135-2 planning and management, while there is slower progress in devolving investment ownership to commune level and in enhancing local institutional capacity . In the area of fiduciary transparency and accountability there is substantial progress in introducing more flexible and efficient forms of procurement but this has been slower than planned . Financial management has become more transparent with quarterly reporting and reconciliation as well as public disclosure of audits and follow -up actions. In the area of monitoring evaluation a rigorous methodology was developed and applied, albeit later than planned. The monitoring information system was established, but only towards the end of the project, and therefore, use of data for timely changes was not possible during this phase . Overall, there were moderate shortcomings . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: There are several risks that might affect results . First, the lack of adequate arrangements for O&M might reduce the sustainability of Program results . Second, despite all good intentions, decentralization of powers and responsibilities from sub-national levels of government to communes has not occurred effectively, and this leaves the framework for management of community investments unclear . Finally, the resistance to community -related procurement may adversely affect Program implementation and results unless government commitment and ownership is strengthened at the intermediate levels . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: Quality at entry was moderately satisfactory . The Bank led the initiative to harmonize processes for participatory across provinces in the country . The credit was carefully designed in collaboration with other key donors and in a manner responsive to client interests. Monitoring arrangements were, however, not fully effective . There was an absence of verifiable indicators, including baseline and end of program targets . The Bank’s policy on DPLs requires a clear analysis of the social impact on vulnerable groups . The Program Document reflects little consideration of these requirements and consideration of gender issues was weak; this is surprising given that the Country Gender Assessment had identified gender and ethnic minority issues as a key constraint to development and that the previous and current CASs had identified the need to address them . Quality of supervision was moderately unsatisfactory . As noted in the ICR, although several implementation problems were evident from the beginning, it was not until the November 2008 MTR that the lack of local capacity, the problems with the production component, and the weaknesses in the M&E system were addressed . The lack of clear monitorable indicators also resulted in tensions with the Government on project ratings . Gender issues did not receive due attention, although the program document acknowledged that women and ethnic minority population would face obstacles to participating in and benefiting from P 135-2, unless special attention was paid to their needs . at -Entry :Moderately Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Performance of the gover nment was moderately satisfactory : There was government commitment and ownership of the DPC. The Government, in particular the Ministry of Finance, coordinated donors well . It ensured that the Program office was staffed adequately . However, ownership was weak with the Minister of Agriculture and Rural Development (MARD) on the activities related to the investment sub -projects. Also, the government did not establish the MIS in a timely manner as agreed . Performance of the implementing agency was moderately satisfactory : The implementation performance of the CEM was weak, particularly during the first two years of the Program, reflecting its weak capacity as well as the delayed and insufficient implementation support from the central level . Despite this, CEM fulfilled its assigned responsibilities towards the latter half of the implementation period, and strengthened its capacity through technical support provided by donors . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: Design : The progress indicators as originally defined in the Program Document were unclear to assess progress in achievement of outcomes. They required a complete revision . Implementation: Late in the program cycle after the November 2008 MTR, these progress indicators were revised using information from the baseline survey . The indicators were reflected in the revised policy matrix included in the MTR, which was formalized in the DPC2 program document. Indicators, although more specific, were still focused for the most part on outputs and processes and not on outcomes . A citizen's score card was prepared for the MTR . Utilization: The MIS was established in late 2008 but the system was unable to produce consolidated reports as of the time of ICR writing and, therefore, utilization has been weak . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): No fiduciary issues are reported in the ICR. Safeguard Aspects: The Program Document had noted that there were measures in existence to protect the access of ethnic minorities to land, and that there was a need to enhance “the coordination between P135-2 and the other sector-based targeted programs (in forestry and rural water supply and sanitation )�. The ICR provides no information on this aspect. The ICR identifies no unanticipated impacts. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate High Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The establishment of a basic MIS (focusing on outcomes) and implementation manuals prior to the start of the Program is necessary to enhance development effectiveness . To ensure that both men and women share in program benefits, it is important to integrate gender considerations during the design of CDD programs; integrating gender issues during implementation is insuffic ient. Ownership of the Program objectives and rules by, and strong capacity of, local governments are important to achieve sustained results in CDD interventions . 14. Assessment Recommended? Yes No Why? To verify the ratings and document lessons from the series . 15. Comments on Quality of ICR: The ICR candidly discusses where progress has been made and where progress has been inadequate. The ICR could have been strengthened if it better addressed social and gender issues and if the progress towards achievement of program impact were also discussed. a.Quality of ICR Rating : Satisfactory