ICRR 12926 Report Number : ICRR12926 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 12/29/2008 PROJ ID : P004595 Appraisal Actual Project Name : Community Based US$M ): Project Costs (US$M): 67.5 42.13 Resource Management Country : Philippines Loan/ US$M): Loan /Credit (US$M): 50.0 35.52 Sector Board : ARD Cofinancing (US$M): US$M ): na na Sector (s): General agriculture fishing and forestry sector (45%) Water supply (20%) Roads and highways (20%) Sub-national government administration (8%) Other social services (7%) Theme (s): Environmental policies and institutions (33% - P) Land administration and management (33% - P) Rural services and infrastructure (17% - S) Decentralization (17% - S) L/C Number : L4299 Board Approval Date : 03/24/1998 Partners involved : Closing Date : 06/30/2004 06/30/2007 Evaluator : Panel Reviewer : Group Manager : Group : April Connelly Ridley Nelson Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: The project objective was to reduce rural poverty and environmental degradation through support for locally generated and implemented natural resource management projects . This was to be done through a ) enhancing the capacity of low-income rural local government units (LGUs) and communities to plan, implement and sustain priority natural resource management projects; b ) strengthening central government systems to transfer finance (as financial intermediaries) and environmental technology, and improve the implementation of environmental policies; and (c) provision of resources to LGUs to finance natural resource management projects . The project was restructured twice, neither of which involved a change to the project's development objectives . The first restructuring occurred during the MTR in 2000 and involved changes in implementation processes and capacity development needs within the implementation agency . A second restructuring was carried out in June 2002 to adapt project design to a more decentralized approach and introduced CDD procurement mechanisms . The PAD included outcome indictors but no targets . According to the ICR, "both restructuring (sic) included the generation of specific targets in relation to the intermediate outcome indicators and in response to the specific planning occurring through the community-led planning process. However, no formally recorded change in the Intermediate Outcome Indicators or targets was registered in the Bank's system ." Formal Board approval was not obtained for the changes to indicators. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Subprojects in Natural Resource Management (Appraisal estimate US$ 46. 34 .3 million, 46 .9 million, Revised US$ 34. Actual US$ 31.31 .43 million ). To finance subprojects involving demand -driven natural resources management investments in the form of loans and grants provided through a new rural window of the Municipal Development Fund (MDF) to the LGU. Eligible subprojects were to fall under three categories : (i) upland agriculture and forestry; (ii) coastal resources and nearshore fisheries; (iii) small rural infrastructure and livelihood projects related to these natural resource investments . Planning and Implementation Support to LGUs (Appraisal estimate US$ 6.7 million, Revised US$ 1.0 million Actual US$ 1.13 million ). To provide planning and implementation support for LGUs and their communities . Specific support would include costs of training including training materials, equipment and allowances and payments to trainers . Initiating an MDF Rural Window and Project Management (Appraisal estimate US$ 7.4 million, Revised US$ 9.52 million, Actual US$ 7.52 million ). To support a project management office which would be incorporated into the MDF's rural window after a transitional period of 3 years. This rural window was to channel funds to low income LGUs for financing rural development investments including the project activities . Environmental Technology Transfer and Policy Implementation (Appraisal estimate US$ 6.5 million, Revised US$ 2.10 million, Actual US$ 2.04 million ). To assist the Department of Environment and Natural Resources (DENR) and Department of Agriculture (DA) to transfer natural resources management technology to LGUs and improve management of environmental policies, particularly policies of land tenure access and security . Specific activities were to include: (i) survey and mapping for resources access and tenure certificates; (ii) technology training; (iii) technical assistance; (iii) policy studies to assist DENR and DA in developing improved land access and tenure . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs, Financing, Beneficiary Contribution : Original project cost was US$ 67.5 million (actual US$ 42.13 m) to be financed by: Central Government US$ 4.2m (actual US$ 2.58m), Local Gov US$ 7.7m (actual US$ 4.03m), Beneficiaries US$ 5.70m (actual 0), IBRD US$ 50.0m (US$ 35.52m actual). The ICR does not comment on why the Beneficiary contribution did not materialize . The region, in its comments on the ICR review, clarified that there was a beneficiary contribution but an estimation of the amount was not attempted by the ICR team . Total loan amount at appraisal was US$50m, but this amount was revised twice . In 2001, US$10m of the initial loan amount was cancelled due to devaluation of the Philippine peso from US$ 1 = Php 34.05 in 1997 to US$1 = PhP 51 in 2001. An additional US$2m was cancelled in 2002 due to savings from further exchange rate shifts in 2002 (US$1= PhP 51.60). The total revised loan amount was US$38m. Actual loan amount was US$ 35.52m. The ICR does not explain the US$ 2.48m undisbursed amount but it notes that by project close the actual disbursement amount exceeded the original projected loan amount in PhP terms even without the two partial loan cancellations . In terms of allocation by component, the allocation for Component 1: Natural Resource Management Sub-projects represents 75 percent of financing, the allocation for Component 3 - MDF Rural Window was increased from 11 percent at appraisal to 20 percent of revised project cost, and Components 2 and 4 - Environmental Technology Transfer and Planning and Implementation Support , were each cut by about 50 percent of appraisal level. According to the ICR the allocation revisions were made to address delays in project implementation and fund disbursement, as at the time of MTR only 20 percent of loan had been disbursed . Dates: The project's closing date was extended twice . An initial two year extension was granted to overcome implementation delays that occurred at the start of the project and a further one year extension was granted to allow time to spend additional funds that became available as a result of exchange rate shifts . 3. Relevance of Objectives & Design: Relevance of Objectives: (substantial) The project's objectives were relevant to the 1996 CAS goals of promoting sustainable management of natural resources and protecting the environment and to the Government's Medium-Term Development Plan objectives of addressing rural poverty and environmental degradation . The Project objectives continue to be relevant to the current CAS (approved 2005), which aims to support the government's cross-sectoral objective of improving devolution of national government functions to Local Government Units . However, the objectives were over ambitious for the planned implementation period given the implementing agency's capacity and the country's policy and institutional framework . The MTR supervision report notes "the project assumed that it would be able to quickly upscale the NRM interventions, when in reality, there is a recognized need to adopt an expanded pilot approach to laying a strong foundation for a longer -term program through phased and strategic NRM interventions and appropriate intuitional arrangements ". Relevance of Design: (modest)The project's components were relevant to the achievement of project objectives but implementation was extremely slow (as of the MTR only 20 percent of the loan was disbursed ) and only moved First the project aimed to work at forward after the project was restructured . The were several design shortcomings . First, the LGU and community level yet implementation procedures were overly centralized and overly complex given local capacity. Second , the choice of implementing agency is questionable . The ICR explains that the MDFO was selected as the implementing agency because it was best placed to channel funds to the LGUs . However, several project documents, including the ICR itself, question the selection of a new agency, with little implementing experience or technical knowledge, to implement a natural resources management project that had a complex implementation arrangement, involving the coordination of multiple implementing partners . The ICR notes operational and disbursement delays arose because of the lack of experience of the MDFO as an implementing agency and its unfamiliarity with technical aspects of NRM . The ICR (pg. 5) also acknowledges that “(i)n terms of technical implementation, the project may have been better placed within DENR (The Department of Environment and Natural Resources)". Third, M&E design was weak. The indicators in the project's logframe do not adequately measure achievement of project objectives . Finally , the initial project time frame did not adequately consider the time required to build implementation capacity at the appropriate level . The Quality of Supervision Assessment undertaken by QAG in Sept 2006, notes that "(d)ealing with CDD issues is a long, process, which should be given adequate time and should not be hindered by complex design . It is possible that for a large country like the Philippines, a barrage of such initiatives, perhaps under an APL umbrella, with generous timing for each operation and collectively, would have been a better approach ". 4. Achievement of Objectives (Efficacy): Reduction in rural poverty appears to be substantial but there are questions about the extent to which the poorest within targeted regions benefited . Both the PAD and ICR use the increase in income of beneficiary households as the measure for poverty reduction . The target set at appraisal was "25% of households demonstrate an increase in income" (no amount of increase specified ), at the MTR this was revised to “incomes for at least 30% of households increase by 10% over 2000 levels�. However, as noted in section 2, this change was not formally approved . The ICR reports that incomes of 65% of the beneficiary households have increased . The reported average annual increase in income was 7% above a reported base average income PhP 38,885 for all project sites. This information was based on a supplemental income study commissioned by the ICR team in 2007. IEG was provided with a draft copy of the study following a meeting with the task manger . According to the income study a sample of 242 individuals were surveyed, which represented 7% of all peoples organizations covered by the project and 8% of all peoples organization members. According to the ICR team, there was no record of total participants for income generating activities only the number of participating peoples organizations . Even though there was no baseline, the study draws its conclusion on the basis of per province household income data available from the Philippines Financial Income Study done in 2000. The ICR's income study (pg. 12) notes that this was the most comprehensive and verifiable data available and that "while income and expenditure rates will have varied considerably with inflation affecting both income and expenditure figures, the comparative household inflow and o utflow provide a context to show the extent to which the CBRMP IGAs (income generating activities) are likely to impact on participating households ". However, it should be noted that though the project targeted regions with a high incidence of poverty, it is not clear that the poorest within the target regions have benefited from the project . Reduction in environmental degradation appears to be substantial but there is little evidence on outcome . The ICR argues that "the physical accomplishments for NRM activities demonstrate the projects impact on environmental protection". The indicators reported in the ICR are : additional forestry and upland areas have been placed under sustainable management, through the establishment of 16,013 ha of agro-forestry, 1,517 micro-watershed areas, and 7,171 ha of trees planted. Additional coastal areas have been placed under community management : 5,220 ha. of mangrove rehabilitation, with a further 2,107 ha of mangrove protection area established; 130 ha. of beach were rehabilitated; 11,585 ha of fish sanctuary; and, 738 units of artificial reef were established . In addition there was 822 ha. of riverbank stabilization, and 500 ha of community managed eco-tourism. All of which are output indicators and do not measure resulting changes in environmental degradation, such as actual reduction in soil erosion . To demonstrate that the outputs reported will lead to the expected outcomes the ICR needed to show additional information, such as survival rates of tree plantings, which is missing . IEG found additional information in the borrower's ICR: 81% weighted survival rate for plantations, reforestation and agroforestry; 49% of NRM areas have high growth performance, 22.2% moderate growth performance, and 29.6% low growth performance; 80% reduction in the rate of deforestation inside the project area (over 2002 levels); a 40% reduction of soil erosion; and 65% net average increase in live coral inside marine sanctuaries . Based on this additional information IEG willing to agree that the objective may have been achieved . None the less, IEG still has concerns on the following issues : (1) The ICR itself points out that targeting and site selection was somewhat problematic . Specifically, "due to the demand-driven approach, sites tended to be fragmented rather than forming the basis for strategic action in most environmentally-degraded areas. Specific site selection was occasionally subject to political influence " (ICR pg. 9). (2) A safeguards study carried out on the project by the WB Environment and Social Department in 2004 raised concerns over the DENR's practice of promoting non -native species in the community forest plantations (see safeguards section). An impact assessment of the project also reported occasions of inappropriate species being planted on some project sites and includes as one of its recommendations the need to conduct species suitability tests before implementing tree plantation and mangrove reforestation projects . The ICR does not provide a breakdown on the species planted by the project but the region, in its comments on the ICR review, notes that the ICR team visits confirmed that species planted were appropriate for each site . (3) Finally, it is important to note that, the full environmental benefits of NRM activities will only be realized if the interventions are sustained over the long term and, as noted in the section on risk to development outcome, IEG has concerns with respect to the sustainability of these investments particularly on sites where the communities have not been granted tenure instruments . The region in its comments states that the development of sustainability plans and issuance of local ordinances shows "substantial commitment by the communities and LGUs to sustain the investments already in place even if the tenurial instruments have not yet been formalized ." The following three objectives represent PDO's of a different order, they are means through which the first two objectives would be achieved rather than ends . (see wording of objectives section 2.a) Enhancing the capacity of low -income rural local government units (LGUs) LGUs ) and communities to plan, implement, and sustain priority natural resource management projects is rated negligible . According to the ICR (pg. 25), "of the 741 communities actively engaged in the project, 89 percent (or 656 communities) have demonstrated active progress in environmental projects ". The ICR (pg. 11) states that “positive improvement in LGU performance was progressively documented and capacity –building initiatives led to accelerated and successful project implementation . Tangible outputs have been achieved by LGUs and communities that demonstrate a level of capacity for planning and implementing�. However, based on the information put forth in the ICR, it is difficult to assess LGU and community capacity. As acknowledged by the ICR, the indicators in the logframe that correspond to this objective (number of of groups receiving training ) are output indicators and do not assess the capacity outcome . The ICR (pg. 11) states that there were no base -line measures of LGU capacity but that "the low level of capacity can be inferred by initial challenges in project implementation " and that "the fact that outputs reached the levels achieved requires a level of organization and management by LGUs that had not been present previously " but no evidence of LGU or community capacity is presented in the ICR, other than reporting that participants in a beneficiary workshops stated that the project improved skills such as project management and financial management which can be used in the future to attract private investors and nongovernmental organization / government support. The ICR also notes that 95 percent of loans made to LGUs were repaid on time (versus target of 90 percent), but the high level of repayment is primarily due to the fact that loan repayment is automatically deducted from the LGU ’s Government Internal Revenue Allotment, which is channeled through the MDOF (the project’s implementing agency). Only 33 percent of loans made to barangays (villages) have been repaid to the LGUs (versus target of 90 percent). The ICR notes that the low repayment level by the barangays is the result of late commencement of income generating activities, LGU ’s amending payment schedules in collaboration with barangay groups (though this was not recorded in the projects database system), and that "some LGUs are not being rigorous in asking for repayments as they see the benefits to the communities of not doing so" (pg. 15). Community capacity is also called into question by the fact that the expected beneficiary contribution to project cost is not reported . The region in its comments states that a beneficiary contribution did materialize but the ICR team did not attempt to estimate the amount . Finally, IEG has concerns the communities' capacity to sustain natural resources management projects (see risk to development outcome ). Particularly the capacity to fund the implementation of sustainability plans and the fact that some communities still lack tenure instruments that are important incentives to sustain NRM investments . Strengthening central government systems to transfer finance (as financial intermediaries ) and environmental technology, and improve the implementation of environmental policies is rated negligible . With respect to transferring finance , the ICR states "(t)he capacity of central government systems to transfer finance (as financial intermediaries) to LGUs has improved. Procedures have been stream -lined, and policies and procedures within the MDF have been amended in line with the lessons learned from the project . The DoF reports that the policies and procedures developed in the project are now being replicated in other development projects . For instance, CDD procurement methods and analysis of LGU liquidations are already being used in other projects .� The ICR (pg. 9) notes that MDFO is developing an operational manual for lending to LGUs and for MDFO project management and that the experiences of the CBRMP along with other Bank -financed projects have provided major input to the process. However, there is no analysis to demonstrate to the reader exactly what experiences of the CBRMP have been incorporated into the operational manual or to show that the project has had "major input" into the process. The project established a loan window in the MDFO to enable the transfer of funds to LGU's for subprojects but according to the ICR (pg. 13) “the government has not committed further funds to expand and further develop the potential of linked environmental and economic development initiatives . In this respect the satisfactory impact of the project has been limited to participating LGUs with little scope for replication in other LGUs .� The ICR also states that lending from project reflows has continued but the loan -grant-counterpart financing mix has not been determined and demand for funding is uncertain (see risk to development outcome, and achievement of provision of resources to LGUs, below). In a meeting with IEG, the task manager noted that the MDF window continues to function and provided IEG with evidence that a subsequent NRM project in the Philippines is using the MDF window to channel GEF grants to the LGUs. However, no evidence was provided to indicate that the central government channels its own fund through the window. In a meeting with IEG, the TTL noted that subsequent WB NRM projects have discontinued the CBRMP approach of channeling loans to communities through the MDF window. The region, in its comment on the ICR, further noted that "the new projects in the Philippines have moved towards better fiscal management – where on-lending by government entities is definitely not a sustainable way to go ". With respect to transfer of environmental technology , the project provided technical support with environmental technologies to LGUs through a memorandum of understanding with the DENR . However, there is no mechanism to ensure sustainable transfer of environmental technology beyond the project (see risk to development outcome ). With respect to improvements in the implementation of environmental policies , the ICR states (pg. 11) “(a)t the national level, there has been progress in policy in improving processes for land tenure security related to community -based NRM and also support for LGUs in environmental conservation activities .� However, it is worth noting that mid- project the DENR halted the issuance of community tenure instruments in public lands . Interviews in the field and with the TTL indicate that this was the decision of an individual Secretary in the DENR . The DENR eventually reversed this decision, due in part to pressure from the donor community, and agreed to proceed with issuance of community tenure agreements for communities participating in the project but as of project close only 65% had been processed. According to the ICR, the DENR has confirmed that incomplete processes for land certification in project areas will be supported by DENR field staff in collaboration with the communities and the LGUs . However, this does not seem to indicate that the procedures for issuing tenure instruments in general have been improved, as it does not improve processing of tenure instruments of non project communities . In fact the ICR includes among its lessons (pg. 19) “the process of issuance of CBFMA and PACBRMA (tenure instruments that provide communities with tenure in public forest and protected areas ) is too slow and needs to be further streamlined .� The Borrowers ICR outlines a number of recommendations from a forest policy study commissioned by the project but provides no evidence that these recommendations were acted upon . The ICR also states that the DENR is involving LGUs more strongly in local NRM initiatives as a result of the project but it is not clear to the reader what the level of LGU involvement was prior to the project and how it changed as a result of the project . Provision of resources to LGUs to finance natural resource management projects is rated modest . As noted above, the project succeeded in channelling resources to LGUs through a rural window created in the Department of Finance through a one time transfer of project funds . In addition, as noted above, the central government has not provided additional finances to fund the rural window, though the window is still operational and is currently used to channel GEF grants to communities under an ongoing WB project . Moreover, according to the ICR the future demand for funds from the rural window is uncertain as the loan - grant finance mix has not been continued and it is unclear if there will be a demand for financing at 100% loan terms. (See also sections on enhancing LGU capacity above and section 7.0 risk to development outcome). 5. Efficiency (not applicable to DPLs): At appraisal the estimated EIRR was 28% and the ICR reports an overall EIRR of 27%. The coverage/scope of the EIRR presented in the ICR is unclear . According to the ICR the recalculated EIRR included all component costs, including project management, training costs, overhead costs and the cost of the LGU equity in the sub -loans. However, according to the task team the overall EIRR calculation was based on two NRM activities : social forestry/agroforestry which had an EIRR of 24% and coastal fisheries which yielded an EIRR of 44%. It does not include returns on infrastructure investments or income generating subprojects due to data limitations . There is, therefore, some uncertainty about the coverage of the figure but the overall EIRR seems substantial . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 28% 100% ICR estimate Yes 27% 0% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project's primary objectives appear to have been achieved and efficiency, based on the overall EIRR, seems substantial. However, there were shortcomings in the achievement of the second order objectives and relevance of design was modest. Therefore, on balance outcome is rated as moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Risk to development outcome is significant for several reasons . First, although sustainability plans were created by all participating LGUs there are several concerns regarding the ability of LGUs to finance the implementation of these plans. According to the last supervision mission in Sept 2006, only about 60 percent of LGUs reported that their sub-project sustainability mechanisms have been institutionalized and incorporated in their Annual Investment Programs. In addition, the ICR (pg. 36) notes that " LGUs do not have the financing for a municipal environmental (MENRO) office, hence there needs to be a continuous lobby for more support to LGUs with a view to greater devolution of responsibility and resources ". Finally, the QAG quality of supervision report questioned the ability of LGUs to raise the necessary taxes to sustain subprojects in the context of incomplete decentralization in the country as given that the project's target LGUs were among the poorest income class in the country . Second, the community repayment rate to LGUs was low (33%, which is substantially below the target of 90%). Third, both the ICR and the QAG QSA acknowledge that many LGUs and POs remain dependent on technical assistance from the various line agencies yet according to QAG "it is not certain that such assistance will continue " due to the lack of clear mandate and budget to support them. The region in its comments notes that the expectation for external support for continued development is an issue in most projects . Fourth, funding for expanding the rural finance window is uncertain . As noted in the section on efficacy, although the window is currently operational, it is unclear how long project reflows will last and whether the central government has committed any of its own fund to expand this mechanism . The region in its comments states that this will not prevent maintenance of activities that have already been established . Fifth, tenure security which is critical for the sustainability of NRM investments in uncertain . According to the ICR, tenure instruments have only been issued for 65% of project sites. The ICR states that DENR is committed to facilitating the processing of the remaining instruments but the 65% figure has not changed since 2005 when the projects impact assessment was carried out . The ICR itself acknowledges (pg. 19) that “(t)he process for issuance of CBFMA and PACBRMA is too slow and needs to be further stream -lined.� Tenure security is further questioned by comments in the Borrower's ICR that indicate that in some project areas there are multiple claims to the same piece of land and lack of appropriate tenure instruments for coastal areas . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: Bank performance in ensuring quality at entry was unsatisfactory . Project implementation was extremely slow and only improved after the project was restructured . There were several problems with the initial project design . First, as noted in the section on relevance, the project overestimated the implementing agency and communities' capacity and was not in step with the institutional framework in the country . Second , in light of country capacity the project set ambitious objectives and set up M&E with indicators that do not adequately assess project objectives. Third, a new agency within the Department of Finance (DOF), which was unfamiliar with the Bank procedures, had little implementation experience and limited technical capacity to implement natural resource management projects, was selected as the lead implementing agency . Various technical line agencies were meant to act as implementing partners but the ICR acknowledges that coordination between multiple implementing partners was problematic. Fourth, the project required a shift in the role of line agencies from providing front line services to providing technical support to LGUs which would in turn enable the LGUs to respond to community needs. The borrower's Project Completion Report (pg. 41) notes that the project underestimated the effort required for this shift to occur . Fifth, the risk assessment presented in the PAD did not identify many of the implementation problems that resulted from the project's design complexities and the low capacity of local partners. Finally , it is evident that some lessons of experience were not taken into account, as the ICR (pg. 17) states “It is notable that a key recommendation from the SECAL project ICR was that ' Substantial delays in project start -up and implementation can be avoided if institutional capabilities are thoroughly assessed as early as project preparation stage and a corresponding agreed action plan formulated with involved institutions for focused capacity .' The Bank’s performance could have been improved if this recommendation had been applied .� (see also relevance of design ) Supervision was moderately satisfactory . The project had a total of 5 different task managers but it is not clear if this affected supervision. That said, supervision missions were conducted every six months, and additional interim technical supervision missions were conducted during the restructuring process to address specific project weaknesses. However, despite the frequency of supervision missions, Bank supervision was not able to turnaround a deficient M&E system . The ICR (pg. 17) states “(e)arlier and more comprehensive assistance with M&E would have addressed one of the weakest aspects of project management � (pg. 17). In addition, there was a lack of focus on monitoring beneficiary contribution to the project . There were also some weaknesses in follow up to supervision missions. The ICR points out that the Bank ’s early missions over-estimated the capacity of the implementing agency to respond to agreed actions . at -Entry :Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government performance was satisfactory on balance . The government provided sufficient counterpart funds and was supportive of project restructuring . However, according to the ICR and at the time of IEG's field mission for the ongoing project assessment, the future loan -grant mix for lending from project reflows through the rural window has not yet been determined, which could have implications for future demand for this mechanism from the LGUs. The implementing agencies performance was moderately satisfactory . The CBRMP was implemented by the Municipal Department Fund Office (MDFO) in the Department of Finance, as the lead implementing agency, in conjunction with several line agencies (The Department of Environment and Natural Resources (DENR), The Department of Interior and Local Government, the Department of Agriculture (DA), and the Bureau of Fisheries and Aquatic Resources (BFAR)) that served as implementing partners and provided technical support to participating LGUs under a Memorandum of Understanding with the DoF . Principal shortcomings in the implementing agencies' performance, on the part of the MDFO, were delays in staffing, high level of staff turnover and overly bureaucratic procedures which negatively impacted project performance . There were also weaknesses in project monitoring (see section 10) and fiduciary concerns including insufficient number of financial management staff and rapid turn -over; and incomplete and late documentation of submissions from LGUs. The ICR notes that all concerns were eventually addressed, though not always in the agreed time frame . According to a multiagency sustainability meeting carried out at project close, which is cited in the ICR (pg. 35), there should have been more effort on the part of implementing partners to improve coordination in early stages of the project. In addition, the ICR states that critical implementation issues, such as the process for the issuance of tenure instruments in project areas and strengthening of market assessments prior to starting income generating activities, did not receive sufficient focus until after 2002. Finally, in the middle of the project implementation period, the Secretary of DENR withdrew all community tenure instruments in the country . DENR eventually began the process of reissuing such agreements /processing new agreements but at project close the issuance of tenure instruments for all project sites was incomplete, which could have implications for sustainability or NRM investments. According to OPCS-IEG harmonized criteria, a rating of satisfactory on one dimension of performance and moderately sat on the other results in an overall rating of moderately satisfactory . a. Government Performance :Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: There were significant shortcomings in design, implementation and utilization of the M&E system . As recognized by the ICR. Design : The indicators in the PAD do not adequately assess the achievement of project objectives . Most indicators were output indicators and no targets were set until the MTR, but the targets were never formally recorded in the projects data system. Baseline data was not established, which according to the ICR was a “serious impediment to the establishment of project monitoring � and created difficulties for the ICR team in assessing project performance . In addition, the M&E system did not monitor the issuance of tenure instruments which are critical for sustainability of the NRM investments supported by the project . Implementation : According to the ICR, systematic data collection only occurred in the last years of the project when there was increasing emphasis on the importance of M&E . The ICR states that during project restructuring the Bank provided additional technical assistance with M&E which included the retroactive establishment of a base -line. None the less, this did not lead to the establishment of a comprehensive M&E system and monitoring continued to focus on inputs and outputs. A computerized MIS system, which according to PAD was meant to monitor project implementation and to guide policy makers, was not installed until 2004 and, according to the impact assessment carried out in 2005, it was not operational in all LGU locations . In addition, the number of project participants and subproject activities were not tracked . The ICR team was unable to provide IEG with a the total number of project participants or a breakdown of the individual subprojects implemented . Utilization : Poor M&E affected project implementation and assessment of project achievements . The borrower's project completion report identified “weak project management controls over monitoring variables � as one of the main factors affecting implementation. According to the ICR (pg. 8) "some data and cross verification gaps remain despite twelve Management Information Training sessions ….there is insufficient data on infrastructure operation to assess its usage, survival rates of rehabilitation planting are not sufficiently verified and reported changes in fish catches are based mainly on anecdotal evidence .� In addition, the M&E system did not capture the repayment rate from people's organizations to LGUs. Consequently the ICR team was not able to assess the project based on M&E alone . They relied on supplemental information such as the impact assessment carried out in 2005 for the borrower's ICR and an additional study carried out in 2007 by the ICR team and the project office to address gaps in data regarding income generation as a result of project activities . a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The project received an unsatisfactory performance rating on its ex post Procurement Review, carried out in 2006, but the ICR reports that the issues raised were satisfactorily addressed . According to the ICR, the project complied with all safeguard policies. However, some instances of less than optimal ecological impacts from the community forest plantations were reported in the Safeguards Thematic review of Decentralized Projects in the Philippines carried out by WB's Social and Environment Unit in 2004 (pg. 21): "Unfortunately, some of these projects appear to promote planting on already vegetated fragile slopes without a clear, well -defined forest management plan. DENR’s promotes mahogany planting without providing sufficient explanation of silviculture practices to local technical and management staff. There was no firm estimate of the length of time required for the trees to reach a girth and height sufficient for marketing. It is not clear in the upland CBRMP subproject activities that planting mahogany seedlings on patchy parcels of public lands is the most judicious action with respect to biodiversity, ecological balance, and wise land use planning...In addition, the Bank’s natural habitats safeguard policy (OP 4.04) raises issues associated with the ecological implications of promoting reforestation with cash crop trees, like mahogany, rather than native tree species." As noted in the section on environmental objectives, an impact assessment of the project also reported occasions of inappropriate species being planted on some project sites . However, the region, in its comments on the ICR review, states that the ICR team visits confirmed that species planted were appropriate for each site . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Significant Significant risks include the ability of Outcome : LGUs to fund sustainability plans, low community repayment rates, lack of mechanism to ensure technical support, and lack of tenure security for some sites. Bank Performance : Moderately Moderately Quality of Entry was unsatisfactory and Satisfactory Unsatisfactory supervision was only moderately satisfactory. Borrower Performance : Satisfactory Moderately According to OPCS-IEG harmonized Satisfactory guidelines, a satisfactory rating on one dimension and a rating of moderately sat on the other results in an overall rating of moderately satisfactory . Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Three relevant lessons were adapted from the ICR : Security of land tenure is important to ensure ownership and sustainability . M&E should include indicators to monitor the progress in issuance of tenurial instruments in cases when they have not been issued prior to project implementation. In a complex project, with multiple implementing partners, a common understanding of project concepts and processes needs to be achieved early in the project . Good coordination processes must be embedded in the normal processes of partner agencies at municipal, provincial and regional levels . Development of a physical mechanism for the transfer of finances to LGUs is insufficient to achieve development objectives. A stronger link between financing mechanisms and the practical requirements for implementation must be established for successful investment of the finances transferred . 14. Assessment Recommended? Yes No Why? This project was already selected for assessment as part of a cluster of 5 projects in the Philippines that are expected to provide input into several ongoing IEG studies . 15. Comments on Quality of ICR: While the ICR had several strong aspects, such as its candid assessment of the project's M&E system, it had a number of significant weaknesses and on balance is rated unsatisfactory . The principal shortcomings were: (1) The quality of evidence. The ICR provides insufficient evidence to draw a conclusion on achievement of some project objectives. Some claims are not backed up with sufficient analysis or concrete evidence . Essentially the project had two overarching objectives: to reduce poverty and to reduce environmental degradation . The ICR does not appear to have explained either quantitatively or qualitatively, the extent to which the poorest within target regions benefitted or the extent to which environmental degradation declined, as the focus in the ICR was largely on outputs . A supporting objective was to improve systems and processes at local and central government levels . The ICR offers little quantitative or qualitative outcome evidence on how far this was achieved and surprisingly made no attempt to assess cost recovery from beneficiaries, a key element in sustainability . To reach a conclusion on some ratings, IEG had to draw on the project files and additional information that was provided by the region . This is a significant shortcoming particularly when some of information is readily available from other sources ; (2) Some information on project financing is missing and the actual beneficiary contribution is not reported . The region in its comments clarified that a beneficiary contribution materialized but the ICR team did not attempt to estimate this amount . This is a significant weakness in an ICR for a project that provides funds to communities and it raises questions about sustainability; (3) IEG had considerable difficulty understanding the components of the ERR calculations, the information was unclear as presented in the ICR . Clarification was obtained through correspondence with the ICR team. a.Quality of ICR Rating : Unsatisfactory