ICRR 12742 Report Number : ICRR12742 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/25/2007 PROJ ID : P076784 Appraisal Actual Project Name : Second Rural US$M ): Project Costs (US$M): 142.9 21.3 as of loan Employment Project cancellation Country : Algeria Loan /Credit (US$M): Loan/ US$M ): 95.0 7.7 Sector Board : RDV US$M): Cofinancing (US$M ): Sector (s): Crops (45%) Forestry (35%) Irrigation and drainage (10%) Animal production (5%) Roads and highways (5%) Theme (s): Participation and civic engagement (29% - P) Land administration and management (29% - P) Water resource management (14% - S) Rural non-farm income generation (14% - S) Rural services and infrastructure (14% - S) L/C Number : L7168 Board Approval Date : 04/29/2003 Partners involved : Closing Date : 06/30/2009 10/04/2006 Evaluator : Panel Reviewer : Group Manager : Group : April Connelly Ridley Nelson Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: The objective of this Specific Investment Loan (SIL) was to generate and stimulate employment in the rural parts of the Project Area through: (i) direct employment to address the short term problem of high levels of rural unemployment; and (ii) asset creation with a medium to long term perspective for sustainable employment by increasing and diversifying agricultural production . The key performance indicators proposed to assess achievement of objectives were : creation of 36,000 person-years of direct employment through labor -intensive activities; increase in rural household incomes and assets through increases in production; indirect employment and PSD effects from increased economic activity and incomes; and, improved capacity in the central and decentralized Ministry of Agriculture and Rural Development (MADR) and General Directorate of Forestry (DGF) administration through training and studies b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Component 1: Soil Erosion Control ($US 50.2m appraisal; $US9.5m as of loan cancellation) (a) Carrying out, through the provision of goods, services and the execution of works, a program of reforestation, including forestry and forage species, and the development and implementation of pastoral management plans; (b) Carrying out, through the provision of goods, services and the execution of works, of river bank management activities, including biological stabilization of river banks; (c) Carrying out, through the provision of goods, services and the execution of works, of a program of terracing to break the force of water flows and biological control to stabilize soils; (d) Carrying out, through the provision of goods, services and the execution of works, of a program to produce plant material for project activities; (e) Carrying out, through the execution of works, of a program of flood control; (f) Carrying out, through the provision of goods, services and the execution of works, of a program of forestry rehabilitation, including the implementation of forest management plans; and (g) Carrying out, through the provision of goods, services and the execution of works, of a program of silvicultural activities, including cleaning up of forest areas . Component 2: Agricultural Development ($US 66.7m appraisal; $US8.8m as of loan cancellation) (a) Carrying out, through the provision of goods, services and the execution of works, of a program of : (i) fruit tree plantations; and (ii) vineyards; (b) Carrying out, through the provision of goods, a program of : (i) windbreak; (ii) land clearing; (iii) development of small livestock; and (iv) activities targeting rural women, including handicraft activities; and (c) Carrying out, through the provision of goods and the execution of works, of a program of rehabilitation and upgrading of rural roads. Component 3: Water Resource Mobilization ($US 15.5m appraisal; $US0.6m as of loan cancellation) Carrying out, through the provision of goods, consultants ’ services and the execution of works, of a program of : (i) construction of hill dams not exceeding fifteen (15) meters in height, for irrigation, potable water supply and erosion control; (ii) construction and maintenance of water points; (iii) rehabilitation of springs; (iv) construction and rehabilitation of shallow wells; and (v) construction. Component 4: Institutional Development for General Directorate of Forestry (DGF) ($US 4.6 appraisal; $US1.8m as of loan cancellation) Carrying out activities to strengthen the institutional capacity of DGF, through the preparation of studies and the provision of training and goods and rehabilitation of deep wells . Component 5: Institutional Development for Ministry of Agriculture and Rural Development (MADR) ($US 4.5m appraisal; $US0.7m as of loan cancellation) Carrying out of activities to strengthen the institutional capacity of MADR, through the preparation of studies, the provision of training and goods and the execution of works, for small scale pilot projects for demonstration purposes within the territory of the Borrower. In addition to the components above, the project was designed to include an unallocated amount of US$ 0.95m. The components were not revised but the ICR reports that targets for the following activities were changed during implementation: Component 1: modifications in volume were made in riverbank management (-4%), terracing (-8%), forest rehabilitation (+3%) and reforestation (+14%); and, Component 2: targets for beekeeping and poultry production activities were increased (+5%); fruit tree plantations areas were slightly reduced (-2%); and, cuniculture activities canceled due to lack of initial reproduction stock and lack of demand as rabbit meat is not traditionally consumed in most parts of Algeria . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Dates . The loan became effective 11/28/2003 and was expected to close 06/30/2009 (an implementation period of 67 months or 5.6 years). However, due to the country's favorable balance of payments and unexpected high revenue from oil exports, the Government of Algeria decided to cancel its outstanding loans with the Bank . Consequently, the loan was closed on 10/06/2006, after an implementation period of 34 months and US$7.66m (8% of original loan) having been disbursed. The non committed balance of the IBRD loan $US 81.56m was cancelled on May 17,2006. The government cancelled the undisbursed balance of the loan, pre -paid the disbursed portion, and continued to finance the project with the country's own resources . As of the date of the ICR (06/27/07), the project was still under government implementation . Total project costs were estimated at appraisal as US$ 142.98m. To be financed by: IBRD $US 95.0m; GoA $US24.22m; and Local Beneficiaries $US23.67m. As of 09/30/06 total disbursements were US$21.31m (15% of estimated total project costs ), broken down by: IBRD $US7.66m; GoA $US11.08m; Local Beneficiaries $US2.57m. 3. Relevance of Objectives & Design: Relevance of the Second Rural Employment Project (SREP) was high. It was designed as a follow on to the Rural Employment Project (REP) which IEG rated satisfactory. Objectives were highly relevant to the 1996 CAS priority of protecting the poor during the period of adjustment by accelerating implementation of well targeted labor intensive programs in agriculture. The objectives remain relevant to current CAS (2003) priority of strengthening natural resource management and establishment of an enabling environment for private sector development, employment generation, and poverty reduction . The project is also consistent with the government's national development agenda which focuses on reforms to sustain growth, generate employment, and reduce poverty and vulnerability . The project's activities were designed to support its objectives . 4. Achievement of Objectives (Efficacy): Due to unusual circumstances in which the Bank loan was prematurely cancelled but the project is still ongoing, this is a challenging project to rate . The ICR focused on project outputs rather than outcome /impact of project activities because the project is still ongoing . IEG rates efficacy substantial because the overall objective of generating and stimulating employment in the rural parts of the Project Area can reasonably be expected to be met. The project's physical accomplishments were below the pro - rated targets for the 34 months that the IBRD loan was effective due to a preliminary phase required for community mobilization, organization and capacity building and the elaboration and approval of Community Development Plans . However, implementation picked up in the five months between loan cancellation and the writing of the ICR offering promise for the future . The ICR does not report on progress towards achievement of the four key performance indicators . By pooling information presented in various sections of the ICR, IEG was able to assess progresses towards achievement of key performance indicators, during the 34 months that the IBRD loan was effective, as follows : Creation of 36,000 person-years of direct employment through labor -intensive activities. As of October 2006, the project generated 1.64 million person-days of employment, equivalent to 6,393 person years; which represents 18% of the final targeted values (36,000 person years) and 74% of the prorated values for year 3 (8,600 person years). Increase in rural household incomes and assets through increases in production . The ICR estimates that as of September 30,2007 the project generated $US12m from direct employment but not targets expressed in dollar amounts were not reported. The region in its comments notes that US$ 12m is a translation of the person - years noted above into a dollar amount based on the current wage rate and that the target for this indicator is the dollar equivalent of 36,000 person- years. Indirect employment and PSD effects from increased economic activity and incomes . The ICR (p.g 10) estimates that the first investments such as tree and vineyard plantations and income generating activities initiated have generated about 7,600 jobs, mainly for production and maintenance and that a total of 11,000 households ( of which 1,800 are women) have benefited from second generation employment . The project supported the creation of 110 new small and medium private enterprises. In addition, 1,800 women received support from the project for income -generating activities; 50% (or 900) beekeeping, 700 received sewing equipment and relevant training . The ICR does not account for the type of support received by the remaining 200 women. The ICR does not report on the resulting changes in income or other outcomes of this activity . The ICR itself acknowledges that the "longer term objective of creating assets for stimulating more sustainable employment was only partly achieved " but it does not substantiate the partial achievement with quantified evidence or report targets . The region in its comments notes that targets can not be reported for this indicator as they were expected to be developed during implementation and the loan was prematurely cancelled before this happened . Improved capacity in the central and decentralized Ministry of Agriculture and Rural Development (MADR) and General Directorate of Forestry (DGF) administration through training and studies . As of loan closing date, one out of four training sessions planned for the MADR was completed : 116 staff were trained in rural development policies and strategies . Of the five types of training planned for the DGF, four were partially completed : Training in Planning and Management Techniques for decentralized projects . Two modules were planned only one was carried out. 26 people (including 9 women) participated in training in M&E methods (planned for 30); Training in Participatory Approach . Three modules planned only one was complete . 14 people (including 3 women) participated in training of trainers in participatory approaches (planned for 7); Training in Soil Conservation Techniques . One of two planned activities was complete . 13 people (3 women) received training (planned for 30); and, Training in Environmental Assessment . 14 people ( 5 women) received training in environmental assessment (planned for 20). 5. Efficiency (not applicable to DPLs): The PAD estimated an overall ERR of 23.6% and an overall FRR of 13.39%. The ICR provides insufficient information to allow the reader to make a judgement on efficiency . The ICR did not calculate an economic or financial analysis because of the IBRD loan was cancelled less than three years after its start date and before project implementation closing. The ICR does not provide a comprehensive discussion of efficiency in use of project funds but based on information presented in other sections of the ICR, there are efficiency concerns for two of the project's activities: (1) the costs for boreholes activities were affected by an unexpected increase in prices, this program was cancelled and the funds were reallocated to fund wells and water points (2) the government's ISR asserts that cost of rural roads was underestimated. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re-estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal Yes 24% 93% ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Based on high relevance and substantial efficacy, but with some uncertainty about efficiency due to lack of evidence, the project is rated satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: Risk to development outcome is assessed as modest. Four of the 13 risks identified at project appraisal have improved and the rest remain unchanged . According to the ICR, as of the fourth year of implementation, the Government was continuing to make all necessary funds available in a timely manner and the pace of implementation had picked up for many activities . The ICR notes that the contract of the M&E specialist should have been extended for an additional year, which raises questions as to the implementing agency's capacity to carry out sufficient M&E. However, the region in its response confirms that six months after the project loan closing date the government's M&E capacity was adequate . The ICR also notes that cooperation has diminished between the implementing agency and the Directorate for Investment Planning and Economic Studies in the Ministry of Agriculture and Rural Development (DPIEE) since loan closure and a change of leadership . The region in its comments notes that, based on the government's progress report for the period ending March 31, 2007, project implementation has not been aversely affected . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at entry was moderately satisfactory . The project was designed as a follow on to REP I which IEG rated satisfactory. However, there were shortcomings in the following areas : The original disbursement schedule was out of step with the experience for this type of project . The ICR itself (pg. 6) notes that “the disbursement schedule should have taken into consideration the fact that little is disbursed during the first year or two of implementation for this type of project because the community -based participatory approach requires, once the project is effective, a preliminary phase of community mobilization ". This was also flagged as an issue by the ICR for the REP I . The livestock subcomponent was designed to include cuniculture even though "rabbit meat is not part of Algerian culinary tradition". The activity was eventually abandoned due to lack of demand and inability to secure breeding stock and the money for implementation transferred to other activities but staff time and resources were spent in the preparation of this activity . The time needed to secure the water resource mobilization was underestimated . The ICR itself points out that the plantation program should have been scheduled to start one or two years after the Water Resource Mobilization component. The time required for procurement staff to adapt to Bank procedures was underestimated . According to Government's ISR viticulture was planned in geographically or socially unsuitable areas, and costs were underestimated for well drilling and rural roads . On balance supervision was satisfactory . Eight supervision missions were carried out prior to cancellation of the loan. The Bank was proactive in providing advice and assistance to improve procurement skills of project staff and redesign the M&E component. There were minor shortcomings: The ICR itself notes that given the interdisciplinary nature of the project it would have benefitted from more frequent supervision missions by larger teams of experts, though that would entail a larger than normal supervision budget. A mid term review was not carried out prior to loan cancellation, which occurred in the project's third year . The Government's ISR (pg 33), noted that the project could have benefitted from a mid term review in year two. at -Entry :Moderately Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: IEG concurs with the satisfactory rating for Government performance. Despite canceling the IBRD loan, the Government has demonstrated strong commitment to continuing project implementation and making funds available in a timely manner. As of loan cancellation, Government financing represented 52% of funds disbursed to the project from all sources. The period immediately following loan cancellation, October to December 2006, the GoA disbursed US$2.67m. The ICR reports that of the US$65.27m allocated for 2007-2008, US$37.27 has been committed. According to the ICR, the government has also indicated that there are plans to extend the project for a year beyond the original closing date to complete all the tasks identified under the project . Financial audits have been carried out regularly . However, the following shortcomings delayed implementation at the start of the project: An initial delay in the release of funds from the Ministry of Finance to the implementing agency . The reason for the delay is not explained . The ICR notes that the supply of funds after this delay was satisfactory . According to the Borrowers ISR, project launch was delayed due to departure of Central Project Management Unit staff who had participated in training sessions during project preparation . In addition there was insufficient staff in some wilayaa units as staff were also managing other activities . Overall, Implementing Agency performance was satisfactory . Implementation was inhibited by the following : Unfamiliarity with Bank procurement procedures on the part of decentralized procurement commissions delayed work in two areas during year 2. Procurement units did not follow approval and signature procedures stipulated in the loan agreement . Bank procedures, which state that bids cannot exceed US$ 500,000, resulted in a large number of contracts and resulted in an increased workload for project management units . Bid documents were published late and were poorly targeted . The ICR reports that project staff were eventually able to make up for initial difficulties . Staff turn over. Government's ISR notes that key WB staff retirements caused an adjustment period for counterpart staff. a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Design . The logframe as designed in the PAD adequately measures achievement of project objectives . The PAD also notes that social impact monitoring was to be an integral part of the project and that several participatory monitoring exercised would be undertaken during project implementation to assess beneficiaries ’ perceptions of the project’s progress in providing adequate channels for participation to target group . Implementation and Use . There is limited discussion of the M&E system in the ICR . Instead, the ICR elaborates on the process of contracting an M&E expert to assist the project management unit, which had limited capacity at project appraisal. According to the ICR, an external "expert" was temporarily hired but after the loan closing date the implementing agency decided that it had sufficient in house capacity and the contract was not renewed . The ICR states that the M&E specialist should have been retained for an additional year "to fully implement a state-of-the-art M&E system" but notes that "the system in currently in place for the project is effective and provides the essential data required for satisfactory project monitoring and evaluation ." However, the analysis presented in the ICR does not substantiate this conclusion . The ICR does not discuss the design of the current M&E system or mention any of the indicators that are actually being measured . In addition, there is no discussion of impact evaluations . The ICR (pg. 19) notes that the baseline survey was completed but it does not state what was measured . The region in its response clarifies that the baseline was conducted in 2005, though it does not clarify what the survey measured . The ICR does not mention the social impact monitoring or participatory monitoring exercises discussed in the PAD . The region in its response notes that the Bank could not assist with establishing social impact monitoring or conduct participatory monitoring because the loan was cancelled when "the project was only 18% complete". a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The project was rated environmental category B in the PAD as activities such as erosion control measures and agroforestry were expected to have positive environmental effects . To comply with Environmental Assessment Policy (O.P. 4.01), an Environmental Management Plan was developed and included a screening procedure to review all subprojects for their environmental impact and propose mitigation measures . As of the loan cancellation date, the environmental study for the rural roads was completed and environmental and feasibility studies for small hill dams were in progress. The building of small hill dams (lower than 15 m) was to comply with the Bank’s Safeguards Policies related to Safety of Dams (OP 4.37). Prior environmental impact studies and prior establishment and training of water users associations were required . According to the ICR, no social safeguards were triggered . Involuntary resettlement (OP4.12) was discussed with the borrower during project preparation and the implementing agency decided to exclude from the project any investment activity entailing displacement and /or resettlement. The ICR notes that "the project put in place a rigorous system for verifying land ownership and use in the suggested areas and hired a consultant to verify if lands donated were indeed done on a voluntary basis ". The Bank reviewed the process as part of supervision missions . Of the 12 sites initially identified for hill dams, only seven passed the Bank ’s OP 4.12 criteria. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory With some reservations about efficiency due to lack to evidence . Risk to Development Moderate Moderate Based on additional information Outcome : provided by the region IEG changed its rating from significant to moderate. Bank Performance : Satisfactory Moderately Shortcomings in quality at entry Satisfactory negatively affected implementation . Borrower Performance : Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The ICR (pg. 15) repeats a lesson learned from the REP, which IEG does not find has been demonstrated by the SREP experience: "SREP has demonstrated that it is possible to simultaneously resolve rural unemployment problems and promote agricultural growth, improve the conditions of the natural resource base and protect the environment." As the loan was prematurely cancelled there is little to no evidence of such outcomes in the ICR . In particular, there is no discussion of agricultural growth resulting from project activities and the ICR itself (pg. 10) admits that the environmental impact of project activities "has not yet materialized and will not before several more years." Specific lessons from the experience of SREP are : The disbursement profile for projects using participatory approaches should reflect the fact that there is a preliminary phase for community mobilization and capacity building . Computerized project management programs may be useful for establishing critical paths . Project design should take into account optimal timing between individual components . In this case, the plantation program should have been scheduled to start one or two years after the Water Resource Mobilization component. 14. Assessment Recommended? Yes No Why? The project is ongoing and an audit will provide the opportunity to assess final outcomes and the monitoring and evaluation system. It also may provide lessons of experience on how the project proceeds when the borrower takes over implementation. 15. Comments on Quality of ICR: The ICR is unsatisfactory. The logic is difficult to follow and it suffers from the following shortcomings : Evidence to allow the reader to assess progress in achievement key development indicators is not presented in a concise manner under the appropriate heading but is scattered throughout the document which makes it laborious to review. There is no information related to efficiency in the use of project funds . The discussion on M&E is limited and does not provide details to substantiate the claim that M&E is functioning effectively. Because the project is still ongoing, the ICR focused more on completion of activities rather than outcomes . None the less, the ICR makes some references to outcomes generated by completed activities but does not provide credible evidence to substantiate these statements . Some lessons are not specific to the experience of SREP . Failure to discuss targeting mechanisms . The PAD includes a detailed social assessment that identifies five specific vulnerable groups of importance to the project . The project was meant to use "a double approach to beneficiary targeting (self targeting mechanisms and direct targeting by the DGF staff ) (to) minimize the risk of elite capture of project benefits and allow vulnerable groups to be the main beneficiaries of project interventions". The ICR is silent on implementation of these targeting mechanisms and only mentions one of the five vulnerable groups (women). The project was meant to be implemented using a participatory approach but references to community participation in the ICR are vague and are not substantiated by hard evidence or even a detailed description of exactly how communities participated in implementation . Data errors. The NCO data sheet erroneously reports the actual closing date as 06/30/09, this is the original closing date. The IBRD loan was officially closed on 10/04/2006. a.Quality of ICR Rating : Unsatisfactory