ICRR 14601 Report Number : ICRR14601 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/13/2015 Country : Morocco Project ID : P069124 Appraisal Actual Project Name : Rainfed Agriculture US$M ): Project Costs (US$M): 41.07 Not Reported in Development Project the ICR L/C Number : L4701 Loan /Credit (US$M): Loan/ US$M ): 26.80 19.97 Sector Board : Agriculture and Rural US$M): Cofinancing (US$M ): Development Cofinanciers : Board Approval Date : 06/10/2003 Closing Date : 12/31/2009 12/31/2011 Sector (s): General agriculture fishing and forestry sector (40%); Irrigation and drainage (20%); Roads and highways (20%); Other social services (10%); Central government administration (10%) Theme (s): Other environment and natural resources management (29% - P); Rural services and infrastructure (29% - P); Improving labor markets (28% - P); Participation and civic engagement (14% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Ebru Karamete John R. Eriksson Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: Original Objectives According to the Loan Agreement (LA, p. 13) "the objective of the Project is to improve the livelihood of rural populations in priority rainfed areas by promoting local participatory development, and fostering sustainable agriculture growth and the development of non -agricultural activities." According to the Project Appraisal Document (PAD, pg. iii) the project's objectives are to "(1) To reinforce, in rainfed areas, institutional capacity for Project formulation and implementation, in partnership with the communes and local organizations. (2) In seven priority action zones, to improve the incomes of the local populations and reduce their vulnerability to drought; and to improve access to public and social services ." The formulation of objectives in the LA is used as the basis of assessment as that is the legally binding document . Revised Objectives and Key Performance Indicators The project objectives and the key PDO indicators were revised on December 30, 2009, to shift the focus from increasing rural incomes to establishing necessary pre -conditions for increasing rural incomes . It was believed that this would make them more realistic, quantifiable and consistent with project activities . Revised PDO : "To promote the development of rainfed based agricultural farming systems in the Priority Rainfed Areas in the Project Area and improve access by local populations to basic infrastructure and socioeconomic services, through an integrated and participatory rural development approach ." (Loan Amendment, p. 2). This PDO entails three sub-objectives: (i) development of rainfed based agricultural farming systems; (ii) improve access by local populations to basic infrastructure and socioeconomic services; and (iii) adoption of an integrated and participatory rural development approach . b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 12/30/2009 c. Components: The project had two complementary components : (A) a capacity building component designed to improve implementation capability for rainfed area development at both national and local levels; and (B) an investment component to finance development activities in the selected priority rainfed areas in seven provinces . Component A Strengthening of implementation capacity for rainfed agricultural development (Appraisal amount US$ 8.9 million; Actual amount not reported in the ICR ) This component financed activities to improve the capacity for implementation of rainfed area development at both the national and local levels. It comprised three subcomponents : A.1- Capacity Building in Ministry of Agriculture and Rural Development (MADR) MADR ) headquarters : - financed support of the programming teams assigned to the seven provinces during the first three years of the project; the development of methodologies for project identification and M&E and evaluation; and the capitalization and diffusion of information on the progress and results of the project . A.2 -Capacity Building at the Local Level : financed (a) rehabilitation of lodging and office buildings; (b) purchase of vehicles and equipment; and (c) training and technical assistance in order to strengthen the capacity for local level programming of the various “partners� of administration to formulate the Communal Investment Programs (PICs) and to monitor their implementation. A.3 -Support to Rainfed Agriculture : (i) Strengthening the implementation capacity of MADR at the central level for developing and managing the rainfed agriculture development program, for adequate M&E and supporting MADR on a restructuring plan for its field offices. (ii) Strengthening the local implementation capacity of MADR, local government and community institutions . The Project was to provide technical assistance and training to help prepare local participatory PICs . (iii) Developing new instruments and institutional capacity to improve support services for rainfed agriculture including drought management and agricultural research . Component B Rural development activities in rainfed priority areas . (Appraisal amount US$ 31. 31 .9 million; Actual amount not reported in the ICR ) This was an investment component that financed development activities in the selected priority rainfed areas in seven provinces . There were three sub-components: B.1 - Agricultural and Pastoral Development : financed soil and water conservation, and land development works aimed at increasing the productivity of agricultural systems, improving natural resource management and mitigating the effects of drought. B.2 - Rural Infrastructure Development : financed small and medium-scale irrigation systems, pastoral water points, rural roads, other non-agricultural infrastructure and the provision of technical assistance to Rural Communes (RCs) for the formulation and implementation of the PICs . B.3 - Support to Local Initiatives : financed education and skills training activities, including financing equipment for on-the-job training and promoting micro credit organizations, farmer associations, and income generating activities to strengthen the capacity of local organizations in identifying their own development needs . The project components remained unchanged throughout implementation, but their scope was scaled back following the December 2009 restructuring. The institutional development component's original emphasis on the broad goals of developing a scalable programmatic approach for rainfed areas development and supporting the restructuring of the Ministry of Agriculture and activities on drought management was scaled back . The ICR does not specify exactly what changes constituted "scaling back" of activities. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs : Total estimated project cost at appraisal was USD 41.07 million; actual project cost at closing were not reported by the ICR (Annex 1 of the ICR reports actual total project costs and each of the cost elements as exactly the same as estimated costs at appraisal, which is clearly an error; see Section 15 below for further discussion). Financing : The project was financed by an IBRD loan of USD 26.80 million, and a beneficiary contribution of USD 2.60 million. The ICR does not provide the actual beneficiary contribution at project closing . According to the ICR, at project closure USD 19.97 million of the IBRD loan was disbursed (p. i, Data Sheet). The Operations Portal indicates that USD 13.16 million of the IBRD loan was cancelled. The cancelled funds reflected activities that could not be completed within the revised Project period, but the specific reasons for this cancellation are not reported . However, these figures are clearly inconsistent . If USD 19.97 million was disbursed, that would leave a balance of USD 6.83 million, which is not consistent with USD 13.16 million of the IBRD loan being cancelled . Borrower Contribution . Government counterpart funding estimated at appraisal was USD 11.80 million The ICR did not report on the actual contributions from the government . There are gaps and inconsistencies in the financing and borrower contribution figures that cannot be reconciled . Dates : The original time frame was 6 years, January 2004 to December 2009. The project closing date was extended by two years to December 31, 2011. The closing date was extended to make up for implementation delays . The Loan Agreement was amended three times : • The first amendment, in April 2005, created a new disbursement category and reallocated funds to finance sub-project grants in partnership with NGOs and /or research and development institutions . • The second amendment, in October 2008, adjusted the project decision making mechanisms at local and provincial levels to align them with the National Human Development Initiative (INDH) that had developed similar decision making mechanisms, all over the country . • The third amendment (December 2009), implemented after the mid-term review, revised the PDO and indicators, to make them more realistic, reallocated loan proceeds, cancelled EUR 1.4 million of the Loan (approximately US$ 2.03 million with December 2009 exchange rates), and extended the closing date for 2 years. Neither the ICR nor the TTL reported on the dates for the remaining loan amount cancellation . 3. Relevance of Objectives & Design: a. Relevance of Objectives: Original Objectives, Substantial . The original objective of improving the livelihood of rural populations in priority rainfed areas by promoting local participatory development, and fostering sustainable agriculture growth and the development of non-agricultural activities, was substantially relevant to government priorities and the Bank's country and corporate strategies. The ICR noted that (p. 1). In the 1990's, prior to the project's preparation, Morocco's agriculture sector experienced ‘negative growth’ of -0.3%, largely due to a succession of droughts . Coupling with the economic stagnation, this led to an increase in rural poverty rates, income inequality and a large and growing gap between the urban and rural population access to basic infrastructure and services . The government's 2020 Rural Development Strategy, adopted in 1999 with assistance from the Bank tried to address these issues through : decentralization; multi-sectoral integration; and stakeholder participation . A series of projects was prepared within the 2020 Strategy to apply its approach to the poorer, largely non -commercial farming areas, including the Bank financed Irrigation -Based Community Development Project that targeted poor mountainous areas and the Rainfed Agricultural Development Project that targeted poor areas with rain -fed agriculture. The Government also set up a dedicated source of finance, the Rural Development Fund (FDR), managed by the Governors (local representatives of the Ministry of Interior ), and with a mandate to finance multi-sectoral integrated rural development . This was consistent with the 2001 Country Assistance Strategy (CAS) of FY 2001-2005, which prioritized inclusion and reducing the vulnerability of the rural poor through integrated rural development . Specifically, the 2001 CAS emphasized: (a) improving the poverty focus of public policies and expenditures, in particular by acceleration of rural infrastructure programs and a set of integrated and participatory rural development programs targeted at the poorest areas; and (b) reducing agricultural vulnerability and environmental threats . The original objectives were also relevant to the following CAS of FY 2006-2009, specifically to Pillar 1: Poverty reduction through Inclusion and Participation and Outcome 1.1 that included better development of poor rural areas (p. 35-36). Revised Objectives, Modest . The revised objective --to promote the development of rainfed based agricultural farming systems in the priority rainfed areas in the project area and improve access by local populations to basic infrastructure and socioeconomic services, through an integrated and participatory rural development approach -- was only modestly relevant to country priorities and strategies . Over the course of the project, there were several changes in the policy and institutional environment . Starting in 2007, Morocco elaborated a comprehensive set of new sector strategies to promote economic and social transformation. This resulted in a shift away from agriculture -led integrated rural development as a mainstream policy in Morocco, as the agenda of the Ministry of Agriculture had become growth and poverty reduction focused through market-oriented agriculture. The new agricultural strategy (Plan Maroc Vert) had two pillars: developing high added value commercial agriculture and increasing incomes of small -holder farms. Although, the project objectives were relevant to the second pillar, this overall strategy shift diminished the relevance of the Project as a pioneer of new approaches to participatory development . In that sense the objectives were partially relevant to the CAS FY 2010-2013, specifically, to Pillar 2, Service Delivery to Citizens, and the outcome of improved public sector support and services to small holders . The CAS did not include any goals in regards to the development of rainfed farming systems. The ICR (p. 8) noted that, following the MTR, it was discussed how to align the project with the National Initiative for Human Development and with new agricultural strategy particularly the second pillar that included non-commercial farming. But no agreement was reached and the Government and the Bank decided to meet the commitments made to communities under the approved PICs (Community Investment Plans prepared with project support), to revise the PDO and define specific relevant outcome indicators . b. Relevance of Design: Original Design, Substantial . The ICR noted that (p. 4), the project was designed to build capability to implement a nationwide program by testing project interventions in the seven areas . The statement of objectives was clear and measurable, and, overall, the results framework presented a logical causal chain between project activities and expected attainment of the objective . The main objective of improving the livelihood of rural populations in priority rainfed areas was supported through an integrated and participatory rural development approach through the following main channels: (i) Agricultural and pastoral development;, (ii) Rural infrastructure development; (iii) Support to local initiatives for human and institutional capacity development . The Project was the first attempt in Morocco to apply participatory planning and to prepare a communal investment plan in a consultative way at the level of the commune so as to respond to the real needs of beneficiaries . In order to promote local participatory development, the design included activities to strengthen the capacity of agencies at central regional and local level including provision of technical assistance to Rural Communes . Capacity building activities also included strengthening the Ministry of Agriculture on implementing rainfed agriculture programs including improving drought management, which was sound, given the frequency of droughts in Morocco . Furthermore, the components tried to address the issue of low level of investments in rainfed rural areas for basic infrastructure, as well as services such as health, education (especially for girls) and agricultural research and extension. Revised Design, Substantial . With the shift in agricultural strategy over time the project ceased to be a pioneer for a nationwide integrated rural development program, and ended up as a small scale rainfed rural development project . At restructuring, the objectives and key performance indicators were revised but project design was not revised in any significant way, although there was scaling back in terms of activities on drought management strategy as well as reduced emphasis on developing a scalable programmatic approach for rainfed areas and supporting Ministry of Agriculture's restructuring (ICR p. 4). The results framework still presented a logical causal chain between project activities and expected attainment of the objective . Project components were still aligned with the revised simplified objectives of developing rainfed systems as well as improving access to basic infrastructure and socio -economic services. Furthermore, the revised outcome indicators were easier to measure and more attributable to the project . 4. Achievement of Objectives (Efficacy): ORIGINAL OBJECTIVE : Improve the livelihood of rural populations in priority rainfed areas by promoting local participatory development, and fostering sustainable agriculture growth and the development of non -agricultural activities. The ICR did not provide any evidence on the achievement of the original objective, therefore it is rated negligible . The M&E system did not generate data on incomes and no other livelihood outcomes were measured . REVISED OBJECTIVE : Promote the development of rainfed based agricultural farming systems in the Priority Rainfed Areas in the Project Area and improve access by local populations to basic infrastructure and socioeconomic services, through an integrated and participatory rural development approach . This overall objective embodies three sub-objectives, the efficacy of achievement of each of which are assessed below . The ICR does not provide an assessment of what independent effects the new agricultural strategy or the National Initiative for Human Development might have had on the revised Project Objective (see for example, the Health Services Outcome under the second sub-objective, below). (i) Promote The Development of Rainfed Based Agricultural Farming Systems, Modest Outputs • Only 9,400 ha of pasture improvement was achieved, versus a target of 20,500 hectares. The shortfall was largely due to land tenure questions and to community inability to provide protective fencing during the establishment period. • 98 agricultural/livestock water points were improved, compared to the target of 80. The ICR argues (p.12) that this achievement demonstrates the effectiveness of the Project in delivering improvements that fit within local socio-cultural and economic conditions . • There is incomplete data on the number of farmers adopting new technologies . Only 30 pilot farmers were reported to have adopted new technologies (target 350 farmers). However, this number only reflects one Project area. The ICR notes (p.26) that the data for other Project areas was not available at the time of the ICR's preparation . Obtaining data for this indicator was affected by delays in the availability of research results, most of which became available only right at the end of the Project . • The project reported 5,005 ha of additional areas were irrigated or restored to crop production . While the revised target of 5,000 ha was met, the ICR notes (p.23) that only some schemes actually showed an extension of the irrigated area or higher productivity, implying that this figure was not reliable . • 22 technologies were made available to farmers (exceeding the target of 10 technologies). The ICR reported that (p. 26) while the target was largely exceeded, a limited number of pilot farmers adopted the improved technologies and there is no guarantee that other farmers will pick them up, considering also the unclear arrangements for future dissemination, given the restructuring of MAPM ’s approach to extension currently underway . • Additional ha of improved agricultural land, forests and pastures and a similar number of hectares with protected soil and water amounted to 17,425 ha (only half of the target of 30,000). The ICR did not provide the reasons for the low achievement of this target . • Works of bush clearing, stone removal and soil conservation were carried out on 3,412 ha (exceeding the target of 3,000 ha). However, the ICR reported that (p. 26) concerns were raised during implementation on the environmental sustainability of the action . • 156 km of irrigation canals (seguias ) were built or repaired (under the target of 200 km). The ICR reported that (p. 26), it is not clear why a target of 200 km was set at the time of the 2009 restructuring, as the PICs were already approved at that date . Outcomes All of the results reported in the ICR are output level achievements . The ICR concludes that (p. 12) " the project contributed less than expected to the demonstration of pathways to improved productivity in purely rainfed farming ". This conclusion was based on the fact that the most sought after investment for communities was one that provided an assured source of agricultural water . In addition the development of new technologies specifically for rainfed areas were delayed and output indicators for "rainfed" investments fell short of their targets . Drought management activities were scaled back or dropped . (page 4). ii) Improve access by local populations to basic infrastructure and socioeconomic services (ii) : Substantial Outputs The project supported the construction or improvement of potable water, health services, education and rural roads . • Eight ambulances were provided for five provinces and 8 health centers were upgraded (target 10).The ICR does not explain the shortfall . • 45 schools were rehabilitated (target 30 schools), one college was constructed, and the two school buses were acquired. The ICR reports however that as of the writing of the ICR the school buses had not yet been used . • 4 bridges and 131 km of rural roads were constructed (target 110 km). Outcomes : The ICR provided evidence that the project contributed to the increased access to infrastructure and basic services in the project areas, therefore this sub -objective is rated substantial . • Potable water : 20 additional douar (rural villages) were provided with access to potable water by the end of the Project (target: 25 douar). The ICR attributes the shortfall to delays in preparation of the studies and procurement and contractual problems. • Health Services : A 30 percent increase in additional visits to health centers was reported (exceeding the target 20 percent). But the ICR questions the reliability of this number as well as the extent to which the project's provision of ambulances and upgrading of health centers has contributed to the increase (p.13). Other factors may have had in effect, such as disease vectors, other Government programs under the INDH . • Education : The number of additional children attending school increased by 93 percent (exceeding the target of 20 percent). The ICR concludes that (p. 12) the project is likely to be responsible for part of the increase in attendance, in particular for girls, as project support included the construction of separate sanitary facilities . • Rural Roads : There was a 29 percent increase in the number of douars that can be accessed by car year round. (slightly below the target of 30 percent). The ICR argues that rural road outputs contributed to the ‘desenclavement’ or reduced isolation of the douar that were supported .. iii ) Adoption of an integrated and participatory rural development approach (iii) : Modest Outputs • Support was provided to MADR provincial and local representations as well as of local government and community institutions. 550 beneficiaries were trained (exceeding the target of 100). Although subject matter of this training was not reported, it was noted that (p. 25) the program was well designed, of good quality and appreciated . The decentralization of training to each Project area created extra costs and led to overlap in preparation of modules and variable quality of training delivered . • 12 Community Investment Plans (PICs) were prepared (exceeding the target of 7) The ICR reported (p. 24) that they are characterized as good quality, and served as a source of proposal for the national programs INDH and PCD. However, caveats such as limited permanent capacity building, long and costly preparation and limited community ownership are identified. Also, it is not clear why the target is 7 PICs, when 12 had already been prepared at the time of the 2009 restructuring. • 31 percent of (Rural Development Fund (FDR) funding was provided for non -agricultural activities (as a percentage of the total PIC amount, this slightly exceeds the target of 30 percent). Although, the indicator suggests that the FDR performed as planned . The ICR reported that (p 24), the mobilization and use of FDR resources caused long delays and cost huge administrative effort . • 45 agreements with local partners were made (exceeding the target of 30). These agreements not only provided a useful mechanism for Project implementation but also helped build civil society capacity but with varying results. Sectors were: livestock 20; agriculture 7; women’s centers 6; micro-credit 3; tourism 2; oil pressing 2; and honey production 2. They also included the distribution of 456,000 tree seedlings in all seven provinces (target 525,000); provision of six oil presses to coops and associations (target nine presses); supply of 1,178 sires (target 925 sires); and supply of hives and bee colonies . • 32 new local associations were formed (exceeding target 15). This is a successful achievement of the Project, although the associations are of variable capability and sustainability . The strongest are local branches of national associations, the weakest are the associations created specifically for a project enterprise . For these associations, sustainability is doubtful and follow -up support is needed. Outcomes • No outcome level indicators were provided for this sub -objective. As an alternative, during the ICR mission an in-depth assessment of the quality of the participatory process and of the Project ’s institutional development impacts was carried out. However, the ICR also did not provide any outcomes regarding the adoption of integrated rural development approach, and reported that (p. 15) the relevance of the Project as a flagship for a new national program in integrated rural development was swept away with the changes in the political and institutional environment; and noted that (p. 7) one particular aspect of the Project approach, the Ministry of Agriculture taking the lead in developing cooperation between agencies at provincial level and in coordinating programs for rural development, was not adopted by the national programs . • The ICR concluded that (p.13), the project had implemented an exhaustive participatory planning process but its heavy reliance on consultants contributed to delays and to a "distancing" of the population to the process and also the diagnostic phase generated overly -complex data that was inaccessible to the vast majority of the population . Also the process could have been shortened and carried out at a fraction of the costs . • It was found that after the diagnostic phase, the involvement of most of the population fell away rapidly and little to none of the knowledge and ideas collected during the participatory process remained in the community, partly because the reports were in French rather than the local language used by communities . • Nonetheless, it seems like the results noted by the ICR on this aspect were mixed . The ICR also reported that participatory process has ‘taken hold’ in the communes involved in the Project based on the fact that the communes are participating actively in the broader national processes of National Initiative for Human Development (INDH) and Commune Level Development Plans (PCD). This was due to the fact that Government and Bank staff involved in the Project contributed significantly to the design of the INDH, drawing on the Project ’s design and early implementation experience. The following elements of participatory planning introduced under the project were mainstreamed into the national programs : the use of the Commune as the unit of planning; the bottom -up participatory approach; the diagnostic -problem-solution paradigm for planning; the delegation to commune level and provincial-level governance organs; and the dedication of a specific budget for each commune . • The strategic capacity of local decision -makers and public services had been strengthened as decision makers gained experience in participatory diagnosis, managing public contracts, putting together cofinancing for large infrastructure, and matching available projects or funding sources to the priority needs of the Communes . Capacity was also built in terms of public services at the provincial level, but to a lesser extent the local level - to work with communities on investment programming and implementation . 5. Efficiency: Efficiency is rated modest , due to lack of economic rate of return analysis at project closing as well as important efficiency shortcomings such as implementation delays and higher costs . The ex-ante economic analysis calculations estimated benefits in terms of higher revenues stemming from diversification towards higher-value crops and increased cropping intensity . The analysis also assumed higher production costs due to increased use of inputs . The financial analysis focused on the profitability of crops supported by the project. Financial Rate of Returns were calculated for tree crops requiring longer -term investments. The results of the models demonstrated that the possible growth of net income with improved techniques would be substantial: more than US$100 per ha per year in most cases or more than 50% growth in gross income. For olives, the FRR ranged from 11% to over 100 %, for almonds from 39% to 56%. The PAD reported an overall Economic Rate of Return of 16 percent. A sensitivity analysis was carried out to determine the robustness of the NPV and IRR to changes in key variables . The results indicated that the EIRR was particularly sensitive to changes in crop yields and agricultural output prices as it dropped below 10% when these parameters decreased by 10% and 18%, respectively, from the base case scenario .. The ERR was not recalculated at closure because there was insufficient data on production impacts at completion . Instead the ICR took into account the costs of several of the project investments compared to other projects . The ICR noted that (p. 15) "Overall efficiency of the Project as a vehicle for delivering the investment program was relatively low, as implementation was slow and costly in relation to the assets and impacts delivered ". The project experienced a number of implementation delays and had higher costs than anticipated . The main shortcomings were: (i) Reliance on consultants in preparation of PICs that contributed to delays and high cost and the excessive detail and length of the participatory planning process . This contributed in turn to delays in investments reaching the communities and a loss of confidence of the population . (ii) Delays in the allocation of Rural Development Fund financing, which was made available late each year that in turn delayed the signing of contracts . (iii) Slow procurement particularly at the local level, despite elaboration of the Procedures Manual and efforts to provide training. Part of the problem lay in the difficulties of coordinating many small procurements at the provincial level and more intensive and targeted training at central and local level might have eased this constraint . (iv) Reduced commitment and varied staff availability to the project that created bottlenecks . Although there was initially strong commitment to the Project in most project units at central and provincial level this commitment diminished as the focus of decision-takers moved away from the integrated rural development concept in part as a result of the new Government agricultural strategy . Staff availability for implementation varied, with most units understaffed and frequent changes of personnel, particularly in the central project management unit . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 16% 100% ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Original Project Outcome : Unsatisfactory . Owing to major shortcomings in the efficacy of achieving the original project objective and to a lesser extent in efficiency, pre-restructuring development outcome is rated Unsatisfactory The relevance of the objective and its design are rated substantial . but the efficacy of achieving the original objective is rated negligible due to lack of outcomes on the achievement of the project objective of improving the livelihood of rural populations in priority rainfed areas. Efficiency is rated modest due to lack of an economic rate of return calculation as well as important delays and cost increases during implementation . Revised Project Outcome : Moderately Unsatisfactory . There were significant shortcomings in the relevance and achievement of the revised objectives and efficiency of the project . Relevance of objectives was rated modest due to the revised objective not reflecting the new political and institutional priorities of the country, but relevance of design was substantial due to a clear link between the project components and the revised objective . Achievement of the first sub-objective- promote the development of rainfed based agricultural farming systems - was rated modest due to limited evidence on the achievement of the objective; achievement of the second sub -objective, improve access by local populations to basic infrastructure and socioeconomic services, was rated substantial; and the third sub -objective, adoption of an integrated and participatory rural development approach, was rated modest due to limited achievement in terms of participatory approach to include communities in planning and decision making and no evidence in terms of an overall integrated approach for rural development . Efficiency was rated modest due to lack of an economic rate of return calculation as well as important delays and cost increases during implementation . Overall Project Outcome : Unsatisfactory . Overall, the project exhibited major shortcomings . The combined outcome rating of the restructured project is weighted according to the proportion of the loan that was disbursed before and after the restructuring . As the original project had disbursed US$ 11.16 million, or 55.9 %, the combined outcome rating is unsatisfactory (i.e. rating for unsatisfactory is 2 and moderately unsatisfactory is 3; if we take a weighted average of the two based on disbursements before and after restructuring, the result is (0.559x2)+(0.441x3)=2.441 which corresponds to rating 2 – unsatisfactory). a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Sustainability of Project Investments on Basic Infrastructure and Income Generating Activities, Moderate . In terms of infrastructure investments, risk is assessed as moderate, since the operation and maintenance of project-financed public goods is the responsibility of the Communes as sub -project sponsors or of the pertinent line ministries. the arrangements for the maintenance of rural road investments was still unclear at project closing as they had not been overtaken by the related Ministry, or the Communes . However, in regards to the other types of investments in health, education and water sectors, the risk is less . In addition for the investments in income generating activities, the profitable projects are likely to have allocations for operation and maintenance . For several failed or at-risk sub-projects, arrangements are being made to recover the assets and to allocate them to other groups. Integrated and Participatory Rural Development Approach, Significant, The ICR notes that (p. 16) the major national programs such as Small and Medium Irrigation Systems (PMV), National Initiative for Human Development (INDH), and Commune Level Development Program (PCD) have paid insufficient attention to the Project . Combined with changes in the policy and institutional environment, the future of the Integrated Rural Development Model is at risk. The ICR also argues that (p. 11), the employment of this approach will depend on how pro -poor agricultural development will fit within the new context provided by the other programs and the role of the Ministry of Agriculture in the rural development agenda . Institutional Capacity, Significant . There is a significant risk for the sustainability of associations and cooperatives at the local level as they need further institutional support in order to be sustainable . Also, risk of lack of sustainability of the capacity created by the project for the public services at the central, provincial and local levels is also significant, as the project teams created with the integrated and participatory rural development approach had already been disbanded after the project was closed . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The project had a long preparation period, during which background analysis was carried out that included technical input from the FAO and extensive engagement of Moroccan counterparts . Project design incorporated lessons from both global and Moroccan experience with participatory rural development . The ICR specifically notes that (p.4) there was solid analysis of the technical, economic, social and fiduciary aspects of project design . Key risks were identified at appraisal and mitigation measures prepared to address them, with the exception of the institutional risks. The ICR also notes that (p. 5) appropriate mitigation measures were identified for institutional risk but they were only partly implemented . Despite these positive aspects of quality at entry there were several shortcomings in quality at entry that negatively impacted project implementation . First, the organizational arrangements for implementing the project were complex (ten different bodies would be working at national level, four in each province, and three at each local level) and depended on deconcentration (the progressive delegation of financial commitment and decision taking powers to the provincial level of line ministries ) and the Rural Development Fund to be operable . But there was only partial consensus within Government on both of these processes at the time of appraisal . There was only partial agreement within Government that the Rural Development Fund should be made permanent . Moreover, its operational modalities were unclear . Also, it was assumed that deconcentration would occur early on in the implementation period. This did not happen and implementation delays occurred . Second, the project lacked a simplified methodology for participatory diagnosis and planning, which adversely affected the timely preparation of Communal Investment Plans and delivery of investments . Finally, there were weaknesses in the indicators originally incorporated in the project ’s M&E system (See Section 10). at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: Supervision was carried out jointly with the Bank's Moroccan partners and was characterized by the ICR as problem solving oriented (p. 18) .The ICR notes that Bank missions were active in ensuring that implementation took place in a timely manner by making sure organizational structures were set up, capacity building was carried out, and initial contracts were awarded . The Bank team reportedly followed up implementation closely and was proactive at identifying and proposing solutions to implementation issues . A fiduciary team in the Country Office provided support for procurement and financial management . Supervision missions were conducted on a regular basis with frequent site visits and were fielded by an appropriate mix of skills. However, there were significant shortcomings with the Bank supervision : (i) The key shortcoming in Bank supervision was at the strategic policy level where an opportunity was lost to address the growing separation of the Project from the strategic context in which it was operating . The ICR notes that (p. 18) once it was clear that the Ministry of Agriculture had moved away from integrated rural development towards a new emphasis on commercial agriculture and that the initiative on participatory planning and community driven development was transferred to the National Initiative for Human Development, the Bank ’s Country and Sector Department management missed an opportunity to engage in a fresh dialogue with Government on a joint pro -poor integrated and participatory rural development strategy and agriculture ’s role within that strategy, which could have allowed for the project to be restructured within this new strategic context . The ICR goes on to argue that the failure to do so was a missed opportunity to turn what was a problem at the level of the Project into an opportunity at the level of the nation. The Bank's technical staff had prepared a paper on how to adapt the Project to the new context of the National Initiative for Human Development and this was shared with the government during supervision but the recommendations were not taken up . The strategic relevance of the project was further weakened when a new agricultural strategy went into affected in 2008. Adapting the project to the new strategic context required a higher level of dialogue beyond technical analysis, which was not carried out . Instead the Bank engaged in separate dialogues and financial support to the 2008 Agricultural Strategy and to the National Initiative for Human Development. (ii) There was a lack of environmental specialists on the team . As a result supervision missions did not pick up on the limited familiarity of stakeholders with the Environmental Assessment requirements and missed an opportunity to suggest mitigation measures . Therefore Bank guidance and follow up on the safeguards management and mitigation was not adequately carried out . The study that was done on environmental impacts was conducted too late, when it was impossible to follow up on mitigation measures . (iii) The ICR also notes that supervision teams could have played a more active role in ensuring that M&E was carried out as planned and followed disbursement requests more closely to ensure a lower rate of disbursement request rejections . (iv) The institutional structure for drought management took a long time to find a home for the proposed activities within the Ministry of Agriculture and therefore never became functional . It was only addressed through preparation delayed strategy that did not drive any specific drought management activity . Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: Overall government commitment to the project varied . The ICR noted that (p. 19), the Ministry of Agriculture's strong commitment to the project during the preparation declined during the implementation period . Several aspects of government performance negatively impacted project implementation and its ultimate achievements: (i) Slow government procedures delayed project effectiveness . (ii) Although the process of deconcentation and the Rural Development Fund were critical to making the organizational arrangements function, there was lack of consensus within government on both . The lack of deconcentration slowed project implementation considerably. (iii) The shift in government policy and strategic direction of the sector away from agriculture-led integrated rural development as a mainstream policy in Morocco affected implementation and diminished the relevance of the Project as a pioneer of new national approach to participatory development . The ICR notes that although there was some attempt on the part of government to make working -level adaptation to the new national program (a parallel initiative of participatory programming, INDH ), there was no effort by government to restructure the project to ensure that it remained relevant within this new strategic context . (iv) Another shortcoming was related to the drought management initiative of the project . As explained in Section 8b, it was the responsibility of the Government to operationalize the drought management activity, which did not happen. For these reasons Government performance is rated Moderately Unsatisfactory . Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: Project oversight and coordination would be carried out largely through existing institutions at the central and local levels. There were several institutions for project coordination at different levels . The overall project monitoring, and implementation of the activities was the responsibility of MADR . A Project Management Unit was established under the Department of Land Development in MADR . At the provincial level, Provincial Directorates of Agriculture facilitated project implementation and monitoring . The ICR notes that (p. 19) implementing agency staff demonstrated a high degree of commitment and effort, but there were several shortcomings : (i) The staff were constrained in general by a lack of support from decision makers. In some cases lack of support from the Governor at provincial level negatively affected implementation (ii) Project units were understaffed and had high turnover with adverse impacts on project implementation . The loss of key staff in the project team resulted in a loss of institutional memory, capacity and weakened ownership of the project. (iii) Procedures were not regularly updated . For example an adequate Procedures Manual was prepared early in the Project but it was never updated . (iv) Financial reporting and auditing was moderately unsatisfactory. The staff struggled with cumbersome procurement, financial management and reporting procedures, resulting in slow implementation and reporting . (v) There was a lack of regular follow up on safeguards management by the central management unit and almost no environmental M&E was carried out (ICR p. 9). Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: Project design specified monitoring of physical and financial aspects of Project activities to be carried out at the local level, involving communities in the process, as well as M&E of outputs and impacts to be conducted by the central Project unit. Many of the outcome and impact indicators were more output in nature (e.g. number of committees established) and some were only vaguely attributable to the project and hard to measure (e.g. incomes have increased and inter-annual variations are reduced ). No indicators were proposed that would have given an idea of progress or results on the strengthening of the implementation capacity component . More precise indicators were established after the midterm review when the project was scaled back . b. M&E Implementation: The original M&E system was designed by consultants but was not implemented and the proposed baseline surveys were never conducted . Instead monitoring was mainly based on administrative reporting . No data on impacts were collected. The ICR noted that (p. 9), the data on outputs or outcomes was not very reliable . There was some qualitative evaluation based on supervision missions and information included in the PICs, on administrative reports, on field interviews and on discussions with the central Project Unit . c. M&E Utilization: The ICR is candid that there is no evidence that M&E served as a management tool, supporting decision -taking and thereby improving efficiency, as had been proposed in the PAD . The proposed participatory M&E, a key element in the participatory approach, could have generated sustained knowledge and engagement of the community throughout the Project period, but was never implemented . M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: The Project was classed as Category B, and triggered OP 4.01 Environmental Assessment (EA) and OP 4.09 Pest Management. The ICR noted that (p. 9), for OP/BP 4.01, the EA was completed and disclosed in March 2002 and the Environmental Management Plan (EMP) Executive Summary was attached in Annex 12 of the PAD. The EMP identified potential positive and negative impacts and a broadly defined set of mitigation measures and also included a Pesticide Management Plan to meet the requirements under OP . 4.09 for chemical land clearing activities . A training program on environmental safeguards and an environmental monitoring checklist was planned . The ICR reported that (p. 9) the safeguard management performance and the compliance with the Ops listed above were rated satisfactory, with the exception of the last ISR, in which the safeguard management performance and the compliance with OP 4.01 were downgraded to moderately satisfactory, justified by the lack of regular follow -up from the Central Project Unit. Notwithstanding this overall satisfactory assessment, there were several shortcomings in the implementation of safeguards policies. The ICR reported (p. 9) that the electronic version of the EA was lost due to changes in the Central Project Unit. Consequently the environmental monitoring checklist was never produced, and the Project was carried out with virtually no environmental M&E . Issues related to by-products of olive oil production that were detected during implementation of subprojects were only partially addressed, and a recommended study was never carried out. No action was taken to address issues reported on drinking water quality in schools and on the environmental impact of bush soil clearing . Nonetheless, the measures specified in the EMP were addressed in the implementation of specific investments (e.g. inclusion of environmental requirements in contracts ). A number of environmental issues were picked up during supervision . Issues related to rural roads and related infrastructure were generally satisfactorily dealt with except in the case of one bridge . The ICR reported that (p. 10), due to limited information on environmental impacts, the Project conducted a study in 2011 to assess the baseline scenario and the environmental impacts of the Project, as well as to produce a guide on how to factor environmental issues into the Project . However, it was hard to understand what would be the use of this study as it was carried out very late, just before project closing . Although the study concluded that the Project had limited negative environmental impacts, with some recommended mitigation measures, it was unlikely that there would be action on these measures . b. Fiduciary Compliance: Procurement : The ICR reported that (p. 10), procurement risk was rated low at appraisal, and a procurement plan for the first two years of the project was prepared and periodically updated during implementation . Nevertheless, there were several weaknesses during implementation : (i) delays in procurement due to late notification of the budget, particularly the Rural Development Fund (FDR); (ii) inexperience in handling procurement at the provincial level; and (iii) lack of competition amongst suppliers of goods and services at the local level . Some training on procurement was provided by the Bank, as supervision missions found training needs particularly at the local level, but it was not clear if this training was sufficient. The ICR noted that post procurement reviews of a sample of contracts revealed significant delays in the procurement process as well as difficulties of obtaining sufficient responsive bids . This led to a rating of moderately unsatisfactory during the last two supervision missions . The ICR also reported that (p. 8) the problem was partially due to difficulties of coordinating many small procurements at the provincial level and concluded that more intensive and targeted training at central and local levels might have eased this constraint . Financial Management : The ICR reported that (p. 10) the Bank’s fiduciary team's assessment of the performance of financial management, financial reporting and auditing was moderately unsatisfactory in December 2011, despite the fact that at appraisal, the financial management arrangements had been assessed as satisfactory, with a ‘medium’ level of risk. The ICR did not provide the reasons for this inconsistency . Also, it was reported that during implementation, risks were managed through detailing of procedures in the Procedures Manual, capacity building, and initial close supervision. The shortcomings were: (i) very slow opening of budget credits, particularly for the FDR which was often not available until the end of the third quarter of the fiscal year; (ii) slow pace of actual spending, due to delays in the procurement process; (iii) sometimes erratic application of the Procedures Manual; and (iv) delayed and sometimes unsatisfactory audit reports . Attempts were made to partly resolve these problems through active follow up by the staff of the Central Project Unit, but performance in the provinces remained weak . As a consequence, the problems could not be resolved . c. Unintended Impacts (positive or negative): Gender Women's vulnerability due to lower education level, and limited land ownership, as well as limited decision Gender: making power within the household, were described in the PAD . Several measures were put in place to help women ’ s voices to be heard during the diagnostic and decision -taking processes. However, during implementation, actual practice appears to have varied or been unclear . For example the ICR concluded that (p. 16),women received direct benefit from investments financed under support to local initiatives, particularly establishment of women ’s cooperatives and associations and the establishment of a number of revenue earning projects, for the most part in agricultural and pastoral development, such as distribution of animals at the household level . Despite this positive assessment, there are questions over sustainability of some investments, due to : (i) the weakness of some of the women’s associations and of the capacity -building mechanisms that should support them; and (ii) the poor design and feasibility of some of the revenue -earning projects. Furthermore, at the time of the ICR mission, it was hard to find a woman who attested to having taken part in any of the diagnostic sessions . Also it was impossible to tell whether gender-specific investments had a higher or lower chance of being selected . d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Unsatisfactory Efficacy of the original objective is rated Unsatisfactory negligible due to lack of evidence on the achievement of the objective and Efficiency is rated modest due to lack of ERR analysis and implementation delays and high costs, these lead to an unsatisfactory Outcome rating for the original project, and although the revised project rating is the same as the ICR -moderately unsatisfactory- the weighted average of the two ratings is unsatisfactory. Risk to Development Significant Significant Outcome : Bank Performance : Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR presents multiple lessons . The most relevant are adapted below : • A participatory planning process can only serve to empower communities if it builds accessible knowledge and transparent decision -making . This requires that key information is made accessible to all participants in a readily understandable format and decisions are made through a transparent and open local governance mechanism. In the case of this project, the bulk of information collected was hidden in reports in a language that the Commune could not understand and was inaccessible to the population at large . This led to a ‘distancing’ of people from the communal plan and from the participatory process in general . The decision making process on the sub-projects to be financed was also not transparent . • An unintegrated rural development approach may not be the best approach for provision of multiple basic services as it is difficult to coordinate between different agencies . However, it may be better suited for implementing activities complementary with agriculture . The project experience showed that it may be better to leave implementation of basic services to the agencies concerned in direct concert with the communes rather than try to deliver them as an integrated package, which led during the Project to considerable implementation challenges. For basic services (education, health, water supply ), the value added of the integrated approach compared to the sectoral approach is questionable, especially considering the difficulties related to coordination between different ministries. There may, however, be merit in integrated implementation of complementary activities having an impact or synergies with agricultural production such as : irrigation, flood protection, pastoral development, rural roads, marketing, agro -industry. • Dependence only on consultants makes the participatory process longer and more expensive and it also reduces the capacity building impact within public services, and undermines the confidence of the population in the process and the consequent decisions . The use of consultants might be best limited to areas requiring specialist expertise such as : supervision of the diagnostic phase; formation of a strategic vision; or in-depth technical studies. • A "rolling " program that is refreshed annually rather than a "once for all " approach may work better for communal investments . A rolling program refreshed annually may generate more participation and ownership than a ‘once for all’ program. The general opinion of the population consulted during the ICR mission was that a ‘small but regular’ approach was more appropriate, perhaps because it made the participatory programming process into a permanent ‘rolling’ one, allowing the participatory habit to grow, mistakes to be corrected in the subsequent years. In the ‘once for all’ approach of the Project, many people seeing an investment delivered in 2011 could not make the connection with the diagnostic in which they had participated in 2005. • When the strategic and policy context involves several sectors and is evolving, there needs to be continuous review of the implications for operations . The chances of the Project having its intended impacts were reduced when the strategic context in which it was conceived changed . The Project might have contributed to identifying and testing options in order to be able to fit pro -poor agricultural investment within a participatory planning framework. Therefore, the Government and Bank should maintain continuous dialogue and strive to re-engineer operations to test new approaches within new strategic contexts . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is comprehensive and the lessons are formulated well . However, there are several important weaknesses with the Quality of the ICR: (i) The ICR presents no discussion of the project's achievements towards the original objectives and targets, whereas the outcomes should be assessed against both the original and the revised objectives and the overall Outcome composed of the weighted average of the original and revised outcomes . (ii) Annex 1 (a) of the ICR erroneously reports the breakdown of actual (or latest estimate) project costs by component, showing instead figures that are identical to the component costs estimated at appraisal . (iii) The financing table (b) of Annex 1 only records the World Bank portion of actual costs; the Government and beneficiary contributions are shown only as not applicable . Consequently there is a discrepancy between the total project costs in table (a) of Annex 1 and total financing presented in table (b). The text of the ICR does not shed light on this . (iv) The ICR does not provide a detailed discussion of how the components were scaled back, and which activity was not implemented . a.Quality of ICR Rating : Unsatisfactory