ICRR 13340 Report Number : ICRR13340 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 05/04/2010 PROJ ID : P043412 Appraisal Actual Project Name : Basic Education US$M ): Project Costs (US$M): 150.55 151.46 Reform Support Program Country : Morocco Loan/ US$M): Loan /Credit (US$M): 80.00 80.91 Sector Board : ED US$M): Cofinancing (US$M ): Sector (s): Secondary education (52%) Sub-national government administration (18%) Primary education (16%) Central government administration (10%) Pre-primary education (4%) Theme (s): Education for all (33% - P) Participation and civic engagement (17% - S) Gender (17% - S) Administrative and civil service reform (17% - S) Rural services and infrastructure (16% - S) L/C Number : L7273 Board Approval Date : 02/08/2005 Partners involved : Closing Date : 06/30/2008 06/30/2009 Evaluator : Panel Reviewer : Group Manager : Group : Alemayehu A. Ambel Susan A. Stout IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: According to the PAD, the objective of the project (PDO) was “to support Government efforts to provide basic education of improving quality to all children aged six to fourteen in a financially sustainable manner �. To achieve this objective, the project -- the Basic Education Support Program (Programme d’appui à la réforme du système éducatif marocaine- PARSEM), using a Sector-Wide Approach (SWAp) -- would contribute to the establishment of practices, mechanisms, and approaches to improve the education system on an ongoing basis . The Loan Agreement states the objective differently : “ to assist the Borrower in the implementation of its Program for the development of the education sector in the Project Area �. This Review assesses the achievement of objectives against the PAD statement since this is more specific about the intended outcomes. The PAD PDO statement is disaggregated into three parts : (i) increasing access to basic education, (ii) increasing quality of education, and (iii) ensuring financial sustainability (see Section 4 below). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): 1. Generalize Basic Education (Cost Estimate at Appraisal: US$95.87 million; Actual US$95.87 million): The objective was to make basic education available to most school -age children by 2008 through increasing the provision of schooling in a cost -effective manner by using new construction norms . Social services (boarding facilities and school transport ) were to be provided to under-served students by promoting partnerships with local stakeholders. Demand would also be strengthened through the promotion of preschool education . Overall, the approach included: (i) promoting pre-primary education through partnership; (ii) achieving full primary education enrollment (grades 1-6); (iii) increasing equitable access to middle school education (grades 7-9); and (iv) mobilizing partnerships towards the expansion of basic education . 2. Improve the quality of education (Cost Estimate at Appraisal: US$42.53 million; Actual:US$42.53 million ): This component sought to improve student learning and reduce dropout and repetition rates through : (i) improving evaluation systems; (ii) improving support services for teachers; (iii) promoting the quality of education at the school level; and (iv) promoting research to improve the quality of education . 3. Build institutional capacity (Cost Estimate at Appraisal: US$12.16 million; Actual US$12.16 million ): This component aimed at building capacity at various levels (central, regional, provincial, and local ) through: (i) supporting the implementation of public administration reform in the education sector; (ii) reinforcing human resource management; (iii) reinforcing finance and administrative management capacity; (iv) improving communication and partnership with education stakeholders; and (v) reinforcing change management in the education sector . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Bank support of 60.4 million Euros (US$80 million) was in the form of an IBRD loan. There is a difference of US$ 0.91 million between the appraisal and the actual estimates due to exchange rate changes . The overall cost of PARSEM is US$275.47 million. This operation is the first of two consecutive loans that the Bank expects to provide in support of the PARSEM program . The closing date was extended twice from June 30, 2008 to December 31, 2008 and then to June 30, 2009, to allow more time for the Government to complete the implementation of policy measures identified in the project . Project disbursements were slow, due to delays in project implementation as well as in data collection and reporting. Beyond the first initial advance to a Special Account in April, 2005, there were a total of three disbursements every two years, in June 2005, June 2007 and June, 2009. 3. Relevance of Objectives & Design: Relevance of objectives (Substantial ). The project directly responds to the Government ’s priorities in the education sector that included, among others, access to quality education . The prevailing Country Partnership Strategy (CPS) [Report No. 50316-A], that articulated program areas for FY 10-13, identifies access, quality, and institutional capacity as key outcomes in the education sector . Relevance of design (Modest ). The components and activities of the project are in general relevant to achieve access and quality objectives . In addition, in a country like Morocco, where progress in particular sub -sectors has reached a plateau (in this case primary education ), increasing access requires specific interventions that address specific bottlenecks.The program design thus included provision of transport services and boarding facilities to under-served groups of children, and plans to monitor service provision at disaggregated levels (by gender and urban -rural location) . In addition, the performance monitoring and evaluation system of the project included output and outcome indicators on access and quality . However, the output and outcome indicators of education quality do not include measures of student performance . Similarly, the design did not include specific arrangements for monitoring financial sustainability, nor did it adequately focus on specifying the institutional changes that would be required to ensure smooth flow of funds and compliance with the Bank's fiduciary and safeguard policies triggered through the project (OP 4.01 and OP 4.12) to decentralized management units, despite a sound analysis of the risks to effective implementation that failure to improve capacity at these levels would entail . 4. Achievement of Objectives (Efficacy): The assessment of the achievement of objectives is summarized under the four aspects of the development objective of PARSEM. These are access, quality, and financial sustainability . 1. Increasing access to basic education (Substantial ) Outputs: The program provided social services (transport and boarding facilities ) to 78,732 schoolchildren substantially surpassing the target of 30,000. Also, by renovating existing classrooms and construction of new ones the program increased the number of rural communities with access to middle schools from 500 to 645. However, there is no information in the ICR on the number of schools and classrooms either renovated or built . Outcomes: The net primary enrollment rate increased from 92 percent (2004) to 93.5 percent (2008). Similarly, during the same period, net middle school enrollment rate increased from 32 percent to 47.4 percent. In both cases of primary and middle schools, increases in enrollments were for both boys and girls . 2. Improving quality of education (Modest ) Outputs: A new in-service training framework was developed and a pre -service training of bi-disciplinary teachers was piloted. In addition, the program promoted school based projects in 90 percent of the schools. The program also successfully supported the first national evaluation of student learning in mathematics, science, Arabic and French, which revealed considerable differences between rural and urban areas and between public and private schools (ICR p. 9). Outcomes: Overall, there was a modest improvement of quality at the national level between 2004 and 2008: End of cycle completion rate at the primary school level increased from 57.8 to 63.1 percent; End of cycle completion rate at the middle school level increased from 43.7 to 53.5 percent; Retention rate at the primary school level increased from 65.8 to 75.8 percent; Repetition rate at the primary school level declined from 13.6 to 12.3 percent; Repetition rate at the middle school level declined from 18.4 to 15.2 percent. 3. Ensuring financial sustainability (Modest ) Outputs: The program upgraded budget and financial management capacity . Also, a new results-based budgetary approach using contracts was developed, and management statistical information systems (MSIS) were reinforced. The program also promoted decentralization in the education sector by improving the capacity of Regional Education and Training Academies (Académies régionales d'éducation et de formation - AREF). More responsibility and accountability were delegated to AREFs . Accordingly, in 2009, AREFs had “a much stronger capacity to independently develop and manage their regional education sector programs � (ICR, p. 9). Outcomes: Financial sustainability was not explicitly defined and discussed in the results framework section of the PAD, and the ICR does not assess the financial sustainability objective . Also, there is no information on cost effectiveness measures such as increasing the number of hours taught by polyvalent teachers . Overall efficacy: Modest 5. Efficiency (not applicable to DPLs): Although, as noted, there is little information in the ICR on cost -effective approaches of delivering quality education . some efficiency enhancing actions can be attributed to the project . These include, “a more efficient use of personnel with the use of polyvalent teachers; more balanced allocation of non -salary expenditures between social and pedagogical expenditures; closer monitoring of teacher / student ratios and their impact on expenditures; and more efficient use of school infrastructure � (ICR p.9). IEG rates efficiency as substantial . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Overall outcome is rated Moderately Satisfactory based on substantial relevance of objectives and efficiency, and modest design relevance and efficacy. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The Government showed strong commitment to further reforms in the education sector with a long -term vision focused on improving the quality of education, and the political will to continue this remains high at all levels . Morocco’s economy is expected to maintain a healthy growth in the coming years . This would create a favorable environment to ensure financial sustainability . In addition, the Bank’s proposed continued engagement in PARSEM will be helpful to further strengthen the impact of reforms undertaken by this program a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Ensuring Quality-at-Entry (Moderately Unsatisfactory ). Output and outcome targets were based on baseline data that included disaggregated information by gender and place of residence . The design linked disbursement to monitoring indicators. The section in the PAD on the quality of education was extensive with detailed information on Morocco’s participation in the Trends in International Math and Science Study (TIMSS) and International Reading Literacy Study (PIRLS). As noted in Section 3, however, although the PAD discussed learning outcomes and measures of student performance, these were not included as outcome indicators of education quality in the monitoring and evaluation plan . In addition, there was no definition of financial sustainability nor any consideration of relevant institutional changes . Moreover, quality at entry was diminished by lack of participation of the AREFs in the design of action plans and implementation procedures, and by over-ambitious program goals. The AREFs lacked the capacity to comply with the Bank's procurement and fiduciary procedures, and no adequate provision was made to build up these capabilities . Issues which arose during implementation relating to fiduciary and safeguards policies in a SWAp operation (see Section 11), might have been more easily resolved had they been raised at the preparation stage . Quality of supervision (Moderately Unsatisfactory ). The Bank conducted regular supervision missions and provided guidance and support to the Borrower . Annual reviews were also carried out regularly . However, despite focusing heavily on fiduciary, safeguards and administrative matters to the partial neglect of technical and policy issues (ICR p. 12), the Bank failed to prevent non -compliance with the Bank's safeguards policies with regard to land acquisition (ICR, p. 6 and Section 11 of this Review). This was due to different interpretations of the stipulations of the legal documents and to poor monitoring by the Government which the supervision missions did not address adequately . There is no evidence that the possibility of loan cancelation or suspension of disbursements was invoked following these failures to comply . The ICR reports only that the overall pace of implementation was negatively affected by these unresolved issues (pp. 4&11). at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government performance (Moderately Unsatisfactory ). The Government was the main financier of PARSEM and its commitment to the program was, and according to interviews with the Task Team, remains strong . The Government was proactive in accelerating reforms in the education sector . It assessed the implementation of the education Charter and also developed a new education action plan . However, Government's disagreements with the Bank concerning interpretation of the legal documents regarding safeguard and fiduciary policy issues, and a failure to resolve them in a timely manner, resulting in non -compliance, slowed implementation of the project . Implementing Agency performance (Moderately Satisfactory ). The National Education Department (DEN) is charged with overall implementation of PARSEM . However, day-to-day responsibility has been largely devolved to newly decentralized structures (the AREFs), which have only slowly acquired the capacity to understand and apply the Bank's fiduciary, safeguards and procurement policies and procedures . Implementation was also delayed by slow data collection and tardy reporting by all implementing agencies . a. Government Performance :Moderately Unsatisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: M&E Design (Modest ). In general, standard outcome and performance indicators were identified to monitor and assess the achievement of objectives . Some baseline indicators are disagregated by level of education (preschool, primary and middle), place of residence (urban –rural), and gender. The indicators are relevant to assess some aspects of the project's objectives . The project had also baseline and target values for most of the output and outcome indicators. However, the monitoring plan did not include indicators to measure quality of education or financial sustainability. M&E Implementation (Substantial ). The Directorate of Strategy, Statistics and Planning (DSSP) built capacity to collect, process and analyze data . Data on most of the project’s output and outcome indicators in Section 4 were collected regularly (ICR p.6), although slow data collection and reporting delayed disbursement (ICR, pp.6&7). The ICR does not have dissaggregated outcome data by place of residence . M&E Utilization (Modes t). The section of the ICR that discusses M&E Design, Implementation and Utilization (ICR p. 6), does not have evidence on M&E utilization . Elsewhere, it is noted that lessons learned from PARSEM were integrated in the Government’s Education Emergency plan, which is a strategic sector document that was put in place to accelerate the implementation of the Education and Training Charter (ICR p. 11). However, no concrete examples of such lessons are mentioned . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): An issue arose during implementation concerning whether Bank or national policies and procedures should be applied with regard to safeguards and fiduciary compliance . The Government maintained that, in a SWAp, national rather than Bank policies would obtain, while the Bank team maintained this would be the case only where national policies ensured an equivalent level of safeguards compliance and fiduciary risk as that expected using Bank procedures. In all other cases, traditional investment project rules would apply . In the end, the Bank's view prevailed, but this remained a sensitive issue throughout the life of the project . In part because of this ongoing sensitivity, there was lack of compliance with Bank safeguard policies on land acquisition (ICR, p. 6). The Bank and the Government differed in their interpretation of a clause in the Loan Agreement, stipulating that all project -financed construction activity should take place on publicly owned land and involve no resettlement. The Government interpreted this to mean that they could purchase land and build on it as long as no physical resettlement was required . This resulted in the Government acquiring new land and accumulating payments arrears to the owners, which is a common situation in Morocco . The Government had, moreover, agreed during preparation to create a data base for monitoring land acquisition issues, but this was not done . AREFs lacked the capacity to ensure compliance with the Bank's fiduciary and procurement requirements and insufficient training was provided under the project to build it . Weaknesses in budget management and procurement were manifest. There is also a need to develop an adequate automated data system to improve financial management. According to the ICR, audits were carried out annually by the Inspection g énérale des finances (IGF) and there were "no major irregularities during 2005-07, the three years of Bank financing of PARSEM ." The ICR further notes that the 2008 audit of PARSEM activities, mainly financed by the Government, has identified a number of financial management issues, including some expenditures that were not properly classified, and that it would be important to address these irregularities and to give proper attention to financial management in future operations . The ICR does not provide further detail. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately There were significant weaknesses in Satisfactory Unsatisfactory design, including a failure to ensure fiduciary management capacity in the implementing agencies and to address the issue of how safeguards and fiduciary policies and procedures should be handled in a SWAp. Similarly, supervision missions failed to monitor adequately compliance with the land acquisition safeguard or to address non-compiance firmly with the Borrower when it became evident. Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: From the ICR (with some modification of the ICR wording): The roles and responsibilities of all parties in a SWAp, particularly concerning the use of country safeguards fiduciary policies and procedures need to be clearly specified during preparation to avoid confusion and misinterpretation both by the Bank and the Borrower teams . The effective implementation of some reforms requires time, especially in the human development sectors . It is therefore important to take into account the time required to build the capacity of executing agencies when preparing the reform implementation plan . IEG would add: Increasing access to quality education by underserved communities needs to go beyond provision of standard school infrastructure. For example, in this project, the provision of transport and boarding facilities to these groups of children was vital. Project objectives need to be matched by relevant output and outcome indicators for effective monitoring and evaluation. For example, in this project, improvements in education quality could not be fully evaluated due to lack of adequate evidence on appropriate learning outcome indicators 14. Assessment Recommended? Yes No Why? A further assessment of this project would provide more insight on multi -component education projects implemented in the context of a SWAp . In particular, it would shed light on issues which may arise in developing and using country safeguards and fiduciary procedures while also complying with the exigencies of Bank policies . A further assessment may also help identify steps the Bank might take to assess better and promote Borrower readiness to pursue programmatic support for sector strategies. 15. Comments on Quality of ICR: ICR quality is satisfactory, although only barely so . The ICR presents a considerable amount of useful information and analysis, and is candid in, for example, discussing the failure to comply with safeguards and fiduciary policies . Nonetheless, it has several drawbacks . Rather than assessing program objectives and outcomes, it evaluates components and outputs. There is lack of discussion on the financial sustainability objective of the project, and incomplete information even on some program outputs, for example, it is not clear whether the school building and renovation work reached its targets or not since no numbers are provided . It would also have been useful to know the proportion of project cost represented by the land acquisitions carried out in non -compliance with Bank safeguards policies. a.Quality of ICR Rating : Satisfactory