Document of The World Bank Report No: 27452 IMPLEMENTATION COMPLETION REPORT (CPL-39510 SCL-3951A SCPD-3951S) ON A LOAN IN THE AMOUNT OF US$37 MILLION TO THE DOMINICAN REPUBLIC FOR A DO- BASIC EDUCATION December 29, 2003 Human Development Sector Management Unit Caribbean Country Management Unit Latin America and the Caribbean Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective ) Currency Unit = Dominican Peso (DMP) 38 DMP (December 11, 2003, 2003) = US$ 1.00 US$ 1.00 = 0.026316 DMP FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS CAS Country Asssistance Strategy GDP Gross Domestic Product IBRD International Bank for Reconstruction and Development ICR Implementation Completion Report IDB Inter-American Development Bank IMF International Monetary Fund LAC Latin America and the Caribbean LCSHD Latin American and the Caribbean Sector of Human Development MIS Management Information System OCI Office for International Cooperation PA Parent Associations PAD Project Appraisal Document PCU Project Coordination Unit POAs Annual Operation Plan SEE State Secretariat of Education SOE Secretariat of Education ROR Rate of Return USAID United States Aid for International Development Vice President: David De Ferranti Country Director Caroline D. Anstey Sector Director Ana Maria Arriagada Task Team Leader/Task Manager: Sam Carlson DOMINICAN REPUBLIC DO- BASIC EDUCATION CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 5 5. Major Factors Affecting Implementation and Outcome 13 6. Sustainability 16 7. Bank and Borrower Performance 17 8. Lessons Learned 20 9. Partner Comments 22 10. Additional Information 27 Annex 1. Key Performance Indicators/Log Frame Matrix 28 Annex 2. Project Costs and Financing 29 Annex 3. Economic Costs and Benefits 32 Annex 4. Bank Inputs 33 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 35 Annex 6. Ratings of Bank and Borrower Performance 36 Annex 7. List of Supporting Documents 37 Project ID: P035494 Project Name: DO- BASIC EDUCATION Team Leader: Samuel C. Carlson TL Unit: LCSHE ICR Type: Core ICR Report Date: December 29, 2003 1. Project Data Name: DO- BASIC EDUCATION L/C/TF Number: CPL-39510; SCL-3951A; SCPD-3951S Country/Department: DOMINICAN REPUBLIC Region: Latin America and the Caribbean Region Sector/subsector: Primary education (70%); Central government administration (17%); Tertiary education (9%); Health (2%); Other social services (2%) Theme: Education for all (P); Civic engagement, participation and community driven development (P); Nutrition and food security (P); Rural services and infrastructure (S); Access to urban services for the poor (S) KEY DATES Original Revised/Actual PCD: 09/13/1994 Effective: 12/31/1996 10/24/1997 Appraisal: 05/15/1995 MTR: 02/28/1999 03/20/2000 Approval: 11/09/1995 Closing: 06/30/2001 06/30/2003 Borrower/Implementing Agency: Government of Dominican Republic/Secretary of Education; Government of Dominican Republic/Arts and Culture Other Partners: STAFF Current At Appraisal Vice President: David de Ferranti Shahid Burki Country Director: Caroline D. Anstey Kenneth Lay Sector Director: Ana-Maria Arriagada Kae Woo Lee Team Leader at ICR: Sam Carlson ICR Primary Author: Sam Carlson 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: S Sustainability: L Institutional Development Impact: M Bank Performance: S Borrower Performance: S QAG (if available) ICR Quality at Entry: S Project at Risk at Any Time: Yes 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The original objectives were to: (a) improve the quality of basic education; (b) increase enrollment and especially completion rates, with priority directed to children from low income families; and (c) strengthen technical and resource management capabilities for basic education. These objectives were clear, realistic and important for the education sector and the Bank's Country Assistance Strategy (CAS) in the Dominican Republic. The objectives were consistent with the Government's 10-Year Education Plan (1990-2000), particularly with the Consolidation Phase of this Plan (1996-2000). They were responsive to borrower circumstances and development priorities. With respect to the CAS (May 1995), increased educational attainment (improved quality, coverage and internal efficiency) is explicitly stated as an objective. Project design benefited from lessons learned during implementation of the first phase of the Ten-Year Plan, as well as from mid-term review of the Primary Education Development Project conducted by the IBRD in January 1995. These lessons included the need to: increase the role of project stakeholders; consolidate 10-Year Plan implementation (including of IBRD and IDB-financed projects) within the Secretariat of Education (as opposed to within project coordination units); and ensure adequate government financing for basic education. However, the project was somewhat complex, with a total of twelve sub-components. While the project was coordinated by a single Project Coordination Unit, it drew on Secretariat of Education (SEE) staff from dozens of departments and units for decision-making and implementation management. There was a single institution involved, with a relatively competent implementing agency, and only one major co-financier (IDB). Unfortunately, both the implementation and co-financing arrangements experienced problems, as will be seen later. 3.2 Revised Objective: None. 3.3 Original Components: The educational development component (US$82.8 million, including contingencies) had seven sub-components: 1. Revision of the national curriculum for basic education (US$3.4 million); 2. Institutionalization of the provision of updated educational materials, including texts and workbooks for all public school students, sales to private schools and provision of complementary materials and supplies for low-income students (US$12.4 million); 3. Provision of training for about 12,600 teachers, school directors and supervisors in core subjects and teaching techniques (US$8.6 million); 4. Improvement of student assessment system (US$3.1 million); 5. Strengthening of the Secretariat of Education's (SOE) capacity to design and manage school nutrition programs (US$3.5 million); 6. Rehabilitation/construction of 6,800 classrooms (including 1,000 for pre-school), and preventive maintenance for 4,900 schools based on a cost-sharing arrangement between Parent Associations - 2 - (PA) and SOE (US$48.1 million); and 7. Support for a pilot program to upgrade private schools serving low-income students (US$3.7 million). Viewed in their ensemble, these seven sub-components were reasonably related to achieving the objectives of improving educational quality, enrollment and internal efficiency at the basic education level, particularly for low-income students. A holistic approach was taken here, including infrastructure, curriculum reform, teaching/learning materials, training, assessment and school nutrition, reflecting lessons learned from the Bank's long experience in the sector. Unfortunately, the design was overly optimistic with respect to the qualitative classroom-level changes in teaching and learning which would be brought about through formal, diploma-based, university-centered in-service teacher training. The management development component (US$16.7 million, including contingencies) focused on gradual de-concentration of responsibilities, training, technical assistance and equipment to regional authorities. It had five sub-components: 1. Strengthening of SEE management procedures and administrative capacity (US$3.6 million); 2. Improvement in use of management information for basic education (US$4.0 million); 3. Expansion of the roles and capacities of parents associations in decision-making to the areas of school planning, performance and monitoring (US$2.1 million); 4. Support for monitoring and evaluation of basic education (US$2.6 million); 5. Project Coordination Unit (US$4.5 million) and IDB supervision and inspection fees (US$0.5 million). These sub-components were directly linked to the project objective of improving technical and resource management capabilities for basic education. They focused on institutional capacity-building at central and decentralized levels, and sought to establish a comprehensive communications system for exchange and processing of information for improved decision-making. Equally important, the project promoted increased parental and community participation for greater accountability and provided incentives for increased local initiatives targeting school improvements through a district-level competitive funding process. Increased stakeholder participation reflects lessons learned from other projects in the Dominican Republic and around the world in the education sector. Finally, the monitoring system aimed at tracking project performance indicators for each component to ensure focus on results (including beneficiary surveys), while the evaluation system was designed to measure the project's impact. Specifically, annual evaluations were intended to assess impact and cost-effectiveness of each project component, as defined by progress on key basic education performance indicators: repetition rates; enrollment rates; completion rates; instructional time; public expenditures on education; and student achievement measured by national assessment examinations. Such comprehensive evaluation indicators are an element of good project design and the importance of focusing on the project's true impact rather than on inputs. However, it should be pointed out that no monitoring or evaluation indicators were established for assessing the project's impact on poor students. While strong in design from a technical perspective, these activities did not fully correspond to the SEE's administrative and financial management capacity at regional levels, nor did they reflect the limited buy-in among central level staff for (i) "de-concentration", (ii) promotion of private schools for low-income families, and (iii) increased community participation. Secondly, true progress in these key basic education indicators was at least partially dependent on the Government's willingness to significantly increase - 3 - recurrent spending on this sub-sector on a sustainable basis, which simply was never fully demonstrated. 3.4 Revised Components: In October 2000 the project design was amended. While established development objectives remained, the project focused remaining funds in the first component (about US$4 M) on three broad initiatives: (1) textbook printing and distribution; (2) implementation of transfers to support school-based maintenance activities; and (3) an early childhood education pilot program. Unallocated resources were assigned to these three initiatives, and the sub-component to support private schools was reduced. The sub-components for classsroom construction and rehabilitation, curriculum reform and training were not changed. This was not a major amendment, but it was significant in that it reflected the incoming Administration's focus on basic education textbook supply, and expanding access to kindergarten (age 5) which was made obligatory in the 1997 Education Law. These revised components promoted increased decentralization through community participation, increased coherence with the IDB loan, and provided an opportunity to finance an early childhood pilot program which greatly facilitated the government's readiness at entry for the Bank's new Early Childhood Education Project. At the same time, the sub-component to support private schools in poor areas was essentially dropped for lack of interest by the new Government for this activity, although efforts were undertaken at the institutional level to strengthen the SEE's knowledge and oversight of private schools. 3.5 Quality at Entry: Overall, a satisfactory performance rating is assigned for quality at entry, although this would not apply to all aspects of project design. Project objectives were consistent with the CAS, government priorities and the Bank's safeguard policies. Project components and sub-components were directly linked to achievement of project objectives, and reflected numerous lessons learned from previous education projects, with the exception of the project objective aimed at prioritizing the needs of poor children (see below). A highly satisfactory performance rating is not assigned primarily because of overly optimistic assumptions regarding (a) the Government's commitment to increasing basic education spending and maintaining project activities, including through presidential transitions, and (b) the ability of the SEE Project Coordination Unit to smoothly implement activities in the context of rigid co-financing arrangements between the World Bank and IDB. Different project timeframes, procurement procedures and institutional priorities between the two banks made it extremely difficult for the PCU to coordinate activities, especially when this required both timely Government counterpart financing and strong implementation capacity within the SOE. Ultimately, this co-financing arrangement was abandoned after the mid-term project review in favor of a parallel financing arrangement. Furthermore, while the Project Appraisal Document pays lip-service to targeting of project investments on children from poor families, no mechanism was put in place to ensure this actually happened. Of the 12 sub-commponents only one ("Sub-Component 7: Support for a pilot program to upgrade private schools serving low-income students") explicitly focuses on poor children, and this sub-component was subsequently abandoned for the most part. The design of the Development Impact Indicators also failed to include any specific information concerning targetting on poor children, leading to the conclusion that this aspect of the project simply was not adequately valued during project design. In other words, there is an apparent disconnect between the specific project objective of prioritizing low-income children, the project design and the way it was implemented. - 4 - 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: Overall, the project's outcome is rated as Satisfactory, having achieved most of its major relevant objectives and development results, with a few shortcomings. The PAD defined a matrix of performance indicators to assess achievement of the project's development objectives and impact. Results are largely, though not uniformly positive, as seen below. - 5 - Table 1: Development Objectives/Impact Indicators Achieved Achieved Impact Indicator Trend Target Objective: Expand Access at the Basic Level Increase in Net Enrollment rate for basic YES YES education (5% per year) Increase in the Percentage of Students YES YES(1) who finish 8th grade (from 25% to 50%) Objective: Improve Quality and Effectiveness of Basic Education Increase in academic achievement in core skills YES YES in Spanish and Mathematics (5% improvement on national tests in 8th grade) Reduction in Dropout YES YES Rate (from 11.7% to 5.1%) Reduction in Repetition YES/NO YES/NO Rate (from 14.7% to (2) 11.7%) Increase in class YES YES instruction time (to 770 hours annually) Objective: Increase Public Expenditure on Basic Education Increase education expenditures as a share YES NO (3) of GDP (from 1.9% to 2.8%) Increase education expenditures as a share YES NO (4) of overall public expenditures (from 13.4% to 16%) Increase per student YES NO (5) expenditure (from $106 to US$186) Footnote 1: The projected 8th grade completion rate for 2003 is 53 percent, although this is based on assumptions of trends derived from 2002 data (Source: Oficina de la Planificacion Educativa, SEE). - 6 - Footnote 2: Repetition declined from 13 percent (1994) to 7 percent (2001), well below the target of 10 percent. But changes in how repetition is calculated appears largely responsible for this. Using new methodology, the repetition rate increased from 5 percent (1996) to 7 percent (2001), below the target but an increase nonetheless. (Source: Oficina de la Planificacion Educativa, SEE) Footnote 3: The 2003 approved budget allocated 2.3 percent of GDP to education, down from 3.0 percent in 2002. (Source: World Bank, Public Expenditure Review, August 2003) Footnote 4: Total education expenditures as a share of total expenditures increased from 13.4 percent (1996) to 16.9 percent in 2000, before declining to 15.6 percent in 2001. Basic education spending as a share of total education spending declined from 50 percent (1996) to 46 percent (2001). (Source: World Bank, Public Expenditure Review, August 2003) Footnote 5: Per student spending (direct costs) increased from US$106 (1996) to US$168 (2001), a 58 percent increase, but still below the target of US$186. If indirect costs are included (administration, pedagogical services, educational planning, etc.), per student spending was US$242 (2001), surpassing the target. (Source: Final Report, Gestion Moderna) With respect to the objective of improving educational quality, mixed results are seen. Student performance in 8th grade in national assessment exams showed significant improvements in mathematics and Spanish (above and beyond project targets), modest improvements in social sciences, and declines in natural sciences. (Caution should be used, however, in using the national assessments to measure student learning over time, as these examinations have changed radically and frequently, and are subject to extensive criticism.) Table 2: Average Scores on 8th Grade National Assessment Exams by Subject and Year Subject 1997/98 1998/99 2000/01 Spanish 59.3 69.1 72.7 Math 59.3 71.5 60.7 Social Sciences 53.8 65.2 56.3 Natural Sciences 60.9 51.5 51.1 Global Average 55.2 64.3 60.2 Source : SEE ­ Department of National Examinations Furthermore, despite major investments in teacher training to improve pedagogical methods and practices, discussions with teachers and classroom observations indicate that while they largely support the goals of the curricular and pedagogical reform, their own capacity to apply their training in their classrooms in concrete ways remains quite weak. This is linked to delays and deficiencies in the supply of learning materials to classrooms, which forced teachers to maintain traditional "chalk-and-talk" teaching practices. Concerning the objective of improving basic education enrollments and efficiency, stronger although still mixed results are observed. Overall project enrollment targets were surpassed ­ gross basic education enrollment increased from 91 percent (1996/97) to 113 percent (2001/02), while net enrollment increased from 73 percent to 94 percent during the same period, largely surpassing the project's goal of increasing enrollment by 5 percent. However, this goal was not reached in all grades of basic education, and poorest provinces showed the least improvement. Provinces with a percentage of poor households exceeding 37 percent increased enrollments by an average of just 3 percent annually, while those with a percentage of poor households below 21 percent increased enrollments by an average of more than 6%, twice as fast. In general, urban zones increased enrollment more than rural areas. - 7 - Internal efficiency of basic education improved significantly, though not quite as much as forecast by the project. Dropout rates declined during project execution (from 16 percent to 5 percent), completion rates increased (from 25 percent to 53 percent), and the percentage of over-age students declined dramatically. Unfortunately, and somewhat inexplicably, repetition rates increased from 5.5 percent (1996/97) to 7.3 percent (2001/02). (Data comes from Educational Statistics Department of SOE. Note that in the PAD repetition rates were estimated at 13 percent in 1996, and forecast to decline to 10 percent in 2000. Thus, while repetition rates increased, they nonetheless remained below the project's stated objective. Data inconsistencies may be at play here.) The net result of all these trends is that the total number of years of schooling required to produce a basic education graduate declined by more than 15 percent, from 13.5 years (1996/97) to 11.4 years (2001/02). By increasing access to basic education and early childhood the project benefited primarily the children from low income families (middle class and upper class students attend private schools). The successful school feeding program which has been expanded to universal coverage for Grades 1-8 reaching 16 percent of the country's total population on a daily basis has also a substantial impact on poverty reduction. Nevertheless the project was less successful in prioritizing. Several project sub-components which were initially designed as targeted on poor children (school feeding, provision of school supplies and textbooks, school facility improvements) were subsequently "universalized, so that all children in basic education could benefit. In addition, the specific subcomponent aimed to reach children attending private schools in poor barrios was ultimately abandoned in favor of the SEE's institutional strengthening of its knowledge and oversight of private schools in general 4.2 Outputs by components: The first component, educational development (representing 83 percent of project expenditures), was achieved satisfactorily. Basic education enrollments and internal efficiency objectives were almost entirely achieved. Qualitative improvements may or may not have occurred - deficiencies in the national testing system (discussed above) and data generated from this system make it very difficult to draw a definitive conclusion. The second component, management development, (representing 17 percent of project expenditures) was also achieved satisfactorily, although not entirely. All management capacity-building initiatives were carried out, over 6,000 parent associations were established, and most evaluation activities were completed. Interviews with SEE staff and education donors all suggest increased use of management information systems for decision-making, and enhanced ability of line departments and divisions within the SEE (as opposed to the Project Coordination Office) to develop and implement annual work plans. However, it is very hard to measure the impact of this enhanced management capacity in terms of the delivery of basic educational services, and it is clear that there is still a long way to go in increasing the participation and effectiveness of community members in school management to improve learning outcomes. - 8 - The following table provides a snapshot of sub-objectives achieved, surpassed or un-achieved, by project sub-component. Table 3: Quantitative Achievement of Project Objectives Number of Number of Number of Sub-Objectives Achieved Sub-Objectives Success Sub-Component Sub-Objectives or Surpassed Un-Achieved Rate Reform of National Curriculum 5 4 1 80% Learning Materials and Resources 9 8 1 89% Human Resource Development 4 2 2 50% National Assessment 5 1 4 20% School Nutrition 4 3 1 75% Infrastructure 7 6 1 86% Support to Poor Private Schools 7 2 5 29% Institutional Strengthening 5 4 1 80% Management Information System 3 2 1 67% Community Participation 6 5 1 83% Monitoring and Evaluation 4 3 4 75% As can be seen in Table 3, project sub-component objectives were largely achieved in the areas of curriculum reform, learning materials, infrastructure and community participation. They were generally un-achieved in the areas of national assessment, support to poor private schools, and monitoring/evaluation. A mixed result is seen in the areas of human resource development and management information systems. Curricular Reform: Rating ­ Highly Satisfactory More than 80,000 educators were trained in the new curriculum, compared to 50,000 projected during project appraisal. 420,000 copies of pedagogic materials were produced and distributed, as opposed to 300,000 projected. 150,000 audio-visual learning materials were produced and distributed, versus 10,000 projected. Distribution of laboratory equipment fell short of project objectives. Distribution of Learning Materials: Rating ­ Highly Satisfactory More than 22 million textbooks were produced and distributed (compared to 18.6 million forecast). All pre-school, multigrade and first cycle classrooms received the projected didactic materials, library books and educational games. Teacher instructional materials were not produced nor distributed as forecast. However, while the total quantity of textbooks distributed exceeded forecasts, there were major problems with distribution such that some - 9 - school districts experienced significant shortages while others struggled to find warehouse space for the surplus. Human Resources Development: Rating ­ Satisfactory 5,745 teachers obtained the "profesorado" level (versus 3,000 planned); 3,536 teachers attained their "licenciatura" degree (versus 6,000 planned) ; and 2,412 technical staff obtained post-graduate degrees (versus 300 planned). This shows a very strong demand for post-graduate study, although it is not apparent this resulted in changes in classroom teaching and learning. Sadly, no database exists of the teachers who participated in the training, nor confirmation that teachers actually attended the training, exposing the risk that teachers were simply paid scholarships and universities were paid for training, which may or may not have been actually conducted. In addition, the decision to expand the list of higher education institutions eligible for contracts to offer in-service teacher training (24 institutions in all), many of which had no prior experience in this area, greatly jeopardized the perceived quality of this training. National Examinations System: Rating ­ Unsatisfactory The provision of training and equipment for national assessments, as well as the conduct of research/analysis of national assessments, fell far below project estimates. School Nutrition: Rating ­ Highly Satisfactory 5 million youth were targeted for the school breakfast program over the six-year life of the project; 4 million youth were effectively reached with project resources. While below forecasts this is nonetheless a major achievement, particularly as the Government subsequently decided to expand this program to universal coverage for all Grades 1-8 (1.6 million students per year) using domestic funds, indicating the sub-component was not only sustainable but also catalytic. Targets for teacher training in school nutrition were surpassed. SEE surveys indicate that malnutrition among basic education students decreased from 19 percent in 1993 to 12 percent in 2002. De-parasitation programs covered 3.7 million students, versus 1.5 million forecast. School Infrastructure: Rating ­ Highly Satisfactory 518 classrooms were constructed (versus 470 planned); 97,000 desks were distributed (versus 70,000 planned); 22,777 classrooms received preventive maintenance (versus 24,500 planned); 5,125 classrooms benefited from minor repairs (versus 5,000 planned); 593 classrooms were rehabilitated (versus 1,600 planned); and 154 pre-school classrooms were constructed (versus 145 planned). However, it must be pointed out that targeting of infrastructure activities was based on a process of "microplanificacion" of schools' needs (engineering criteria), rather than on the needs of schools located in the poorest 40% of school districts (poverty criteria), as was planned during appraisal. Pilot Program to Support Private Schools Serving Low-Income Families: Rating ­ Unsatisfactory The design of this sub-component was changed substantially in response to shifting government priorities, but implementation of training (teachers and school directors), distribution of learning materials, and financing of critical maintenance activities were all far below projections. The Government did conduct a census of private schools, developed an MIS system for categorizing private schools, and strengthened the Private School Department of the SEE with two vehicles and office technology. 200 private schools also received a computer and some library books. The revolving loan program was never attempted. Institutional Strengthening: Rating ­ Satisfactory Quantitative input targets for training of school directors and SEE staff were largely met, and the number of projected trucks and motorcycles to be purchased was surpassed. Interviews with project staff and co-financiers indicate that SEE's institutional capacity has improved greatly over the last six years, although it is difficult to measure and observe this ex-post. - 10 - Management Information System (MIS): Rating ­ Satisfactory Twice as many SEE staff received training in the use of the MIS than forecast, all eight information systems were designed and rolled out, 600 schools received MIS systems, and 66 (out of 112) regional or district educational offices obtained Internet access. Information and educational statistics are now readily available, although it is not apparent this is used for decision-making on a regular, systematic basis. Community Participation: Rating ­ Unsatisfactory Although the project achieved the impressive objective of establishing Parent Associations (PA) for every basic education school (6,295 of them), most of these PA are nominal and largely inactive. No model for promoting community participation was designed or implemented. While training of SEE staff in community participation surpasses project targets, almost all of the trained (and equipped) staff were replaced with the change in Government in 2000, a significant loss of capacity. Indeed, 89 percent of Parent Association members surveyed at the end of the project indicated that they had never participated in any training. Finally, none of the 86 local educational projects designed by the Parents Associations were approved for financing, nor was the Escuela de Padres y Madres ever established. . Monitoring and Evaluation: Rating ­ Satisfactory The project performance indicator system was developed by the end of the project, but this should have been achieved much earlier and it is not apparent this was used to provide feedback for project execution. Of the 10 semesterly financial control exercises forecast 8 were carried out; of the 13 special impact studies forecast only 4 were undertaken. Qualitative Achievement of Project Components With respect to the first objective of improving basic education quality, multi-variable regression analysis of the determinants of educational quality was carried out, to determine the impact of different project sub-components on student learning. The results are interesting, although contradictory and inconclusive. The presence of a strong parent association was the only explanatory variable with a statistically significant positive impact on math scores. Availability of audiovisual resources and strong community participation were the primary explanatory variables with a statistically significant impact on Spanish language achievement. Somewhat counter-intuitively, the percentage of qualified teachers had a statistically negative impact on Spanish language scores, while it was the only variable with a statistically significant positive impact on Social Science scores. Equally strange, the presence of school latrines had a positive impact on natural science scores and a negative impact on mathematics scores. Interestingly, the proportion of girls in total enrollment was a very strong explanatory variable with statistically significant positive impact on Spanish and Natural Science scores, with somewhat negative impact on Math and Social Science scores. Finally, there was no clear strong influence of the types of teaching techniques used on student learning outcomes: at the 4th grade level "traditional" teaching methods had a negative influence, while at the 8th grade level they had a slightly positive influence. Based on this analysis, it is difficult to draw definite conclusions regarding the determinants of educational quality and the role played by the different project components in promoting educational quality. Weaknesses in the national testing system (one of the sub-components under the project which was supposed to be strengthened) are probably responsible for these inconsistencies. - 11 - 4.3 Net Present Value/Economic rate of return: The PAD projects three types of economic benefits: l Internal Efficiency Savings (from reduced repetition or student-years saved); l Benefits from completing basic education (rate of return, ROR, model, a la Psacharopoulos and Ng); and l Unmeasurable higher employment and productivity of basic education graduates For purposes of the ICR, the first benefit cannot be measured by a reduction in repetition rates, which actually both decreased and increased during the project (see Footnote 1 for an explanation of methodological issues). However, the benefits of improved educational internal efficiency can be measured by quantifying the financial benefits (savings) of reducing the number of years required to produce a graduate of basic education. During the project this figure declined from 13.5 years (1996/97) to 11.4 years (2001/02). Using a unit cost per year of US$168 this represents a savings of US$336 per graduate. With more than 100,000 graduates per year of basic education, internal efficiency savings are equivalent to US$33.6 million annually. At the beginning of the project, estimates of the social and private rates of return of the project were calculated, using standard ROR methodology (Psacharopoulos and Ng) and salary data from 1991. Upon project completion, these rates were re-calculated, using 2000 labor market data. Table 4: Estimated Rates of Return, Before and After the Project Beginning of Project End of Project 4th Grade Social Rate of Return 18% 25% Private Rate of Return 20% 30% 8th Grade Social Rate of Return 14% 15% Private Rate of Return 15% 18% Note: beginning of project calculations based on 1991 earnings by educational level data. End of project calculations based on labor market surveys by the Department of National Accounts and Economic Statistics of the Central Bank. It is important to note that the period of project implementation was marked by strong sustained economic growth, which benefited workers at all educational levels. Analysis indicates that those with higher education levels enjoyed greater employment opportunities and greater salary increases during this economic expansion, relative to those with no education. These rates of return are for educational levels overall, and are not directly attributable to project investments. Nonetheless, they indicate strong positive and increasing rates of return to basic education investments. 4.4 Financial rate of return: N/A - 12 - 4.5 Institutional development impact: The institutional development impact of this project was modest, less than what the projected aimed to achieve but significant nonetheless. Perhaps most notable was the formation over more than 6,000 Parent Associations (PAs) covering every primary education school in the country. PAs were offered training in school management and some of them are functioning actively. These PAs helped school districts manage almost US$3 million in Bank-financed cash transfers for school maintenance, and have increased accountability of teachers (particularly in rural areas where teacher absenteeism is a major problem). In addition, the technical departments and divisions within the SEE improved their institutional capacities considerably, with the transfer of project planning and execution responsibilities from the Project Coordination Unit to the SEE over the course of the loan. Within the SEE, MIS systems are routinely used for research, planning and scenario-forecasting necessary for policy-making. In addition, many of the 23 institutions of higher education contracted under the project to provide in-service teacher training increased their institutional capacity in important ways (technological, pedagogical and managerial). This increased the downstream multiplier impact of the teacher training financed under the project, as new students and teachers continue to participate in training activities financed from other sources. Finally, the Project Coordination Unit itself was transformed into the Office for International Cooperation (OCI), and now manages World Bank, IDB, USAID, GtZ, Japanese, Taiwanese and other donor projects. There is strong capacity within the OCI to efficiently implement Bank procurement, disbursement and financial management guidelines. Actual "results on the ground" impact of these institutional development investments are much harder to measure. No objective indicators were defined to evaluate the effectiveness of the institutional strengthening component of the project. Staff turnover related to changes in Presidential administrations certainly affected the project (e.g. 100 community participation agents trained and equipped at the beginning of the project were released when a new Administration took over in 2000). Efforts to strengthen regional and district level educational administration fell short of capacity-building goals, and many of the Parent Associations mentioned above remain "on paper", lacking in capacity and unable to genuinely participate in school management. In short, the SEE has yet to institutionalize concepts of genuine accountability of school administrators and teachers vis-a-vis the consumers of education (parents and students), and it is impossible to link the project's "input-oriented" institutional development initiatives with improvements in student learning outcomes. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: One of the key factors affecting project implementation and outcomes was the co-financing structure of this project, between the Bank and the IDB. This structure alternated between (i) stipulated fixed percentage co-financing from each bank for certain project components and sub-component, and (ii) definition of certain project components and sub-components which would be entirely financed by one bank or the other. Confusing co-financing arrangements combined with different procurement procedures and project timelines to render project execution constrained and inefficient. Ultimately, following a review of project execution in 2000 (at which time project disbursements were well behind schedule), a decision was reached to "de-link" Bank and IDB financing, although it should be noted that the IDB was not consulted during the process and only informed after the fact. The resultant parallel financing model was easier to implement and resulted in significantly improved project implementation. - 13 - 5.2 Factors generally subject to government control: The key factor subject to government control was basic education spending and provision of counterpart funds. Despite generally favorable macroeconomic conditions, sustained economic growth, clearly defined sector strategy and policies favoring basic education, and official rhetorical commitment, minimum expected levels of Government funding for basic education were not reached. Indeed, the Bank's Public Expenditure Review (August 2003) indicates that central government education expenditures averaged 2.6 percent of GDP during 1997-2002 (2.3 percent in 2003), which compares unfavorably with the 3.5 percent average for the LAC region. As a share of this funding, basic education expenditures actually declined during project execution, from 50 percent (1996) to 46 percent (2001). However, it should be pointed out that the period of project execution was one of sustained rapid GDP growth, meaning that education spending had to increase in absolute terms just to keep up with GDP growth. Indeed, nominal spending on basic education in increased an average of 20 percent per year between 1996 and 2002, which was also a period of low inflation. (Source World Bank Dominican Republic Public Expenditure Review, August 2003) On balance, spending on basic education increased significantly during the life of the project, although still not as much as was hoped would be the case. While it is difficult to specify exactly how insufficient basic education funding affected project implementation and outcomes, it is safe to say that low funding for qualitative inputs such as pedagogical supervision and educational materials (which averaged less than US$4 per basic education student in 2000/01), certainly contributed to poor educational quality, as reflected in student test scores and internal efficiency indicators. With respect to counterpart funds, delays were experienced during the early years of project execution. In March 1999 the Project Coordination Unit estimated "arrears" of US$2 million in counterpart funds due to lack of funding for this in 1998. The change in government in 2000, which included the nomination of the country's vice-president as Secretary of Education, greatly improved matters in terms of the amounts and timeliness of project counterpart funds. Finally, discussions with PCU staff and former World Bank task managers indicates that the Minister of Education micro-managed the project in its early days, often to extreme levels, with the result that project execution was extremely slow. PCU staff were insufficiently empowered to make decisions and implement them, as all matters had to be approved by the Minister. This would appear to be a case of excessive centralization. However, following negative supervision ratings and intensive support from both the World Bank and IDB in the second half of 1999, the Minister engaged pro-actively in project execution and presided over bi-monthly implementation meetings, which resulted in the project's real "take-off" (the positive side of centralization). 5.3 Factors generally subject to implementing agency control: Project supervision and back-to-office reports in 1998 and 1999 repeatedly describe deficiencies in the implementing agency's capacity to manage procurement and financial management processes. This lead to both delays in project execution and processing of loan disbursements. Despite extensive World Bank-financed training of PCU staff in 1999, new staff were assigned to the PCU with the change in government in 2000 which required additional (duplicative) training for PCU personnel in procurement and financial management. By the end of the project, however, PCU staff were operating smoothly and efficiently, with excellent and highly efficient relations with the Bank task manager, procurement officer, disbursement officer and other staff. Clearly, important project implementation capacity was developed - 14 - under the project. On the technical side, the PCU and executing departments within the SEE took a long time to develop the smooth flow of information and communication necessary for efficient project execution. Progress was made in the joint development of the Annual Work Programs (POAs), but more could have been done on a day-to-day basis to speed things up. The lack of information regarding the process of section, registration, and attendance of teachers in the in-service training program, and the lack of transparent regulations and procedures regarding training scholarships and the obligations of trainees, is an example of this void of information and communication. Finally, with the conversion of the PCU into the International Cooperation Office (OCI), the focus shifted somewhat from project execution and coordination to the development of new investments (loans and grants) in Secondary Education, Basic Education and Early Childhood Development. 5.4 Costs and financing: There were no major cost changes during the life of the project. As of the time of this ICR total project expenditures were US$97.53 million, versus an estimated cost of US$100 million (the IDB loan closes on December 31, 2003, and some additional final expenditures are expected). As for the World Bank loan, out of the original principal of US$36.4 million, US$35.4 million was disbursed and US$1 million was cancelled. Project scope, scale and design for the major cost items (e.g. classroom construction and rehabilitation) was accurate, and where actual costs were lower than estimated this permitted procurement in greater quantities (e.g. textbooks, audio-visual materials). Inflation was low through the project implementation period and exchange rates were stable. In general, major contractors (infrastructure, training and MIS systems) performed satisfactorily. There were changes in financing of certain project sub-components. Specifically, the Loan Agreement was amended in 2000 to reallocate US$3.7 million (much of which was initially intended to support a pilot program to upgrade private schools serving low-income students) towards a pilot program for Early Childhood Education, increased financing for textbook printing and distribution and implementation of transfers to support school-based maintenance activities. Indeed, actual Bank-financed expenditures on educational materials and resources were US$6.73 million instead of the US$2.95 million projected at appraisal. No Bank-funds were spent on the private school sub-component, versus a projected US$1.62 million. Other expenditures which were lower than projected were for national curriculum reform (no Bank funds were used for this component, which was ultimately financed by the IDB), human resources development (50 percent below expected), and monitoring and evaluation (70% below expected). Project appraisal estimated a 5 1/2 year project execution (1/1/96 - 6/30/2001), while actual project execution lasted nearly six years (10/24/97 - 6/30/03). Delays in project effectiveness reflected an unrealistic implementation schedule and slow start-up on the part of the Government (which was in transition in 1996/97). Disbursements began in October 1998, more than two and one-half years behind schedule. Additional delays in project implementation were related to a decision by the Government to place sole decision-making authority for the project within the office of the Secretary of Education (a poor management decision) during the first three years of the project. Deficiencies in provision of counterpart funding, project procurement and financial management caused additional delays, but these were "re-couped" during the last three years of project execution. Thus, while the project completion date was extended by a full two years, the period of actual project implementation was not significantly different - 15 - from expectations at appraisal. 6. Sustainability 6.1 Rationale for sustainability rating: This project's sustainability is assessed as "likely". The Government's commitment to further improvement of basic education is clearly reflected in the recently published 10-year Education Development Plan (2002-2012). This plan and its overarching policy framework has received strong support from a wide variety of stakeholders in government, civil society, academia and even opposition political parties, which suggests that it will be maintained. Secondly, the plan has been well-received by all donors, several of whom have committed significant funding (the European Union has committed 100 million euros and the IDB has committed US$80 million (with strong possibility for a follow-on program), both for basic education). To reinforce improvements in the quality of basic education, the World Bank has also launched a US$42 million Early Childhood Education project which will improve readiness for primary education and should lead to further improvements in internal efficiency and quality of basic education. In terms of recurrent financing, the Government has committed to minimum funding levels for classroom maintenance, educational materials and school feeding programs necessary to sustain project investments, as part of World Bank and IDB-financing social sector adjustment lending in 2004. The main risk to sustainability relates to the Dominican Republic's extremely tight public expenditure constraints in 2004 (as part of an agreed-upon IMF plan), which could jeopardize overall education spending in the short run. Additional limitations to the project's sustainability include the lack of local participation (parent associations) in school management and the SEE's continued reluctance to promote greater decentralization and local initiative of at the basic education level (in part because of lack of training and capacity-building of the parent associations). Thirdly, it must be recognized that the Dominican Republic has a tradition of major changes in civil service personnel which accompany transitions in government, and there is a risk that much of the SEE's institutional capacity developed under this project will be lost when a new Administration takes charge in August 2004. Fourthly, the project investments in management information systems carry an estimated annual maintenance cost of US$235,000-525,000 over the next four years, which may be difficult for the Government to sustain. Despite these factors, the commitment of all stakeholders to the 10-Year Education Development Plan and very strong donor support for this plan are likely to overcome these limiting factors, thereby justifying the "likely" rating. 6.2 Transition arrangement to regular operations: The Government has moved to transition project investments to regular operations in various important ways: SEE's institutional capacity was strengthened through specific project investments (training, information systems, etc.) and through transfer of responsibilities from the Project Coordination Unit to the appropriate departments. The conversion by the Government of the Project Execution Unit into a Project Coordination Unit, which placed in the different departments of the SEE full responsibility for planning, budgeting and executing ongoing activities, has facilitated this process. Secondly, the School Maintenance Committees, which are integrated into the School Councils ("Juntas"), have increased the commitment of parents and community members in ensuring ongoing maintenance of - 16 - infrastructures financed under the project, although not as much as might be expected. In addition, as part of a social sector adjustment loan, the Government has committed to the World Bank and IDB to provide through the national budget in 2004 and 2005 sufficient financing for the maintenance of 10,000 classrooms each year (at a projected cost of US$11.6 million per year). It is expected that a significant share of these funds will be executed via up to 500 financial transfers to the School Maintenance Committees. Thirdly, while it is difficult to precisely measure the impact of the substantial project investments in teacher in-service training, there is no question that teachers' knowledge of the new curriculum and ability to apply new teaching methods (developed through the project-financed in-service training) has improved substantially. While the overall assessment of the in-service training component may be that it was not as effective as was hoped (a more critical view would be that it served as a "diploma factory" which simply resulted in higher teacher salaries), conversations with teachers do indicate both strong satisfaction with the training received and increased teaching effectiveness in the classroom. A related point is that this component greatly strengthened the capacity of the in-service teacher training institutions themselves (Universidad Autonoma de Santo Domingo, INTEC and others), through the many project contracts fulfilled. This kind of human and institutional capacity building has a positive multiplier effect on all the other trainees/students in these institutions, which reinforces sustainability. Fourthly, the Government has integrated several important project investments into its own recurrent activities. For example, the school feeding program supported under this loan has been fully institutionalized in the Student Welfare Department, and has quadrupled in size using domestic resources to reach all basic education students in the country (1.6 million rations per school day are distributed, at an annual cost of US$42 million). The financing of educational materials and textbooks has been partially integrated into the 2004 SEE budget, and the Government has committed to provide US$14.2 million annually in 2004 and 2005 for this purpose. Fifth, while education financing as a percentage of GDP did not increase as hoped during the life of the project, funding for basic education increased steadily in nominal terms between 1996-2002, at an average annual rate of 20 percent. These steady and significant increases reflect a commitment to support the expansion and qualitative improvement of primary education. It is hoped these increases will continue so as to further ensure the sustainability of project investments. 7. Bank and Borrower Performance Bank 7.1 Lending: The Bank's performance in identification, preparation assistance and appraisal of the project was satisfactory. Explicit consideration of the Government's 10-year plan (strategies and priorities) for education was demonstrated during identification. This plan focused on qualitative improvements throughout the system, especially for basic education, and aimed to improve both internal and external efficiency of the educational system. As discussed earlier, project sub-components were coherently linked to achievement of project objectives. The Bank's preparation assistance was generally adequate in terms of technical and economic design, but occasionally simplistic technically and overly optimistic in terms of financial and institutional aspects. A broad, somewhat unfounded, assumption was made during project preparation that qualitative improvements in student learning and quantitative improvements in internal efficiency would be achieved - 17 - through in-service and pre-service teacher training delivered by national universities. However, no careful assessment of the demonstrated capacity of the national universities to deliver these results was carried out. What transpired was a costly series of contracts between the SEE and local universities, and thousands of training diplomas distributed to Dominican teachers (the SEE linked salary incentives to completion of these programs), with little observable impact in terms of teaching practices or learning outcomes. The mid-term evaluation highlighted a number of problems with this component, including deficient programming, participant selection, cost control, monitoring and evaluation. Indeed, it can be argued that this was an expensive mistake. By the end of the project, a new model was developed for in-service teacher training, which emphasizes peer-based facilitation, classroom observation of other teachers and other forms of on-the-job training. This, at least, shows a process of learning. Financially, the overall project was projected to cost US$100 million, with the Bank's portion projected at US$37 million. The IDB's financing was projected at US$52 million, but the co-financing mechanisms were poorly designed. Project cost tables in the PAD do not specify what IDB would finance ­ the PAD simply states that the banks would "share the financing of major components, though discernible subcomponents will be financed by each." (PAD, page 26) This oversight became a serious problem during project implementation, and resulted in a cumbersome process of financial allocation for the PCU. With regards to institutional aspects, the design of the management structure for project implementation was confusing. The PCU had both a technical unit (intended to support technical execution of project components) and an administrative unit (responsible for project personnel, procurement, finance, accounting and auditing activities). The division of responsibilities between the PCU technical unit and the relevant SEE departments was unclear. The PCU was also authorized to pay regular SEE staff for extra hours worked on the project, which created tensions between technical unit staff and their SEE counterparts. In addition, the government's commitment to support the development of private schools assisting low-income families was grossly overestimated during project preparation, and the rotating loan fund designed during appraisal was inappropriate. During appraisal staff estimated the cost of this program at US$3.4 million. In fact, very little was undertaken or accomplished in terms of training (teachers/school directors), infrastructure or distribution of materials. Ultimately, this sub-component was largely abandoned in favor of improving the SEE's institutional oversight of private schools. Otherwise, Bank performance was satisfactory in terms of its appraisal of: the commitment by government, implementing agency and the project beneficiaries; the capacity of the implementing agency; recognition of project risks; appropriateness of the financial package; adequacy of performance indicators; suitability of the lending instrument; consistency with the Bank's safeguard policies; and degree to which lessons learned from previous projects in the country and education sector were taken into account. However, lack of clear coordination with the key co-financing agency (IDB), particularly with respect to financing arrangements and procurement procedures, resulted in significant project implementation problems. 7.2 Supervision: The Bank's supervision performance is rated as satisfactory overall, although it is argued that supervision was unsatisfactory during the early years of the project. Specifically, lack of attention to the different procurement, disbursement and financial arrangements between the two financiers contributed to the project's implementation declared as "unsatisfactory" by the Bank in February 1999. This resulted in much more intensive supervision, including involvement of the LCSHD Implementation Improvement Support Group, which resulted in the project's return to "satisfactory status" in April 2000. - 18 - Surprisingly, the Bank did not participate actively in the mid-term evaluation of the project, despite clear commitments to do so in the PAD and Loan Agreement. The midterm evaluation was essentially carried out by the Government and the IDB (in March 2000, one year later than specified in the Loan Agreement), with all evaluation staff hired by the IDB. Indeed, the mid-term evaluation noted a lack of supervision by the Bank. The Bank did organize a workshop after the mid-term evaluation to discuss implementation to date and necessary changes, but this was done by an interim task manager who provided little continuity. Indeed, between the end of 1999 and October 2000 there were two changes in supervision responsibilities, which resulted in a supervision "vacuum". Interviews with SEE authorities revealed that the lack of a World Bank office in the country during the initial years of the project significantly hampered communications with the Bank and delayed both procurement and disbursement processes. Project supervision improved markedly following in October 2000, with a change in task managers that resulted in positive modifications to project design, including an amendment to the Loan Agreement. This coincided approximately with the opening of a Bank Country Office in Santo Domingo and a new Government, both of which provided greatly increased attention to project execution. Finally, Government authorities highlighted the Bank's responsiveness in providing the training required for procurement and financial management functions within the PCU. This training was provided on multiple occasions during the life of the project. 7.3 Overall Bank performance: Taking into account the ratings given to the Bank's lending and supervision performance, overall Bank performance was satisfactory. Deficiencies in project design (both institutional and financial) were corrected during project implementation, with strong assistance provided by Bank task managers and the LSCHD Implementation Improvement Support Group once serious problems were identified. Finally, senior Government education authorities interviewed as part of the preparation of this ICR stated emphatically that they greatly appreciated the Bank's flexibility in adjusting project implementation and sub-components in order to achieve the project's development objectives. Borrower 7.4 Preparation: The Borrower's performance during preparation is also rated as satisfactory, for several reasons. The Borrower ensured the project was consistent with education sector development strategy and sub-sectoral priorities. In addition, the Borrower mobilized sufficient levels of funding (both external and domestic) to achieve very ambitious project objectives, via a holistic approach to improving educational access and quality. In short, the most essential aspects of loan preparation were successfully carried out. However, the Government did not sufficiently involve stakeholders (particularly parents and teachers) in project preparation, nor was consensus generated within the SEE in favor of decentralization and increased parental involvement in school management. Lack of reliable statistics and information within the SEE during project preparation resulted in project design which was not fully adapted to reality, particularly with respect to the sub-components "support to private schools" and "national evaluation". These weaknesses are not such that an unsatisfactory rating should be attributed to the borrower's preparation, but they do highlight some critical areas which need to be corrected in future project preparation. - 19 - 7.5 Government implementation performance: Overall Government implementation performance is rated as satisfactory, although weak during initial years of the project. For example, for the first three years of the project, the Secretary of Education mandated that all personnel and expenditure decisions related to the project had to be approved by the Secretary's office, which slowed down administrative processes considerably and "dis-empowered" project staff. Secondly, the establishment of new procedures by the Comptroller General for accessing international funds deposited in the Central Bank, and for accessing corresponding counterpart funds, created delays and confusion for PCU financial management staff. Bank supervision documents also attest to occasional reluctance on the part of PCU staff to fully adhere to World Bank procurement procedures, in some cases for lack of knowledge/training and in other cases because IDB procedures were preferred. On the positive side, despite the sweeping changes of SEE personnel related to the arrival of a new Government in August 2000, there were only minor changes of PCU staff. This continuity was critical for project implementation. Appointing the Vice President of the country as Secretary of Education in August 2000 greatly strengthened implementation performance, as well. The new Secretary provided strong political support, ensured counterpart financing, and accelerated administrative procedures. 7.6 Implementing Agency: PCU performance is also rated as satisfactory. Despite some delays and weaknesses observed in the implementing agency's financial management and procurement functions, the PCU continually improved its effectiveness and efficiency. The creation of the Office for International Cooperation, in which the PCU operates, further strengthened the capacity of the PCU as it consolidated expertise and experience with World Bank and IDB loan execution. Procurement, infrastructure, financial management and monitoring/evaluation functions improved markedly over the last three years of project implementation. 7.7 Overall Borrower performance: Satisfactory. 8. Lessons Learned 1. While far from a "new" lesson learned, the project illustrates once again the need for sustained increased public basic education spending if this type of investment is to fully succeed. The Bank needs to be even more proactive in insisting not only on timely and sufficient allocation of project counterpart funds for project execution, but also on overall increases in funding for the sub-sectors targeted for investment. 2. Institutional capacity-building is more than a question of finances, information technology and training. It is a cultural shift which requires strong leadership by example at all levels, including demonstration of the use of information for decision-making and clear designation of roles and responsibilities for accountability. Secondly, it requires continuity of staff and a degree of immunity from political changes above. So long as shifts in presidential power result in massive sweeps and/or relocations of civil servants, based on political party affiliation, efforts to improve institutional capacity are unlikely to produce lasting results. 3. Community Participation requires more than rhetoric to make it real, and is stimulated when - 20 - financial resources and decision-making authority are combined. Simply encouraging parents and community leaders to get involved in school management is not enough. Parents require training, empowerment (authorization) and some control over financial resources to get engaged. 4. School nutrition programs (desayuno escolar) are an important incentive for student attendance and retention at school. Teachers, school directors and parents interviewed for this report were unanimous in this assessment. That public primary schools are attended in large part by students from poor families (many better off families prefer to send their children to private schools) provide further justification for this program as a poverty reduction/social safety net intervention. 5. In-service teacher training programs which focus on certification and/or diplomas may produce teachers who look good on paper but who are unable to apply their training in the classroom. Training programs should be grounded in the practical teaching and learning realities of classrooms, rather than based on prevailing academic theories of education. Secondly, a clear definition of the teaching/learning objectives sought through in-service training is necessary to ensure both selection of high-quality institutions capable for providing this training and downstream impact measurement. There must be some mechanism to assess school performance before and after training, including links to teacher performance. Finally, training of teachers may be more effective when some personal commitment is required beyond time - offering both scholarships and higher salaries for diplomas certainly encourages teachers to participate in training, but actual training impact is likely to be higher and more cost-effective if teachers participate financially, as well. 6. Co-financed projects require very careful project design and particular attention to the distribution of financing by components, sub-components and categories of investment. Ensuring synchronization of financier timelines is also essential to avoid debilitating "starts and stops" in project execution. Equally important for co-financed projects, the harmonization of World Bank and IDB procurement procedures is a much needed area for future work between the two banks, which would greatly facilitate project implementation and effectiveness. 7. Efforts to improve project implementation through initiatives such as the Implementation Improvement Support Group can indeed be very effective, particularly when this is combined with flexibility on the part of Bank task managers to make needed adjustments on its side. 8. Project design must allow for considerable flexibility over a 5-6 year implementation period, given the high probability of changes in Government, task team leaders and external environments. In particular, while the logical framework is an important management tool it needs to be open to continuous revision as project targets and indicators change. 9. Ensuring that Secretariat of Education personnel are ultimately responsible for project execution (NOT the project coordination unit) significantly improves internal management capabilities of the SEE. This requires development of skills related to planning, budgeting, personnel, monitoring and evaluation, and the promotion of a culture within the SEE which encourages these skills. This difusion of good management practices within the SEE increased both institutional capacity and suggests greater abilities to implement changes in the future. - 21 - 9. Partner Comments (a) Borrower/implementing agency: Translated Summary: The enrollment and access objectives for basic education were achieved. However, there still exist inequalities and deficiencies in basic education enrollment, with poor and rural children still lacking access to all eight grades of basic education. The major investments in teacher training increased teachers' understanding of the new curriculum, but has not dramatically improved teaching methods nor created a new dynamic of teaching and learning. Low socio-economic conditions of teachers, many of whom live just above the poverty line, are an obstacle to their further development. Internal efficiency and quality of basic education improved somewhat. Overage students decreased more than expected. Repetition decreased then increased. Math and spanish test scores improved more than expected, while improvements in social science scores were modest and science scores actually declined. Lessons Learned: 1. Increased, sustained investments in basic and early education are necessary. Without more resources and qualified teachers, with high performance standards for teaching, school facilities and management, basic education results will change little. 2. The in-service teacher training model needs to be revised and re-thought. University-based programs lack connection to practical needs of teachers and their working conditions. Increased monitoring of improvements in teacher performance and of the effects of their training on student learning outcomes is urgent, given the large amount of resources invested in this activity. 3. To improve management capacity at ALL educational levels, additional efforts are required. Clear performance objectives for each educational level should be established, with increased autonomy and responsibility given to school and district levels. 4. Internal efficiency improvements can be promoted through definition of educational goals at the school level which are shared between the school and community. Local involvement must be stimulated. Resumen Ejecutivo El Segundo Programa de Mejoramiento de la Educación Básica e Inicial se proponía aumentar la equidad, la eficiencia y la sostenibilidad de la Educación Inicial y Básica de los niños y las niñas dominicanas, como una estrategia de apoyo a la Segunda Fase del Plan Decenal de Educación. Las metas generales del Programa, en términos de indicadores de los efectos de las intervenciones, fueron las siguientes: 1. Gasto Gobierno Central en Educación Inicial y Básica como % del PIB aumenta a 2% en año 1, 2.2% en año 2, 2.4% en año 3, 2.6% en año 4 y 2.8% en año 5. 2. El gasto (inversión + corriente) por alumno en Educación Inicial aumenta de US$105 en 1995, a US$150 en el año 5. 3. El gasto promedio (inversión + corriente) por alumno en Educación Básica aumenta de US$164 en 1995, a US$166 en año 1, US$170 en año 2, US$175 en año 3, US$180 en año 4, y US$186 en año 5. 4. Mejora en más de un 5%, en 8º Grado, el rendimiento de estudiantes en las Pruebas Nacionales a partir del año 2000. - 22 - 5. La tasa de repitencia se reduce de 14.7% en 3 puntos porcentuales a partir del año 2000. 6. Reduce la tasa de deserción del 11.7% al 5% para el año 2002. 7. Aumenta el porcentaje de egresados de la Educación Básica de 25% a 50%, en el 2002. 8. La matrícula proporcional se incrementa en un 5% en todos los grados de Educación Básica, a partir del 2000. 9. Tasa de retención de Educación Básica aumentada en 3% a partir del 2000. En términos generales, la iniciativa asumía las siguientes hipótesis: 1. Que un mejoramiento de la calidad de la Educación Básica era viable a partir de la implementación de un nuevo currículo para este nivel, caracterizado por ser más abierto y flexible que el que se venía aplicando con anterioridad, donde se fomentara la participación de todos los actores relevantes del proceso, y sustentado por una efectiva capacitación de directores de centros, maestros, técnicos y miembros de la comunidad organizados alrededor de cada centro educativo. 2. Se asumía, asimismo, que la distribución de libros de texto, manuales, guías y otros materiales didácticos; que el desayuno escolar, la rehabilitación y construcción de aulas, el equipamiento y mantenimiento de escuelas, conjuntamente con mejoras en el dominio de contenidos y de las metodologías de enseñanza por parte de los docentes, previstos como parte de las intervenciones prioritarias del Programa habrían de conducir al logro de los referidos indicadores de calidad. En términos específicos, mediante las intervenciones implícitas en las hipótesis señaladas, el Programa fue dirigido a mejorar los niveles de competencia y eficiencia en la gestión institucional operativa de la Secretaría de Estado de Educación; a ampliar los servicios que en materia educativa se orientan a beneficiar a estudiantes provenientes de familias de bajos ingresos, tanto en las escuelas públicas como en las privadas; y a incrementar la matrícula y tasa de retención a nivel Inicial y Básico. También el programa perseguía aumentar la tasa de promoción, reducir la tasa de repitencia y la deserción escolar; apoyar la aplicación del Nuevo Currículo; y capacitar (en servicio) al personal técnico, directivo y docente tanto a nivel central como regional, distrital y local, así como a miembros de la comunidad organizados en las APMAES. Evaluacion y Metodología El estudio evaluativo del Segundo Programa de Mejoramiento de la Educación Básica ha sido abordado tomando en cuenta que ha consistido en un conjunto de intervenciones en el marco de un sistema, el sistema educativo dominicano, una de las estructuras de servicio y gestión pública más extensas, complejas y de larga data en la historia dominicana. El paradigma que orienta el estudio evaluativo del Segundo Programa se basa en la lógica de sistemas y una concepción de los programas y proyectos sociales como intervenciones cuasi experimentales Martinic, Sergio: La Sistematización y sus relaciones con la evaluación y la investigación social, Fundación Universitaria Luis Amigó-CEAAL, Santiago, Chile, 1998.. Se ha seleccionado este enfoque sobre la base del criterio de que al ejercicio de cualquier proyecto o programa que pretenda afectar procesos sociales y marcos institucionales tan amplios como los que pretendía afectar el Segundo Programa se debe exigir la posibilidad de generar un caudal significativo de experiencias y lecciones aprendidas que "sistematizadas" (valga la redundancia) contribuyan a enriquecer el repertorio de políticas públicas y herramientas de gestión necesarias para impulsar el desarrollo. - 23 - La evaluación ha sido concebida como un intento de reconstruir: a) Las hipótesis de intervención, los supuestos y condiciones así como los criterios y estrategias presentes en quienes diseñaron el Segundo Programa; b) Una delimitación lo más precisa posible de las intervenciones propiamente dichas, las variables que se introdujeron en el sistema durante la ejecución del programa; c) Lo que efectivamente se hizo al ejecutar el programa, ya que si (a) y (b) definen lo que se quería hacer, entre esto y lo realmente hecho suele haber discrepancias; d) Lo que sucedió como resultado de las intervenciones, los efectos o la eficacia de las variables de intervención sobre las situaciones intervenidas, que incluyen procesos, marcos organizativos, niveles de actuación y actores en una dinámica de conexiones e interacciones constantes; e) La valoración general del Programa como estrategia de acción y política pública en el sistema educativo. Mediante la compilación de datos procedentes de la Oficina Nacional de Presupuesto, Banco Central de la República, Centro Nacional de Salud Materno Infantil y Departamento de Estadísticas Educativas, la Dirección de Pruebas Nacionales, Departamento de Presupuesto , la Sub Dirección de Bienestar Estudiantil y el Departamento de Monitoreo y Evaluación, la Sub Secretaría de Informática y otros departamentos y unidades de la SEE se ha tratado de reconstruir, en parte, lo que efectivamente se hizo en el curso de la ejecución del Segundo Programa. Resultados Grado de cumplimiento cuantitativo de las metas La ejecución del programa alcanzó un alto grado de cumplimiento de las metas desde el punto de vista de la ejecución cuantitativa. Componente Cantidad Metas % Metas no % de metas cumplidas cumplidas o superadas Reforma del currículo nacional 5 4 80% 1 20% Materiales y recursos de aprendizaje 9 8 89% 1 11% Desarrollo de recursos humanos 4 2 50% 2 50% Evaluación nacional del rendimiento 5 1 20% 4 80% Apoyo a colegios privados pobres 7 2 29% 5 71% Infraestructura 7 6 86% 1 14% Fortalecimiento institucional 5 4 80% 1 20% Sistema de información gerencial 3 2 67% 1 33% Participación comunitaria 6 5 83% 1 17% Nutrición escolar 4 3 75% 1 25% Monitoreo y evaluación 4 1 25% 3 75% La ejecución presupuestaria también se ajustó en gran medida a la programación que se había realizado. En cuanto al alcance de las intervenciones, la meta de gasto público en educación básica e inicial no pudo ser alcanzada, antes bien, la ejecución quedó por debajo del 50% de lo previsto. - 24 - El costo por estudiante, calculado sobre la base de costos indirectos, sí alcanzó la estimación programada, mientras que calculado mediante costos directos solamente, se produjo un incremento considerable, pero por debajo de lo previsto. El mejoramiento de la planta física escolar se manifestó, a la fecha de los datos disponibles (2000) en el hecho de que la mayoría de los directores de centro consideraron que las condiciones físicas de sus escuelas eran adecuadas. La dotación de materiales educativos y equipos, sin embargo, era insuficiente. Impacto Las cobertura de la educación básica ha crecido hasta alcanzar las metas previstas. Persisten, sin embargo, deficiencias en la distribución de esta cobertura, tanto en términos territoriales como en lo que respecta al acceso de los grupos más pobres de la población a la educación básica. La capacitación del personal docente no se ha manifestado en una efectiva renovación de los métodos de enseñanza predominantes entre el profesorado ni en la creación de una nueva dinámica en la enseñanza-aprendizaje. Se percibe que las condiciones sociales y económicas de los docentes, gran parte de los cuales se encuentran en el nivel de pobreza, obstaculiza su desarrollo. Con limitaciones de alcance, las intervenciones han mostrado cierto impacto en la eficiencia interna y la calidad de la educación básica. La tasa de deserción ha disminukdo, mientras que la tasa de egresados ha aumentado, conforme a lo previsto. La de sobreedad se redujo mucho más de lo previsto, pero la de repitencia se incrementó, con lo que la variación en la eficiencia interna de la educación básica ha mejorado en alguna medida, pero por debajo de las expectativas. El rendimiento de los estudiantes de cuarto y octavo grado en las pruebas nacionales mejoró en las asignaturas de matemática y español por encima de lo que se había programado. En ciencias sociales el incremento del rendimiento fue modesto y en ciencias naturales se produjo una regresión en el rendimiento. Lecciones aprendidas La inversión en educación básica e inicial debe aumentar de forma considerable y sostenida. Sin una notable mejoría en la provisión de recursos educativos, de personal docente calificado y con altos estándares de desempeño, de infraestructura y capacidades de gestión es muy poco el cambio que se puede producir en los resultados del ciclo básico de educación. Las metas de inversión en Educación Básica e Inicial que se fijó el Segundo Programa y que eran una de las bases de la estrategia de desarrollo del sistema educativo fijadas por el Plan Decenal siguen pendientes de cumplimiento. La capacitación del personal docente y directivo en los niveles de base del sistema educativo (profesores y profesoras, directores de centro, técnicos y directores de distrito) debe ser revisada y repensada. No se trata sólo de problemas de metodología, sino de visión y estrategia general. A la par que se invierte en capacitación con programas y metodologías revisados, es imperativo que se den pasos en la dirección de mejorar las condiciones del personal docente y de atraer recursos humanos más calificados hacia la práctica docente. Sólo una mejoría creciente y constante del financiamiento público a la educación básica podrá producir el último de los procesos citados. La forma de operación de la capacitación de los recursos humanos que ha sido predominante en el Segundo - 25 - Programa, a través de programas universitarios, deberá ajustarse a diseños y estrategias metodológicas más efectivas. Se requiere de un ejercicio de capacitación con más apego a las necesidades prácticas de los docentes y a las condiciones reales en que desarrollan su labor. Un monitoreo adecuado de las ganancias en desempeño por parte del personal capacitado y de los efectos que la capacitación vaya produciendo en el rendimiento de los estudiantes y la calidad de la educación básica parece urgente para ajustar la magnitud de los recursos y esfuerzos invertidos en capacitación y los resultados alcanzados. Para reducir la brecha entre la capacidad de gestión del sistema educativo en todos sus niveles y las necesidades de desarrollo del país se requerirán esfuerzos adicionales y diferentes a los ya hechos. La capacidad de gestión debe tener como parámetro de eficacia el cumplimiento del compromiso de calidad en todos los niveles, y el compromiso de calidad no se obtiene con sólo fijar metas cuantitativas de alcance de cierto tipo de intervenciones sino estableciendo parámetros cualitativos de desempeño, umbrales de calidad de gestión, que den cuenta de cambios en las destrezas, el rendimiento y la conformidad de las prácticas de gestión con unos resultados pre establecidos. Los niveles superiores del sistema educativo deberán avanzar hacia el cumplimiento de estándares cada vez más rigurosos de desempeño en la gestión, apegados a criterios gerenciales avanzados, rigurosos. Los niveles de distrito y centro deben ser estimulados hacia roles más proactivos, más autónomos en la definición de metas locales que debe alcanzar cada centro y cada distrito en cuanto a rendimiento y calidad del proceso educativo. Establecer el compromiso o aporte de cada nivel al logro de las metas generales y hacer que ese compromiso sea responsabilidad compartida, evaluable, de cada actor es una condición para que los procedimientos y herramientas de gestión que sean servidos a través de la capacitación y el desarrollo del sistema de gerencia de la educación sean incorporados y asumidos como una responsabilidad y una necesidad de la operación diaria de cada distrito, cada centro y cada educador. El uso de las herramientas y procedimientos adecuados no debe seguir siendo algo que se promueve, debe ser una necesidad del funcionamiento rutinario. Hacer que no se pueda trabajar en el proceso educativo sin utilizar herramientas de gestión adecuadas es un reto que debe imponerse el sistema educativo. La eficiencia interna del sistema educativo puede ser impulsada si se estimula, permite y fomenta que cada centro tenga el compromiso con este propósito y defina sus metas locales, su plan de centro, generado y compartido con la comunidad. Hay que estimular la proactividad y el dinamismo local. Cada centro debe cubrir una cuota de avance en la búsqueda de una educación de calidad y con la eficiencia necesaria. La interacción entre establecimiento de metas e imperativos locales de calidad y el mejoramiento de las herramientas de gestión, con un contexto de mayor autonomía y de rendición de cuentas, parece ser una de las exigencias y condiciones para el mejoramiento sostenido de la calidad y los resultados del proceso educativo. (b) Cofinanciers: [Note: a draft of this report was shared with the IDB for comments. A meeting was held to discuss the IDB's comments which have been largely incorporated into the ICR document itself. (c) Other partners (NGOs/private sector): n/a - 26 - 10. Additional Information N/A - 27 - Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate Improve Quality of Basic Education Indicator 1: Reduce repetition rates Indicator 1: 10% (baseline: 14.7%) Indicator 1: 7% Indicator 2: Reduce drop-out rates Indicator 2: 5% (baseline: 11.7%) Indicator 2: 5% Indicator 3: Classroom hours Indicator 3: 770 hours (baseline: 680 hours) Indicator 3: 770 hours Increase enrollment -- especially completion rates Indicator 1: Gross enrollment rate Indicator 1: 97% (baseline: 74%) Indicator 1: 113% Indicator 2: Completion rate in 8th grade Indicator 2: 50% (baseline: 25%) Indicator 2: 35% Strengthen technical and resource management capacity for Basic education Indicator 1: public expenditure on education Indicator 1: 2.9 % (baseline: 1.9%) Indicator 1: 2.6% as % of GDP Indicator 2: public expenditure on education Indicator 2: 16% (baseline: 13.4%) Indicator 2: 16% as % of yearly national budget Output Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate Indicator 1: New classrooms built Indicator 1: 270 Indicator 1: 518 Indicator 2: Clasrooms with preventive Indicator 2: 24,500 Indicator 2: 22,777 maintenance Indicator 3: Refurbished classrooms Indicator 3: 600 Indicator 3: 593 Indicator 4: Integrated Early Childhood Indicator 4: 50 Indicator 4: 154 classrooms functioning Indicator 5: Teachers trained/in training to Indicator 5: 6,000 Indicator 5: 9,281 obtain a teaching degr 1End of project - 28 - Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Component US$ million US$ million National Curriculum Reform 3.10 3.54 114 Educational Materials and Resources 11.40 16.47 144 Human Resources Development 7.90 6.41 81 National Student Assessment 2.90 1.54 53 School Nutrition Programs 3.20 1.41 44 School Facilities (infrastructure) 43.10 49.57 115 Pilot Program to Support Private Schools Assisting 3.40 1.93 57 Low-Income Populations Institutional Strengthening of SEE 3.20 5.20 163 Management Information System 3.70 4.19 113 Community Participation 2.30 0.99 43 Monitoring and Evaluation System 1.90 1.02 54 Project Coordination Unit 4.10 5.46 133 IDB Supervision 0.50 0.45 90 Total Baseline Cost 90.70 98.18 Physical Contingencies 1.90 Price Contingencies 7.40 Total Project Costs 100.00 98.18 Total Financing Required 100.00 98.18 - 29 - Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 13.10 3.00 16.50 32.60 (0.00) (11.80) (2.40) (16.50) (30.70) 2. Goods 1.80 2.00 1.80 11.20 16.80 (1.80) (1.40) (1.80) (11.20) (16.20) 3. Services 0.00 0.00 6.60 6.60 13.20 Consulting (0.00) (0.00) (6.60) (6.40) (13.00) 4. Other Goods 0.00 0.00 1.10 13.40 14.50 (equipment/vehicles/pharm aceuticals) (0.00) (0.00) (0.80) (12.60) (13.40) 5. Maintenance Materials 0.00 0.00 4.40 4.30 8.70 (0.00) (0.00) (4.00) (0.30) (4.30) 6. Training 0.00 0.00 6.20 8.00 14.20 (0.00) (0.00) (6.20) (8.00) (14.20) Total 1.80 15.10 23.10 60.00 100.00 (1.80) (13.20) (21.80) (55.00) (91.80) Figures in parentheses are the respective amounts financed by the IBRD and, for Non-Bank Financed, by the IDB. Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 16.92 5.24 16.99 39.15 (0.00) (13.20) (4.70) (0.00) (17.90) 2. Goods 6.91 0.00 0.31 13.30 20.52 (6.36) (0.00) (0.28) (0.00) (6.64) 3. Services 0.00 0.17 3.99 9.35 13.51 Consulting (0.00) (0.17) (3.99) (0.00) (4.16) 4. Other Goods 1.82 0.00 0.25 12.67 14.74 (equipment/vehicles/pharm aceuticals) (1.65) (0.00) (0.21) (0.00) (1.86) 5. Maintenance Materials 0.00 0.00 2.10 1.18 3.28 (0.00) (0.00) (2.10) (0.00) (2.10) 6. Training 0.00 0.00 3.39 8.68 12.07 (0.00) (0.00) (3.39) (0.00) (3.39) Total 8.73 17.09 15.28 62.17 103.27 (8.01) (13.37) (14.67) (0.00) (36.05) 1/Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff - 30 - 2/ of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (in US$ million equivalent) Percentage of Appraisal Component Appraisal Estimate Actual/Latest Estimate Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF. National Curriculum 1.22 0.77 2.05 0.00 0.31 3.23 0.0 40.3 157.6 Reform Educational Materials and 2.95 1.10 8.38 6.73 0.46 9.09 228.1 41.8 108.5 Resources Human Resource 4.00 0.94 4.50 2.02 0.18 4.18 50.5 19.1 92.9 Development National Student 1.43 0.15 1.56 1.41 0.08 0.00 98.6 53.3 0.0 Assessment School Nutrition Programs 1.16 0.25 2.06 1.02 0.35 0.00 87.9 140.0 0.0 School Infrastructure 18.98 7.68 21.47 21.17 6.13 21.87 111.5 79.8 101.9 Pilot Program to Assist 1.62 0.38 1.72 0.00 0.02 1.91 0.0 5.3 111.0 Private Schools assisting Low-Income Populations Institutional Strengthening 1.31 0.02 2.22 1.07 0.65 3.28 81.7 3250.0 147.7 Management Information 0.50 0.55 2.98 0.00 0.34 3.85 0.0 61.8 129.2 System Community Participation 0.52 0.46 1.53 0.01 0.09 0.89 1.9 19.6 58.2 Monitoring and Evaluation 1.01 0.13 0.96 0.30 0.06 0.66 29.7 46.2 68.8 Project Coordination Unit 1.91 0.56 1.91 1.41 2.21 1.84 73.8 394.6 96.3 IDB Supervision Fee 0.52 0.00 0.00 0.45 86.5 TOTAL 36.60 11.46 51.92 35.40 10.88 51.25 96.7 94.9 98.7 Note: These appraisal figures are from IDB Project Appraisal Document. The World Bank PAD did not include this information. The actual figures are from the Project Coordination Unit on World Bank, Government and IDB Contributions. - 31 - Annex 3. Economic Costs and Benefits The Project Appraisal Document projects three types of economic benefits: § Internal Efficiency Savings (from reduced repetition or student-years saved); § Benefits from completing basic education (rate of return, ROR, model, a la Psacharopoulos and Ng); and § Unmeasurable higher employment and productivity of basic education graduates For purposes of the ICR, the first benefit cannot be measured by a reduction in repetition rates, which actually increased during the project. However, the benefits of improved educational internal efficiency can be measured by quantifying the financial benefits (savings) of reducing the number of years required to produce a graduate of basic education. During the project this figure declined from 13.5 years (1996/97) to 11.4 years (2001/02). Using a unit cost of US$168 this represents a savings of US$336 per graduate. With more than 100,000 graduates per year of basic education, internal efficiency savings are equivalent to US$33.6 million annually. At the beginning of the project, estimates of the social and private rates of return of the project were calculated, using standard ROR methodology (Psacharopoulos and Ng) and salary data from 1991. Upon project completion, these rates were re-calculated, using 2000 labor market data. Table 1:Estimated Rates of Return, Before and After the Project Beginning of Project End of Project 4th Grade Social Rate of Return 18% 25% Private Rate of Return 20% 30% 8th Grade Social Rate of Return 14% 15% Private Rate of Return 15% 18% Note: beginning of project calculations based on 1991 earnings by educational level data. End of project calculations based on labor market surveys by the Department of National Accounts and Economic Statistics of the Central Bank. It is important to note that the period of project implementation was marked by strong sustained economic growth, which benefited workers at all educational levels. Analysis indicates that those with higher education levels enjoyed greater employment opportunities and greater salary increases during this economic expansion, relative to those with no education. These rates of return are for educational levels overall, and are not directly attributable to project investments. Nonetheless, they indicate strong positive and increasing rates of return to basic education investments. - 32 - Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, 1 FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 09/14/1994 2 OPERATIONS OFFICER, COUNTRY OFFICER 03/06/1995 2 OPERATIONS OFFICER, COUNTRY OFFICER Appraisal/Negotiation 05/15/1995 3 OPERATIONS OFFICER, EDUCATION SPECIALIST, NUTRITION SPECIALIST 10/11/1995 5 OPERATIONS OFFICER, LAWYER, DISBURSEMENT OFFICER, PROCUREMENT OFFICER Supervision 10/19/1996 1 OPERATIONS OFFICER (1) U S 12/13/1997 4 TASK MANAGER (1); S S EDUCATION ECONOMIST (1); FINANCIAL SPECIALIST (1); PROCUREMENT SPECIALIST (1) 05/09/1998 4 TASK MANAGER (1); S S EDUCATION ECONOMIST (1); EVALUATION SPECIALIST (1); ECONOMIST (1) 10/16/1998 3 TASK MANAGER (1); S S CONSULTANT (1); PROCUREMENT (1) 02/13/1999 2 TASK MANAGER (1); EDU. U S ECONOMIST (1) 04/30/1999 2 TASK MANAGER (1); EDU. U S ECONOMIST (1) 07/30/1999 2 EDUCATION SPECIALIST (1); U S OPERATIONS SPECIALIST (1) 11/11/1999 1 TTL (1) S S 10/13/2000 4 TASK TEAM LEADER (1); S S EDUCATION (1); OPERATIONS SUPPORT (1); OPERATIONS (1) 02/17/2001 4 TEAM LEADER-EDUCATION S S (1); EDU- INST. DEVELOPMENT (1); PROCUREMENT (1); OPERATIONS (1) 07/19/2001 6 TASK MANAGER (1); S S - 33 - FINANCIAL MANAGEMENT (1); OPERATIONS (1); SR. ARCHITECT (1); ENGINEER (1); SR. ENGINEER (1) 12/07/2001 5 TASK TEAM LEADER (1); S S PROCUREMENT SPECIALIST (1); EDUCATION SPEC. (1); TEACHER TRAINING SPEC. (1); OPERATIONS ANALYST (1) 06/07/2002 4 PROJECT MANAGER (1); JPA- S S EDUCATION (1); EDUCATION (1); PROJECT ADMINISTRATION (1) 11/01/2002 3 ED. SPECIALIST (1); JPA (1); S S PROCUREMENT (1) 03/07/2003 4 TEAM LEADER (1); S S FINANCIAL MANAGEMEMENT (1); PROCUREMENT (1); EDUCATION (1) ICR (b) Staff: Stage of Project Cycle Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation 467.00 Appraisal/Negotiation 117.0 Supervision 452.0 ICR 41.0 Total 1,077.00 - 34 - Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating Macro policies H SU M N NA Sector Policies H SU M N NA Physical H SU M N NA Financial H SU M N NA Institutional Development H SU M N NA Environmental H SU M N NA Social Poverty Reduction H SU M N NA Gender H SU M N NA Other (Please specify) H SU M N NA Private sector development H SU M N NA Public sector management H SU M N NA Other (Please specify) H SU M N NA - 35 - Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating Lending HS S U HU Supervision HS S U HU Overall HS S U HU 6.2 Borrower performance Rating Preparation HS S U HU Government implementation performance HS S U HU Implementation agency performance HS S U HU Overall HS S U HU - 36 - Annex 7. List of Supporting Documents 1. World Bank Project Appraisal Document, Dominican Republic Basic Education Project, Loan DO-3951A, October 1995 Legal Agreement, Loan DO-3951A, October 1995. 2. Inter-American Development Bank (IDB), Project Appraisal Document for Dominican Republic Basic Education Project, Loan 897/OC-DR, August 1995. 3. Informe Final: estudio evaluativo de la contribucion de las intervenciones prioritarias del Segundo Programa de Mejoramiento de la Calidad de la Educacion Basica sobre indicadores de calidad, eficiencia y matricula educativa, prepared by Grupo Gestion Moderna, SA, August 2003. 4. DRAFT, Project Completion Report, Basic Education Improvement Project, Second Phase, IDB, July 16, 2003. 5. Informe de Resultados: Evaluacion de Medio Termino (Mid-Term Evaluation of Basic Education Improvement Project, Second Phase), IDB, March 2000. 6. World Bank Project Supervision Reports #5-19. 7. All Aide Memoires of World Bank missions. 8. Plan Estrategico de Desarrollo de la Educacion Dominicana, 2003-2012, State Secretariat of Education, SEE - 37 - - 38 -