Documentof TheWorldBank Report No.: 35012 PROJECT PERFORMANCE ASSESSMENT REPORT URUGUAY VOCATIONAL TRAINING AND TECHNOLOGICAL DEVELOPMENT PROJECT(L1594-UR) BASICEDUCATIONQUALITY IMPROVEMENT PROJECT(L3729-UR) SECOND BASICEDUCATION QUALITY IMPROVEMENT PROJECT(L4381-UR) January 23,2006 Sector, Thematic, and Global Evaluation Division Independent Evaluation Group Currency Equivalents (annual averages) Currency New Peso (Nur$) AppraisalYear: US$1 = 5.5 Nur$ CompletionYear: US$l = 174Nur$ ExchangeRate Effective December 17,2001 Currency Uruguayan Peso W P l = US$0.072 1JS.Zl =TNP 13.95 ExchangeRate Effective July 30, 2004 WU1=US$0.0340 US$1- WU29.38 Abbreviations and Acronyms ANEP Administracion Nacional de Educacion Publica COCAP Consejo de Capacitacibn Profesional (Professional Training Council) CODICEN Consejo Directivo Central (Central Directive Council) EFA Educationfor All FTI Fast-Track Initiative to achieve Education for All GDP Gross domestic product ICR ImplementationCompletion Report IDA International Development Association IADB Inter-American Development Association IEG Independent Evaluation Group LATU Laboratorio Tecnologico de Uruguay (Technological Laboratory o f Uruguay) LCC Latin America & Caribbean Region Countries MECAEP Mejoramiento de Calidad de la Educacion Primaria M I S Managementinformation system NGO Nongovernmentalorganization OECD Organization for Economic Cooperation and Development PISA Programme for International Student Assessment PPAR Project Performance Assessment Report PCR Project Completion Report PCU Project CoordinationUnit P A D Project Appraisal Document PRSP Poverty Reduction Strategy Paper QAG Quality Assurance Group S A R Staff Appraisal Report TVET Technical and vocational education and training U K A S UnitedKingdomAccreditation Service UNESCO UnitedNations Educational, Scientific, and Cultural Organization FiscalYear Government: January 1-December31 Director-General, Evaluation : Mr.VinodThomas Director, Independent Evaluation Group : Mr.Ajay Chhibber Manager, Sector, Thematic, and Global EvaluationDivision : Mr.AlainBarbu Task Manager : Ms. HelenAbadzi 1 IEG Mission: Enhancing development effectiveness through excellence and independence inevaluation. About this Report The IndependentEvaluationGroup assessesthe programsand activities of the World Bank for two purposes: first, to ensurethe integrityof the Bank's self-evaluationprocessand to verify that the Bank's work is producingthe expected results, and second, to helpdevelopimproveddirections, policies,and proceduresthrough the disseminationof lessonsdrawnfrom experience.As part of this work, IEGannually assessesabout 25 percentof the Bank's lendingoperations. In selectingoperationsfor assessment, preferenceis given to those that are innovative, large, or complex; those that are relevantto upcomingstudies or country evaluations;those for which Executive Directors or Bank managementhave requestedassessments;andthose that are likelyto generate importantlessons. The projects, topics, and analytical approachesselectedfor assessmentsupport largerevaluationstudies. A Project Performance Assessment Report (PPAR) is based on a review of the Implementation Completion Report (a self-evaluation by the responsible Bank department) and fieldwork conducted by IEG. To prepare PPARs, IEGstaff examine project files and other documents, interviewoperational staff, and in most cases visit the borrowing country for onsite discussions with projectstaff and beneficiaries.The PPAR thereby seeks to validate and augment the information provided inthe ICR, as well as examine issues of special interest to broader IEG studies. Each PPAR is subject to a peer review processand IEG managementapproval. Once cleared internally,the PPAR is reviewed by the responsible Bank department and amended as necessary. The completed PPAR is then sent to the borrower for review; the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System The time-tested evaluation methods used by IEG are suited to the broad range of the World Bank's work. The methods offer both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluationcriterion (more information is available on the IEGwebsite: http://worldbank.org/oed/eta-mainpage.html). Relevance of Objectives: The extent to which the project's objectives are consistentwith the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Possible ratings: High, Substantial, Modest, Negligible. Efficacy: The extent to which the project's objectiveswere achieved, or expected to be achieved, taking into account their relative importance. Possible ratings: High, Substantial, Modest, Negligible. Efficiency: The extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. Possible ratings: High, Substantial, Modest, Negligible.This rating is not generally applied to adjustment operations. Sustainability: The resilienceto risk of net benefitsflows over time. Possible ratings: Highly Likely, Likely, Unlikely, Highly Unlikely, Not Evaluable. lnstitutionalDevelopment Impact: The extent to which a project improves the ability of a country or region to make more efficient, equitable and sustainable use of its human, financial, and natural resources through: (a) better definition, stability, transparency, enforceability,and predictability of institutional arrangements andlor (b) better alignment of the mission and capacity of an organization with its mandate, which derives from these institutional arrangements. InstitutionalDevelopment Impact includes both intended and unintended effects of a project. Possible ratings: High, Substantial, Modest, Negligible. Outcome: The extent to which the project's major relevant objectives were achieved, or are expected to be achieved, efficiently. Possible ratings: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Bank Performance: The extent to which services provided by the Bank ensured quality at entry and supported implementationthrough appropriate supervision (including ensuring adequate transition arrangements for regular operation of the project). Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower assumed ownership and responsibility to ensure quality of preparationand implementation,and compliedwith covenants and agreements, towards the achievement of development objectives and sustainability. Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory. ... 111 Contents PrincipalRatings ................................................................................................................ v Key StaffResponsible ........................................................................................................ v Preface .............................................................................................................................. .. vi1 Summary ........................................................................................................................... ix 1 Background . .................................................................................................................. 1 Bank Sector Strategy................................................................................................ 2 2 . Objectives andImplementationoftheVocationalTrainingandTechnological DevelopmentProject (Ln.1594-UR) ........................................................................... 3 Results of the Vocational Training and Technological Development Project.........5 Train adults to improve employment prospects -partly achieved .......................... 5 Providing technological research and quality control to strengthen export capacity - achieved............................................................................................ 5 3 BasicEducationQualityImprovementProjectsIandI1 . ......................................... 6 Basic Education Quality Improvement Project I (Ln. 3729-UR known as MECAEP I)........................................................................................................ 6 Basic Education Quality Improvement Project 11(Ln.4384-UR known as MECAEP 11)....................................................................................................... 6 4 ResultsofMECAEPIandI1 . ...................................................................................... 9 5 Ratings . ........................................................................................................................ 20 Project Outcomes................................................................................................... 20 Institutional Development Impact.......................................................................... 21 Sustainability.......................................................................................................... . . 21 Bank Performance ................................................................................................. 22 Borrower Performance .......................................................................................... 23 6 QualityEducationfor All inUruguay . ..................................................................... 23 Maximizing Poor Students'Achievement.............................................................. 24 Balancing the Costs and Benefits of Insewice Teacher Training. SomeInsight from Mission Observations .............................................................................. 25 This r e p o iwas preparedbyHelenAbadzi who assessedthe project inFebruary 2005 The report was edited by WilliamB .Hurlbut.andPilar Barquero provided administrative support . . iv 7 Lessons . ......................................................................................................................... 26 References ......................................................................................................................... 27 Annex A Implementationofprojectcomponents . ........................................................ 31 Annex B SupplementalTables . ....................................................................................... 35 Annex C Statements andIssuesRaisedinMissionInterviews . ................................... 39 Annex D BasicDataSheet . .............................................................................................. 41 Figures Figure 1 CompletedBankeducation projects inUruguay .. ................................................. 3 Figure2 Percentage of sixth gradersperforming sufficiently inlanguage byincome level ................................................................................................................................... 13 Figure 3 ..Percentage o f sixthgradersperforming sufficiently inmathby income level.. 13 ................................................ 14 Figure 5 Figure 4 Number ooffstudentsattending full-time schools low income sixth graders infull-time schools inlanguage ............15 Figure6 Progress o flow income sixth graders infull-time schools inmath..................16 Figure7 Grade 1repetitionratesbyincome level ........................................................... 17 Figure8 Grade 1repetitionbytype o f school ................................................................. .. ..Progress 17 Tables Table 1 Objectives o f the Basic Education Quality Improvement projects ....................... Table 2 Preschool enrollments by income quintile .. ............................................................ 9 6 Table 3 Student performance insixthgrade (2002) according to mother's educational . attainment andpreschool attendance . ......................................................................... ........................... 10 11 Table 5 Full-time schools constructed according to socioeconomic environment ..........14 Table 4 Sixth grade test store changesby socioeconomic background Table 6 Average repetition rates inprimary schools .. ....................................................... 16 V PrincipalRatings - --- -_*___I. k P - - --_a__ ICR* ICR Review* PPAR I". __I(--_ l--.--"ll__-~~l ___--_.l_~ - ~ _ . Vocational Trainingand TechnologicalDevelopmentProject (Loan 1594-UR) - ~ - - __ 1__-1 l^l^lll"lll_ ---I- Outcome Not Rated Not Rated Satisfactory Institutional Not Rated Not Rated Substantial DevelopmentImpact Sustainability Not Rated Not Rated Highly Likely Bank Performance Not Rated Not Rated Satisfactory Borrower Not Rated Not Rated Satisfactory Performance Basic Education QualityImprovementProject (MECAEP I; Loan 3729-UR) - - --____.__ _______I Outcome Satisfactory Satisfactory Satisfactory Institutional Substantial Substantial Substantial DevelopmentImpact Sustainability Highly Likely Likely Likely Bank Performance Satisfactory Satisfactory Satisfactory Borrower HighlySatisfactory Satisfactory Highly Satisfactory Performance Second Basic Education QualityImprovementlll.ll-- Project (MECAEP 11; Loan 4381-UR) -----.---. ---- -- Outcome Satisfactory Satisfactory Satisfactory Institutional Substantial Substantial Substantial DevelopmentImpact Sustainability Likely Likely Likely Bank Performance Satisfactory Satisfactory Satisfactory Borrower Satisfactory Satisfactory Highly Satisfactory Performance -- * The ImplementationCompletionReport(ICR) is a self-evaluationbytheresponsibleoperational divisionofthe Bank. The ICR I__I Review is an intermediateIEGproductthat seeks to independentlyverify the findings ofthe ICR. Key StaffResponsible --- .a=- Task Manager/ Division Chief/ Country Director ~. ._i Leader Sector Director ~ Vocational Trainingand TechnologicalDevelopment Project (Loan. 1534-UR) Appraisal Jasdip Singh MiguelSchloss S. M. L. Van der Meer Completion J. St. Germain Douglas Keare Robert Picciotto Basic Education Quality ImprovementProject (MECAEP I; Loan 3729-UR) Appraisal Juan Prawda Ping Loh Julian Schweitzer Completion MyrnaAlexander William Experton RicardoSilveira (Acting) 1 _ _ _ 1 ~ - Second Basic Education QualityImprovement Project (MECAEP /I; Loan 4381-UR) Appraisal RicardoSilveira DonaldWinkler MyrnaAlexander Completion RicardoSilveira Ana MariaArriagada Axel van Trotsenburg vii Preface This i s the Project Performance Assessment Report (PPAR) on three education projects inUruguay. The Vocational Training and Technological DevelopmentProject (Loan 1594- UR)was approved for aUS$9.7millionloaninJune 1978. The loanclosed onJune 30, 1986, after extensions totaling 36 months; US$1.2 millionwas canceled. The Basic EducationQuality Improvement Project (MECAEP I;Loan 3729-UR was approved for a loan o fUS$31.5 million inJanuary 1994. The loan closed as scheduled on June 30,2001; US$O.9 million was canceled. The SecondBasic EducationQuality Improvement Project (MECAEP 11; Loan. 4384-UR) was approved for a loan o fUS$28 million inJuly 1998. The loan closed on February29,2004, after extensions totaling 7 months; US$0.25 millionwas canceled. The projects inUruguay were selected for assessment inorder to studythe effectiveness o f Bank strategy ina middle-income country that i s expanding its system to low-income populations. The assessmentcontributes to backgroundwork for an ongoing IndependentEvaluationGroup (IEG) study o fthe Bank's assistanceto basic education. The PPARi s based on the following sources: Project and Implementation CompletionReports (ICRs), StaffAppraisal Reports (SARs), LoanAgreements for the projects, andproject files, particularlythe supervision reports. An IEGmissionvisited Uruguay inFebruary2005 to interview officials andbeneficiaries, observe instruction in schools, and collect other pertinent information. Fieldvisits took place invocational training institutions, in-service training venues, and inschools o f Montevideo, and the departmentsofFloridaandColonia. The author thanks the government officials who received the mission for their extensive cooperation. Following standard IEGprocedures, copies o fthe draft PPAR were sent to the relevant government officials and agencies for their review and comments. No comments were received. ix Summary Since the 1970s, Uruguay has implemented three projects inthe education sector. e The Vocational Training and Technological DevelopmentProject (Ln.1594- UR)was approved for a US$9.7 millionloan inJune 1978. The loanclosed on June 30, 1986, after extensions totaling 36 months; US$1.2 millionwas canceled. e The Basic EducationQuality ImprovementProject (MECAEP I; Ln.3729- UR)was approved for a loano fUS$31.5 millioninJanuary 1994. The loan closed as scheduled on June 30,200 1;US$O.9 millionwas canceled. e The Second Basic EducationQuality Improvement Project (MECAEP 11; Loan4384-UR) was approved for a loano fUS$28 millioninJuly 1998. The loan closed on February 29,2004, after extensions totaling 7 months; US$0.25 million was canceled. All three projects aimed at improving educational quality andperformance of studentsor trainees. The projects were implemented as planned, most o fthe activities were carried out, and objectives were substantially met. The institutions supported by the Vocational Education and TechnologicalDevelopmentProjecthave fulfilled their purposes as originally intended. The Professional Training Council (COCAP) didnot train as many learners as expected, but it produces quality courses o f some demand by learners andby organizations willing to pay. The Laboratorio Tecnologico de Uruguay (LATU) became able to conduct research andprovide to Uruguayanindustriestechnical assistance that has helpedincrease the competitiveness o fUruguayanimports. Uruguay achieved Education for All inthe 1990s, and in 10 years o fBank support to primary education, the country has consolidated its progress. Primary education coverage and completionrates have reached 98 percent, while dropout rates are less than 1 percent. The parts o fthe systemmost likely to benefit the poor have expanded. In2004, preschool coverage reached 90 percent o f 5-year olds and 85 percent of 4-year olds. Special schools catering to the disadvantaged through longer days and enriched programs increased from 58 in 1996 to 106by 2004. Overall, student achievement has improved. Between 1996 and 2002, the share o f sixth-graders reaching a "sufficient" level o f performance rose fiom 57 to 66 percent in language and from 35 to 48 percent inmath. The achievement gap between students inthe lowest and highest income levels has beenreduced. In 1996, the `pass' rates o f the lowest and highest income groups had a gap o f48 percentage points inlanguage, butby 2002 it hadbeenreducedto about 35 percentage points. Similarly inmath, the `pass' rates o fthe lowest and highest income groups had a gap o f 50 percentage points in 1996, butby 2002 it hadbeenreduced to about 35 percentage points. Preschool attendance was found to be effective inreducingrepetition among low-income first graders. However, national repetitionrates have remained highingrades 1and2 (19.9 and 14percent in2002), and despite some improvement, they have not beenreducedto the target o f 10percent. According to teachers interviewed, the most commonreasonfor repetitioni s failure to learn reading, a skillprescribed bythe grade 1curriculum. The government has adopted a teaching methodology that inother countries has been shown to prolongthe process o freading acquisition among disadvantaged students. Alternative X methods that teach the readingbasics more efficiently might help reduce further the social inequitiesineducation as well as the expenditures associatedwith grade repetition. All three projects largelymettheir targets, andtheir outcomes are ratedsatisfactory. The MECAEP Iand I1projects succeededinincreasing equitythrough improved participationo fthe most disadvantaged students inpreschool education and full-time schools. The Vocational EducationandTechnological DevelopmentProject achieved its long-term labor-market support goals. Institutionaldevelopment i s ratedsubstantial for all projects, becausethe institutions expanded their capacity to matchimplementationneeds. Sustainabilityi s rated likely; the net benefits o f the vocational education project have provedresilient over time, and the MECAEPprojects have resultedinlong-term achievement increases. For all projects, Bank performance i s rated satisfactory. Borrower performance i s ratedsatisfactory for the vocational education project and highly satisfactory for the basic education projects. This assessment confirms anumbero fIEGlessons from the education sector: 0 Preschool attendanceoffers significant academic benefits to low-income students in terms o freduced grade repetition as well as long-term academic achievement (para. 4.4) 0 A longer school daymay offer important academic andsocial benefits to disadvantaged students, includingimproved test scores. To maximizeperformance inhighergrades, however, studentsinlower gradesmightspendpartofthe extra time practicingthe basic skills that serve as prerequisites for hrther learning (paras. 3.10,4.13, and 6.5). 0 Quality o f education depends on a strong and functional supervisory chain. If government policies and learningactivities are to be carried out effectively at the school level, teachers must be supervised closely by knowledgeable andinterested staff. Although supervision i s often costly interms o f salaries and vehicle support, it i s a prerequisite for improved learning outcomes (paras. 3.8 and 5.10). 0 Project efficiency may be enhanced through instructional methods that enable most o fthe students to achieve educational objectives within the prescribed instructional time. Efficient instruction may bemore important for disadvantaged children, who receive less home support and who often have higher dropout and grade repetition rates. Furtherresearch and experimentation would help determine which instructional activities are more cost-effective inthe Uruguayancontext (paras. 5.3 and 5.9). 0 Vocational-technicaleducation project appraisals may overestimate the number o f likely graduates and not adequately take into account dropout rates or changing demand for training. Appraisals o f future projects should take into account country experience on dropout rates as well as demand changes duringtimes o f economic hardship (paras. 2.2-2.4,2.6) Vinod Thomas Director-General Evaluation 1 1. Background 1.1 This document presents evidence regarding the achievement o f objectives o fthe education projects implemented inUruguay between 1978 and 2005. It also presents evidence regardingthe extent to which Uruguay has achieved its goal o fproviding quality and equitable education to all primary-school students. 1.2 With aper capita income o fUS$3,790,' Uruguayi s amiddle-income countrywith a long-termpolicy o f social protection for its populationo f 3.4 million. Ithas hadthe highest per capita spendingrates on social sectors among the countries o f the Latin America and Caribbean Region(LAC), thoughmost of it hasbeendevoted to social safety nets. The country's traditional ability to provide social services to its citizens largely stems from its industrialsector that was vigorous throughthe 1990s. Quality control and research have beenkey factors inexportingproducts such as wood, wool, and dairy. To provide workers for the industry, vocationaltraining institutions have complemented the formal education system since the 1970s. 1.3 InUruguay, free anduniversalaccessto all levels ofeducation was introduced early inthe 20thcentury. The country attained universalprimary education inthe 1960s, andits adult literacyrate is 98 percent. Educationspendingtends toward eq~ity,~ 85 and percent o fprimary schools are p ~ b l i c This i s important becausemost children in . ~ Uruguay tend to beborn inlower-income households; 20 percent o f the poorest families have 42 percent o f the country's childrenwhile the richest families have 7 percent o f the children5Primary-schooldropout i s negligible (1,668 students in2003), butpoor students are prone to low performance and highgrade repetition.6 In2002, the average primaryschool repetitionratewas 10.3 percent, compared to 5.3 percent inArgentina and 5.4 percent inChile. InUruguay, repetitiontends to be concentrated inearly grades; the average for the first two years was 17.4 percent in2000.' The country has sought World Bank assistance to prepare young children for school and improve internal efficiency. 1. Atlas method, 2002 (World Bank 2004a). 2. Public educationinUruguayis the responsibility ofautonomouspublic entities, andnot the Ministry ofEducation andCulture, which ismainly responsiblefor cultural activities. Primary and Secondaryeducationare managedbythe National Administration for Public Education(Administraci6n Nacional de EducacionPdblica, ANEP), which receives guidanceand support fromthe CentralDirective Council (CODICEN) ofANEP. CODICEN is constitutedoffive membersappointedbythePresidentandconfirmed byCongress. Itsdependencies includecouncils for primaryand secondary education, teacher training, andVocational Education(COCAP) for short-durationtraining. The secondary- and post-secondaryleveltechnicalschools (Universidad de Trabajo) provide semi-formaltraining administeredbythe Council ofVocational and TechnicalEducation.Vocational educationconsists of shorter courses for adults. 3. For the years 1999,2002-2003, the Gini coefficient for the education sectorwas 0.27 and for healthwas 0.44 to .45. About 51.6 percentofthe educationbudgetwas directed at the poorest20 percentofthe population. (OPP 2004, p. 17- 18; Table B-9). For budgetarydata on Uruguay see tables B-6 to B-9. 4. In2003 there were atotal of 2,834 schools (15 percentprivate) that accommodated about 469,114 students(ANEP Statisticalannual 2003). 5. EncuestaContinuade Hogares, InstitutoNacional de Estadisticas- 1996. 6. ANEP-MECAEP, 2002~. 7. World Bank 2002a. 2 1.4 Despitethe expense involvedineducational programs, public expenditure on education remains relatively low. Itwas about 3.6 o f GDP in2004, considerably less than the 4.5 averagepercentage inLAC and lower than the 5.3 percent average o fhigh- income countries. Quality o f education interventions for the poor are neededat all levels o f the system.* Bank Sector Strategy 1.5 World Banklendingfor education inUruguay started in 1978 with vocational- technical education. Altogether, the Bank has financed four education projects in Uruguay, the first three o fwhich are the subject of this report (Figure 1, Annex Table A- 4). Bank assistance has been focused on supporting the government's educational objectives and providing economic and technical advice to vocational education inthe 1980sand to primary education since the 1990s. 1.6 With World Bank support, the government has implementedsince 1997a 10-year educational reformthat focuses at the primary level on improving poor children's academic achievement, psychomotor, and social skills, and on reducing repetition rates. Universalization o fpreschool was considered necessaryto improve the skills o f low- income childrenand lower their particularlyhighrepetition rates. To further support the achievement ofthe poor and limit harmful environmental influences, the government developed the strategy o f keepingthem inschool for 7.5 hours. A series o fthree Basic EducationQualityprojects (known as MECAEP I,11, and 11) has focused on carrying out these goals. 1.7 Uruguaywill continue to receive financing from the Inter-AmericanDevelopment Bank (IADB) for secondary, technical, and some higher education. Inparticular, IADB has supported the government's reform at the secondary andhigher education level and i s currently financing a project on the modernizationof secondary education andteacher training (UR-0132, US75.0 million, FY2001A). lo The country also receives some bilateral aid on technical education issues from Japan and Germany. 1.8 Dueto the economic downturnofareacountries Uruguayfaces the challenge of sustaining the current investments andimproving them." The education sector has received assistance from the Public Services and Social Sectors StructuralAdjustment Loan and Special StructuralAdjustment Loan Project (Ln.7104, P078726, FY03) inan effort to protect educational investments from adjustments necessary to overcome the effects o fthe 2001 economic crisis. (See more inthe Sustainability section.) 8. World Bank 2005. The percentage o fthe education budget devoted to primary and preprimary public education i s about 44.7 percent, secondary education about 34.5 percent, tertiary andhigher education 20.7 percent, andhigher education only 15.1 percent). 9. MECAEP 2004. 10. Inter-American Development Bank 2000. 11. Incontrast to many L A C countries that have decentralized education and given more regional autonomy, Uruguay remains fairly centralized. One reasonmaybe the relatively small size o f the country and its small andrelatively homogeneous population. (PREAL. 2001). 3 1.9 How effectivehas the Bank financing beeninimproving the quality o f education inthe subsectorsithas supported?Havelearninggains andlower repetitionrates accompanied the consolidation o fprimary school enrollments? Inaddition to assessing the completedprojects, this report presents evidence regarding these questions. Figure1.CompletedBankeducationprojectsinUruguay Vocational Education& Technological Development Project Basic EducationQuality Improvement Project I .-al2 cI 0 (MACAEP I) Basic EducationQuality n. trnprovementProject II (MECAEP II) Basic EducationQuality Improvement Project111 (MECAEP Ill) 8~G38 8 8 8 G ~ ~ ~?9* 0 ~ ~ * 6 * f f 8 8 8~ 08 8 ~ 0 8 4 70 8 70 70 8 8 8 $0 $0 $0 ~oo~oo~07 6 Year 1.10 The three projects under review focused on improving access and quality o f education inthe subsectorsthey supported. Their implementation experience and results are presented below. 2. Objectivesand Implementationof the VocationalTraining and TechnologicalDevelopmentProject (Ln.1594-UR) 2.1 The project was to enhance the competitiveness o fUruguayan exports by increasing labor productivity, improving product quality, andpromoting the use o f technologies well suited to the country's requirements. Project objectives were: (a) Train adults to improve employment prospects (originally stated as `Finance the physical facilities and technical assistancenecessary for the Vocational Training Council (COCAP) to beginoperations") and (b) assist the government indevelopingdomestic capability inthe areas o f technological research, dissemination o f information, and technical assistance to entrepreneurs insupport o f its export-oriented industrial development strategy. The project was to finance (a) the expansion o fthe Laboratorio Tecnol6gico o fUruguay (LATU), a parastatal organizationcarrying out these hnctions since 1965 and (b) establish the Professional Training Council (Consejo de Capacitacibn Profesional - COCAP). 2.2 Both components (which were not linked) suffered long delays inconstruction andprocurement due to a lack o f advance preparation. By 1982 only 4 percent o fthe loan had disbursed, andthe project neededextensions totaling four years. Despitevarious disagreements andproblems, most expected activities were carriedout (see Annex Table A-1). Bothinstitutionshadconsiderable continuity instaffandmanagement, andthis made it possible to gather information 19 years after project closing. The managers o fthe 4 two institutions showed the IEGmissionthat the buildingsas well as much o f the old equipment were still beingusedand ingood condition. 2.3 Vocational Training Council (Consejo de CapacitacidnProfesional - COCAP). In an era o fmanpower planningandsupply-based training, the project helped set up an institution offering demand-based short vocational training courses, mainlyto industry workers. For the first decade o f its operation, the institution designedcourses o f short duration for various industrialoccupations. It expanded, movedinto largerpremises, and opened two provincial centers COCAP has usedpart-time instructors who work inthe industryduringthe day, payingthem acceptable salaries. In-planttraining inparticular was to meet the needs o fpublic ports, railways, electricalpower, petroleum, and fishing agencies as well as several largeprivate manufacturers. Project completiondocuments in 1988 show that COCAP was meetingindustrydemand for in-servicetraining and recuperating about 100percent o frecurrent expenditures (Table B-l).'* C O O was to finance its work partly throughan export tax, butneededlegislation was never enacted, and the expected hndswere never received. Instead, ad hoc subsidieshavebeengivenover the years. 2.4 The appraisal overestimated the capacity o fCOCAP to carry out the expected course load, labor market demand, andthe government's politicalwill to impose export taxes for the operation o fthe institution. The institution raninto multipleproblems in 1997 when the National Employment Council ceasedoperations andno longer paid for COCAP courses. Halfthe staffwere laid off, the institution lost a newbuildingbought for its operations, and it has returned to the original Bank-financedpremises. Nevertheless, the institution functions as intended, albeit with a reduced scope. The missionvisitedeveningcourses of COCAP, where interviewswere heldwith three groups o f adult learners (two students studyingautomaticity, 13 students o f office work subsidized bythe NationalEmployment Council, 14 ingraphic design). All students indicatedthat the training i s usehl and o fgood quality. The students o f automaticity reported that their employer paid for the course (12,000 pesos for six months), but the unemployedstudentswere subsidized at 70 pesosper day to take courses. The students pointedout that the selection o f courses offeredby COCAP in2005 was limited. Several made it clear that they didnot expect work but took the courses for personal enhancement and could afford to wait for suitablypayingwork. 2.5 Technological Laboratory of Uruguay (LATU).For this parastatal companythat has certifiedthe quality o f exported goods since 1965, the project financed equipment, fellowships, and technical assistance to improve quality control means and find new manufacturingprocesses through four pilot plants.LATUusedits own funds to buildthe new premises on the basis o fplans acceptable to the Bank (Annex Table A-1). These are located on 700 hectares, surrounded by an industrialpark. The project design benefited l3 from o f a consultant who envisioned pilot plantsand laboratories inneeded fields. Ithas 12.Nevertheless, the institutionhas largelyrecuperatedrecurrentexpendituresover the years (TableB-1).Until 1990, the institutionrecuperated83 percent ofrecurrentexpenditures. Accordingto reports of COCAP staff, currentcost recoveryrateis about 50 percent, andthe institutiongets ad-hoc subsidies insteadof a steadyincome fromtaxes as plannedduringappraisal. 13. The appraisal of thisprojectreceivedtechnicalassistancethroughan industrialdevelopment project(Ln. 1176-UR; US$20.8 million). 5 developed into a large organization with linkages to other international agencies o f quality control and with broad-ranging activities inthe investigationand training for improvedproductionmethods. L A T Ureceives hndingthrough taxes on exports and generates considerable income. RESULTSOF THE VOCATIONALTRAINING AND TECHNOLOGICAL DEVELOPMENT PROJECT Train adultsto improveemploymentprospects-partly achieved 2.6 Staffinterviewedinthis organization reported that over the years COCAP has providedtraining to more than 100,000 people (versus approximately 300,000 according to appraisal targets; Annex Table A-1). Some in-plant training was carried out for various state agencies, such as petroleum and water. There was more demand, however, for training youth andunemployedmiddle-aged people out o fwork who may be subsidized bytheNationalEmployment Council to take courses (Annex Tables B-1andB-2). For example, in2000, 1,130 persons studied in 101courses and in2001,933 studiedin84 courses with a dropout rate o f about 10percent. However, employability o fjobless trainees was limited. Inan informal tracer study conducted by COCAP,about 28 percent had obtained work. Providingtechnologicalresearchand quality controlto strengthenexportcapacity - achieved. 2.7 Staffinterviewedconsider the Bank financing a great achievement and a leap forward for the institution. The project design and assistance gave the impetus for the l4 development o f a world-class institute ofquality control and export certification. LATU i s accredited by the United Kingdom Accreditation Service (UKAS) to carry out over 200 analyses and collaborates with other international quality control agencies.l5Countries importingUruguayanproducts thus accept the quality control ofwool carried out by LATUwithout additionaltesting. The pilot plantshelp findbetter ways o fprocessing materials (such as tomato sauce, wood chips, uniformity o f cheese production) for exports. l6 2.8 LATUhasdeveloped into areference institution for LAC; ithas credibility and does international consulting. In addition to Bank financing LATUhas receivedhelp from Japan and the Inter-AmericanDevelopmentBank as well as a grant administered by the World Bank (InfoDev grant IIV017). It employs about 300 people. LATUalso supports small andmedium-size companies and incubationo f new businesses and carries 14.Audit reports showednoproblems(AuditoriaMariiio Montevideo: Estadode Fuentesy Usosde fondos, 1983) 15. Otherinternational services supportingLATUincludeSQS (Switzerland, OQS and OVO (Austria), DGQ (Germany), INMETRO(Brazil), CSIRO(Australia), INTI(Argentina), SIRIM(Malaysia). LATU: EspacioCiencia (undated informationbulletin) 16.Also for cellulose, pulp, paper, andplasticswww.latu.org.uy LATUfunds exports from alevy of0.3 percenton the value ofimportsof specifiedprojects, and 1percentofthe CIF value ofcertain goods andmachinery. 6 out industrial training, such as productionprocessesfor ruralwomen and then offers expositions. Its multipleactivities include an expositionpark for children. 3. BasicEducationQuality ImprovementProjectsIand I1 BasicEducationQualityImprovementProjectIan. 3729-URknownas MECAEPI) 3.1 The project was to improve the quality, equity, and efficiency o fthe primary education system (Table 1). It constituted the first large-scale effort to support primary education within a context of Uruguayansocial policies andto concentrate resources to the lower-performing schools. Its components aimed at distributingtextbooks, training teachers, and developing a sample-based standardized achievement test. Through construction, training, andmaterials, the project also aimed at increasing the net enrollment o f 4- and 5-year old childreninpreschoolfrom 51 to 55 percent andreducing the repetitionrate for grade 1from 17.8 to 11.9 percent. (See Annex Table A-1). ~ Table 1. Objectivesofthe BasicEducationQualityImprovementprojects Objectives I Components 0 Expand preschool coverage inareas with unsatisfied j Improvingthe efficiency, quality, and equity o f basic needs; primary education, through textbooks, education 0 Enhance the quality o f preschool education to improvement projects, and teacher training increase elementary school readinessand reduce 3 Expandingpreschooleducationaccessandquality repetition inthe first two grades; through construction, teacher training and materials Enhance sector productivity and strengthen overall 3 Institutionalstrengtheningandefficiencythrough sectoral management establishment o fa management information system and decentralization actions. Expand coverage and improve quality o fpreschool j Universalization o fpreschool education and elementary education; 3 Expandingaccesstofull-timeschools Achieve greater social equitybytargeting the expansion o f the new (preschool and elementary) 3 Projectadministrationandmonitoring. full-time school model for socially disadvantaged - children. Source: Technical and legal documenfafionof respectiveprojects 3.2 The 6.5-year project was implementedas scheduled, andnearlyallthe planned activities were completed. Lack o f stakeholder consultationduringappraisal created some mishust among teachers and administrators, which project authorities had to overcome. Early inthe life o fthe project auditing and accounting issues arose, and on occasion ineligible expenditures were financed. These problems were addressedquickly, and audit reports have been satisfactory since 1996 despite some delays. BasicEducationQualityImprovementProjectI1(Ln.4384-UR knownas MECAEP11) 3.3 As with MECAEP I, objectives were concerned with quality and equity. MECAEP I1continued to implement the educational reform introduced inMECAEP I, but also included the `hll-time' school model. A curricular design, special training, and 7 support were developed to accommodate disadvantaged children ina school day that lasts 7.5 hours andincludes three meals. As an enrichment activity, 28 hll-time schools also introducedbilingual education inPortuguese (inborder areas) andEnglish, teaching half the time ineachlanguage. Theproject also aidedregular schools inlow-income areas. LikeMECAEPI, emphasized teacher trainingandintensiveteacher supervision bya it cadre o fwell-trained inspectors who are responsible for about 14 schools each and visit themfreq~ently.'~The project carried out several studies (see References). Most activities under MECAEP I1were completed (Annex Table A-2.) 3.4 MECAEP Iand I1hadmany common activities and implementation issues, which are described below with achievement o f specific targets where applicable." 3.5 Classroom construction and rehabilitation. The projects faced difficulties with land acquisition. Thus, the government preferred to attach classrooms to existingschools rather thanbuildnew ones. MECAEP Ibuilt 87 preschool classrooms (exceeding a target o f 71) rehabilitated another three eventually increasing coverage to 44,000 additional students; 50 primary-school classrooms were also built. MECAEP I1built 109preschool classrooms (rather than the targeted 200) because the project populationwas reduced. It also decided to builda smaller number o f 120new classrooms (11new full-time schools) rather thanthe projected 280. Though designs were decided centrally, efforts were made to the needs o f specific users, such as orienting doors or changing the number o f bathrooms. The localresidents took care o fthe construction andnotifiedMECAEP if there were problems. 3.6 Textbooks.MECAEP Idistributed4.2 milliontextbooks, exceeding targets by 68 percent (Table B-2). MECAEP I1complemented the effort and distributed many instructionalmaterials (Table B-3). Textbooks were made according to precise specifications for a unit cost o f about US$1.2, teachers andparents were consulted, and the process was eval~ated.'~Students receive them free andreturnthemat the endo fthe year. Publishers own the authors' copyrights and have sold the textbooks inthe market for about US$lO. Though textbooks became available for all studentsin 1995, they have not been gettingreplaced as they wear out. A 2000 study showed that 80.3 percent o f schools hadthe neededbooks. Low-income schools tended to have more scarcity, and 8 percent o f schools had a severe shortage.20This means that students must share books and are often unable to take them home to study. The situationhas improved. The IEG mission found only one school whose staff stated that they had an adequate numbero f textbooks. However, the textbooks are usually unavailable for sale inthe market, so even parents who can afford them cannot buy them. 3.7 School improvementprojects. This mechanism o fresponding to school needs through teacher initiatives started in 1994and has become very popular. MECAEP I awarded 931 projects and the practice continuedMECAEP I1and 111.In2005, schools 17. There are 18,644 teaching staff and235 inspectors, bringingthe average for a six-grade schoolto about 14 school per inspector. 18.World Bank2002b. 19. ANEP 1998a. 20. ANEP 2003f. 8 have 320 projects to implement. In 1995,36 percent o fthe projects were inscience, while in1997,42 percentwere inlanguage.?'Theprojects havebecome moreinstructionally relevant. Frequent examples are development o fnewspapers, plant nurseries, web pages, art history and critique. Evaluations o f school improvementprojects have shown some positive outcomes.22Teachers thought that they improved: activities inclass (73 percent), reflectionregardingschool reality (75 percent), generated new forms o fwork inthe school (70 percent), showed that teachers' work transcends the classroom (60 percent). Teachers reported that they were more likelyto collaborate andparticipate (57 percent), more capable o fworking ingroups (55 percent), closer links with other teachers (55 percent), less absenteeism (13 percent). For students, the school improvementprojects reportedly improvedvalues (70 percent), school motivation (67 percent), relationship with families ina specific class (56 percent), relationshipwith families inthe school (50 percent), relations with community (46 percent), discipline (47 percent), learning difficulties (43 percent), absenteeism (24 ~ercent).?~A studyhas shown apositivethough limitedrelationship between academic achievement to the executiono f school improvementprojects (Table A-3). 3.8 Teacher training. MECAEP Icarried out extensive teacher training, reaching 17,000 teachersby 2000 and surpassing targets. However, training inlarge auditoriums that consisted mainly o f lectures didnot have a sufficient impact on classroom behavior. InMECAEP 11, training was limitedto teachers offull-time schools (extensive courses of 150hours), preschools, and some teachers ofthe teacher training institutions. Using salaries o f teacherswho hadbeenabsent, it was possible to pay for teachers to attend training on Saturdays. Supervisors and inspectors gave close follow-up to teachers to help them modifytheir teaching behaviors. Convincingteachers o f the needfor change was found to be an important component for the improvement o f teacher training. Specific traininghas taken place on the full-timeschool meth~dology~~to emphasize the skills o f teachers taking the children's knowledge and expanding it, gettingthemto think about and question issues such as why teeth fall out, why people gain weight. Linkageswith the community have been an important part oftraining. 3.9 Achievement tests.An achievement test unitwas created through MECAEP I (Unidad de Medicion de Resultados Educativos) in 1995. Through a participatory process,25it developed and administered tests approximately every three years: to all sixth graders in 1996, a sample o f third graders in 1998, a sample o f sixth graders in 1998, an evaluation o f fourth grade andpreschoolachievement as well as grade 6 in2002. About 20,000 educators receivedtraining inthe interpretationo f test results, andthis activity increased awareness o ftesting outcomes under MECAEP 11. 3.10 Full-time schools. Such schools had existed inUruguay since 1992 but were intended for special education. Students spend four hours inthe morningon the regular 21. ANEP 1999b, p. 31 22. ANEP 1999b,p. 38 23. ANEP 1999b, p. 39 24. ANEP-CODICEN-CEP-MECAEO-BIRF' 1999,2004. 25 Benveniste 2000. 9 academic curricula and 3.5 hours inthe afternoon on sports, evaluation and group organization, andworkshops (or projects) on language, science, and math. There i s no supervised homework time duringthe school hours. Children are expected to go to bed soon after returninghome, so homework i s often limited to information gathering. The schools that teach Englishor Portuguese do so by teaching part o fthe curriculum inthese languages, including math. Full-timeschools are larger buildingsthan regular schools and have recurrent expendituresthat are about 78 percent higher than regular schools (Table B-4). Teachers work two shifts, but are paid only the basic salary for the second. MECAEP I1carried out promotionalcampaigns to enrol childreninthese schools as well as inpreschool. 4. Results of MECAEPIand I1 4.1 These two projects were implementedinsequence and their objectives partly overlap (Table 1). Objectives related to preschoolexpansion and full-time schools were infact meansto achieve improvedperformance, increasedequity, andreducedrepetition. Grouped together, the objectives o fMECAEP Iand I1are presentedinterms of: (a) expanding preschool coverage inareas o f unsatisfied basic needs, (b) enhancing the quality o fpreschool and primary education, (c) achieving greater social equitythrough full-time schools, (d) reducinggrade repetition, and (e) improving sectoral management. (a) Expandingpreschoolcoveragein areas of unsatisfiedbasic needs achieved - 4.2 Preschool education has expanded considerably, and in2004 there were about 85,628 preschool students. According to the 2002 household survey, coverage o f 5-year olds reached 90 percent; coverage o f4-year olds has quadrupled since 1994, and was about 85 percent in2004. Increases inenrolment rates were most noted for lower-income students. Between 1996 and 2001, the attendance o f five-year olds inthe lowest income quintile improvedby 13 percentage points, while the attendance o f four-year olds improvedby 23 percentage points (from 36 to 59 percent; Table 2). Table2. Preschoolenrollmentsby incomequintile Source:ANEP (AdministracionNacional de Educaci6n Ptiblica) Continuoushouseholdsurvey - representativesample - of urban population(5,000 or more inhabitants). (b) Enhancingthe quality ofpreschoolandprimary education-partly achieved 10 4.3 Preschool education was to improve inquality inorder to increase primary school readiness and reduce repetitioninthe first two grades. It i s unknown whether this was achieved becauseno tests measuredwhether disadvantaged studentshad a higher commando f age-appropriate language, motor, or psychosocial skills between 1996 and 2002 or whether attendance improvedthese skills. 4.4 Nevertheless, preschool attendance seems to reduce failure; the repetitionrate o f low-income first graders who didnot attendpreschool in 1996-2001 was 47 percent compared to 27 percent for similar children who had attended A study showed that children were more likely to repeat grades ifthey hadnot attended preschool. Childrenwho attended preschool had a lower tendency to repeat at least one year, evenwhenmothers hadhigher education (Table 3). Also, childrenwho had attendedpreschool hadhigher test scores ingrade 6. Perhapsthere i s no difference among preschool models inUruguay. Second graders who had attendedthe preschool model developed through MECAEP scored about the same as a national sample o f second graders who had attended traditional kindergartens. *' Table3. Studentperformanceinsixth grade (2002) accordingto mother's educationalattainment and preschoolattendance Attended 37 12.1 14.3 Completed primaryschool Did not attend 47 11.6 13.7 Technical education or Attended 18 14.0 16.1 incomplete secondaryschool Did not attend 34 12.8 14.4 Completed secondary or Attended 9 15.7 17.7 higher education Did not attend 23 13.8 15.7 de Aprendizajes - ~~ Source: Evaluaci6nNacional en 6 O de Primaria 2002. 4.5 Overall student achievement in the sixth grade has improved inthe years o f MECAEP I and I1implementation. Between 1996 and2002, the share o f students who reached a `passing' or "sufficient" level o fperformance (defined as a 60 percent rate of items correct incriterion-referenced test scores) rose from 57 to 66 percent inlanguage and from 35 to 48 percent inmath (Table 4; see Annex figures B-1and B-2 for raw test scores). 4.6 Improvements aside, the absolute levels o f achievement scores are still low. Masterycriteria typically indicate a highrate o f achievement on objectives that have been taught.28 Fewer thanhalfo f the sixth graders attain even 60 percent o fthe objectives 26. World Bank2002b, p. 8 27. ANEP 2002b. 28 Popham 1978,Berk 1980; The 60% sufficiency criterion mayhavebeenempiritcally set to resemblethe grading systemthat ranges from 0-10. The technical annex of learningassessment documents showsno evidence that this cutoff point was developedto conformto acertaintolerable error rate or bepegged to the performanceo fknown 11 correct inmath and slightly more than half attain 60 percent o f the objectives inlanguage. Schoolprincipals and supervisors were informed o fthe scores, and teacher training in some areas focused on outcomes. However, the school-level data were not made public, as it happens insome other countries, and an opportunity was missedto increase parental involvement andteacher accountability. Table 4. Evolution of academic achievement inthe sixth grade With satisfactoryperformance(14- With highlyunsatisfactory performance(0-6points) 5.2 3.4 3.3 10.9 8.3 7.9 Failing -Unsatisfactory 42.9 38.7 33.7 65.4 59.2 51.7 100.0 100.0 100.0 100.0 100.0 100.0 (c) Achieving greater social equity - achieved 4.7 The education system has become more equitable. Test scores showed that students from more disadvantaged backgrounds made greater progress thanthose of advantaged backgrounds. For example, the percentage o f students obtaining a "sufficient" score increased by 12.5 percentage points among the disadvantagedbetween 1996 and 2002 inmath, while it increased by only 5.9 points among the most advantaged. Overall, the poor have made more substantial gains between 1999 and 2002. An equity index, consisting o fthe difference between the percentage o f the highest and lowest socioeconomic groups passing or obtaining "sufficiency" showed that equity increased between 1999 and 2002. Inlanguage, the `pass' rates o fthe lowest andhighest income groups had a gap o f 48 percentage points in 1996, but by 2002 the gap hadbeen reduced to about 35 percentage points. The language pass rates o fthe most disadvantaged students increased by 17.7 percentage points in 1996-2002, while those o fthe most advantaged increasedby only 2.3 percentage points. Inmath, the `pass' rates o fthe lowest and highest income groups had a gap o f 50 percentagepoints in 1996, but by 2002 it hadbeenreducedto about 35 percentage points. The mathpassrates o fthe most disadvantaged students increasedby 19percentage points in 1996-2002, while those o f the most advantaged increased only 5.9 percentage points (Table 5, Figures2 and 3).29 Table 5. Sixth grade test store changes by socioeconomic background `masters' (e.g. successful grade 7 students). Inprinciple, it ispossibleto havehard items anda low criterion, but the meanitemdifficulty index ofthe tests is average to easy, only 0.56-0.64 (p.40, ANEP 2002b). 29. ANEP 2003b, pp. 22,23 r 12 I T e r y advantag;: Advantaged Average 1996 1999 2002 Language 85.4 88 Difference 1996- 12.5I 17.7 i IMath 66.4 71.2 72.3I46.2 51.5 56.4 I34 39.4 52.5I24.1 27.9 39.1I16.7 27.9 35.7 Difference 1996- 5.9 10.2 18.5 15.0 19.0 2002 Source: MECAEP I1ImplementationCompletion Report, p 7. 13 Figure 2. Percentageof sixth gradersperformingsufficientlyin languageby incomelevel ! Ioo1 ~ 70 60 54.6 50 46.7 48.5 10 O L ~ t ~ \ l l l ~ ~ ~ ~ 1996 I999 2002 I996 19992002 1996 19992002 1996 I999 2002 1996 1999 2002 Very Very disadvantaged Disadvantaged MId Advantaged advantaged Source:ANEP 2003b, p. 17 Figure3. Percentageof sixthgradersperformingsufficientlyin mathby income level 8 0 - 80 - 72.3 70 - + 60 - 6 6 . 4 71'2 56.4 5 0 - p 52.5 40- 30 - , , , , , , , , ~ I:] , I I I I I I I I I 1996 19992002 1996 19992002 1996 1999 2002 1996 I999 2002 1996 1999 2002 very Mid -"laged advantaged I Source:ANEP 2003b, p. 18 4.8 Full-time schools expanded. The number o f full-time schools increased from 58 in 1996 to 98 by 2003 (Table 6). The Implementation CompletionReport showed an enrollment of 26,900 students in2003, approximatingthe target of 30,000 students (Figure4). 30 Initial targets hadbeen higher, but the government decided to limit the expansion ofthese rather expensive schools. 30. ANEP StatisticalAnnual 2003, Table 31.1. The 2003 statisticaltables mention 96 schools. MECAEP I1 ImplementationCompletion Report, (World Bank2004, p. 5) mentions26,900 studentsbut government statisticsshow enrollmentin2002 at 13,3 18andin2003 at 18,700 ingrades 1-6. Data informally provided mention 25,086 students for 2004. The variability seems due to including preschoolersinthe statistics. 14 Table 6. Full-timeschools constructedaccordingto socioeconomicenvironment 1996 1997 7998 1999 2000 2001 2002 2003 2004 Advantaged 1 1 1 1 1 1 1 1 Medium 4 4 4 7 7 8 8 8 9 Disadvantaged 14 14 15 20 20 21 23 23 23 Very disadvantaged 38 38 38 40 42 50 51 55 60 Not specified 1 1 1 3 3 10 10 10 11 Total 58 58 59 71 73 90 93 98 106 Note: Schools built in2004 were constructed by MECAEP 111. the effect size o f attending full-time schools. Only a 35000 few schools were 30000 - sampled, some o fwhich 25000 - offered only regular 20000 - classes in 1996.32 It i s 15000- difficult to draw 10000 - conclusions from -- performance changes in 5000 the most disadvantaged 0 I populations; statistical 1996 1997 1998 1999 2000 2001 2002 2003 2004 4.10 Given these limitations, however, Figures 5 and 6 suggest that achievement o f disadvantaged students inthe sampled schools i s better infull-time schools. Inthe 1996 test for sixthgraders, the percentage o f students performing at a "pass" level inlanguage was reportedas significantlyhigher insingle-shift, full-time schools, than inregular schools of areas considered disadvantaged or highly disadvantaged areas (60.7 versus 49.3 percent for the former, and 42.6 versus 37.5 percent for the latter).33Furthermore, the sixthgraders who stayed longer infull-time schools have higherperformance than those who arrived morerecently. The differences from one testing year to the next show that 5 to 9 percent more students attain satisfactory scores insixthgrade. Furthermore, the change rate in `pass' percentagesfrom 1996 to 2002 seems about the same for disadvantaged students inregular and infull-time schools. Although some schools that 31.ANEP 2003b and 2002e 32. The resultsreported are basedon samples o f4 schools inunfavorable circumstances and 5 invery unfavorable circumstances (ANEP 2003b, p. 70). To draw reliable conclusions from the data, it would help to correct statistically for regression towards the mean, follow paired samples o f students acrosstime, analyze within- as well as between- school variance, or compare difference-in-difference scores among various combinations o f schools and income levels. 33. ANEP 2003b, p. 42 and 45, graphs 12 and 14. 15 converted to full-time status in 1999 show better performance, the scores o f other full- time schools show somewhat lower progress. 4.11 The effects of full-time schools are also ambiguous inthe critical early grades. A national assessment coveringpreschool, first, and second grades found no difference in performance between first graders inthe national sample and full-time school students. However the latter were better inreading a variable that teachers use to decide which students repeat grades. It i s possible that the measurable benefits o f full-time schools for students are small over briefperiods o ftime but significant over the duration o fprimary school. Figure5. Progressoflow incomesixthgradersinfull-time schoolsinlanguage 75 70 65 60 55 50 45 40 35 30 1996 1999 2002 +Msadvantaged-natlonal sample --e Long-term full-the schools +Full-time schools slnce 1999 U N a t l o n a lsample Source: EvaluacibnNacionalde Aprendizajesen Lenguajey Matemhtica, 2003, p. 42 34. ANEP 2002b. 16 Figure6. Progressoflow incomesixth graders infull-time schools inmath -- I 53.9 50 45 40 35 30 25 20 24.1- 15 ." I 1996 1999 2002 -4- Msadvantaged-national sample -8- Long-term full-time schools -A- Full-time schools since 1999 +National sample Source: ANEP 2003b, p. 47. Long-termfull-timeschools have had this program since 1996or earlier (d) Reducinggrade repetition-partly achieved 4.12 Limited reduction in repetition rates. The strategies pursued inMECAEP Iand I1 were partlysuccessful inreducingrepetitionrates. Average primary-level repetitionrate was reduced from 10.8 to 10.3 percent duringthe two projects, and was nearly double the 5 percent envisaged inthe appraisal documents. At first grade, overall repetitionrates decreasedfrom 21.O to 19.9 percent between 1996 and 2002 and to 18.9 percent after project closing (Figure 7, Table 7, B-5); insecond grade, they remainedalmost stagnant at 14percent. Efforts to convince teachers to fail fewer students succeededina reduction o f the averagerate to 8.6 percent in2004. Table 7. Average repetitionratesinprimaryschools ProjectAppraisal 7990 7996 2002 2004 Document Target Nationalaverage for all grades 11.7* 10.8 10.3 8.6 5.0 Full-timeschools all grades nla 9.8 7.3 6.1 5.0 Nationalaverage for first grade 20.2 21.o 19.9 16.9 10.0 Fulltime schools first grade nla 28.1 14.6 12.4 10.0 Source: ANEP (Note*: datum from 1992) 4.13 Project efforts to reduce the highrepetitionamong the poorest areas have been partly effective. The most successful intervention inlowering failure rates has been preschool attendan~e.~~Full-time schools show a progressivelylower repetitionrate over time (Figures 7-8). The repetitionrate offirst graders attending these schools in1996was 28 percent but hadbeen reduced to 12percent by 2004 (Figure 8, improvement happened 35. World Bank2002b, p. 8 17 duringMECAEP111).The full-timeschools havehigher scoresthanregular schools in poor areas, but many o f their students are not low-income. As with performance, the mixedpopulation and changing clientele of full-time schools make interpretation o ftrend data difficult. Nevertheless, studentsinfull-time schools o f disadvantaged areas seem to perfonn better thanthose o f regular schools indisadvantaged areas. In2001, for example, the former had a slightly higherpromotion rate, 89.4 percent infull-time schools versus 86.1 percent than the latter.36 4.14 Overall, first graders inregular schools o f low-income (critical) areas had repetitionrates that fluctuated but didnot show much improvement (28 percent in 1996 and 25 percent in2004). Itis unclear how many o fthe students inthis sample hadbeen to preschool. Figure 7. Grade 1repetition rates by income level 30.0 1 +Very Favorable --CFavorable -&-Average s +Disadvantaged +Very Disadvantaged +Total 5 0 - Source:ANEP Figure 8. Grade 1repetition by type of school 35.0 1 I 15.0 - 10.0 - 0.05.05 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 &RegularUrban +Regular Rural +Critical areas --)lt Fulltime &Total Source:ANEP 4.15 Repetitioni s emotionally and financially costly. In2003, the total number o f primary school repeaters was 28,3 18, o fwhom 61 percent were infirst grade (11,025) and second grade (6,224). Considering an average class size of 26 for primary education, ~~ 36. MECAEP 2002, p. 5 18 about 1,089 extra teachers are neededannually to teach repeaters.37At an average monthly salary o f about US$300, the annual cost o frepetition i s roughlyUS$3.9 million. Ifrepetitionjust inthefirst andsecondgradeswere reducedto thetargeted 10percent, thenthe repetitioncostsinsalaries alone wouldbe59 percent lower, roughlyUS$2.3 million. Thus, there would be savings o f at least US$1.6 million annually. 4.16 Some government staff interviewedbythe mission attributed the highfailure rates to a lack o ftraining and distorted incentives. Teacher training duringMECAEP Itraining was not very effecti~e,~*while duringMECAEP I1it was only givento full-timeschools andpreschools. Teachers receive a 25 percent bonus for working inlow-income areas andmay inadvertentlyattempt to keep studentnumbers highand theirjobs secure. (Efforts after 2002 to convince teachers to fail fewer students may bepartly responsible for the improving rates.) However, teachers interviewed duringthe IEGmissionreported that they recommend grade repetition for the students who do not learnhow to read. Some expressedconcern regardingthe effectiveness o fthe reading methods they have been instructed to use (para. 5.3). A study has shown that most childrenlearn to read later than curricula would specify; only 23 percent o f first graders and 66 percent o f second graders read fluently bythe end o f the school year; 35 percent o f second graders could only read syllable^.^' 4.17 Overall, the data presented inthis report suggest that preschool has a notable effect on reducing grade repetition among all students, particularly among the poorest. Bycontrast, full-time schools have aweaker effect onrepetition. Bothinterventions seem to have little effect on test scores inthe early grades, but by grade 6, disadvantaged students performbetter ifthey have beento one o fthese programs than other students. MECAEP has not analyzed data on the interaction betweenpreschool and full-time schoolingand the additive effects are not known. (e) Improving Sectoral Management - achieved 4.18 The studies and activities carriedout bythe projects (such as the development o f standardized achievement tests (Annex Tables A-2 and A-3) have providedusable input inthe improvementofthe sectoral outcomes. Staffreportedto the IEGmissionthat achievement test data inparticular have providedvaluable feedback indeciding future courses o f action. 4.19 Mission Obsewationsfor MECAEP1and 11. The IEGmissionvisiteda total o f 10primaryschools inrural and inlow-income urbanareas.4oThe visit hadbeen 37. MECAEP Iwas to reduceor maintain the preschoolstudent-teacherratio, but due to high enrollmentsthepreschool student-teacher ratio increasedfrom 34 to 37. The primarystudent-teacherratio increasedfrom 25 to 26. (World Bank 2002b, p. 18) 38. World Bank2002b. 39. ANEP 2002b, p. 43. The countryOperationsDepartmentnotesthat "the finding that 66 percentof second-grade students are readingfluently inUruguay, while criticized bythe PPAR, i s indeedimpressiveby international standards." 40. The schoolswere: full-time schoolsno. 259,260 inMontevideo, andthree full-time schools ofFlorida andLaCruz (dept. laFlorida). Inthe department o fColonia, the schoolsvisited on the secondday of classes were full-time schools nos. 98 (Colonia) and inCarrnelo full-time schools nos. 138, 117.Also visited inthe same region were regular schools 19 announced previously, and an inspector accompaniedthe mission. Five schools were visited before classes started, and groups o fteachers were interviewedregardingthe benefits o fthe MECAEPproject. Fiveother schools were visited duringthe second day o f classes. Inthe latter, the missionobserved instructionalactivities taking place inclass at the moment o f IEGmissionentry and asked 3-4 studentschosen at random to read and answer simple comprehension questions. The missionalso observed two inservice workshops for teacher trainers inmathematics and science. School observations confirmedthat the projects hadmade efforts to serve the disadvantaged through quality education but also pointedto continuing challenges: 0 All teacherswere present inschool andoccupiedininstructionor inthe care of children. They seemedwell educated and trained and articulately answered instructionalquestions. The vast majority o f students were engaged inlearning activities at the moment o fmission entry, and instructional time seemed efficientlyused. Few studentswere seen unoccupied or uninvolved. 0 Lower-grade childrenwere articulate hut had limitedreading skills; only two o f 14 second and third graders who were askedto readcould do so fluently. (Findingswere consistent with arecent Thus, ina multigrade class where students shouldwork independently third grade students could not readthe text o f arithmetic problems well enough to solve them. However, most teachers expressed satisfaction with student performance and class achievements. 0 There was a textbook shortage inall but one of the schools visited, and it was unclear when more books would be available. Because o fthe shortage, textbooks are mainly for classroom use (even inthe advanced grades when students should read several pages daily) or students take turns taking them home. They cannot regularlybe usedfor homework. The MECAEP editions are not available for sale, butmost teachers expressedthe opinionthat at U S 1 0 each, students wouldbe unable to afford them evenifthey were. 0 The new schools are large and architecturally appealing, with kitchens, dining rooms, and auxiliary areas. The staff were o f the opinion that they were generally o f satisfactory q~ality.~' nos. 5 and 111(latter multigrade). The samplewas purposive; schools were selected on a conveniencebasis and some were inrural but not very poor areas. 41.ANEP 2000, p. 43 42 The missionheardconcernsabout exchange ofmaterialson sitewith those of lower quality and allegations that some principals hadbeenaskedto certify that work was completedwhen it was not. The region subsequentlyinformed the mission that the latter allegationwas investigatedandwas found not to pertain to aMECAEP school. Also no irregularities were found. 20 4.20 The missionalso interviewed government officials and donor staff involved with the projects assessed inthis report and obtained opinions about actions and outcomes (see Annex C and sections on Bank andborrower performance). 5. Ratings ProjectOutcomes 5.1 The outcomes o f all three projects are ratedsatisfactory. The strategies and project designs were relevant to the country's needs andwell suitedto its implementation capacity. The institutions supported by the Vocational EducationandTechnological Development Project have fulfilled their purposes as originally intended with substantial efficacy. COCAP never trained as many learners as expected, but it i s a viable institution that produces quality courses o f some demandby learners and organizations willing to pay. LATUacquired the capacity to engage intechnological research, provide technical assistance, andmarket informationservices for Uruguayanindustriesand has helped increase the competitiveness o fUruguayanimports. The MECAEP Iand I1projects were successful inincreasing equity inthe systemthrough improvingparticipationo f the most disadvantaged students inpreschool education and full-time schools. 5.2 For all three projects, relevancewas high andefficacy was substantial. The strategy o f increasing the achievement o fpoorer studentsthrough preschool education and instructiono f longer durationprovedrelatively effective. Eflciency for all three projects is ratedsubstantial. The strategy o f increasing the achievement o fpoorer studentsthrough preschool education and instructiono flonger durationprovedeffective. Inputswere also provided efficiently. Nevertheless, some outcomes fell short. The training quality o f COCAP courseshasbeengood, but the limitedcourse selections sometimes prevented unemployedtrainees from getting training and findingjobs in sectors ofhighdemand. And despite efforts, the MECAEPprojects were not able to reduce the highrepetitionrates inthe early grades andreap systemic savings. 5.3 The potential for efficiency improvement can be illustratedthrough international research. European studies o f children taught through simple synthetic phonics show nearly that all German, Italian, Greek, Spanish, or Turkishlearners become accurate and fluent readersbythe endofgrade 1.43 Bycomparison, only 23 percent o fUruguayanfirst graders read fluently by the end o fthe school year.44Children cannot learncontent from textbooks (including math) untilthey read fluently, and early delays reduce the efficiency o f subsequent grades and o fthe entire system. One determinant may bethe limited textbook supply. Another may be the "whole word" readingmethod adoptedby the government. This method, which i s popular inLatinAmerica, has been the focus ofmuch 43. Readingbasics are taughtin4-6 months, andSpanish students at the endofgrade 1readwith 96 percent accuracy at 45 wordsperminute(Seymour et al. 2003). For other languagessee Seymour et al. 2003 aswell as Cossu 1999, Harris andHatano, 1999. SomeUruguayanofficials interviewedthought that poorreadingmightbe linkedto child malnutritioninUruguay; however, there is no documented linkagebetweenmalnutritionandbasic reading achievement levels. 44. Ofsecondgraders 66percentreadfluently(ANEP2002b, p. 43). The regionnotes that "the findingthat 66 percent of second-grade students are readingfluently inUruguay, while criticizedbythe PPAR, is indeedimpressiveby internationalstandards." 21 internationalconcern since the early 1990sbecauseit requires very skilledteachers, time, andparental support. 45 Since disadvantaged students often lack the needed support at school and at home, research incountries such as UK,US, and New Zealand has found that they are more at risk o f failure.46The same problems may arise inUruguay, since home practices havebeen found to affect student~erformance.~'The "whole-word" methodmay havebeen adopted on the basis o f limitedinformation, as the IEGmission didnot locate studies inUruguay or pilots to test its efficiency. InstitutionalDevelopmentImpact 5.4 The institutionaldevelopment impact for all projects i s rated substantial. The institutions that were supported or created through the Vocational Education and TechnologicalDevelopmentProjectimprovedtheir capacity to carry out their activities. MECAEP Iand I1supported capacity buildinginthe entire sector. Units for standardized testing, textbook production, and curriculum were created, staffed, andcontinuedto operate effectively for a decade. Many studies were carried out, and efforts were made to implement and disseminate their findings. Sustainability 5.5 The sustainability o f all projects is rated likely. The Vocational Education and TechnologicalDevelopmentProjecthas sustained most expected benefits for about 20 years. The educational benefits o fMECAEP Iand I1are likelyto be maintained as the more disadvantaged studentsbecome able to performandcontinue on to secondary school, ultimately improving their earning capacity. 5.6 Inpoliticalterms, resilience ofprojectbenefits inchangingcircumstances is likely. A new government took power inMarch2005, anditplansto maintainthe current policies that target the poor. Inthe short term, the government plans to improve the physical condition o fthe schools by forming an "impact plan" through the Ministryo f Educationand Culture.48There i s also a medium-termplanto hire more teachers and reduce class sizes. Ineconomic terms, however, sustainability i s uncertain. Although educational investment inUruguay is considered debts are large, and fiscal constraints are real. Though the percentage o f education inthe social budget has somewhat increased (Table B-7), expenditures on the social sectors have beenreduced 45 Somemanualsdiscussingthe methodologyare ANEP 2002b, ANEP undatedb. The methodadvocates literacynot by learningindividualletters but through analogies and context. Also, muchemphasisi s put on listening comprehension. While these activitiesare desirableinclass, extensiveuse mayprovetime-consuminganddifficult, and some studentsmayget insufficientreadingpracticeto acquirefluency (Tunmer et al. 2003, Nicholson1999). 46 E.g., Stevens 1993; Greenwood 1991, Tunmer et al. 2003, Nicholson 1999, Crouchet al. 2005. 47. Uruguayanhomepracticesdeterminedthe readingmaterialsavailable, practicereadingto children, and sending themto preschool. All variables except for inquiriesregarding school events were shownto influencestudent outcomes (Duthilleul 1997). 48. Brigadesofworkers will behiredto improvethem. The two-monthplanwill befinancedby the amountsremaining after the sale ofthe containerterminalinMontevideo. (`Plan de impact0de laizquierdapararefaccionarliceos y escuelas' El Observador, February25,2005.) 49. World Bank2005. 22 since 2002 (Table B-6). Staffreport that the Structural Adjustment Loan (Ln.7104) that was supposedto protectthe social sector budgets may not be succeeding and that the government has reduced education spending by about 29 percent since 2001. Sufficient funds maybe enteredinthebudget,but lower amounts are actually allocated andpaid. The MECAEP I11project mustoften apply to the treasury for the hndsneededto carry out its activities, causing payment and execution delays." Bank Performance 5.7 Overall, Bank performance for the three projects i s satisfactory. Bank performance on the vocational education project dates from an earlier era. The design corresponded to the needs o fthe institution, but some problems were identified. Enrollmentprojections for COCAP were unrealistic and didnot consider the alternative o fvocationaltraining provisionthoughprivateproviders. The Bank didnot have at that time the technical competence to supervise the LATUcomponent. Informationfrom project files indicates disagreements between the Bank's architects andUruguayan staff, and the Bank delayed approval on the construction o f LATUbuildinginan effort to force questionable changesto it. The project was better suited for the industryrather thanthe education sector. 5.8 For the MECAEP Iand I1projects, there was close collaborationwith the government and much exchange o fideas. Government staffpraised the Bank for its rigor, competence, level o fknowledge, follow-up, and frankness. Itwas reported that Bank staff respectthe identity o fthe country andhave the capacity to transmit experiencesfrom other countries. Furthermore, there was task manager continuity, so government staffdidnot have to explainthe country's particularities repeatedly. However, the MECAEP Idesign didnot include stakeholder consultations. This omission created difficulties anddelays in the execution o fMECAEP I. the appraisal o fMECAEP I1stakeholderswere During consultedmorebroadly. However, civilwork needs were not realisticallyestimated during appraisal, and itbecame necessaryto change targets duringimplementation. 5.9 Although international researchhasraised concerns regarding the effects o f the `whole word' approach since the early 1990s, documents and interviewswith staff showed that these were not brought to bear inthe discussions leadingto MECAEP I.By 1999, mher studiesandinternational consultations resultedinwidelypublicizedguidelines by U S NationalReading Panel infavor o fphonics instruction, particularly for poorer students.'* But there i s no evidence that these studieswere discussedwith the government duringMECAEP I1andMECAEP 111, despitetheir potentialrelevanceto Uruguay's high grade repetitionrate andimpact on the efficient use o fresources. The Bank couldhave usefbllyadvised the government to evaluate current teaching methods againstalternatives. 50 The IEGmissionvisited one schoolwhose additional constructionhadbeenstoppedright beforeschool starteddue to a lack of funds, renderingpart of the existing structure unusable. 51 National Institutesof Child Health and Development. 2002. National ReadingPanel Findings andDeterminationsof the National ReadingPanelby Topic Areas (http://www.nichd.nih.gov/Dublications/n~/ftndings.htm). 23 BorrowerPerformance 5.10 Borrower performance for the vocational educationproject i s ratedsatisfactory. For MECAEP Iand 11,performance i s rated highly satisfactory. The staff worked with dedication, andnearly all activities were substantially completed as planned. Educational activities were closely supervisedby a well-staffed group o f supervisors. 6. Quality Educationfor All inUruguay 6.1 In10years o fBankinvestmentinprimaryeducation, Uruguayhasconsolidated its Educationfor All achievement. At the primary level, coverage and completionrates have reached 98 percent, while dropout rates are less than 1percent. The parts o f the system most likely to benefitthe poor have expanded. Universalpreschool enrollment o f five-year olds was attained in2001, making Uruguayone o fthe first middle-income countries to reachthis milestone. 6.2 Achievement increases across the boardhave enabledUruguay to score highly in internationalcomparisons. Inthe 2003 Program for InternationalStudent Assessment (PISA), Uruguay hadthe highest score among L A C countries (Table 8). Despitemany students' relativepoverty, the willingness o fthe countryto make long-term investments inthe educational system seems to havebornefruit. Table8. PISAtest scores for various countries(2003 unlessotherwisenoted) Country Language Math Science Hong Kong-China 550 510 539 Finland 544 543 548 Korea 542 534 538 Japan 534 498 548 New Zealand 523 522 521 OECD average 500 494 500 Poland 490 497 498 Spain 485 481 487 EuropeanUnion 483 495 491 Portugal 466 478 468 Greece 445 472 481 Turkey 423 441 434 Thailand 417 420 429 Uruguay2003 434 417 438 Chile (2001) 410 400 415 Argentina (2001) 418 399 396 Mexico 400 364 405 Brazil 403 333 390 Peru (2001) 327 230 333 Source:OECDandANEPg 2003. Primerinforme nacional PISA, p. 28 24 6.3 The country expects to provide efficiently high-qualityprimary and secondary education for all. Futurepolicy measuresinclude universalizingpreschool education to four-year olds and extending attendance downwards to reachpoor children at ever younger ages.52The goal o fthe current MECAEP I11project i s to increase equity, equality, and efficiency inthe provision o fpreschool andprimaryeducation through (a) expanding the full-time school model which focuses on students from socioeconomically disadvantaged backgrounds, (b) improving the quality o fpreschool andprimary education by enhancing the teacher training system and introducingnew teaching and training instruments inthe classrooms, and (c) increasing the efficiency o f educational institutions. 6.4 The government would also liketo universalize coverage insecondary education through more differentiatedsecondary school curricula, but student performance i s a problem. About 90 percent o fprimary-school graduates enter secondary school, but only about 50 percent o fthem graduate. Those who repeat inprimary school often tendto drop out o f the system, partly because they cannot pass examinations. Itremains to be seen whether the strategy o f offering two or more years ofpreschool will reduce repetition rates inthe longrun. MaximizingPoor Students' Achievement 6.5 I t i s hard to learnmuch information from books or read and solve mathproblems untilstudents are fluentreadersandcomprehend text. Giventheimportance o fearly reading for schoolwork and a script that i s exceptionallyeasy to master, the government might consider settingreadingbenchmarks, such as a goal that all childrenshould become fluent readersby the end o f grade 2. (See norms and goals o fthe U S and other Hispanic countries. 53) The government might also conduct studies to estimate the resources neededto make disadvantaged students fluent readersby the end o f grade 2 given efficient instructionalmethods. Full-time schools might devote some o f the afternoon time to activities involving systematic readinginthese early grades. Regular schools might offer extra readinghours inthese early grades. 6.6 Supplantingcritical skillsfor the disadvantaged. Educationalresearchpoints to other compensatory activities that improve the performance o f the poor. One is direct instructionthat involves choralrepetitionof complex sentencesto help children master a 52. The target populationof MECAEP I11is the estimated12,300 children of age 4 and 5 who have not yet participated inpreschooleducation. Theserepresentabout 20percentofthepreschoolandelementaryschoolage children ina condition ofpoverty. Other children are estimatedto be enrolled inprivate schools and inspecialeducation. The project will expandthe preschooleducationcoverage to the remaining 10,700 children(10.3 percent)inurbanareas through constructionofnew classrooms, andto 1,600 children(4.5 percent) livinginisolatedruralareas through alternativestrategies. 53. Benchmarksand proposednormsfor reading: Inthe US 30-70 words orally per minute for grade 1,60-100 words silently per minute for grade 2 (Barret al. 2002, p. 76). Inthe ChileanRedMaestros-de-Maestrosprogramofthe MinistryofEducationgoalsfor grades 1and2 are 30 and70wordsperminute.ForChileanNGOEducandoJuntos goalsare around34 and64 wordsper minute for grades 1and 2 respectively.Reportedaveragesin Spainfor grades 1 and2 are about 50-55 and about 75 words per minute, respectively(Equipo de OrientacionEducativade Marbella 2003). Among the low-income Spanish-speakersthe US, readingonly 30-60words per minute inSpanish inGrades 1 and 2 is usedas an index ofdisadvantage(see de la Colina et a12001). (Chilean goalsin http://www.rmm.cl/index-sub.php?id-contenido=ll28&id-seccion=3 lO&idqortal=75 http://www.educandojuntos,cl) 25 level o f linguistic complexity they often lack.54The interest o fUruguay inmusical and cultural traditions also points to another set o fusable skills: music, and inparticular youth orchestras. Not only has music education been shownto improve intellectual development andverbal memory,55youth orchestras inVenezuela may have been effective inreducing crime among the Balancingthe Costs andBenefitsofInserviceTeacher Training-SomeInsightfrom MissionObservations 6.7 Observations o fMECAEP I11training events taking place duringthe IEGmission raised some concerns regarding the efficiency o fthe activities, particularly giventheir sizeable cumulative expenditure~.~'These observations might be significant giventhe limitedteacher training effects reported duringMECAEP I.58 with teachers Meetings usedtime inefficiently, allowing lengthy commentaries on general or abstract topics. There was limitedrecord-keeping or attempt to summarize and concretize actions. Some workshops involved homework, most didnot. Teachers met and conducted group projects, which shouldprovide elaboration o f concepts and therefore make them more memorable. 6.8 Teachers are not evaluated on the knowledge acquired and infact have little incentive for paying attention or learningduringtraining. Some teachers interviewed duringthe IEGmissionhave suggestedthat coursesberigorous and include homework as well as a final exam, not merely aproject. Others suggestedthat promotions be made contingent on passing exams on training. This way there would be a reasonable incentive to actually learn duringtraining. 6.9 Teacher training under MECAEP I1was found to be usable inclass due to follow- up. Hopefbllythis trend will continue. Loanfunds mightbebest usedto finance informationthat i s transmissible, that effectively cascades to beneficiaries. It may be worthwhile to formulate innovative ways to measure the effectiveness o fvarious types o f "software," cost the informationprovided and the impact ofits use, and assess the probability that it will be retained and flow to students. Suitable andreadily usable materials that connect with school curricula might link the informationto classroom practices. 6.10 Teacher training institutions do not give textbooks to student teachers. As aresult, teacher trainees must study from notes and library books, and they may thus use their time less effectivelyand learnlessmaterial. Itmaybe feasible to bindthe content needed 54. Researchstudies are availableat the web site of the National Association for Direct Instruction, www.nifdi.org, www.adihome.org 55. Schellenberg2004. 56. InNovember 1997, IADBapprovedaUS$8 million loan for youth orchestras inVenezuela, aprogramdesignedto improve the lives of low-income children. 57. Expensesincludetravel, hotel, and food. (625 pesos per week for those inMontevideo and 1125 for those fromthe interior). 58 World Bank 2002b, p. 6; World Bank 2004b. 26 for pre-service teacher training, even ifstudents receive cheap editions or loose-leaf collections. 7. Lessons This assessment confirms a number o fIEGlessons from the education sector: 0 Preschool attendanceoffers significant academic benefits to low-income students in terms o freduced grade repetitionas well as long-term academic achievement (para. 4.4) 0 A longer school day may offer important academic and socialbenefits to disadvantaged students, including improved test scores. To maximizeperformance inhighergrades, however, studentsinlower gradesmightspendpartofthe extra time practicingthebasic skills that serve asprerequisites for further learning(paras. 3.10,4.13, and 6.5). 0 Quality o f education depends on a strong and functional supervisory chain. If government policies and learning activities are to be carried out effectively at the school level, teachers mustbe supervisedclosely by knowledgeable and interested staff. Although supervision i s often costly interms o f salaries and vehicle support, it i s a prerequisite for improvedlearningoutcomes (paras. 3.8 and 5.10). 0 Project efficiencymaybe enhanced through instructionalmethods that enable most o fthe students to achieve educational objectives within the prescribed instructional time. Efficientinstructionmaybemore important for disadvantaged children, who receive less home support andwho often have higher dropout and grade repetition rates. Further research and experimentation would help determine which instructional activities are more cost-effective inthe Uruguayancontext (paras. 5.3 and 5.9). 0 Vocational-technical education project appraisals may overestimate the number o f likely graduates and not adequately take into account dropout rates or changing demand for training. Appraisals o f future projects should take into account country experience on dropout rates as well as demand changes duringtimes o f economic hardship (paras. 2.2-2.4,2.6). 27 References ANEP. 2003. Statistical Annual. ANEP. 2003a. "Sintesis de 10s Principales Resultados del Estudio de Evaluacih de Impacto de la Educaci6n Inicial en el Uruguay." Montevideo. ANEP. 2003b. "Evaluaci6n Nacional de Aprendizajes en Lenguaje y Matematica. Resultados en Escuelas de Tiempo Completo y Escuelas de Areas Integradas." Montevideo October. ANEP. 2003c. "Ahora me Toca a Mi.Uruguay: Educaci6ninicial para todos; recopilacionde las principales acciones." Montevideo. ANEP. 2003d. "Programa de Desarrollo Escolar." MECAEP. Montevideo. ANEP. 2003e. "Proyectos de Mejoramiento Educativo (PME)." Montevideo. ANEP. 2003f. "Evaluaci6n del Sistema de Adquisicion y Distribucih de Libros de Texto de MECAEP." ANEP. 2003g. Primer infome nacional PISA 2003 Uruguay. Montevideo. ANEP. 2002a. "Evaluaci6n en el Primer Nivelde la Escolaridad.(Separata I,11,III)." September. ANEP. 2002b. "Los Nivelesde Desempeiio a1Iniciode la Educaci6nPrimaria." Estudio de las competencias Linguisticas y Matematicas. September. ANEP. 2002c. "Proyectos de Mejoramiento Educativo (PME): Unapropuestade cambio." Montevideo. ANEP. 2002d. "Evaluacih Nacionalde Aprendizajes en Lenguaje y Matematica". Montevideo. ANEP. 2002e "Evaluaci6n Nacional de Aprendizajes en Lenguaje y Matematica. Resultados en Escuelas de Tiempo Completo y Escuelas de Areas Integradas." Montevideo. November. ANEP. 2001. "La pruebay otros aportes. Evaluaci6nautonoma de aprendizajes ensefianza primaria - cuarto afio". March. ANEP. 2000. "Evaluaciones Nacionales de Aprendizajes en Educaci6nPrimaria en el Uruguay (1995-1999) ."Montevideo. ANEP. 1999a. "Estudio del Lenguaje en 10s Nifios de Cuatro M o s del Uruguay." Proyecto MECAEP. ANEP. 1999b. "Proyectos de Mejoramiento Educativo Evaluacih del Impacto: Hallazgos Principales del analisis cualitativo." Montevideo ANEP. 1998a. "Especificaciones tecnicas. Para sustento de 10s libros de texto de l o y 20 grados de escuelas primarias." 28 ANEP. 1998b. "Proyectos de Mejoramiento Educativo. Evaluacion del Impacto." Resumen, December. ANEP. Undated a. "Primer Ciclo en EducacibnPrimaria: Una experienciacurricular." ANEP. Undatedb. "Guia de Evaluacihy Selecci6n de (PME)." ANEP-MECAEP.2000a. "Educacibn Inicial. Propuestaspara el us0 del material didactico." ANEP-MECAEP. 2000b. "La Oralidady la Escritura desde el Aula." May ANEP-MECAEP. 1999. "Programa de educaci6n Inicialpara 3,4, y 5 Mos." ANEP-CODICEN-CEP-MECAEP-BIRF.1999. "Tejiendo vinculos para aumentar la equidad." Aristimuiio, Adriana, Ruben Kaztman, LuciaMonteiro. 2003. "iC6mo se usa y que impacto tiene la informationempirica sobre las evaluaciones nacionales de aprendizajes en el mejoramiento de la EducacionPrimaria p~blicaen Uruguay?" Fondo de Investigaciones Educativas Informe Final. Montevideo. 31de Julio. Barr, R; C. Blachowicz; C. Katz; and B.Kaufman. 2002. ReadingDiagnosisfor Teachers:An Instructional Approach (4th ed.). Boston, MA: Allyn and Bacon, p. 76. Benveniste, Luis.2000. "Student Assessment as a PoliticalConstruction: The Case o fUruguay." Education Policy Analysis Archives 8(32). July 11. Berk,Ronald. 1980. Criterion-ReferencedMeasurement: TheState of theArt. Baltimore: Johns Hopkins University Press. Crouch, L.,H.Abadzi, M.Echegaray, C. Pasco, and J. Sampe. 2005. "Monitoring Basic Skills Acquisition Through RapidLearning Assessments: A Case Study from Ped." WESCO Prospects (forthcoming). de la Colina, Maria G., Richard Parker, Jan Hasbrouck, and Raphael Lara-Alecio. 2001. "Intensive Intervention inReading Fluency for At-Risk Beginning Spanish Readers.'' Bilingual ResearchJournal. Vol25, No. 4, pp.417-452. Duthilleul,Yael. 1997. "DOParentsMatter? Practices OnFourthGraders' Reading Comprehension Achievement inMontevideo Public Schools". LASHD Paper SeriesNo. 14May. Equipo de Orientacion Educativa de Marbella. 2003. "Evaluacion de la velocidad lectora oral y An6lisis de la correlacih de esta variable con la nota global dejunio." Consejeria de Educaci6n y Ciencia, Junta de Andalucia. "Estirnacion del gasto por alumno en Educaci6nPrimaria P~blicay costos comparados de la educacion comb y de tiempo completo." Working document, April 2004 (author Unknown). Gathercole, SusanE., SusanJ. Pickering, Benjamin Ambridge, and HannahWearing. 2004. "The Structure o f Working Memory From4 to 15 Years o fAge." Developmental Psychology, 40(2):177-190. 29 Gavens, Nathalie andPierre Barrouillet. 2004. "Delays o fretention, processingefficiency, and attentional resources inworking memory span development." Journal of Memory and Language 51:644-657 Greenwood, C. 1991. "Longitudinal Analysis o f Time, Engagement and Achievement inAt-Risk Versus Non-Risk Students", Exceptional Children, 57 (6):521-535. Harris, M. and G. Hatano (Eds) 1999. Learning to Read and Write: A Cross-Linguistic Perspective. Cambridge, U.K:.Cambridge UniversityPress. Inter-American DevelopmentBank. 2000. "Country Paper: Uruguay." Washington, D.C. Lijtenstein, Sergio andJuanBogliaccini.Undated. "Estudio sobrebuenaspracticasy lecciones aprendidas en escuelas de tiempo completo." MECAEP.2004. "Mejoramiento de la Calidad de la EducacionPrimaria." Montevideo. MECAEP. 2002. "Escuelas de Tiempo Completo: Una `performance' alentadora." Montevideo. National Institute o f ChildHealthand HumanDevelopment. 2000. Report of theNational Reading Panel. Teachingchildren to read: An evidence-basedassessment of thescientijk research literature on reading and its implicationsfor reading instruction: Reports o f the subgroups (NlHPublicationNo. 00-4754). Washington, DC: U.S. Government Printing Office. Nicholson, Tom. 1999. Riskfactors inlearning to read. In Preventing and Remediating Reading Difficulties, Barbara R. Foorman, ed. Baltimore: York Press. Oficina de Planeamiento y Presupuesto (OPP). 2004. "El gasto pcblico social en el Uruguay 1999-2003." October. Popham, J. W. 1978.Educationalevaluation. EnglewoodCliffs, NewJersey: Prentice- Hall, Inc. PREAL.2001. "Quedandonos atrhs. Uninforme del progreso educativo enAmCrica Latina." Santiago: PREAL. PCrez Villar, JosC. 1996. "LCbmo lee mipaciente?: Contribucih a la metodologia del examen direct0 en psiquiatria de niiios." Revista Cubana dePediatria 68 (3). `Plan de impact0 de la izquierda para refaccioar liceos y escuelas'. El Obsewador, February 25, 2005. Rayner, Keith, Barbara R.Foorman, Charles A. Perfetti, DavidPesetsky, and Mark S. Seidenberg. 2001. "How Psychological Science Informs the Teaching o f Reading." Psychological Sciencein thePublic Interest 2: 1-33 SchellenbergEG. 2004. "Music Lessons Enhance IQ". Psychological Science 15(8): 511-514. Schiefelbein, Ernesto. 2005. "External support has not improvedBolivian classroomprocesses." Paper presented at the annual Comparative and International Education Society conference, Palo Alto, California, March 23. 30 Shaywitz, Sally. 2003. OvercomingDyslexia. New York: Alfred Knopf. Stevens, F. E. 1993. "Applying an Opportunity-to-Learn Conceptual Framework to the Investigation o f the Effects o f Teaching Practices via Secondary Analyses o f Multiple- Case-Study Summary Data", TheJournal of Negro Education 62(3):232-248. Tea, Deloitte, Touche. 1996. Estudio de auditorial del personal docente administrativo tCcnico de servicios de la educacibnpreescolar y primaria piiblica. Tunmer, W.E., J.W. Chapman, and J.E. Prochnow. 2003. Preventingnegative Matthew effects in at-risk readers. I n Preventing and Remediating Reading D$jculties, B.R. Foorman, ed. Baltimore: York Press. WorldBank. 2005. "Uruguay -Opciones de Politica 2005." World Bank. 2003. "Program Document For A Proposed Public Services And Social Sectors Structural Adjustment LoanInThe Amount OfUS$151.52 MillionAnd Special Structural Adjustment LoanInThe Amount OfUS$101.02 MillionTo The Oriental Republic OfUruguay". March25,2003. Report No. 25012-UR World Bank. 2002a. "Uruguay. Third Basic Education Quality Improvement Project." Project Appraisal Document, March 2002. Report No: 23825-UR. World Bank. 2002b. Uruguay: Implementation Completion Report, Basic Education Quality Improvement Project. Report no. 23566, January 30,2002 World Bank 2004b. Uruguay: Implementation Completion Report, Basic Education Quality Improvement Project. World Bank. 2004a. World Bank Country at a Glance. 31 Annex A Annex A. Implementationof projectcomponents TableA-1. Vocational TrainingandTechnologicalDevelopmentProject Components/ Outcomes subcomponents Activities Targets to be achieved outputs Info obtained during mission COCAP (US$4.9 million Total 16.200 araduates Enrollmentsvaried Industrial training provision cost) annually from-all but were always far remained limited facilities; below this level most trainees were new entrants 14,860 already employed 8% new entrants Construction 320 trainee places Construction After expanding to larger building, 6500 workers annually complete and in COCAP is moving again to the use Bank-financed building Furniture and Acquired as Has been maintainedwell and equipment expected much remains in use 6 mobile training 1-3 week courses in 11 Units procured, Did not prove very useful. Some units occupations to 6400 rarely used equipment stolen rural workers In plant- training 2.5 staff years for Partially completed Limited in-plant training has taken 1300supervisors place over the years For 6500workers annually Technical 9.7 staff years in-plant ILO and COCAP acquired the capacity to assistance (TA) training CINTERFOR gave design and deliver courses 11 staff years some TA institutional development TA amounts partly spent Fellowshipsfor 7 staff years Not carried out No impact textiles and leather Training needs Methodologywas unsuitable surveys Technological Support for 11 of 36 Acquired the capacity to engage in Laboratoryof Uruguay exporting industries number of sectors technological research, technical (LATU)final cost assistance, and market US$18.1 million information services for Uruguayan industries Construction (paid New building and land Size increased Extensivegrounds serve the by LATU) during needs of the institution well implementation Office Procured as Furnitureand equipment remained Mechanical and expected in use electrical equipment Laboratories 1 microbiological All constructed and Labs test and certify the quality of functional export products; results respected 4 sector-specific and certifications accepted 1materials testing worldwide 1metrological Pilot plants 5 pilot plants 5 pilot plants Has developed the capacity to selected for food, certify quality and find better ways textiles, leather of processing exports such as products, process wood, milk and leather products, industries fruits, vegetables. according to criteria Technical 12staff years Partly used, free ITA brought needed expertise assistance TA obtained fellowships ~~ 24 staff years Completed as Staff trained have worked in the planned institution (many since retired) Management Needs for additional 115 Completed as LATU became a large organization committee staff planned with close industry linkages Industrialadvisory 32 Annex A I I Components/ Targets to be Outcomes n..tmrr4c subcomponents Activities achieved outputs Info obtained during lnfnnhtsinarl rlrrrinn mission ImDrovina ekcienci quality, equity of primary education Classroomconstruction 50 classrooms 100classrooms Classroomsare of satisfactoryquality (US$22.5million) Textbook production 3 booksfor gr 1-2 Textbooks developed as Textbooksavailablefor 4 books for gr. 3-6 planned printing Textbook and materials 3.3 million 4.3 milliontextbooks Every student hasthe purchase 12,325teaching 12,325teaching packages textbookseries packages Qualityimprovement Fundingfor 800 931 subprojects Involveteachers in the Droiects . . subprojects . . school, slightlyrelatedto betterstudentachievement Inservicetrainingin the 16186educationalstaff About 17,000 (8,200in Effectsunknown use of textbooks low-incomeschools) IInservicetraining 3200 educationalstaff About 20,000 I I I did notchangewith Teacher classroomskills - training Expanding Buildingconstruction 50 classroomsfor 4- 87 built Classroomsare of preschool and expansion year olds 3 rehabilitated satisfactoryquality education 11classroomsfor 5- year olds, 23 classroomsrehab (US$16.6 million) Teaching materials 12,300sets distributed 12,300sets distributed Used in schools III I Trainina 3000 teachers 10,500 trained Training had limitedeffects in influencingclassroom I 185directors (4500 in low-income 24 supervisors schools) Iinstructionand improving student learning Strengthening Developmentof Tests developed I Functional ITests have beenusedfor education I standardized cognitive monitoringpurposes institutions achievement tests (US$2.3 million) Management Development System incomplete Teachersdo not enter informationsystem studentdata directlyon Operational manuals development I Completed ~ Vehicles for 20 20 vehicles I Vehicles procured; some inspectorates were leased transportinginspectors, Maintenancefinancially who actually visit schools extensively Sector management Resultsinterestingbut not productivitystudy I Targetsto be met Increase netenrollment From41% to 44% Increasedto 93% of 4- and 5-year olds Dreschoolersless likelvto be retained in first grade Increaseenrollmentin 4-year olds 22% 4-year olds 57% Lower-income low-incomeareas 5-year olds 52% 5-year olds 85% preschoolersless likelyto Increasenetenrollment From 91.7% in 1993 Increasedto 98% in 2001 of 6-11year olds Increaseprimary From 92% in 1997 Increasedto 97% in 2001 school completion Increase per student From US$305unitcost Increasedto US$600 cost Improvesixth grades From 57.1% in 1996 Increasedto 61.3% in Spanish testscores 1999 Improvesixthgrade From 34.6% in 1996 Increasedto 40.8% in Achievementimproved math testscores I Source: Project documents and informationobtained duringthe assessment mission 33 Annex A rableA-3 Bas educationqur ity improvementprojectI1(MECAEP 11; Components/ I Outcomes subcomponents Activities Targetsto be achieved I outputs Info obtained during mission Universal Accessfor the remaining 13,318 additional4-5 year Of cohortswith about preschool 10,700 childrenaged 4-5 in olds enrolled 50,000 are enrolled education urban areas 93% of 5-yearolds US$15.4 m) and 73%of 4-year olds Classroom 200 classrooms 109classrooms Reducednumber construction correspondsto needs Constructionsof good guality Equipmentand For 200 classrooms For 109newlyconstructed Materialsused in materials 53 existingclassrooms class Teachingmaterials 100setsfor 200 75 sets for 150classrooms Materialsused in classrooms 100music boxes class 29,000 booksfor rural children A new preschool No specifictargets Curriculumdevelopedand With follow-up and curriculum supervision, training is Teacher training more likely to influenceclassroom instructionand result in student learning Promotional Impactunknown campaignsfor preschool Impactevaluation Resultsdisseminated to parents Teachers hired 400 working %time 200 placed in schools All students are All certified attended Access to full-time Classroom Refurbishmentof 25 30 double shiftschools About 18700children schools (US$23.2 constructionand 280 classroomsbuilt converted enrolled in full-time m) rehabilitation,from 55 pre-existing schools by project 58 schoolsto about rehabilitated end. 105 120classrooms(13 new Access for about 30% schools) (33,500)childrenage 4-11 in poor areas 100%of full-time schools created Equipmentand Providedto 400 Providedto all 330 Used in class learningmaterials classrooms classroomsavailable Books distributedto 23 Trainingand Teachers and technical teachers and principals supervisors report assistanceto satisfactionwith schools training Promotional Resultedin increased campaigns preschoolenrollments Studies Beneficiary Carriedout Resultsdisseminated assessmentof teachers and printed and families on the Impact unclear implementationof the new model Impactevaluationof Carried out Resultsdisseminated the full time school model and printed on learningoutcomes Impact unclear Case study of successful Carried out Resultsdisseminated and and printed unsuccessfulfull time Impact unclear schools. Project Technical ANEP capable of administrationand assistance to PCU monitoringprojects monitoring 3 strengthen and disseminating 4NEPs monitoring results apacity 35 Annex B Annex B. SupplementalTables Table B-1. Coursesofferedby COCAP Area Number of courses Percentage Industrial 79 68.1 Computertechnology 4 3.45 Commerceand services 25 21.5 HotelslFood 7 6.03 Agrarian 1 0.86 SourceCOCAPdata (2004) FigureB-1COCAP cost recovery percentages. I Source:COCAP TableB-2. Courses Offeredby COCAP Source: COCAP.Note: (1) Coursesoffered to 9/30/2002 Annex B Table B-3. Language and math test scores related to schoolimprovementprojects in 1996 Socioeconomic status Language Math WithPME No PME With PME No PME Advantaged 15.5 15.6 13.7 13.2 Medium 14.3 13.7 12.1 11.4 Disadvantaged 13.6 13.2 11.1 10.9 Very disadvantaged 12.1 12.5 11.0 10.7 Source Schoolimprovementproject(PME) impactstudy, p. 51 (on266 matchedschools) Figure B-1. Evolution of sixth grade test scores in Figure B-2. Evolution of sixth grade test scoresin languageby socioeconomiclevel mathby socioeconomiclevel ("i 1'"1 18.1 1; 1996 1999 2002 1996 19992W2 1996 19992002 1996 1999 2002 1996 1999 2002 1996 19992(Kn 1996 19992wZ 1996 19992wZ 1996 I999 2002 1996 I999 2002 very very dlsadvantaged Disadvantaged Middle Advantaged advantaged very Very disdvantaged Disadvantaged Mld Advantaged advantaged Source: EvaluacibnNacionalde Aprendizajes en Lenguajey Source: Evaluaci6nNacional deAprendizajesen Lenguajey MatemBtica, 2003, p. 19 MatemBtica,2003, p. 20 Note: scores vary from 0 to 24 Note: scoresvary from 0 to 24 Table B-4. Full-time school unit costs (in US$) ~ ~~ ~ Programs Cumulative Unit cost No extra programs (regularschool) 373 Teacher trainingand materials 457 84 Recreation 468 11 Bilingualeducation 638 170 Linkageswith community 664 27 Computer technology 701 37 Total 701 329 37 Annex B Table B-5: Repetition rates in first grade by socio-economic level (1990-2004) 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Category Urbancommon sld sld 21.4 21.0 20.8 18.9 18.5 19.0 18.7 14.2 16.9 19.0 19.3 17.1 16.30 Rural common sld sld 27.1 28.1 25.0 25.5 24.9 23.9 23.8 16.7 21.2 20.0 18.0 16.6 16.25 Lowestincomeareas . , 28.9 28.9 27.6 29.3 23.9 28.1 30.1 30.5 27.4 24.98 FullTime . 29.2 29.7 26.3 28.1 24.0 26.4 19.2 22.6 20.3 14.6 12.4 11.95 Total 20.2 21.1 22.0 21.8 21.5 21.6 21.3 21.2 21.2 18.2 21.7 20.9 20.1 17.9 16.91 Source: MECAEP Table B-6. Percentage of Gross Development Product devoted to Social Sectors Sector 1999 2000 2001 2002 2003 % % % % % Education 3.2 3.1 3.3 3.3 3.2 Health 3.0 3.1 3.0 3.0 2.8 Socialsecurity and assistance 15.7 15.9 15.9 15.9 13.9 Housingand community 2.8 2.7 2.6 2.4 2.2 service Total GDP insocialsectors 25.4 25.9 25.5 25.1 22.4 Source: El gasto pbblicosocial en el Uruguay 1999-2003(p. 18) Table B-7. Percentage of the Social Budget devoted to Social Sectors Sector 1999 2000 2001 2002 2003 % % % % % Education 24.9 24.4 25.5 24.8 27.5 Health 17.9 18.7 17.5 16.7 18.0 Socialsecurity and assistance 49.7 51.7 51.O 53.4 49.8 Housingand community service 4.2 3.7 3.9 3.3 3.0 Overallpercentage of educationexpenditures 6.3 6.3 6.3 6.2 6.1 in the government budget Source: Elgastopbblicosocial en el Uruguay 1999-2003 (p. 13) Table B-8. Education budget breakdowns Categoriesof Education 2003 2003 US$ million % Educationbudget 402.0 Primary-preschooleducation 153.5 0.38 Secondary education 124.1 0.31 Highereducation 63.2 0.16 Nonformaland other education 7.0 0.01 Other educationservices 60.9 0.15 Source: Elgasto publico social en el Uruguay 1999-2003 (p. 41) 38 Annex B Table B-9. Budgetfor the primaryeducationand annualexpensesper student in 2002. Average of Uruguayanpesos 2002 - Salaries and performance Investments Total CEP 3,490,137,823 12,484,768 3,502,622,591 CODICEN - prorrata (a) 333,564,560 39,882,159 373,446,719 MECAEP 45,021,599 99,012,281 144,033,880 Total 3,868,723,982 151,379,208 4,020,103,190 Expensesper student US$2002 9,582 375 9,957 Average expenses per student (US$)2002 452 18 470 Expenseper student US$2003 (b) 11,439 448 11,887 (a)A proportionalquota of the CODICEN budgetwas allocated. The percentage is the CEP (peso presupuestaldel presupuesto) on the total of CEP+CES+ProfessionalExpert, (b) Updated by IPC Source:: Elaboracibn propia en base a Proyectode Rendicibnde Cuentasy Balance de Ejecucion Presupuestal,Ejercicio 2002, ANEP, junio 2003, 39 Annex C Annex C. Statements and Issues Raised inMission Interviews Interviewstook place individually or insmall groups as the circumstances dictated. The persons interviewed included: e 6 officials and staff inCOCAP and LATU e 6 government officials and staffo f MECAEP e 10principals, and 3 inspectors inthe schools o f areas visited. e Teachers were interviewedindividually as well as ingroups (9 inschool no. 359, 13 inschool no. 360, 58 ina teachers' meetinginLa Florida). The table below reflects the number o frespondents who indicateda view on each question; persons couldraise one or more issues but an opinionbrought up repeatedly by the same person counted as one comment. Not all staff had opinions about all questions and sometimes only one person ina group expressedopinions, so reply statistics are approximate. Teachers were only asked questions regarding inputs and effectiveness, since most didnot know who hadpaid for the inputsthey hadreceived. The questions posed to respondents were: e What were the benefits o fthe project inyour school, geographic or sectoral area? e What problems did you face inimplementingthe project? What were its disadvantages? e Which components worked best inbringingabout results, which didnot? (Some persons interviewedwere asked about specific components, as appropriate.) e How effectivewere the Bank staff or consultants who worked on the project? e What training didyou receive through the project? Didit teach you what it was supposedto? e Were there financial irregularitiesinyour area o fjurisdiction? e What would be different inthe education sector ifthe project hadnot existed? e Other issues and observations. 40 Annex C Table C-1Interviewresponsesand opinions Frequency of response Issue Education Vocations' MECAEP I MECAEP I1 Pmject benefits Helped establishindustrialquality controlcapability 3 Made it possibleto investin reachingthe poor 4 4 Increasedsectoral capacity 3 2 2 Developed governmentunitsthat could respond to needs fast 2 2 Mostand leasteffectivecomponents Much traininghelped; opened the mindsof teachers 4 5 Justteachers in project-supportedactivitiesget trained 4 Mostdifficult:supportschools with computers, maketeachers enter student data directlyon computer MIS unfinished-absenteeismdata not readily available 1 1 1 1 Achievement testingwas most useful 6 6 Mere trainingwithout class follow-upprovedto bringaboutlimitedchange 2 0 The biggest problemwas the slow expansion of full-timeschools 1 New schools are of low quality because of budget concerns and are very expensive to maintain;should buildsturdierand more expensive schools 2 2 Bilingualeducationwas poorlyconceived 3 Effectivenessof Bank staff, Bank policies Excellent 4 5 5 Stablestaffwho communicated well 2 5 5 Visionary projectdesign, laid ground for great progress 2 IADB is less effectivethan the World Bank in implementation and design of its 1 Qualifiedaudits, mismanagement Lossesinfrequent;some materialstheft by teachers I 1 Contractorsovercharge, put low-qualitymaterials 2 Principalswere askedto sign that civilworks were complete while they were 2 not Counterfactual -if projecthad notexisted The government would not havethe money to train poor studentsbetter 1 1 IftheWorld Bank had notfinancedthe project,the lnteramericanDevelopment 1 1 Bankwould have financed it General observations MECAEP managedto take advantageof resources 3 3 MECAEPcreatedexcellentschoolsfrom those that had low prestige 2 Some staff gained internationalrecognitionfrom MECAEPwork 1 1 The new textbooksare briefwith limitedinformation;earlier editionsgave more 1 1 Teacher promotions shouldbe made contingenton goingto trainingbut also passing exams in it 1 Buildingsare usuallyin good condition,withjust minor faults 4 4 Constructivismis good for the middle class but notthe poor 2 Classroom time use is poor 9 Children performparticularlypoorlyin language 4 4 There is an ethicalconcern if the poor are givendifferenteducationalmethods 3 3 Total number of comments receivedfrom 25 people 15 51 76 41 Annex D Annex D. Basic Data Sheet URUGUAY VOCATIONAL TRAININGAND TECHNOLOGICAL DEVELOPMENT PROJECT (LOAN 1594-UR) Key Project Data (amounts in US$million) ---<-a P A X . . ---a urn_-- Appraisal Actual or Actual as % of Estimate current estimate appraisal estimate Loan amount 9.7 8.5 88 Total cancellation 1.2 12 Total project cost 16.5 22.3 135.15 t Dates ----- l-P-.- Original -----Actual -_1 ____lll____,_l___~l.- - _I--- ~ - _ Board approval 06/23/78 1 06/23/78 I Signing 06/23/78 06/23/78 Effectiveness 04/27/79 04/27/79 -Closing date - 06/30/86 -0 _i___ .,_ . 06/30/83 z *i -1 Mission Data --*> _- 2 - _._ N" of people and ~ a . __*_ Date No. of Staff weeks (month/year) days Specializations represented Identification1Preparation 10176 7 1 1 Appraisal 10177 25 a 28.5 Post appraisal 03.78 5 2 1.5 Supervisions 1 05/79 5 2 AD 1.5 2 11/79 5 2 AD 1.5 3 06/80 1 E 1.o 4 05/80 5 3 2.0 5 I2/80 2 1 A 0.3 6 03/81 5 2 DF 1.5 7 04/81 1 F 5 a 05181 5 2 CE 1.5 9 06/81 2 1 B 0.3 10 12/81 7 1 G 1 11 I2/81 12 2C,G 1.5 12 06/82 7 1 G 1 13 06/82 7 1 G 1 14 I2/82 5 2 GH 1.5 15 I2/82 5 2 GH 1.5 16 06/83 6 1 GI 1 17 07/83 6 1 G 1 18 I0183 2 2 JB 0.5 19 I1/83 6 1 GI 1 20 07/84 7 1 G 1 21 I2/84 6 I G 1 22 09/85 6 2 GK 2 23 I1/85 1 K 24 03/96 1 K 1 25 06/86 a 1 G 2.5 a. Actual work on Loan 1594-UR as reported on 590' Forms b. A =Architect; B = Loan Officer; C = Consultanton Technical Buildings; D =Vocational Training Specialist; E = Technical Development Specialist; F = Operations Officer; Q = Senior Technical Educator; H = Deputy Division Chief; J = Division Chief; K = Senior Educational Planner. 42 Annex D URUGUAY EDUCATION BASIC QUALITY IMPROVEMENT PROJECT(LOAN 3729-UR) Key Project Data (amounts in US$million) =- - Aipipraisal Actual or Actual as % of estimate current estimate appraisal estimate Originalcommitment 31.5 30.6 97 Total cancellation 0.9 3 Total projectcost 45.00 44.77 99.5 I -._D_Xn__--- -_____Le- Original Actual _ l _ _ _ l ~ " _ ~ Board approval 05.10311994 05.10311994 Signing 0911611994 0911611994 Effectiveness 1211511994 1210911994 Closingdate 0613012001 0613012001 --_____n_i wr .- ~ *u LI --.*- = * II us$us$(looo) _ R _ _ . N" Staff weeks 7P *-- IdentificationlPreparation 81.8 199.0 Appraisal1Negotiations 20.9 51.6 Supervision 97.4 264.5 ICR 1.2 29.1 Total -------- 20.3 544.2 P 1__ p_u__l_--- Mission Data I__/ m- Date No. of Sipecializations represented Performance rating (monthlyear) persons `"lmplementatio Deve n progress objectives Identification1 March 1992 5 1 Educ. Specialist, Preparation May 1993 1textbooksspec., 1 preschoolspc., 1 cost spec., 1 economist AppraisalINegotiati November 1993 6 Missionleader, on 2 educationspec., March 1994 1 infrastructurespec., 1 institutionalstrength.Spec., 1 disbursementspec. Supervision December 1994 1 Mission Leader S S May 1995 3 Missionleader, S S 1 educationspec., 1 managementinformation system November 1995 4 Missionleader S S 2 educationspec. 1 infrastructurespec. September 1996 2 Missionleader S S 1 procurementspec. November 1996 3 Missionleader S S 1educationspec. 1 infrastructurespec. May 1997 1 1 infrastructurespec. June 1997 7 Leadspec. (Human 43 Annex D -- a_ Date No. of Specializationsrepresented Performance rating (montwyear) persons lmplementatio Development n progress objectives Development) task manager deputytask manager Educationspec. 2 procurementspec. 1 task assistant October 1997 3 Missionleader, 1 infrastructurespec. 1educationspec. January 1998 1 Missionleader March 1998 5 1 missionleader S H S 2 educationspec. 1 infrastructurespec. 1 procurementanalyst September 1998 1 1 infrastructurespec. November 1998 7 Missionleader S H S Sector leader 1 principalprocurementofficer 1financial mgmnt.Spec. 1economist 1 infrastructurespec. 1 educationspec. November1999 4 . Missionleader, 1 program HS H S advisor, `1 infrastructruespec., 1financialspec. June 2000 8 Missionleader HS H S 1educationspec. 1 programadvisor 1operationofficer 1 procurementspec. 1 operationsofficer 1disbursementanalyst 1 procurementspec. November2000 4 Missionleader HS H S 1 disbursementofficer 1 textbooksspec. 1 educ. Spec. ICR December2000 3 2 educationspecialists 1operations/education specialist September2001 1 1 educ.specialist S S October2001 5 1educationspecialist S S 1operationsspecialist 2 educationconsultants (preschool,studentcognitive assessmentsystem) 1operationsconsultant 44 Annex D SECOND BASICEDUCATION QUALITY IMPROVEMENT PROJECT (LOAN 4381-UR) Key Project Data (amounts in US$million) ---- ___ *---r- P w - " Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Originalcommitment 28 27.75 99 Total cancellation 0.25 1 Total projectcost 40.0 40.74 101.85 -- ---- _c ui.-,___l -Ai -- - Original- __ Actual _- - __II- X^_I__ ~" ----I Board approval 07/30/1998 07/30/1998 Signing 10/06/1998 10/06/1998 Effectiveness 01/29/1999 --- 01/29/1999 02/29/2004 ----a-- --_--- - r _ l - - - v m --- - N" Staff weeks us$us$(looo) IdentificationIPreparation 12.2 42.6 Appraisal1Negotiations 3.1 10.9 Supervision 59.2 181.5 ICR 9.8 38.8 Total -. -- 84.3 273.8 ~ Mission Data Date No. of Specializations represented Performance rating (month/year) persons lmplementatio Development n progress objectives Identification/ 10/20/1997 4 1Task manager Preparation 1educator 2 consultants AppraisallNegotiati 03/20/1998 8 1Task manager on 4 consultants 1 procurementspecialist 1financialspecialist 1programassistant 11/20/1998 12 1task manager 1sector leader 2 educationspecialist 1 procurementspecialist 1environmentalspecialist 1financial management specialist 4 consultants Supervision 05/21/1999 7 1 task manager S S 1sector leader 1educator 1 disbursementofficer 1disbursementanalyst 1 procurementanalyst 1 projectassistant 12/07/1999 4 1 task team leader S S 45 Annex D -* Date No. of Specializationsrepresented Performance rating (montMyear) persons - lmplementatio Development n progress objectives 1 programassistant 1financespecialist 1 school architect 06/30/2000 11 1 economist S S 2 operations 1 educationspecialist 1 team assistant 1 projectmanager 1 evaluation 1financedldisbursements 2 procurement 1 disbursements 12/12/2000 4 1 Sr. economist S 2 educationconsultants 1 operationsanalyst 10119/2001 6 1 Sr. economist S 1operationsofftcer 4 consultants 06/07/2002 3 1TTL S 1operat. consultant 1 educ. consultant 12/12/2002 7 1TTL S 1operationconsultant leducation consultant 1operat. analysVenviron 1 social dev. Manager 1 civilsociety spec. 1architect 05/21/2003 3 1 task team leader S S 1operationsspecialist 1 educationspecialist 11/13/2003 3 1 task team leader S S 2 consultants ICR 06/13/2004 4 1task team leader S S 2 consultants 1finance specialist