Document of The World Bank FOR OFFICIAL USE ONLY Report No. 19430 IMPLEMENTATION COMPLETION REPORT BRAZIL INNOVATIONS IN BASIC EDUCATION PROJECT (Loan 3375-BR) June 28, 1999 Brazil Country Management Unit Human and Social Development Sector Management Unit Latin America and the Caribbean Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS Local Currency Unit: Real (R$)1 Rate at Appraisal: US$1.00= Cz$230.34 (March 1991) Cz$1.00= US$0.004 Cz$l million= US$4,000 Rate at Completion: US$1.00= R$1.18 (December 1998) WEIGHTS AND MEASURES Metric System ABBREVIATIONS AND ACRONYMS CENPEC Center for Research on Education, Culture and Community Action Centro de Estudos e Pesquisas em Educa,cio, Cultura e A ,do Comunitdria DRE Regional Education Divisions (Divisoes Regionais de Ensino) DSE School Food Provisions Department (Departamento de Suprimento Escolar) FDE Foundation for Educational Development (Fundaqdo de Desenvolvimento de Educa,do) FIPE Economic Research Institute Funda,co Instituto de Pesquisas Econ6micas F UNCRAF Foundation for the Study and Treatment of Craniofacial Disorders (Fundaadopara o Estudo e Tratamento das Deformidades Crdnio-Faciais) FUTNDAP Foundation for Development of Public Administration (Funda,ao de Desenvolvimento da Administra,cdo Publica) FUNDEF National Basic Education Fund (Fundo para Ensino Fundamental) GSP Greater Sao Paulo IEB Innovations in Basic Education Project (Inova,ces de Ensino Basico) IT Information Technology NGO Nongovernmnental Organization PEC Continuing Education Program (Programa de Educa,do Continuada) SAE Sistema de Administracao Escolar SAEB Educational Evaluation System of Brazil (Sistema de Avalia,do do Ensino do Brasil) SARESP Educational Evaluation System of Sao Paulo (Sistema de Avaliaccio do Rendimento Escolar de Sdo Paulo) SEADE Foundation for Data Analysis (Funda,cao Sistema e Analise de Dados) SEE Secretariat of Education of the State of Sao Paulo (Secretaria de Estado de Educa,do) UNDP United Nations Development Program Vice President: Shahid Javed Burki Sector Director: Xavier E. Coll Country Director: Gobind T. Nankani Task Manager: Robin S. Horn The Cruzado (Cz$) was in use at the time of Project appraisal. The Real (R$) was introduced in 1994. FOR OFFICIAL USE ONLY IMPLEMENTATION COMPLETION REPORT BRAZIL INNOVATIONS IN BASIC EDUCATION PROJECT (Loan 3375-BR) TABLE OF CONTENTS Page no. PREFACE EVALUATION SUMMARY i PART I: PROJECT IMPLEMENTATION ASSESSMENT 1 A. Background 1 B. Project Objectives, Components and Design 3 C. The Implementation Record 6 D. Achievement of Project Objectives 8 E. Use of Consultants 22 F. Project Sustainability and Future Operations 22 G. Bank Performance 23 H. Borrower Perfonnance 24 I. Assessment of Outcome 25 J. Findings, Lessons and Future Actions 26 PART I: STATISTICAL INFORMATION Table 1: Summary of Assessments 35 Table 2: Related Bank Loans 37 Table 3: Project Timetable 38 Table 4: Loan Disbursements: Cumulative Estimated and Actual 39 Table 5: Key Indicators for Project Implementation 41 Table 6: Studies Included in Project 42 Table 7: Project Costs and Financing 44 Table 8: Status of Legal Covenants 45 Table 9: Project Costs and Benefits 50 Table 10: Bank Resources: Staff Inputs 50 Table 1 1: Bank Resources: Missions 51 PART III: SUPPORTING DOCUMENTS ICR Mission Aide Memoire 52 Borrower Completion Report, 3375-BR (Executive Summary) 53 MAP This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authonzation. I IMPLEMENTATION COMPLETION REPORT BRAZIL INNOVATIONS IN BASIC EDUCATION PROJECT (Loan 3375-BR) PREFACE 1. This is the Implementation Completion Report (ICR) for the Innovations in Basic Education Project, for which Loan 3375-BR in the amount of US$245.0 million was approved by the Board on June 26, 1991, and became effective on January 7, 1992. 2. The Loan Closing Date was extended by ten months and the Loan closed on April 30, 1999. Total disbursements were US$243.1 million and final disbursement (on funds committed at Project Closing) took place on May 14, 1999. 3. The ICR was prepared by a team composed of Kate Hovde (Consultant), Edward Bresnyan (LCSES), Adriana Weisman (LCSPR), Alberto Rodriguez (LCSHD), Robin Horn (LCSHD), Wilsa Maria Ramos, Claudio Rosemberg and Helena Jubany (Consultants). The ICR was reviewed by Anna Roumani (LCSES). 4. The ICR is based on material in the Project file, discussions with Bank staff and consultants who worked on the Project, a Borrower Completion Report prepared by the Secretariat of Education of the State of Sao Paulo (SEE) and an ICR mission to Sao Paulo which took place during March and April 1999. The Borrower Completion Report and a series of consultants' reports detailing implementation experience on specific project components are available in the project files. I IMPLEMENTATION COMPLETION REPORT BRAZIL INNOVATIONS IN BASIC EDUCATION PROJECT (Loan 3375-BR) EVALUATION SUMMARY Introduction 1. Despite an initial period of poor performance, the Innovations in Basic Education project (IEB) experienced a dramatic turnaround in its final three years of implementation, primarily due to strengthened commitment by State government and tighter, proactive Bank supervision. The Project was approved in 1991 and was initially developed to support the implementation of two educational reforms in Greater Sao Paulo (GSP). The first, ciclo bcisico, was initiated in 1984 and entailed a curricular reform of primary grades 1 and 2. The second -jornada i2nica - began implementation in 1988 and extended the school day to five instructional hours for ciclo bcasico students. In 1995, a new administration was appointed to the State Secretariat of Education (SEE) and ushered in a series of policy reforms to improve the quality of basic education in Sao Paulo. The IEB was reformulated at that time in support of these reformns. The Loan closed on December 31, 1998. Project Objectives 2. The objective of the IEB was to improve primary school learning and retention among children of poor and migrant families in Greater Sao Paulo in a manner that could eventually be adapted to other Brazilian states. This objective was to be achieved through support for: (a) a new curricular approach (the ciclo basico) and extended school day (jornada uinica) for the first two years of primary school; (b) expanded provision of preschool to poor children; (c) strengthened school health programs based on the new decentralized health system; and (d) the strengthening of institutions which manage primary education at the state and municipal levels. The project also provided for the extensive evaluation of these approaches and dissemination of the research results and implementation experience to the rest of Brazil. 3. The IEB had six components. In School Quality Improvement (80.0 percent of Base Cost) supported school construction and expansion; teacher training; purchase of classroom materials; and improvements to the school lunch program. Preschool Education (11.9 percent of Base Cost) aimed to increase preschool enrollment of poor children and broaden Nongovernmental organization (NGO) participation. School Health (5.7 percent of Base Cost) programmed nine health-related interventions, inter alia, annual vision testing and preventive oral health. Project Evaluation (0.6 percent of Base Cost) assessed implementation monitoring and process, analyzed project and education sector costs and conducted a longitudinal impact evaluation of the Project. Institutional Development (1.4 percent of Base Costs) supported technical assistance in planning and student assessment, as well as studies on the municipalization of primary education and strengthening public/private partnerships in education - particularly for preschool. Finally, Project Administration (0.3 percent of Base Cost) supported project management through the Project Coordination Unit. ii Total Project Cost (with contingencies) was estimated at appraisal to be US$600.0 million, including a Bank loan of US$245.0 million. 4. The Project's holistic approach and involvement of multiple agencies demanded strong administrative capacity from the SEE, while capacity was recognized as a risk. Subsequent implementation problems indicate that this risk was underrated. Compliance with guidelines for the preschool component - especially in regard to NGO participation and annual municipal preschool plans - was seen as risky and, in the final analysis, proved to be a significant weakness in project implementation. Implementation Experience and Results 5. The Project suffered from a disjointed preparation history and severe implementation problems for the first four years of its execution (1991-95). During this period, there were significant procurement bottlenecks. Later, a shortage of counterpart funds signaled weak political commitment of State government for the Project during a period of financial hardship. 6. At the time of the Mid Term Review, the Project had disbursed only 16 percent of the US$245 million loan. Primary school construction and rehabilitation - the largest project component - was substantially behind appraisal targets, due to the earlier (and now largely resolved) procurement bottlenecks and the frequent turnover within the SEE department responsible for the implementation of this component. In January 1995 and under new leadership, the SEE sought to restructure the IEB to bring it in line with new policy priorities, including (a) a school reorganization into buildings serving grades 1-4 and those serving grades 5-8, (b) an administrative reorganization decentralizing responsibility to local offices and increasingly devolving the responsibility for grades 1-4 to the municipal level, (c) a large-scale teacher development initiative; (d) special classes and efforts to meet the needs of children falling behind in school; (e) a second curricular reform, creating two "cycles" of material for classes 1-4 and 5-8; and (f) the development of a system to track children's learning achievement. However, implementation continued to languish until December 1995 when the Bank, with less than 20 percent of the loan disbursed, communicated its desire to cancel the undisbursed balance. 7. Subsequently, the Bank and the SEE negotiated and signed a revised project implementation agreement, based on time-bound quarterly benchmarks consisting of four types of indicators: (a) the bidding documents to be published each quarter, (b) the contracts to be signed each quarter, (c) the amount of financial resources committed each quarter, and (d) the amount of the financial resources spent each quarter. It was furthermore agreed that partial cancellation of the loan would be requested at the end of any quarter in which these benchmarks were not achieved. 8. Project execution and loan disbursement improved dramatically after January 1996. This resumption of the project was due to several factors. First, and most importantly, the new SEE administration was convinced of the relevance and importance of core IEB objectives, had successfully negotiated a number of project alterations with the Bank and was iii deter-mined to integrate project funds into ongoing educational reforms. Second, many procurement problems were resolved by this time, and the hiring of United Nations Development Program (for procurement of educational materials and teacher training) helped avoid others. Third, the Bank impressed upon the Government the need to improve significantly the project performance profile. This benchmark system clearly helped to drive implementation. The Loan closed on April 30, 1999 having disbursed 99 percent of the original US$245 million. Achievement of Project Objectives and Project Outcome 9. Dropout and repetition rates in GSP decreased dramatically over the decade of Project implementation, although direct attribution to IEB investments is not possible. Evidence on learning achievement is mixed; however, the State's development and implementation of an ongoing student assessment system is one of the main achievements of the Project. The Project surpassed its original targets with regard to investments in teacher training and educational materials, which helped to consolidate the ongoing policy reforms. The model for supporting students with a history of repetition is correctly being promoted nationwide. Project investments in information technology contributed to improved efficiency in education management. School construction and rehabilitation financed under the Project exceeded original poverty targeting objectives. The School Health and Preschool components fell substantially short of their implementation plans. 10. The project StaffAppraisal Report did not include an Economic Internal Rate of Return (EIRR). The Project is assessed as having reached its development objective and is rated Satisfactory. Sustainability and Future Operations 11. Sustainability of the investments in school construction, educational materials and computerization of the education system is dependent on adequate funding from SEE for (a) maintenance and replacement, (b) teacher and staff training for effective use of new materials and computers. There is also a strong role for Parent-Teacher Associations, regional SEE offices and the SEE's civil works department in ensuring the full useful life of these investments. Programs such as Accelerated Classes (Classes de Acelerafdo) are targeted, low cost, and have clear positive outcomes; they also work to reduce future expenditure on repetition. In a period of increasingly scarce resources, the educational testing program (SARESP) could be scaled back to every two years, without jeopardizing its objectives. School and parental involvement in the testing program remains the best assurance that the program with continue into future administrations. The Continuing Education Program (PEC) is being reviewed by the SEE for possible continuation; some follow-up in this area will be important in order to consolidate the gains made from its initial offering. Project Costs, Financing and Timetables 12. At Loan Closing, total project cost was US$601.3 million, compared with US$600.0 million estimated at appraisal. Bank loan funds (US$243.1 million) represented 40 percent of total project cost. The State Government contributed counterpart funds totaling US$349.1 iv million (58 percent) and municipalities providing financing of US$9.1 million (2 percent). The Loan Closing Date was extended by ten months to April 30, 1999. Loan Covenants 13. In general, covenant compliance was satisfactory. Municipal obligations in the preschool component as well as expectations regarding NGO participation were made explicit in the covenants (3.01(e),(f)). Of 39 municipalities eligible under the preschool component, only 26 ultimately entered into subproject agreements with the Project Coordination Unit and none of these 26 included NGOs in their municipal preschool plans. The Midterm Review was postponed by one year from its original date, due to slower than expected implementation of the project. The contracting and execution of project impact evaluation was completed on time as set out in the covenants, but was completed prior to substantial Project implementation, thereby weakening its ability to gauge Project impact. Performance of the Bank and the Borrower 14. Bank Performance. The Bank's preparation and appraisal, though lengthy, were satisfactory, with the exception of inadequate attention to procurement-related issues. This proved to be a major stumbling block during the early years of project implementation. During the implementation phase of the Project, Bank supervision was overly optimistic in its assessment of the State's ownership of the Project, later reflected in the difficulty in securing required counterpart funds. Only after four years of implementation and less than 20 percent of the loan disbursed did the Bank recommend partial cancellation. Earlier and stronger Bank action was warranted. Bank actions which contributed to improved project performance were: (a) the posting of Bank task management to Brasilia in 1995; and (b) the establishment with the SEE of time-bound, quarterly implementation benchmarks in 1996. 15. Borrower Performance. Frequent State-level staff turnover during preparation and appraisal led to a protracted project preparation process. Borrower commitment to the Project was weak up to 1995. The advent of a new State administration in 1995, which brought changes in education policy and significantly raised the level of interest and commitment to the Project, was chiefly responsible for the IEB's eventual satisfactory performance. Summary of Findings, Lessons and Future Actions 16. From the IEB experience, a number of generalizable lessons can be gleaned: (a) Early attention to procurement is vital. The IEB experience confirmed the importance of early and up-front training on procurement issues for borrower agencies, as well as the requirement that model bidding documents be approved prior to Board presentation. v (b) A flexible project design can speed implementation and avoid delays. The rigid design of the IEB (particularly Schedule 1 of the Loan Agreement) made it difficult to adapt its various components to the policy shifts in successive SEE administrations. Future projects should adopt a design that provides consistency in its objectives, yet can adapt rapidly to changing circumstances. (c) There is no substitute for strong monitoring and evaluation. The setting and attainment of quarterly benchmarks for IEB implementation after the Mid Term Review helped to jump-start the project and provided a transparent monitoring mechanism for both the SEE and the Bank. Earlier establishment of perforrnance indicators could have facilitated tighter supervision, ultimately improving the project's disbursement profile. _ Poor coordination between evaluation and other project components resulted in a "completed" Project impact evaluation prior to its substantive implementation. (d) Strong client ownership makes a difference. The drarnatic turnaround in implementation from January 1996 to December 1998 demonstrates how strong client ownership (in this case by the SEE) can catalyze overall project performance. The lack of such ownership was itself a major factor in the previous four years (1991-1995) of unsatisfactory performance. Proactive client consultation implies that the final project design is fully agreed and "bought into" by both the Borrower and the Bank. Any components for which such mutual agreements cannot be reached (e.g., preschool and school health) should be seriously reconsidered before their inclusion. (e) Decreasing project complexity can increase its chances for success. The holistic approach of the IEB and its diverse components demanded a sophisticated level of institutional articulation, not only within the SEE but also with other State ministries, NGOs and the private sector. Rather than a multiplicity of components, it may make sense to channel diverse yet worthwhile investments into separate loans. (f) Projects should more effectively assess local political cycles. Municipal and state elections and shifts in policy agendas must be more adequately incorporated in project designs and implementation schedules. Again, sufficient flexibility must be present in project design to allow for potential adjustmnents during administrative transitions. There are also lessons to be learned from several innovative activities undertaken by the State of Sao Paulo and supported under the IEB Project. In the following pages, case studies of the following innovations are offered: (a) Accelerated Classes, (b) Teacher Training, (c) Student Assessment, (d) School Reorganization, (e) Information Technology and (f) School Lunch Program. vi Correci Studet Flws CsesdAcelerafedo Acceeratd Clsses The issue. High repetition ratesin 3Brazil have resulted in significant age-grade distortions. According to a 7 MCreport; 63% of all stidents li grades 1-8 are older than they shouldbe for their corresponding grade. Multiple repetitions often jead to drout, as older students increasingly comparethe benefits of employmnt to participation in a poor quality system with little relevance tO twir nees. Repetition and dropout rates result in only two out of three children who enter first grade being expected to complete en grade, spending on average 10 vears to do so. 'Response. S4o Paulo took on its challenge i n 196 with heCsses de Acleroe progran. The program is an innovative example ofaddriss retiona dropout rtesby focussingon children Wi a history of repetition. Devel theSo PulobsCenter fResearch in Education, Culture, And Cmn Ai( P the Classes dis ull-out" program: children who have beehldback ianrare rt thepeers are -streamed into special 00classes with different materials an:dsp iatrin te s and thenmaIinstreamed$bak into the system after having "caugtup" to m Course materls follow a odlar f0om t, allowing students to progress at theirko wn- pce.aMatrialstarealso designed toflect the intrest and general knowlgebaseoan older-th goup of childre. Ovr first three years of its itplementain in0$ o Paulo, some 1, schools significant age-grade distortion paticiipated in the program, reachin atota of 01 chidre in grades 1 -4,,and a r 53,000 in Results. Existing statistics indicate if i hostated in "Acelraqa&r Pfrom 1996199S (chideingas1ad2 thcdobsk) 3RB percent were tracked back into 4" giade 19.6 pent were tracked it g and . percet were tracked into 3rd gre. t About one: -fih of chiidren (213 percent)xmved on to "eri 1I and the overall drtopout rate was 8.4 e rent - highe ant tatwi average, n o when cosidering that this is a population at particularly higrisk for doout. Of nin 0Acelrd1, 2.5 percent went on to 4k grade 76.3 percent wenton-to,5i grade, 2.8 perce in th"sA celera¸iio I1I or hadc some other e edal w ,d63*cendropped tout. Lessons learned. Lessons learned include the importance of: (a) school-wide familiarity and aceptance ofthep;gram,to smooth thewayfoir stjdents stransition back into "normal" classes; (c) good materials - the :CENCj materialsare wildlyypopular with.teachers and studentss alike-;(c,) seleting experienced and motivated teahers and givin-g themincentives (such as splecialized training) toparticipate; and0(d)buikldingstudent self-esteemalongwith content knwledge and skills (aided by small classes (25students), th emodular curricuum, fand continuous assessment rather than high-stakes tests). Ftutre Actions. The Classes de Ace`lerqj; model and CENPEC materials ae currently in use in several .other frazilian states besides S Paulo (including Para And Goias- Bahia an'd Ceara also have 0similar programs). The program i's sbei ngpromt inationally by the federal Ministry of Education, and was waarded a UCtEF national prize. There is as yet lle informaxtion available Xregardingthe educational outcomes of Classes d 4ra students, they are tracked back into the "regular" system. Evaluationof these longer term joutcomesf thEe program will be important in Sgaugingits overall effectiveness; however, inif}tal indications are pfromfAffising. vii Teacher Training: The Programa de EducaVo Continuada (PEC) The issue. The administration entering the SEE in 1995 had serious doubts about the effectiveness of ongoing teacher training efforts. Courses were generally centrally defined,, and were not necessarily responsive to school or regional needs. Most were "one-off' courses - classes of 20-30 hours duration, with little opportunity for.follow-up. Response. The PEC was an unusual example of trying to radically change the character of teacher training by working with traditional training providers but shifting the locus of power and the traditional "rules of the game," Most PEC courses met regularly over one year (96 hours), allowing course participants to model new content and practices in their schools. Previously, course content and modules were centrally planned by the Secretariat and negotiated with universities and other training providers; the PEC decentralized these activities to the local-level SEE offices (delegacias de ensino). SEE training organizers divided the state's 143 delegaias into 19 geographic areas (polos) each of which was matched with a training provider. The delegacias within eachpolo then negotiated with their designated training provider for training activities to meet their identified needs. All training activities were expected to incorporate certain cross-cutting themes and desired training results. Results. Results from the PEC varied enormously among the differentpolos. In areas where the local university or training provider had a history of working closely with the public school system, the negotiation and subsequent courses generally went smoothly. In others, the negotiation and implementation of courses proved difficult, either because the delegacia was unclear about its needs, or the designated training provider lacked the expertise desired. In a few cases, delegacias failed to reach agreement with the designated provider, and either implemented training activities from their own budgets or obtained SEE permission to negotiate with the provider assigned -to a neighboring polo. Despite these problems, the PEC achieved substantial coverage. Twelve training institutions were involved state-wide, covering l9polos, 143 delegacias, and 425 municipalities. These institutions developed&245 distinct training activities, involving 104,215 participants, in 2,761 groups for a total of 252,000 class hours of training between 1996 and 1998. While flaws in evaluation make it impossible to know the degree to which desired attitudes, behaviors, and content were actually adopted in the classroom, a majority of participants reported that PEC courses had helped theni better integrate new theories and practices into their ongoing work. The detegacias gained experience in identifying local training needs, and in negotiating with providers to make sure that courses actually met the needs identified. Although not originally an objective of the PEC, a very important ancillary benefit was the realization by many universities and training institutions of how little they knew of the realities and needs of public school teachers and administrators. As a result, several universities are currently re-thinking their pre-service teacher training and licensing programs. Lessons Learned. These include the realization that: (a) local empowerment (in this case of the delegacias) takes time; (b) there must be flexibility in matching training supply and demand; (c) planning needs to be as inclusive as possible from the outset; and (d) the decentralized provision of training requires particular attention to issues of monitoring and evaluation. Future Actions. As a model, the PEC is most directly relevant for areas which have a well developed system of universities and/or other training institutions which can reach the level of the school. Certain aspects of the program, however, such as its decentralized implementation, the identification of training demand closer to the school, a concem with reaching all members of the school team, and a concern for evaluation, are relevant to a much wider array of training designs. viii S: udn Assessment -the Sistema de Avi* do Rendimentto Escolar (SARSP) The issue. Thet: development af a stae student assessment system had been a goal of the I EB project from the outset. Education leaders increasingly felt that improving quality in education depended upon relable and constant information on studenit learning across the systiem. Yet until the early 1990s not testin system for gathering such infrmation existed, in Sao Paulo or in Brazil. jesponse. The SEE had experimented with standardized testing of selected schools in 1993 and 10994 butin 1t995 ittset about developing astate-wide system with the technical assistance of the Funda0 do W ( Cctrlos Chag. The first SARESP test was done in 1996, with a second round in 1997 and fa third in 1998. I0 SA S was designed as a census test (meaning all students in all schools in a particlar grade weretested) and asX a iongitudinal one (meaning the same cohort has been tested three yeSars running,to evaluategrtinelearning achievement). Th e SARSP test was applied thlrough a stroiollabration t tits regional offices, an&dschools. Students answer qest e wshch incld th aicifevement test and background informaation. The process is clevoylydeig&ed to: (I) guarantee reliabiity of the information, (2) have schools heavily involved in te prowess, aid (3 provide: Instant fe k: to schts and teachers regardi e perfrmance of their studets(stuetsltransfer answers foro booklets to an answer sheet which is sent to and corrected by the SEE, whe the bowokets sy with and are also corrected by the school). The first two rounds oftest applications were supported by a mass-media campaign, explaining the objectives of the test and encouragi student participation. Parenta] involvement was also encouraged through the form n of school-basedtlesting oversight committees. Resul t.ThSAkESPis technically sound, with a number of innovative fatures. Unlike the raziliannational assessment test (SAEB) whicl is conducted on a sample basis, SARESP's census design allows student-evel, classroom-level, and school-level analysis for all schools. SARESP's l dinal design also provides a strong measurement of school effectiveness, although it does raise questions in schoos(y always the sa kds) andissuesoftesting bias. Most importantly, SARESP is both 'a sunmaive and a formativet evaluation. Like most large assessment tests, tSARES isssummative in that provides information on the accumulated learning of students at a given powint.tthas playd a formatTive role within the education system, however, because through it teahers,parents and school leaders have learned to apply, utilize, and value assessment tests and the statisticaio nthat teiranMayses can provide. Lessons Learned; (a) There are strong benefits to parental and school involvement in the testing prcess; (b) :outside expertise is critical, but there are advantages to technology transfer over time; (c) thedevelopment ofCan asseassment systemn involves political will and leadership; (d) longituditnal desin has; both benefts andweak nesses; and (e) non-linkage to national testing efforts risk making SA Pa parallel effrt, and pehaps more politically vulnerable in a change of administration ; Future Actins. Saveil Brazilian states currently desiging assessment systems would do well to consimerincootg elements of A ESP' strategy for school and community-level participation. Overall,the downsides of the longitudinal strategy seme to outweigh its advantages an alternative would be to fbcus on testing only one or two grades on a consistent basis (for example, 4t and 8th grades).0 Cos$t, administrative burden, and the danger of information overload would also suggest that testi everytwo years, rather hifn every year, might be a better strategy. This spacing would also allowfor abetter pacing of national (&SA;EB) and state-level testing efforts. Finally, linking of state- level testing efforts with national ones is hiohly recommended. ix School Reorganization The issues. There were three main issues behind Sao Paulo's school reorganization. First, there was the issue of space. Overcrowding had long been a problem in Sao Paulo schools: in 1995, 19% of schools were still operating with more than two day-time shifts of students. Studies revealed that with a better distribution of enrollments between schools serving ensinofundamental (grades I -8) and secondary schools, a good deal of the pressure for space could be eliminated. Second, the administration entering in 1995 believed that separating younger and older children would permit a better pedagogical focus for each group, including the installation of age-appropriate pedagogic spaces, or sa/as ambientes. In contrast to most Brazilian states, the majority of schools in S3o Paulo housed grades 1-8 under one roof. Finally, greater municipal responsibility for ensinofundamental, as mandated by the 1988 Constitution, was complicated by the prospect of divided responsibility for the same physical buildings and school staff, Response. In response to these issues, Sao Paulo initiated in 1995 a massive reorganization of the school system into schools serving grades 1-4 and 5-8. Plans for the reorganization began with a state- wide student census. This infornation was supplemented by an assessment of community attitudes toward education, and focus-group opinion polling on the proposed reforn. The, SEE thus had an accurate basis for planning both the logistical and political aspects ofthe reorganization. In preparation for the reform, the SEE used mass-media (radio, television) and other means (booklets distributed to schools) to communicate the goals of the reform to schools, parents, and ,the public. The planning unit developed manuals illustrating how to conduct the reorganization of schools and students within municipalities of different size, while the actual planning of the reorganization in each municipality was left to the SEE local offices (delegacias do entsino) in conjunction with municipal authorities. While this strategy risked some delegacias doing a poorer job than desired, it avoided the much greater risk of mass-rebellion against decisions made from above, and saved an enormous amount of time. Results. Given the level of complexity (the Sao Paulo state system in 1995 had over 6,000 schools), the school reorganization went remarkably smoothly. Space gains were immediate: from 1995 to 1996, the number of schools with more than two daytime shifts dropped by 10 percent. Teachers and school officials confirm that, while the reorganization has been difficult, the narrower age-range of students facilitated their work, particularly in schools serving grades 14. In a public opinion survey conducted in 1998, nearly 80 percent of parents, 85 percent of students, and 71 percent of teachers. thought learming conditions had improved. The transfer of responsibilities for schools now serving grades 1-4 has also taken off. as of 1998, 283 municipalities had signed "partnership" agreements with the State, and the percentage of students in grades 1-4 in municipal schools had increased from 16% in 1990 to 31% in 1997. Lessons Learned. While the actual division of SAo Paulo schools into (1-4) and (5-8) is irrelevant for most Brazilian states, the reorganization is an excellent example of the planning and execution of a major systemic reform. Lessons learned include the importance of: (a) starting from a good information base; (b) communicating goals; (c) involving the system in making the change; and (d) maximizing the use of existing resources. Future Actions. Beyond learning from the reorganization as a planning example, the other potential relevance it may have for other states and countries is in the notion of creating specialized pedagogic spaces for different age groups (the salas ambientesj. Though potentially worthwhile, there is as yet little information about the actual use and educational contribution of these spaces. The cost of creating such spaces may also be prohibitively high for many states. x Thisue The admioiion ee in 95 ha a strna tiulated pos for improving9 th fetvns n iinc feuainlmngmn. taei betvsicue shikn h ieo h stae' edcational manageettu tur throug cosldto and00 decnrlizton-o funtion:s to oa fie dig4cas doensino), an strngteing planning,i0000; managemet, ndevlatinfn ction aX hecetrl,lcal ad schol eves. norer#00 fo ths eae depart e t n ev l; a ()ready access to re leva t n u -o -d t n formation for0 planning and 0 deiso maig.Inestents in informationtcnlg ee seen s as arttial answer to these neefds. Repue The SE bgnIts lnforaionrlutio~a h n d of 199 whn i odutda taewide stue: ces s inf antcpioof th reorganiatio of th :olsse.Studnsadschools were 0 assgnd uniqu idenifcationnumbrs anadatabsewas devielopd by the state dtaroessingf 00 coman, PRODEP. I 199, te SE deveope a; copehniv ehnlgyPa, pciyn $ystem St, uctu th the SEE w forced to. make so a g t this pa fome h bs for ie o o system. etaie cae fcmuesadrltdeupetwr made f rorutheSE ad4 USSOits LeAi*oU ofwd Mis: 0 cruia; dee,t d-t 0 ,., 0" d: scolsfr h deceAntralie purchas, of coples pnters adeltedo equipmnt The3 SEas re thes$ wtA e l;ElXqieait the aditr tio sl coinractThe.uSEE liaiieoatedthcnt y aft abut t285,000 nonexiteirost"stdns, tu s a s f a nfsc Intriews withi school adinistaosparticiatn in heS ilt also indicat that 6 both.0 th ne sytmadte ato ces to coputershas r sutditmesavngwth egardt routine reporting dele acsanthSE via thie Intere, n lsoalemaueo intrEn3 ae-govermna ollaboration.0 fLessons Leane. Tes iclud hesson tht:() sopisticatod manageen of ITis crucial, (b).:0t you i muthv opeesv lnbfr lanhngitoI * finvestet;()whre the markeXt: is wel FutireSS Acio . TheSEEis movngaha;d withte0 ful impentation of AE-EC priobably wthin the nxtt year. Scol ilb connected1 to jcxortesstmsva telntere ndth EE has X alread devlopdawebste (w.ducco.o.r wihifomto onitvaious pogas ad pbicit : access to oseveralg dataases(oabythtuet cesus dataan SAEP evauaion dat). ltoughin 0 ftheBailia cntxtS Palisntnecesaiy in th frfrn ofusof I n euctin,giend~ .the logtr trendtwrd inceae us f?wrdie aloseprec a rvd soe usefult ideasand eS ssons foro. t h;0;::0;aers.. 0t j;0t0q0-;:0.;0;^00tf.00ff; }t0-.0X 00;;f :0y ; f xi Enhancing Student Nutrition: The School Lunch Program (Merenda Escolar) The issue. In the early 1990's, the system for school lunch provision was based in large part on centrally procured, industrially formulated foods that were expensive and not well-accepted by students. Menus were outdated and, in general, procurement of foodstuffs was highly political and lacking in transparency. Response. The DSE has been proactive in constructing a transparent, analytical foundation for improving nutritional content and student acceptance of the merenda, Nearly three-fourths of the students are now served primarily through a decentralized approach, passing financial resources to the local level, empowering municipalities .and schools to take charge of implementation. A centralized program continues to serve most schools within the city of Sao Paulo. DSE also instituted a computerized management informi-ation system to track costs and control inventory, providing essential feedback on school lunch performance. The program took a critical look at menus, revising and enhancing offerings in ord,er to expand student acceptance and reduce wastage. A program of field supervision - in partnership with university-level nutrition intems - took stock ofexisting facilities, assessed nutritional content and provided on-site training for food preparation personnel. In order to improve nutrition and acceptability of the school lunch program, DSE also instituted a program for the purchase of fresh produce to supplement the processed foods in the menu. Technical specifications for foodstuff procurement were updated and made widely.available (e.g., listing on various websites). A study was commissioned to assess the poverty and nutritional profile of students statewide. Results. Greater efficiency in school lunch provision was achieved through less wastage and better cost control. In an 1998 evaluation, 97 percent of schools surveyed showed high student acceptance ofthe merenda, in part duetothereformstothe program. Some 11,300 merendeiros received a20- hour training module in food content, human nutrition and food storage and accident prevention. School kitchens were refurbished in 100 percent of centralized schools and in about 20 percent of decentralized schools. Results from the nutritional profile are contributing to ongoing menu analysis and enhancement. Lessons Learned. These include: (l) Decentralization is only as effective as the information which drives it, (b) a strong'sense of accountability raises the probability of success, and (c) partnerships can be cost-saving and mutually beneficial. Future Atons. The revised technical specifications for centrally purchased foodstuffs have already had an impact in the school lunch programs of other Brazilian states. In loco supervision of the centralized program contributed to quality enhancement; it remains to be seen how the decentralized program can benefit from similar supRision efforts. IMPLEMENTATION COMPLETION REPORT BRAZIL INNOVATIONS IN BASIC EDUCATION PROJECT (Loan 3375-BR) PART I: PROJECT IMPLEMENTATION ASSESSMENT A. BACKGROUND 1. From its conception in 1988 to its closure in 1998, the Innovations in Basic Education (IEB) project spanned a remarkable decade in Brazilian education. Although access to basic education grew steadily through the period of the military dictatorship (1964-1986), the resumption of civilian rule focussed new attention on the social sectors, particularly on issues of equity, financing, and management. The 1988 Constitution mandated increased decentralization of social services, for the first time giving guidance regarding the role of each level of government in the finance and provision of education. The new Constitution also paved the way for a new Federal Education Law, promulgated in 1996, and a bold attempt to equalize per- student spending on basic education within each state by radically revising federal and state education financing. Education, particularly basic education, also became increasingly important for democratically-elected state governors and mayors, spawning numerous localized efforts at reform.2 2. This wave of policy reforms was prompted by Brazil's very low education levels. In 1990, formal schooling for the adult population averaged only 3.9 years, approximately the same as El Salvador, Guatemala, and Nicaragua, countries with less than half the income level of Brazil. In part, this statistic reflects past problems of access to basic education which have been largely superceded today.3 Of increasing concern, however, were very high rates of grade repetition and dropout in ensino fundamental (I" to 8h grades). By the late 1980s, about 90 percent of all Brazilian children were entering primary school, but only 38 percent of them advanced to the eighth grade. This epidemic of school failure was a result of inter alia (a) wide disparities in per-pupil spending, (b) poor educational management, (c) poor coordination between state and municipal education systems (both of which are involved in the financing and provision of primary education), (d) poor teacher salaries and preparation, (e) inadequate educational materials and (f) poverty-related factors. 3. Bank Involvement and Sector Strategy. Bank involvement in Brazilian education began in the early 1970s with projects supporting general secondary, technical, and agricultural education. The Bank's involvement in primary education began in 1980 with its support for rural basic education in the Northeast (Ln 1867-BR) and for urban basic education in the North and Center-West (Ln 2412-BR). At IEB project preparation, the Bank's education strategy was as follows: to help improve the efficiency and effectiveness of government and private 2Basic education in Brazil includes preschool, ensino fundamental (Istto 8th grades), and secondary education. Primary education in this report refers to the first four grades of ensino fundamental. Access to grades 5-8 remains a problem in some parts of the North, Northeast and Center-West regions. 2 resources, so as to expand the supply of educated manpower that Brazil requires for competition in a world market characterized by increasingly sophisticated production. Within basic education, the Bank aimed to help end the long history of dropout, repetition, politicization, inefficiency and misdirection of funds (IEB SAR, para. 3.2). Specifically, the Bank's education strategy sought to (a) increase efficient use of federal government funds; (b) assist states and municipalities in strengthening the management and organization of their educational system; and (c) support innovative education programs - which could be adapted elsewhere in Brazil - to reduce dropout and repetition. The IEB project fit into the Bank's overall strategy at the time through: (a) its poverty orientation (targeting the children of poor families in Greater Sao Paulo); (b) its focus on basic education, with high social returns; (c) its focus on efficient use of public and private resources for the provision of social services; and (d) its ability to model reforms that could be replicated elsewhere. 4. Primary Education in Greater Sao Paulo. The State of Sao Paulo has long been Brazil's industrial center. While the region generates a great deal of wealth, there are wide disparities in income among its residents, particularly within the Greater Sao Paulo (GSP) Metropolitan area.4 Within the GSP region alone, the state education system in 1990 served approximately 1.8 million primary students, while the Municipality of Sao Paulo served another 400,000. Statewide, the Secretariat of Education (SEE) in 1990 directly employed more than 160,000 primary school teachers for almost 4.2 million students throughout the state, and had an annual budget of more than US$3 billion. The state and municipal shares of primary enrollment have been gradually shifting over the last few years as a result of an ongoing decentralization of responsibility for grades 1-4 to the municipal level. Whereas the state was responsible for 90 percent of primary enrollment in 1990, by 1997 this share had declined to 82 percent. 5. In the early 1980s, massive student failure, repetition and dropout characterized primary education in GSP. To address the problems of early grade failure, the state initiated in 1984 the ciclo basico - a major curricular reform of grades one and two in all State primary schools. The ciclo basico involved a new pedagogical approach to teaching reading and writing, automatic promotion from grade one to grade two, and continuous student evaluation during these first two grades in order to target special assistance to slow learners. In 1988, the State sought to further improve the quality of primary education by lengthening the school day to five hours of in- school instruction (jornada unica) for students in the ciclo basico. Due to severe overcrowding, however, in March 1991, only 62 percent of the State schools in GSP were able to implement the jornada uinica. The IEB was initially developed to support the full and integrated implementation of these two reforms. 6. By the time the IEB was signed in 1991, a new State administration had assumed power. While adhering to many of the policy goals and strategies the IEB supported, the new Administration proposed and implemented a third education initiative: the Escolas Padroes or Standard Schools. These schools were given more autonomy over their own budgets, a six hour school day, more time for teacher preparation, and additional educational materials. Schools constructed or repaired under the IEB were typically designated as Standard Schools. Again in 1995, another state administration entered with its own policy agenda, designed to tackle the persistent problems of repetition and dropout. The new administration universalized many of the Standard School features, such as increased budgetary autonomy, more teacher preparation time, and a longer school day for all students (five hours). The new administration undertook 4 Composed of the municipality of Sao Paulo and 37 surrounding municipalities. 3 other initiatives, including: (a) a school reorganization into buildings serving grades 1-4 and those serving grades 5-8, (b) an administrative reorganization decentralizing responsibility to local offices and increasingly devolving the responsibility for grades 1-4 to the municipal level, (c) a large-scale teacher development initiative; (d) special classes and efforts to meet the needs of children falling behind in school; (e) a second curricular reform, creating two "cycles" of material for classes 1-4 and 5-8; and (f) the development of a system to track children's learning achievement. The implementation and evaluation of the IEB should be understood within this overall political and policy context. B. PROJECT OBJECTIVES, COMPONENTS AND DESIGN 7. As defined in the Staff Appraisal Report (SAR), the objective of the IEB was as follows: to improve primary school learning and retention among children of poor and migrant families in Greater Sao Paulo in a manner that could eventually be adapted to other Brazilian states. This objective was to be achieved through support for: (a) a new curricular approach (the ciclo basico) and extended school day (iornada zunica) for the first two years of primary school; (b) expanded provision of preschool experiences to poor children; (c) strengthened school health programs based on the new decentralized health system; and (d) the strengthening of institutions which manage primary education at the state and municipal levels. The project also provided for the extensive evaluation of the effects of these approaches and dissemination of the research results and implementation experience to the rest of Brazil. 8. Total Project Cost was estimated at appraisal to be US$600.0 million. The IEB had six components. * In School Quality Improvement (80.0 percent of Base Cost) supported the implementation of the ciclo basico and jornada zunica through school construction and expansion (for a total of approximately 4,150 new classrooms), giving priority to low- income areas; training of ciclo bdsico teachers and school principals in GSP; purchase of educational materials for ciclo basico classrooms; and improving the school lunch program in terms of nutrition, acceptability to children, and cost efficiency. * Preschool Education (11.9 percent of Base Cost) aimed to increase preschool enrollment of poor children (from families earning less than two minimum salaries) by 120,000, improve its pedagogical quality and broaden participation of NGOs (not less than 15 percent of new spaces). Support to municipalities included funds for preschool construction, adaptation or expansion, teacher training, and technical assistance. * School Health (5.7 percent of Base Cost) looked to improve the health of approximately 1 million preschoolers and first and second graders in GSP municipal and state schools. Nine basic interventions were defined: (a) the integration of health and nutrition themes into the regular curriculum; (b) a physical exam for pre and primary school entrance; (c) annual vision testing; (d) completion of childhood immunizations; (e) preventive oral health practices; (f) a pilot study to measure the prevalence of vitamin A and iron deficiency, as well as a supplementation program; (g) physical and mental health referrals to the local health center; (h) school environmental assessments; and (i) training of health center personnel in pre and primary health and of teachers in health education. 4 * Project Evaluation (0.6 percent of Base Cost) was covered by four types of program evaluations: (a) implementation monitoring; (b) implementation process; (c) cost analysis; and (d) a longitudinal impact evaluation. * Institutional Development (1.4 percent of Base Cost) supported technical assistance in planning and student assessment, as well as financing studies on the municipalization of basic education and strengthening public/private partnerships in education - particularly for preschool. * Project Administration (0.3 percent of Base Cost) supported project management through the creation of the Project Coordination Unit. 9. Risk Assessment. One of the hallmarks of the IEB design was its holistic approach to addressing childrens' learning difficulties. Although the involvement of multiple agencies (e.g., SEE, Secretariat of Health, municipalities) was acknowledged as a risk, Sao Paulo's administrative capacity was thought good enough to make the risk a small one. Subsequent implementation problems - as detailed later in this report - indicate that this risk was underrated. Furthermore, the original proposal by the State of Sao Paulo focussed on support for improvements at the primary level alone; the preschool and school health components were included in the project at the Bank's suggestion. The implementation history of these two components indicate they lacked sufficient institutional ownership within the SEE and the Secretariat of Health. The SAR identified several risks associated with the preschool component, particularly that the participating municipalities would not be able or willing to comply with component guidelines, including the participation of Nongovernmental organizations (NGOs). Unfortunately, the project's mechanism for dealing with this risk - a year-to-year approval of municipal plans - was never fully used. 10. Project Design Alterations. The core IEB objective - improving learning and retention in primary school, particularly for poor children - held its relevance throughout the sequential waves of educational policy reforms in Sao Paulo. It is a testimony to the basic soundness of the project that it neither became irrelevant over the course of a decade nor required radical redesign. Nonetheless, a number of alterations were made within the framework of the original project design throughout its implementation, although not all of them were formally codified in the Loan Agreement. The original nine actions for the School Health component were reduced to four areas (i.e., dental health, vision and auditory screening and treatment, and a study of morbidity among school-age children) in 1993. At this time, the decentralization of the health system meant that the school health activities were being implemented by municipalities rather than by the State, and these areas were ones in which municipalities (as well as a new management team in the Secretariat of Health) registered the most interest.5 1 1. Additional changes were made by the new State administration in 1995. By this time, the ciclo basico had been implemented in Sao Paulo for a decade. Studies done both within the SEE and by an expert hired with IEB funds showed that demand for school placements was shifting to -th 6h 5 through 8th grades (as opposed to ls'to 4th grades).6 To maximize the efficiency of existing 5These were also the areas with financing through the Loan - the rest of the activities being financed with 100% counterpart funds. 6 See Klein, Ruben. 0 sistema de ensino regular no Estado de Sdo Paulo, 1983-94 (1995) and SEE/CIE. Exercicio analitico de politica educacional: 0 caso de Sao Paulo (1991). 5 schools (as well as for pedagogical reasons), the new administration proposed a reorganization into schools serving grades 1-4 and those serving grades 5-8.7 12. With this new configuration of the school system, the number of schools with more than two daily sessions fell from 19 percent in 1995 to 10 percent in 1996, allowing 90 percent of schools to fully implement the jornada unica of five hours for all students, not just those in the ciclo bdsico. Given the adjustment in demand for new classroom space, IEB funds for civil works were reallocated. Some funds for new school construction were reallocated for repairs and alterations to existing classrooms. The new SEE administration also introduced programs to improve teaching and learning, among these: (a) accelerated classes (Classes de Aceleraqdo) for children who through repetition have fallen behind their age-appropriate levels; (b) additional materials such as computers for new pedagogic spaces (salas ambientes); and (c) the Continuing- Education Program (PEC), which aimed to improve teacher training statewide through extended course hours with course content identified by local-level SEE offices.8 13. Simultaneously in 1995, an administrative decentralization of the State school system delegated more administrative authority to local SEE offices (delegacia) and encouraged greater municipal financial and management responsibility for schools serving grades 1 to 4. The IEB's institutional development component contained some financing for technical assistance on planning and studies on municipalization and changes were made to permit the financing of investments in information technology in support of these reforms. 14. The original project design was further altered in the dissemination of project activities and procurement. Originally, dissemination activities sought to share project study results with other states; alterations included activities to generate public support and understanding for ongoing educational reforms in GSP. Procurement shifted toward greater decentralization. The new administration sought to increase school autonomy by transferring funds directly to schools through parent-teacher organizations for the purchase of educational materials. From 1995 through project closing, educational materials under the Project were purchased under a combination of centralized and decentralized procurement procedures. Though it proved a difficult negotiation, in the end the Bank was convinced that decentralized procurement of a majority of items by the schools themselves would in fact result in greater expediency and less waste. 15. Alterations to the Project target area in 1996 extended coverage to two additional regions adjacent to GSP (Sorocaba and Vale do Ribeira), both with particularly low socio-economic indices. Although the IEB focus moved beyond the ciclo basico from 1995 onward, the focus on improving learning and retention in the early years of ensino fundamental was retained. 16. Finally, the implementation arrangements under the original design included the use of the SEE's implementation arm (FDE) across a number of components, including civil works, the contracting of consultants and teacher training. At Project reformulation, it was agreed that the United Nations Development Program (UNDP) would undertake the procurement of educational materials, teacher training and consultants, with FDE remaining responsible for civil works. In contrast to most states in Brazil, where primary education for the most part was divided into schools serving grades 1-4 and those serving 5-8, in Sao Paulo a majority of schools housed grades 1-8 in the same building. Development of the PEC, in 1996, was initiated slightly later than the other programs. 6 C. THE IMPLEMENTATION RECORD 17. Despite an initial period of poor performance, the IEB experienced a dramatic turnaround in its final three years of implementation, primarily due to strengthened commitment by State government and tighter, proactive Bank supervision. The Project suffered from a disjointed preparation history and severe implementation problems for the first four years of its execution (1991-95). By its midterm review in 1994, the Project had disbursed only 16 percent of the $245 million loan. From 1996 onwards, with a new State administration and the Bank's move to cancel a large part of the Loan, the Project made a dramatic recovery. The Loan Closing Date was extended by ten months to April 30, 1999 and all remaining funds were committed by that time.9 18. Certain implementation deficiencies can be traced to the original project design. The IEB was a traditional investment project - a "blueprint" approach - with the allocation of funds detailed by individual component and expenditure category, making flexibility in implementation difficult. At the time, both economic tunnoil and constant changes in the education and health sectors warranted a more flexible approach (perhaps with annual reviews and yearly allocations for project financing). The Project was also, in some instances, under- designed and not ready for immediate implementation. There was no operational manual, though few social sector projects had them at the time. At Board presentation, the school health component still lacked an implementation plan (which was included as a condition of disbursement for the component). Similarly, the financial flows of project money to municipalities under the preschool component were not clearly detailed. Monitoring and evaluation indicators were not explicitly defined, but were left to be specified during evaluation studies to be contracted under the Project.'0 19. Other implementation problems can be traced to the project's preparation history. The project was identified in December 1987 and appraised in February 1989. A dispute between the State and Federal governments over Sao Paulo's debt levels delayed negotiations until May 1991. After Board approval of the Loan, a new State administration took power and brought in new technical staff that had not assisted in the IEB preparation. There was also a switch in Bank supervision staff early into project implementation. With no institutional memory and very little in the way of detailed project implementation plans beyond what was contained in the SAR, the new project coordination team was in many ways "starting from scratch". Though the Bank did provide procurement training to the new SEE staff during a project launch in November 1991, the lack of early attention to procurement and failure to agree on bidding documents prior to Board approval also led to significant implementation bottlenecks. 20. From 1991 to 1995, the Project experienced significant procurement problems; from 1992 to 1994, there was an increasing shortage of counterpart funds. In hindsight, the project appeared to lack the requisite political commitment in a time of financial hardship. Within the SEE, the introduction of the escola padrdo in early 1992 took some of the focus away from IEB 9 The last disbursement of Loan funds was made on May 14, 1999, after which US$ 1.9 million were canceled from the original balance. For more extensive documentation of the implementation history of each project component, please refer to the unabridged version of the Borrower's Completion Report. 10 No overall project monitoring plan with agreed tracking indicators was developed. 7 implementation, and as procurement problems persisted, the Project may have been viewed as more of a problem than a solution to education needs. The housing of the Project Coordination Unit apart from the regular line units of the SEE may have also exacerbated the IEB's relative isolation. 21. Mid Term Review. In June 1994, following 30 months of project implementation, a mid term review of the project was conducted. At this time, the long-standing procurement bottlenecks had largely been resolved, while counterpart funding availability remained a potential problem. School construction was by now substantially behind expectations established at appraisal, due to several factors inter alia (a) the earlier effects of procurement delays, (b) the currency conversion from cruzeiro to reais, (c) personnel changes within the FDE and (d) the lengthy review of school construction subprojects by other government agencies (e.g., environmental reviews). It was discussed at the time - and later agreed with the Borrower as a formal amendment to the Loan Agreement - that the effective contribution by the Bank toward civil works would move from 31 percent to 40 percent.11 Overall, despite the extremely low disbursements at mid term, the administrative capacity of the Project Coordination Unit and the implementation capacity of the SEE was deemed sound, echoing earlier supervision assessments. 22. State elections for Governor were held shortly after the Mid Term Review. In January, 1995, a new Government was sworn in. A new Secretary of Education, who was both a national leader in education policy reform and politically astute, requested of the Bank the formal restructuring of the IEB in order to align it with the Government's new policy priorities. A delegation from the SEE met with the Bank in Washington in April 1995, the outcome of which was an agreement (in principle) to undertake substantial project design alterations. Despite the Administration's interest, project implementation continued to languish during the restructuring of the project design. From April to December 1995, loan performance actually deteriorated, with no disbursements processed over this time period. Furthermore, most of the time-bound actions agreed with the Bank during the April 1995 mission were not met. 23. Proposed Cancellation. In December 1995, with less than 20 percent of the Loan disbursed after four years of implementation, the Bank communicated to the Government of Sao Paulo its desire to cancel the undisbursed loan balance. At the same time, the Bank was reviewing other problem projects in the Sao Paulo portfolio: the IEB was not the only one facing implementation problems. In January 1996, the Bank, the Government of Sao Paulo, and the Federal Government undertook a unique, comprehensive review of all of the Bank-financed projects under implementation in Sao Paulo with the objective of cleaning and improving the entire portfolio. 24. Agreement on High-Stakes Project Implementation Benchmarks. During the comprehensive portfolio review, the Bank and Borrower agreed to give the IEB project one last chance. During this review, the Borrower inforned the Bank that it would request cancellation of the loan if the SEE failed to meet detailed and mutually acceptable implementation targets. I IThe currency conversion, combined with delays in payments to contractors, effectively lowered the Bank's planned participation in civil works. The increased percentage was therefore designed to compensate for greater than anticipated participation on the part of the State of Sao Paulo. 8 During the subsequent month, the SEE and the Bank task management team negotiated a new agreement on project implementation. Working backwards from the expected project completion date (September 1997) with 100% of the project implemented, the SEE and the Bank agreed on a project implementation timeline with detailed, quarterly implementation benchmarks. This final benchmark agreement, signed at the end of February 1996, included four types of indicators established across each of the components of the IEB: (a) the bidding documents to be published each quarter, (b) the contracts to be signed each quarter, (c) the amount of financial resources committed each quarter, and (d) the amount of the financial resources spent each quarter. The Borrower also stated that a partial cancellation of the Loan would be requested at the end of any quarter in which these benchmarks were not achieved. 25. Project execution improved dramatically after January 1996. This resumption of the Project was due to several factors. First, and most importantly, the new SEE administration was convinced of the relevance and importance of core IEB objectives, had successfully negotiated a number of project alterations with the Bank and was determined to integrate project funds into ongoing educational reforms. Second, many procurement problems were resolved by this time, and the hiring of United Nations Development Program to coordinate procurement of educational materials and teacher training helped avoid others. Third, the Bank impressed upon the Government the need to significantly improve the project performance profile. The loan closed on April 30, 1999 having disbursed 99 percent of the original US$245 million. D. ACHIEVEMENT OF PROJECT OBJECTIVES 26. Learning and Retention. It is notoriously difficult to attribute changes in learning levels to any one intervention, or even a series of interventions such as those supported under the IEB project. The project impact evaluation sought to demonstrate such linkages through a quasi- experimental, multivariate design which isolated project effects by holding other factors constant (e.g., the socio-economic status). Unfortunately, the impact evaluation was completed in 1995, prior to most of the IEB implementation; as such, it sheds little light on how project-financed inputs affected learning levels. While the impact evaluation did compare learning achievement of children in schools with different levels of inputs (schools implementing ciclo bdsico but without the jornada unica, schools with both ciclo basico and jornada zunica, and schools chosen to be "escolas padrao"), these inputs were for the most part not financed through the Project. The study was therefore far more useful as an evaluation of policy directions and effectiveness than as a project evaluation per se. 12 27. It is possible to describe trends in learning levels without direct attribution to the project. There are two main sources of information on learning levels in Sao Paulo: the Educational Evaluation System of Brazil (SAEB), a national test developed in the early 1 990s and given every two years, and the Education Evaluation System of Sao Paulo (SARESP), the State's own student assessment test, given every year since 1996. Comparing SARESP results from 1996 and 1997 (1998 results are not yet available), it is clear that, within the cohort of children being tested, these same children's learning levels are increasing annually. There was also a larger 12 The Bank placed such importance on the impact evaluation that its design was a condition of project effectiveness and the contracting was included in the Loan Agreement as a dated covenant. In fact, this strategy ended up backfiring, as the impact evaluation got off to a much earlier start than the rest of the project. Whereas the impact evaluation concluded in 1995, actual project implementation only began to gain momentum in 1996. 9 jump in mastery of new content and skills in the transition from 3rd to 4h grade than from 7h to 8th grade, and among children moving from 3rd to 4th grade in GSP as compared to the rest of the state. 13,14 Overall scores for the project region were slightly lower than those for the rest of the state, which is not surprising given the higher concentration of poor children in GSP. 15 28. The SAEB test, which is a national cross-sectional (as opposed to longitudinal) test, was given in 1995 and 1997. Results of testing 4th graders in Sao Paulo in both Math and Portuguese have been basically stable over this time period, with no statistically significant increases or declines. At the 8th grade level, however, results declined in both Math and Portuguese. 29. While these results may be an anomaly of the cohorts tested, they may also be in part attributed to the state's success in improving student repetition and dropout rates. Statewide, both failure and dropout rates for I stt 8h grades (ensino fundamental) have decreased steadily. over the last decade, with marked reductions beginning in 1995. Student flow analysis for Sao Paulo confirms a decline in repetition and dropout rates from 1987 to 1996, while revealing that most of the improvements in dropout have been in the early grades.'6 Within GSP, promotion rates for ensino fundamental increased from 68.5 percent of students in 1987 to 90.2 percent of students in 1997, while failure rates fell from 20.3 percent to 4.3 percent, and dropout fell from 10.2 percent to 5.5 percent during the same time period (see figure 1 below).17 With fewer failures and dropouts, the universe of students being tested has expanded, particularly to include many slower learners who might have been retained or opted out of the system in past years. Figure 1: Trends in Promotion, Retention and Dropout in GSP (1987-1997) E 100 E 80 40 0 60 ..... ..m fg 40 0 - - C*~~~~~~~~~~~rpouts 20 a. 0 87 89 91 93 95 97 Years Source: SEE - Centro de Informacoes Educacionais (CIE) 3 However, this relationship was reversed in the higher grades (7 " to 8t) 14 Children moving from 3rd to 0 grade improved mastery of Portuguese competency by 24% and Math competencies by 22%. Those moving from 7th to 8* grade improved Portuguese competency by 14% and Math by 9%. 15 To give an idea of skill levels, the mean score in Math for 4h'- grades in the project region tested in 1997 was 56.29. A score of 60 indicates that students possess (among others) the following competencies: (a) division of a number of three digits by a number of one digit, (b) completion of a numeric sequence made up of odd numbers; (c) resolution of simple word problems involving addition, subtraction, and multiplication; and (d) preliminary notions of geometry, including the measurement of sides and angles. 16 See Klien, Ruben, " Uma Analise do Sistema Educacional do Estado de Sao Paulo." 1999. Unpublished. 17 With regard to promotion, retention and dropout rates, rates for GSP are very close to those of the entire state. 10 30. Poverty Focus. Of all IEB activities, only school construction had explicit poverty targeting goals. The initial poverty targeting was based on 1988 data which facilitated the disaggregation of the local SEE offices (delegacias) in GSP into three distinct, homogeneous groupings.18 Over the course of the subsequent ten years, changes in parameters outside the control of the Project (e.g., increased urbanization, population shifts within GSP, demographic shifts in primary education, municipalization, school reorganization) as well as project-specific issues (e.g., slow implementation, revised numbers and locations of new school construction) called into question the continued validity of these earlier targeting efforts. At project closure, a follow-up study was commnissioned to assess the success of poverty targeting ex post in regard to school construction using recent (1998) employment data and specifying per capita household income as the principal variable of analysis.19 31. Under the poverty targeting at appraisal, 75.3 percent of new classrooms were to be constructed in Region III, the poorest of the three, 22.4 percent in Region II, and the remaining 2.3 percent in Region I, the wealthiest among the three (see Table 1). In 1995, following reformulation of the Project, new classroom construction for Region III was increased to 82.0 percent, Region II decreased to 17.7 percent, with Region I accounting for less than one percent. At Loan Closing, a total of 3,387 new classrooms had been constructed, 75.8 percent of which were located in Region III. Table 1: New IEB Classrooms Constructed, expected and actual (percentages in parenthesis) Project Appraisal Reformulation Loan Closing (1989) (1995) (1998) Region I 98 (2.3) 12 (0.3) 10 (0.3) Region II 929 (22.4) 619 (17.7) 455 (13.0) Region III 3,129 (75.3) 2,861 (82.0) 2,567 (75.8) Vale do Ribeira - - 55 (1.6) Sorocaba - - 300 (8 9) TOTAL 4,156 (100.0) 3,492 (100.0) 3,387 (100.0) 32. The analysis of per capita household income confirmed the original 1988 poverty targeting under the Project and, in fact, showed that the IEB actually exceeded these targets. GSP per capita household incomes declined from R$420 in 1988 to R$402 in 1998.20 Region III per capita household income in 1998 was 7 percent below the 1988 level. Income disparities also grew among the three regions over the project period. In 1998 Region III per capita income was slightly more than one-quarter of that for Region I and about 60 percent of that for Region II. 33. Of the non-construction activities funded under the IEB, the Classes de Aceleraqao showed the strongest implicit poverty link. Education research in Brazil has long established that poor children are disproportionately at risk of school failure, repetition, and dropout. Given that the Classes de Aceleraqdo focus on schools with a large percentage of age-grade distortion and are concentrated on children who have fallen behind their peers through repetition, it can be assumed that the program disproportionately benefits poor children. Other project resources 18 SEADE. 1989. Analise comparative das caracteristicas do atendimento educacional na Grande Sdo Paulo segundo regioes homogeneas. Sao Paulo. 19 SEADE. 1998. Pesquisa de Emprego e Desemprego - PED. 20 Per capita household incomes expressed in December 1998 Reais. 11 were distributed primarily on an "equal footing basis" although under the teacher training (i.e., PEC) guidelines, poorer areas were guaranteed training for 1 00 percent of their teachers. The school lunch program has also begun to integrate geographic nutritional profiles and poverty information into its food selection and distribution criteria. 34. School Construction. School overcrowding significantly decreased over the project period, due to both the construction funded under the Project and the school system reorganization. Overall, the IEB did well in meeting its construction targets. A total of 3,387 additional primary classrooms were built under the project, 97 percent of the 1995 revised estimate, and 81 percent of those expected at appraisal. In addition, 695 schools were remodeled or completely renovated. By 1998, schools with more than two daily sessions had decreased to 5.8 percent (343 schools), allowing the implementation of jornada unica of five hours in 94 percent of all schools for all students (not just those in the ciclo basico). 35. The school system reorganization also had pedagogic goals. By separating children according to age group, the SEE hoped to focus on age-specific learning needs, achieve greater school team integration, and introduce age-appropriate specialized pedagogic spaces (salas ambientes).21 Anecdotal evidence suggests that this strategy is working. Teachers and school officials confirm that, while the reorganization has been difficult, the narrower age-range of students facilitates their work, particularly in schools serving grades 1-4. In a public opinion survey conducted in 1995, just before the school system reorganization, 62 percent of parents and 35 percent of teachers thought the change would improve learning. In another public opinion survey conducted in 1998, nearly 80 percent of parents, 85 percent of students, and 71 percent of teachers thought learning conditions had improved.22 36. Evidence on the use of salas ambientes is mixed. In the same 1998 survey, 70 percent of school principals thought that salas ambientes had improved teaching. However, in a sample of teachers participating in the PEC program, 28 percent did not have or did not use the salas ambientes. It would be useful for the SEE and the FDE to more closely examine actual and desired use of space, as it has significant cost and pedagogic implications. In part because of construction quality improvements and architectural alterations to accommodate these new pedagogic spaces, construction costs under the IEB project nearly doubled.23 If these spaces are not being fully used, either their use should be encouraged or they should be eliminated in the future. 37. Teacher Training. Quality improvement in teaching was a priority of the SEE administration from 1995 onwards.24 Efforts in this regard included: (a) extra time for lesson preparation and collective discussion of teaching issues;25 (b) reform of teachers' career and 21 These included video rooms, laboratory space, an arts room, more exterior play space, and other alterations. 22 The 1995 survey was conducted with 6 focus groups in GSP and 8 in the interior of the State; the 1998 survey was conducted with a sampling of parents and school staff in municipalities served by the delegacia of Araraquaquara. 23 These alterations were first made in 1992 to accommodate the escolas padrdo. The architectural model for the escolas padrdo, however, is close to that currently in use and includes the salas ambientes. 24 Some training, mostly of ciclo basico teachers, was completed prior to 1995. In all 20,671 teachers and other school professionals were trained through workshops, seminars and courses, and about 20,000 participants through distance training from 1993-95. 25 Horario de Trabalho Pedag6gico Colectivo 12 payment structure; (c) teacher training for Accelerated Classes (Classes de Aceleraqao); and (d) the PEC. IEB funds were used in the implementation of the Classes de Aceleraqdo and in the design, implementation, and evaluation of the PEC, both of which are detailed below.26 38. The Classes de Aceleraqdo program was developed by SEE in collaboration with a Sao Paulo-based NGO (CENPEC). The program focuses on students who have fallen behind in school through repetition, accelerating them to their age-cohort and reintegrating them into mainstream classes at a higher grade level. Course materials follow a modular format and cover skills and subjects that would normally be addressed in ensino fundamental. Materials are also designed to reflect the interests and general knowledge base of an older-than-average group of children. 39. The SEE began the Classes de Aceleraqdo in 1996 with a sample of 160 schools that showed significant age-grade distortion in the primary grades. CENPEC was contracted to train the 160 pedagogic coordinators and 417 teachers in the selected schools in the program's objectives, approach, and materials. The program was expanded in 1997 to an additional 640 schools, and in 1998 to another 940 schools. In 1998, the SEE began experimenting with a similar methodology for grades 5-8, and training oriented to teachers of the Classes de AceleraVao was integrated into the PEC. To date, 133,000 students in grades 1-4 and another 53,000 in grades 5-8 have participated in the Classes de Aceleragdo. 40. Statistics indicate that the Classes de Aceleraqdo are working. Of children who started in Aceleraqdo I (children in grades 1 and 2 - the ciclo bdsico), 37.8 percent were tracked back into 4th grade, 19.6 percent were tracked into 5t grade, and 1.4 percent were tracked into 3rd grade. About one-fifth of children (21.3 percent) moved on to Acelerafao II and the overall dropout rate was 8.4 percent - higher than the statewide average, but not so high when considering that this is a population at particularly high risk for dropout. Of children in Aceleraqdo II, 2.5 percent went on to 4th grade, 76.3 percent went on to 5h grade, 2.8 percent stayed in the Aceleraqdo 11 or had some other remedial work, and 6.3 percent dropped out. 41. In its design, the Continuing Education Program (PEC) represented an important departure from previous training initiatives. Rather than a series of short (20 - 30 hour) courses, most PEC courses were intended to meet regularly over one year (most were 96 hours), allowing course participants to model new content and practices in their schools and return to class with questions. Previously, course content and modules were centrally planned by the SEE and negotiated with universities and other training providers; the PEC decentralized these activities to the local-level SEE offices (delegacias). SEE training organizers divided the state's 143 delegacias into 19 geographic areas (polos) each of which was matched with a training provider.27 The delegacias within each polo then negotiated with their designated training provider for training activities to meet their identified needs. All training activities were expected to incorporate certain cross-cutting themes and desired training results (e.g., participatory school management; school and classroom organization; creative use of curriculum; use of library and laboratory; continuous evaluation of students; use of educational statistics and information in work plan preparation). 26 Training for the Classes de Aceleracao came under project financing in 1998 as part of the PEC. 27 Some universities and training institutions were contracted directly (sole-source) by the State, while others were subject to a selection process. 13 42. Results from the PEC varied enormously among the different polos. In areas where the local university or training provider had a history of working closely with the public school system, the negotiation and subsequent courses generally went smoothly. In others, the negotiation and implementation of courses proved difficult, either because the delegacia was unclear about its needs, or the designated training provider lacked the expertise desired, or simply resisted the new arrangements.28 In a few cases, delegacias failed to reach agreement with the designated training provider, and either implemented training activities from their own budgets or obtained SEE permission to negotiate with the training provider assigned to a neighboring polo. Others delegacias (estimated at 20 or 30) dropped out of the program after only one course module. Despite these problems, the PEC achieved substantial coverage. Twelve training institutions were involved state-wide, covering 19 polos, 143 delegacias, and 425 municipalities.29 These institutions developed 245 distinct training activities, involving 104,215 participants, in 2,761 groups for a total of 252,000 class hours of training between 1996 and 1998. 30 43. Unfortunately, evaluation of the PEC started after courses were initiated. As a result, there was no ex ante, ex post classroom observation of teacher behavior which would indicate whether the teacher training had the desired effects. The lack of uniform monitoring instruments further complicated evaluation efforts.31 The evaluation did apply questionnaires to a sample of training participants. A majority reported having benefited from the PEC courses, and indicated that they were attempting to integrate course content and methodologies into their ongoing work. 44. Educational Materials. Nearly 28 percent of IEB loan proceeds went toward the purchase of educational materials - the second largest area of expenditure after construction. In general, the SEE did well in meeting implementation targets. Under centralized procurement, between 1993 and 1995, virtually all schools in the Project area received packages of supplemental reading books and other materials appropriate for use with grades 1-4. In 1996, all schools received a package of didactic books and videos, and schools serving grades 5-8 began receiving magazine subscriptions. In 1997, schools serving grades 5-8 received materials for the salas ambientes, including musical instruments, laboratory equipment, models and supplies, and computers, software and accessories for classroom use.32 45. The SEE's experience with decentralized procurement of educational materials was also positive. The first transfers of funds to Parent-Teacher Associations for the purchase of educational materials and furniture took place in 1996. In preparation for the transfer of funds, instructional booklets were produced and distributed to schools, and a mass media campaign launched to inform Parent-Teacher Associations of their responsibilities. These booklets outlined what kinds of materials could be purchased and their technical specifications, gave 28 Those institutions subject to a competitive selection process proved on the whole to be more flexible than those selected through sole-source procedures. Interestingly, some of the "premier" institutions also were among the most resistant to the change in training modalities. An administrator of the University of Sao Paulo, for example, complained that under this new system, he was expected to "send his Phd.s to the slums....." 29 Training was restricted to staff of state schools and "municipalized" schools. 30 Due to flaws in program monitoring, it is not possible to say exactly how many individual teachers were trained. 31 Evaluating institutions were the Universidade Federal de Rio Grande do Sul, Cesgran Rio, the FundaqJo Jodo Pinheiros in Minas Gerais, and the Fundagdo Carlos Chagas. 32 Each school serving grades 5-8 initially received 5 computers, to be complemented with another 5 through a different financing source. 14 guidelines on prices and local shopping procedures, and explained record-keeping and accounting procedures for purchases. Funds were transferred to 2,346 schools serving grades 1- 4 and 1,717 schools serving grades 5-8 for the purchase of a pedagogic kits, and to 2,187 schools serving grades 1-4 and 806 schools serving grades 5-8 for furniture purchases. Schools serving as teacher training centers also received an additional transfer for the purchase of didactic books and videos in 1996, for equipment purchases in 1997, and for professional development activities in 1998. 46. Both the quality and cost of procured educational materials under the Project were reasonable. Information on the use of these materials at the central level is mostly anecdotal. The SEE has been regularly publishing a school bulletin on computer use in the classroom, and it appears that at least some schools have been successful in incorporating the use of computers into lesson objectives. Teacher training in classroom computer use was also included under the PEC in 1998. Currently, SEE is implementing a methodology and an information system at the central level to monitor the regular use of computers. Given its innovative nature, relatively high upkeep and replacement cost, an evaluation of computer usage will help in charting the future direction of the program. 47. School lunch. While project disbursements for school lunch were miniscule (less than 1 percent of total project disbursements as opposed to an expected 2.7 percent of total loan at appraisal), the School Food Provision Department (DSE) within the SEE nonetheless made major advancements in the school lunch program in GSP during project implementation. When the IEB was developed, the school lunch program was a centralized program run by the State, financed with both Federal and State funds. By the early 1990's, school food procurement had been largely decentralized to the municipal level, with the exception of part of GSP (which nonetheless still contained a significant number of schools and students).33 Initially, both the scope and content of municipal participation in the school lunch component posed a stumbling block to program development, as did the reluctance of the DSE to permit outside scrutiny of its centralized program. 48. In 1996 a new DSE coordinator was appointed, a diagnostic study of the school lunch program was completed, and a number of important changes were introduced over the 1996-98 time period. Most of the changes occurred in the centralized program and included: (a) revised technical specifications for food procurement; (b) official menus on a trimestral basis; (c) a program for enhancing the school lunch; (d) field supervision of the school lunch program; and (e) purchases of kitchen equipment. Within this overall programmatic reform, IEB funded the training of food preparation personnel (merendeiros) and the computerization of DSE operations. To improve monitoring of both centralized and decentralized programs, the DSE computerized its information base and installed a cost monitoring system. Also, studies of the poverty and nutritional profile of students in the State helped to orient DSE's nutritional and distribution policies. 49. Today, some 626 decentralized municipalities receive Federal funds (R$0.13 per student per class day) combined with R$0.06 from DSE/SEE from which to provide school lunches for approximately 3.8 million students in nearly 3,200 primary schools. The DSE procures nonperishable foodstuffs for the remaining 19 centralized municipalities (approximately 1,400 primary schools with some 1.3 million students) - roughly following R$0.19 per capita limit. 33 Financing remains a joint responsibility of the federal and state governments. 15 With the aim of fortifying and increasing the acceptability of student lunches, 86 percent of centralized schools also participate in the School Lunch Enrichment Project (PEME), using earmarked funds for the purchase of fresh fruits and vegetables, meat and eggs.34 Preschools - under the budgetary responsibility of the municipality - receive a per capita supplement from the Federal government of R$0.06 for their school lunch programs. 50. Bank funds financed a complete inventory of existing cooking facilities in some 6,000 primary schools during 1997/98 and developed a standard kit of fifty kitchen items to be provided to each school kitchen. Using State fumds of approximately R$4.0 million, the DSE refurbished 100 percent of the centralized schools with these standard kits as well as some 20 percent of the decentralized schools. Additionally, DSE conducted intensive field supervision visits to all schools in the centralized school lunch program from 1996 to 1998. Student interns from university nutrition programs in Sao Paulo conducted these supervisions and evaluated the schools on food preparation, menu construction, nutritional content and student acceptability. Comprehensive training of school lunch personnel followed these supervision visits. Beginning in 1997 and continuing through mid-1998, some 11,300 merendeiros received a 20-hour training module in food hygiene, food content, human nutrition, food storage, accident prevention and interpersonal relations.35 5 1. Preschool. The cost (without contingencies) of the preschool component at appraisal was US$62.5 million, of which US$41.5 million was Bank financing (17 percent of total loan amount). At project closure, only US$4.9 million (2 percent of total loan amount) was disbursed toward this component. While it was expected at appraisal that 120,000 preschool vacancies would be created via construction financed under the project, this goal was dramatically scaled back to only 32,000 new vacancies (in 99 preschools with a total of 429 classrooms) at project reformulation.36 At project closure, only 53 preschools had been constructed with Project funds, yielding 241 new classrooms which created nearly 16,000 new preschool vacancies, well below the revised goals. Of the 39 municipalities eligible to take part in the preschool program, 32 signed Cooperative Agreements for the development of municipal plans, and 26 municipalities completed plans and signed subproject agreements (convenios) with the State for the receipt of funds. By 1997, only 11 of the original municipalities chose to continue in the program. 52. Implementation by the municipalities was delayed due to a relatively long planning period, a six month delay in anticipation of municipal elections in 1993 and counterpart funding difficulties. In accordance with the Loan Agreement, the rules for municipal participation were published, a seminar was held with interested municipalities, and a model developed to help municipalities complete the required preschool plan. To aid municipalities in plan development and the project management unit in its analysis, a database was created to analyze municipal demand for preschool and assess poverty and other socio-economic indicators. A group of 34 The funds earmarked for PEME are not additional, but rather specifically allocated within the existing budget envelope of R$0. 19 per capita. The National Program for School Lunch (PNAE) has established a minimum content of 350 Kcal with at least 9 g of protein for the school lunch. 35 Of these, some 3,500 merendeiros within the centralized schools were trained directly, with the remaining 7,800 merendeiros in the decentralized schools reached through 113 change agents (multiplicadores) in a training-of- trainers approach. 36 The original estimates of demand for preschool (based on 1988 data) were revised downward during the planning process with muncipalities and based on current data from 1990 and 1992 for census and the education sector, respectively. Independently, several municipalities had also dropped out of the program by 1995. 16 consultants - contracted through the project coordination unit - worked with municipalities to prepare their plans. The first transfer of funds, however, occurred only in 1994. 53. Participating municipalities initially had difficulties with disbursement and accounting procedures; those municipalities that could not accurately report project expenditures incurred delays in receiving additional funds. Training provided by project consultants largely solved these problems. Despite the project's provision of a simple architectural model for preschool construction, many municipalities opted for their own architectural plan, resulting in a significantly higher cost of construction. Two cost parameters for preschool construction were established at appraisal: aggregate construction cost (construction, furniture and equipment) - set at US$369 per square meter -- and cost per student - US$518. Evaluation of ex post construction costs reveals that aggregate cost increased by more than one-third (US$490 per square meter) while per student cost rose by 80 percent (US$934). The State counterpart financing was restricted to 70% of the estimated cost of construction at appraisal, regardless of the design ultimately chosen by the municipalities. Given the restricted financial participation of the State, combined with the onerous reporting requirements for reimbursement, many municipalities decided to drop out of the program. 54. No municipal plan included NGO participation -- a major departure from the original project design -- despite the rule that NGOs provide 15 percent of new preschool spaces. Plans were approved on an exceptional basis for the first year, with the issue of NGO participation to be revisited when renewal of the plans took place in year two. A study on NGOs was commissioned and a workshop held in 1994 with local NGOs to try to solicit their participation. The issue was not pursued further, however, and ultimately none of the preschool spaces under the project were provided through NGOs. 55. The original project design envisioned a state role in assisting municipalities to improve preschool education quality and content through curriculum development and training. A consultant team - hired by the SEE and financed under the Project - developed a new curricular proposal in 1994, leading to a training seminar with municipalities in early 1995. During 1995, however, the SEE gradually withdrew from this role, terminating its preschool coordinator, disbanding the group of consultants working with the municipalities, and dismantling the permanent SEE preschool pedagogical staff. From this point on, no training was conducted under the preschool component. 56. School Health. The school health component fell short of its intended objectives. Its implementation also suffered from delays, due in part to the ongoing process of decentralization of the health system. Health activities agreed under the Project with the State Secretariat of Health had to be renegotiated with all 38 municipalities in the GSP region. Lack of agreement between the State and the municipality of Sao Paulo over the terms of decentralization proved to be an initial stumbling block, as did delays in producing a coherent implementation plan.37 An implementation plan was finally approved in 1993, after a change in administration and project coordination within the Secretariat of Health. At this time, Bank-financed school health activities were scaled back to four main areas: oral health, visual screening and treatment, auditory screening and treatment, and a study on morbidity. 37 Submission of an implementation plan acceptable to the Bank was a condition of disbursement for the component. 17 57. Of these four areas, only oral health (school-based training of teachers and children in tooth brushing, flossing, and application of topical fluoride, as well as a system for monitoring activities) and auditory screening made significant progress. Visual and auditory screening plans were halted when it became clear that there were insufficient specialized personnel at the municipal level to adequately treat (or in some cases accurately diagnose) problems encountered.38 With regard to auditory screening and treatment, this difficulty was overcome with the signing of a contract with a private specialized institution (FUNCRAF) to conduct screening and treatment. An attempt to follow a similar strategy for visual screening met with Bank objection due to sole-sourcing. The design for the morbidity study was contracted out in 1994, but the actual proposal was not presented until 1996, and then at such an exorbitant cost that it was not considered viable. Later, negotiations during 1997 brought down the cost of the study, but by that time there was insufficient time for its completion.39 In all, about 1.7 million children in the project area benefited from project-funded activities in oral health, and 19,637 children received auditory screening. 58. Evaluation and Assessment. With regard to evaluation and assessment, the IEB failed to meet some of its goals and surpassed others. As mentioned earlier, the fact that the timing of the project impact evaluation and overall project implementation were not synchronized meant that the impact evaluation was more useful as an examination of policy directions than as a project evaluation per se. The process study suffered from similar limitations in that it concluded in 1996 - two years before Loan Closing. Monitoring was strong in some areas (e.g., civil works) and weak in others (e.g. teacher training).40 The original idea behind the project evaluation studies was to have them proceed in tandem, allowing for a constant flow of monitoring and process information to implementing agencies and providing the basis for conducting a cost- benefit analysis of project inputs. In fact, the studies were contracted and implemented over different time periods and the possibility of a cost-benefit analysis was discarded.41 And while the post -1995 inclusion of an external evaluation of the PEC program was commendable, the actual design and execution of the studies was in places flawed. 59. On the other hand, the evaluation studies that were done were used as a basis for policy decisions and actions - more than can be said for many a flawless study. The findings of the impact evaluation helped ground the policy proposals of the new SEE administration in 1995 .42 38 The Bank's goal in the school health component had been the promotion of simple, low-cost preventive screening measures to be undertaken largely by school personnel. The SES (perhaps rightly) declined to promote screening when they were not sure of the ability of the larger health system to respond with treatment. The SES attributes some of the difficulties it experienced with project implementation to the fact that communication with the Bank was routed via the Project Coordination Unit in the SEE. It was not informed, for example, of the possibility of the extension of the project closing date in time to permit the contracting of the morbidity study, and on at least one occasion a Bank no-objection to a major procurement process was delayed for months because the Project Coordination Unit failed to send on the request in a timely fashion. In short, the working relationship between the two agencies at the project level was not an easy one. 40 With regard to project monitoring, the Project Coordination Unit opted to rely on implementing line departments for information on physical and financial execution of the project, summarizing financial information in a computerized tracking system, and contracting specialized monitoring as needed. The benchmarks system in use from 1996-98 (though crucial) was limited to procurement and financial monitoring. 41 The impact study was contracted with the Fundaf do Carlos Chagas (FCC) in 1992, the cost study with FIPE in 1993, and the process study with NEPP in 1994.4' Contracting delays were for the most part due to negotiations with selected institutions over proposed designs and associated costs. 42 In part, this was serendipity - the team leader for the impact evaluation, Rose Neubauer, was appointed as Secretary of Education in 1995. 18 While diverging from its original intent (providing the basis for the cost-benefit analysis in conjunction with the results of the impact evaluation), the cost study evolved into the development by the Fundaqdo Instituto de Pesquisas Econ6micas (FIPE) of a computerized model for monitoring educational costs.43 SEE planning staff were trained in the operation and upkeep of the system, which is currently in use for planning purposes. The experience that the Funda,do Carlos Chagas obtained while implementing the impact evaluation also helped lay the foundation for the subsequent design and implementation of the Educational Evaluation System of Sao Paulo (SARESP), the State's own ongoing student evaluation system. 60. The development of a state assessment system had been a goal of the IEB project from the outset. The SEE had experimented with selected testing of the escolas padrao in 1993 and 1994, but in 1995 it set about developing a state-wide system with the technical assistance of the Fundaqdo Carlos Chagas. Education leaders felt that effective interventions for quality enhancement could only take place once reliable and constant information on the status of educational performance across the system was available. The first SARESP test was done in 1996, with a second round in 1997 and a third in 1998. SARESP was designed as a census test (i.e., all students in all schools in a particular grade were tested) and as a longitudinal one (i.e., the same cohort has been tested three years running, to evaluate growth in learning achievement). IEB funds were used both in the development and implementation of the SARESP. 61. The SARESP was applied through a strong collaboration between the SEE, its regional offices, and schools. Students answer questionnaires, which include the achievement test and background information. The process is cleverly designed to (a) guarantee reliability of the information, (b) have schools heavily involved in the process, and (c) provide instant feedback to schools and teachers regarding the performance of their students. The first two rounds of test applications were supported by a mass-media campaign, explaining the objectives of the test and encouraging student participation. Beyond the infornation provided to central policy makers, awareness and use of SARESP results appear to be growing at the level of schools and the immediate supervisory institution, the delegacias. References to SARESP results were made during several school interviews, and several delegacias made sophisticated use of results in identifying their training needs under the PEC. Through SARESP, PEC and other initiatives, the SEE appears to be gradually inculcating the use of information for planning and evaluation, an objective that extends well beyond the project's goals. 62. Institutional Development and System-wide Reform. The new SEE administration in 1995 had a strong and articulated proposal for improving the effectiveness and efficiency of educational management. Strategic objectives included: (a) shrinking the size of the State's educational management structure, through consolidation and decentralization of functions to local SEE offices (delegacias) and gradual transfer of financial and managerial responsibility for grades 1-4 to municipalities; (b) computerizing administrative functions and providing easier access to information at the school, delegacia, and central levels and (c) strengthening planning and evaluation functions at both the central and local level. IEB funds were used to support these objectives through the contracting of studies, software design, training, and the purchase of computer hardware and software. 43 According to Project staff, the FIPE study ended up as an input into the discussions of a base-line per-student cost under the FUNDEF, as few actual cost (as opposed to expenditure) analyses were available at the time. 19 63. Decentralization. Prior to 1995, education management in Sao Paulo had four tiers: (a) the SEE itself, which housed central budgetary, administrative, planning, policy, and regulatory functions; (b) 18 Divisoes Regionais de Ensino (DRE), which had some education supervision functions, but for the most part were administrative units handling personnel and financial issues; (c) 143 delegacias, which had administrative functions and were responsible for direct school supervision; and (d) the schools themselves. 64. In 1995, the state Government issued a decree eliminating the DREs, and reassigning administrative responsibilities and financial powers to the delegacias.4 Though the elimination of the DREs generated some cost savings, the larger implications of the move were political and managerial.4 The DREs had long served more of a political than educational function, and in eliminating them the administration removed a site of potential opposition to its pedagogic and other reforms.46 From a managerial standpoint, the move brought financial and administrative decision-making power closer to the schools, allowing for faster resolution of problems. 65. The decentralization of functions to the delegacias also created needs: (a) delegacia staff needed to be trained regarding their new responsibilities and functions; (b) to the extent that more planning, as well as administrative functions were being decentralized to the delegacias, local staff needed access to educational planning information and the tools to manipulate it; and (c) the SEE needed greater facility in communication with its local offices. To help meet these needs, the SEE used IEB funds to: (a) hire the FundaVdo de Desenvolvimento Administrativo (FUNDAP) to develop and conduct a training program for delegacia staff; (b) hire the Fundaado Sistema e Analise de Dados (SEADE) to develop a user-friendly research data base containing educational statistics and information (completed in 1998); and (c) launch a program to computerize the entire state educational system, including the school, delegacia, and central levels of administration. 66. On a parallel track, the SEE also pursued decentralization to the municipal level. Sao Paulo had begun a municipalization program in 1989, with the initial emphasis on transferring the responsibility for all state-run preschools to the municipal level. Whereas several Brazilian states (e.g., Parana) had begun experimenting in the early 1990s with the transfer of financial and managerial responsibility for grades 1-4 to municipalities, Sao Paulo lagged in part because the majority of its schools housed grades 1-8 under a single roof. The school system reorganization removed this particular barrier, and the passage of the 1996 Education Finance law (FUNDEF)gave an enormous financial incentive to municipalities to accept responsibility for more students.47 67. The result has been a surge in municipal enrollments. As of 1998, 283 municipalities had signed "partnership" agreements with the State, and the percentage of students in grades 1-4 in 44 Decree # 39,902, 1995. As of April 1999, the SEE announced a second wave of reform, consolidating the 143 delegacias into 89. 45 Largely through the elimination of office expenses, since personnel were for the most part transferred to the delegacia level. 46 Certainly the move could not have been made without strong support from a popular govemor. The timing of the change (early in the administration) was also probably crucial. The federal Fundo para Ensino Fundamental (FUNDEF), seeks to redistribute per-student funding for primary education within each state. Under FUNDEF, municipalities with large tax receipts but few students find themselves surrendering a portion of their required educational spending to the State. Similarly, poor municipalities with many students will receive additional funds through the State. 20 municipal or "municipalized" schools had increased from 16% in 1990 to 31% in 1997. To help municipalities better understand the consequences of the new education finance law and prepare for assuming more responsibility in the area of primary education, the SEE contracted (with IEB funds) FUNDAP to develop a series of training and management tools and materials for the SEE and municipalities. The long-term trend (as mandated by the Constitution and the new Federal Education law) is to have municipalities increasingly responsible for the provision and financing of primary education whereas the State retains responsibility for the supervision of primary education and the direct financing and management of secondary and the upper grades of ensino fundamental. It is still unclear to what extent the State will continue active oversight of municipalized primary education or adopt an increasingly perfunctory stance (as in the case of preschool) as time evolves. 68. Information Technology. The SEE began its information revolution at the end of 1995, when it conducted a state-wide student census in anticipation of the school system reorganization. Students and schools were assigned unique identification numbers, and a data base was developed by the State data-processing company, PRODESP.48 In 1996, the SEE developed a comprehensive Technology Plan, specifying system structure, applications, technological specifications, and organization. Although the SEE was forced to make some alterations along the way,49this plan formed the basis for implementation of the system. Centralized purchases of computer and related equipment were made for the SEE and 143 delegacias, networks linking the SEE and delegacias were established, and funds were sent to all state schools for the decentralized purchase of computers, printers and related equipment.50 A private software development company was initially contracted to develop a school-based information management system, and to train school, delegacia, and SEE personnel in the use of computers and basic software packages (e.g., Windows, Word, PowerPoint). 69. Although the development of a comprehensive Technology Plan was clearly a crucial first step to such a massive undertaking as the SEE proposed, the lack of internal Information Technology (IT) personnel proved a weak point. Ideally, the SEE should have hired more specialized IT personnel during the development and implementation of such a plan, complementing the consultant team formed by a few professionals. This was only partially done in 1998 when a specialized IT department was formed. This gap contributed to problems in the administration of the software development contract. Although the basic computer and commercial software training under the contract was done, the SEE was extremely dissatisfied with the development of the school-based information management system. The SEE abrogated the contract a year after it had begun, and chose instead to adopt the Ministry of Education- developed school-based management system (i.e., SAE).51 There are currently 200 schools in Sao Paulo participating in a pilot for the implementation of the SAE. 48 The census work by PRODESP (as a state entity) was not funded under the IEB. 49 In terms of structure, the original plan was to equip the delegacias to give technical support to schools. This structure was superceded by state-wide governmental mandates on the implementation of information technology, and the SEE is now relying on the superior technical expertise of the Fazenda and State Public Security Secretariat local offices in giving technical assistance to schools and delegacias. This intra-govermnental collaboration is unusual, and commendable. 50 All purchases were made with IEB funds. 51 The development and implementation of the SAE is being supported by the World Bank through the Northeast Basic Education II and III projects and the Fundescola projects. 21 70. The process of computerization of the education system in Sao Paulo is still far from complete, but there is both concrete and anecdotal evidence that it is making a difference. For example, the 1995 school census resulted in the elimination of about 285,000 non-existent or "ghost" students, producing significant cost savings on books, materials, and food through the school lunch program. Interviews with school administrators participating in the SAE pilot indicate that the new system and access to computers have resulted in time savings with regard to routine reporting functions. According to one school administrator interviewed, the fact that the school bulletin can be generated by computer "makes the students feel good - they want to take it home to show their parents because they think it's cool."52 Once schools are linked via the Internet to SEE databases, the potential for two-way flows of information for planning and communication purposes will also vastly increase. 5 71. Strengthening Planning. The SEE has made great strides over the last few years in making sure that relevant planning information is available, encouraging people to use it, and giving them at least some tools (such as computers) and skills with which to do so. Stylistically, this has been very much a "learning-by-doing" process. One example of this approach is the implementation of the PEC, in which the delegacias do ensino were thrust into the position of detennining their own training needs and negotiating them with local training providers. Some did well, others not so well, but all learned something. 72. Another interesting example of the comprehensive planning of a major reform is the school system reorganization into schools serving grades 1-4 and 5-8. The idea for the reorganization was rooted in: (a) the conviction that the separation of younger and older children would result in pedagogic benefits; (b) statistical information and analysis indicating that such a reorganization would substantially reduce school overcrowding; and (c) the political reality that it would be far easier to proceed with the municipalization of entire schools than "split" ones. Preparations began in 1995 with a statewide student census. A study of student flow in the Sao Paulo school system, and a beneficiary assessment study examining community attitudes toward education, were contracted and incorporated into the planning process.54 A mass-media campaign, involving television, radio, and the distribution of pamphlets, was developed to inforn parents and schools of the goals behind the impending change - a particularly crucial step, considering the potential of a political backlash. 73. The planning process also involved all parts of the system. SEE planning staff prepared a training manual and began the training of staff in the 143 delegacias do ensino on how to conduct the school reorganization within each municipality under their area of responsibility. The actual responsibility for analysis of local conditions, designation of schools and reassignment of students and teachers was given to the delegacias, acting in conjunction with local municipal authorities. The reorganization itself took place at the end of 1995, and the 1996 school year began with the new configuration of schools. As with the implementation of the PEC, the reorganization went more smoothly in some municipalities than in others. Overall, 52 Interview with administrator - 1ttp:v/'www.saeniec.iov.br - casos de sucesso - SP. 53 The SEE is exploring with the private institutions the possibility of their providing internet access for schools - an interesting example of public-private partnership. The SEE has also established its own website (http:"'www.educaca.sp.gov.br), though direct access to data bases is not yet available. 54 Klien, Ruben. "O Sistema de Ensino Regular no Estado de Sdo Paulo, 1983-1994". August, 1995. Funda,cdo Carlos Chagas. "A Importancia da Participa,co ComunitMria na Questdo da Educa,cao e da Pobreza. " Sao Paulo, 1995. Both studies were financed through the IEB. 22 however, there were far fewer problems than anticipated, considering the dimensions of the change. E. USE OF CONSULTANTS 74. Consultant services were used for all components of the Project. UNDP and FDE were used as implementing agencies for various sub-components such as the construction of schools; teacher training; acquisition of educational materials, including computers and software; parts of the school lunch sub-component; and evaluation and dissemination. Consulting services were essential to the preschool, teacher training and evaluation and dissemination sub-components. - Consultants were used to provide on-going project evaluation to monitor execution, analyze costs and processes, and assess the project's impact through the evaluation of children's learning and retention in school over time. Dissemination services included several public information campaigns, inter alia SARESP, decentralized purchase of education materials and training of school lunch personnel. F. PROJECT SUSTAINABILITY AND FUTURE OPERATIONS 75. As with any project, the sustainability of IEB project investments and reforms is in part contingent on the political and economic environment. On the political front, the reelection of the Covas government in 1998 and the continuation of the same administration in the SEE should go a long way toward ensuring the consolidation of reforms and the incorporation of new activities and practices into the life of the institution. On the economic side, however, Sao Paulo like the rest of Brazil is facing yet another period of economic turbulence and tightening budgets. Given this context, it is possible that some project-supported activities will have to be scaled back. 76. With regard to school construction, educational materials, and the computerization of the education system, the main sustainability concerns are regarding maintenance and (in the case of materials and computers) training, technical assistance and replacement. Although the quality of construction under the Project has been satisfactory, regular maintenance needs to be assured (both financially and institutionally), either through the training and transfer of maintenance funds to Parent-Teacher Associations, delegacias, an FDE post-construction inspection program, or contracting with third party providers.55 Regarding educational materials, although schools are undoubtedly better equipped than they have been in earlier years, funds will still need to be budgeted and transferred for replacement of items. Given the rapidity with which both hardware and software grow obsolete, the SEE will also need to make regular planned purchases or leasing arrangements to replace equipment and software bought through the IEB.56 Training and technical assistance for the school-based information system is guaranteed through the link with 55 At least one project school seen by a consultant two years after construction was in poor shape. Although the FDE does have a program to promote community involvement in school maintenance, this is currently only done after the construction is completed. Experience elsewhere has shown that community involvement in planning of schools leads to a greater sense of ownership and lower maintenance costs. 56 The SEE's current estimate is that they delay on a general re-equipping of the system until 2001 at the latest - resources are being budgeted for replacement. 23 SAE and the Ministry of Education, and information technology support from other government institutions is likely to continue at least through the current administration. Meantime, the SEE is changing its internal structure and defining its own information technology team, as per the Technology Plan. 77. With regard to other programs, including the Classes de Aceleraqdo, the testing program (SARESP) and the continuing education program (PEC), the most likely candidates for a scale back are SARESP and the PEC. Programs such as Accelerated Classes (Classes de Aceleraqdo) are targeted, low cost, and have clear positive outcomes; they also work to reduce future expenditure on repetition. Though the SARESP program is (rightly) one of the flagship programs of the current administration, it would be possible to make some changes to the design and test every-other year (as opposed to every year) without seriously compromising the program's objectives. This move would save money and allow more time for the SEE and schools to process information resulting from the test. Continued school and parental involvement in the testing program, however, remains the best assurance that the program will continue into future administrations. The SEE appears to have decided to pause and evaluate the PEC before deciding whether to continue training initiatives along these or other lines. Some follow-up to the PEC will be important, however, both for the credibility of the current administration and consolidation of some of the gains made.57 G. BANK PERFORMANCE 78. Successive identification and preparation missions in the late 1980's led to the formulation of the project consistent with the educational policy at the time. However, SEE staff turnover during the period was frequent. This impeded Project development since Bank staff frequently met with new SEE project staff at each successive phase of preparation and appraisal. Despite three appraisal missions completed in 1991, agreement on project components was at best tacit with regard to Preschool, School Health and School Lunch. The main project implementing agencies (the SEE and the FDE) had no prior experience with procurement under Bank guidelines, and the Bank did an inadequate job of covering procurement during project preparation. The adequacy of FDE procurement procedures for local competitive bidding of civil works was examined only in the final stages of project preparation. The Bank initially concluded that the procedures and bidding documents were adequate, but reversed its position once project implementation had begun and it became clear that the procedures were not generating sufficient competition. The failure to require a project operation manual also contributed to the early lags in implementation; such a manual should have included the bidding documents to be approved by the Bank and used by the project. Until 1997 significant delays in the provision of "no objection letters" occurred; however, with the decentralization of the CMU to Brasilia these delays were reduced. 79. The Bank was overly optimistic in the early years of project implementation regarding the State's ownership of the project and its desire to overcome bottlenecks in implementation, particularly counterpart contributions. It was not until 1995 - after four years of implementation resulting in less than 20 percent disbursement of Loan funds -- that the Bank recommended 57 Among school personnel, there are already some complaints that if the PEC does not continue it will be "just like every other training - there is no point in calling it 'continuing education' if it does not continue." 24 partial cancellation of the Loan. Earlier and stronger Bank action was warranted. Nonetheless, the Bank did demonstrate appropriate flexibility both in the redesign of the project in 1995 and the closer supervision afforded the project by placing an education staff member in the Brasilia office. R. BORROWER PERFORMANCE 80. Procurement. Procurement proved a major stumbling block in the early years of the project (1991-94) and was responsible for a significant portion of project implementation delays. The FDE in particular proved extremely reluctant to change its procedures to meet Bank guidelines, causing months of delay on the civil works component. To make matters still more complicated, Brazil's own procurement laws were under revision at the time, leading to multiple revisions of bidding documents in the attempt to meet both Bank guidelines and local law. By 1994, final agreement on local and international bidding documents and procedures had been reached. The first International Competitive Bidding processes were published, and the advertisement and bidding of civil works resumed. UNDP was contracted in 1995 to carry out the procurement of educational materials and teacher training (in both cases, centrally procured).58 81. Single source contracting of consultants was common throughout the Project, yielding mixed results. With the heavy emphasis in the IEB on human resource requirements, single source selection of consultants was often used, primarily because there were few specialized institutions that were capable of providing training for teachers or evaluating the training being provided. In some cases (e.g., the evaluation studies) prolonged negotiations over design and cost might have been avoided under a more competitive format for contracting these consultants. In other cases (e.g., contracting of teacher training providers), single sourcing permitted a quick start to project implementation. It should be noted, however, that single sourced providers also proved less flexible in adjusting training content to expressed demand. 82. Audit. In compliance with Bank requirements, audits were performed annually during 1992-98 and the auditors considered the accounts to be adequate and reasonable in all significant aspects. Although usually completed without significant delay, the audit reports for 1994 and 1995 were late due to project reformulation. All of the audit reports were issued without qualification. The Project staff responded promptly to the Bank Auditors' recommendations in the audit report covering 1997. Specifically, the Project Coordination Unit incorporated in its 1998 statement of funds the mechanism used by the Bank to disburse funds to the Project as recommended in the 1997 audit report. In reviewing the 1997 report, the Bank's Auditors also recommended tighter internal controls, especially treatment of gains and losses in currency exchange caused by the use of currency pool loans, delays in submission of some accounts, and cases of non-compliance with procedures regarding the submission of accounts by contractors. In all cases the Project Coordination Unit demonstrated compliance with the recommendations. 58 According to Project staff, the hiring of UJNDP was a "necessary evil" given the FDE's intransegence in procurement on the one hand, and the need for swift action (made possible with UNDP) in order to meet project benchmarks. 25 83. Covenant Compliance. In general, covenant compliance was satisfactory. Municipal obligations in the preschool component as well as expectations regarding NGO participation were made explicit in the covenants (3.01(e),(f)). Of 39 municipalities eligible, only 26 ultimately entered into convenios with the State for preschool in some form or another and none of these 26 included NGOs in their municipal preschool plans. The Midterm Review was postponed by one year from its original date, due to slower than expected implementation of the project. The contracting and execution of project impact evaluation was completed on time as set out in the covenants. Yet ironically (as stated earlier) the impact evaluation was completed prior to substantial project implementation. I. ASSESSMENT OF OUTCOME 84. The IEB did not calculate an Economic Internal Rate of Return (EIRR) during appraisal nor was one estimated at Project Closure.59 Nonetheless, the Project contributed in a positive way to improved learning conditions for poor children in Sao Paulo, which was a principal project objective. Given the project's early poor implementation, the fact that it fully disbursed the remaining 84 percent of loan funds in its final three years is remarkable. 85. Nearly one-half (47 percent) of disbursements supported school construction, remodeling and renovation, primarily in those areas were poverty is concentrated in GSP (i.e., Region III). Additionally and importantly, effective targeting of school construction implemented under the Project was confirmed. Another 28 percent of disbursements supported the purchase of educational materials for schools with grades 1-4, as well as selected books and videos for all schools (1-4 and 5-8) and magazine subscriptions for grades 5-8. Evidence is scant, however, on the use of these purchases and how they have actually impacted learning. The salas ambientes proved a more costly investment than initially envisioned and again, their subsequent utility is uncertain. Computer equipment for classroom instruction was widely procured and installed; evidence (though minimal) is positive in regard to its effectiveness in student learning. 86. Teacher training (PEC) was successful in its outreach and it its effort to more closely link teacher training demands with available training providers. While the matching of training demand and supply was not always precise (due to scarcity, poor articulation of demands, incompatible training providers, etc.), this attempt to decentralize teacher training was indeed a step in the right direction. By removing constraints on negotiations in future training events, better training fit can be achieved between desired content and timely provision of services. Greater attention to evaluation of teacher training in the future will also aid in confirming the anecdotal evidence to date of its effectiveness. 87. The Classes de Aceleraq ao also reached a broad spectrum of at-risk children in over 1,800 schools with targeted interventions to raise them to the level of their age-grade group. Statistics from the program indicate that more than 90 percent of these children are successfully mainstreamed into their appropriate age-grade groups after completing the accelerated program. However, as in the case of educational materials, we know little to date as to how these children have faired subsequent to their return to mainstream classrooms. 59 This is customary in social sector projects 26 88. Neither Preschool nor School Health achieved a sufficient degree of ownership within the SEE, eventually contributing to their poor implementation record. Performance for the Preschool component was significantly less than expected. Ambiguity concerning municipal participation, the absence of long term, six year municipal preschool plans (as agreed at negotiations) and SEE's abdication of teacher training for the preschool component hindered successful implementation of this component, even after the project reformulation. Conversely, while School lunch performed below expectations in terms of disbursements, SEE nonetheless made great strides in improving the quality of the school lunches through menu enhancement, improved technical specifications for food procurement, training of personnel, kitchen refurbishment and extensive field supervision. J. FINDINGS, LESSONS AND FUTURE ACTIONS 89. From the IEB experience, a number of generalizable lessons can be gleaned: (a) Early attention to procurement is vital. One lesson that was quickly internalized into the subsequent Brazil education lending portfolio was the need for early and up-front training on procurement issues for borrower agencies. On a larger scale, procurement related problems in a range of sectors finally prompted the Bank and the Federative Republic of Brazil to work together to develop a set of model bidding documents in Portuguese for local competitive bidding of goods and works within Bank-financed projects. (b) A flexible project design can speed implementation and avoid delays. In retrospect, the rigid design of the IEB (particularly Schedule 1 of the Loan Agreement) made it difficult to adapt its various components to the policy shifts that occurred with each successive SEE administration. Future projects would do well to adopt a design that provides consistency in its objectives while simultaneously adapting rapidly to changing circumstances. (c) There is no substitute for strong monitoring and evaluation. The setting and attainment of quarterly benchmarks for IEB implementation helped to jump-start the project and provided a transparent monitoring mechanism. It also provided both the SEE and the Bank with common ground for discussing bottlenecks as they occurred. Earlier attention to establishing performance indicators could have facilitated tighter supervision, ultimately improving the project's disbursement profile. While the IEB sought to evaluate project impact on learning achievement, poor synchronization between evaluation and other project components resulted in a "completed" impact evaluation prior to substantive implementation. (d) Strong client ownership makes a difference. Many of the early implementation difficulties with the IEB can be traced to a lack of SEE ownership. In fact, the dramatic turnaround in implementation from January 1996 to December 1998 demonstrates how strong client ownership can catalyze overall project performance. The lack of such ownership was itself a major factor in the previous four years (1991-1995) of unsatisfactory performance. Proactive client consultation implies that the final project design is fully agreed and "bought into" by both the Borrower and the Bank. Any components for which such mutual agreements cannot be reached (e.g., preschool and school health) should be seriously reconsidered before their inclusion. 27 (e) Decreasing project complexity can increase its chances for success. The holistic approach pursued under the IEB would present an implementation challenge even in an ideal situation. The diversity proposed in the project's components requires a fairly sophisticated level of effective institutional articulation, not only within the SEE but also arnong it and other State ministries, NGOs and the private sector. Rather than a multiplicity of components, it may make sense to channel diverse yet worthwhile investments into separate loans. (f) Projects should more effectively assess local political cycles. Municipal and state elections and shifts in policy agendas must be more adequately incorporated in project designs and implementation schedules. Again, sufficient flexibility must be present in project design to allow for potential adjustments during administrative transitions. 90. The IEB also fostered specific innovations which can serve to increase the effectiveness of other Bank-financed projects. These innovations are (a) The Continuing Education Program (PEC), (b) The Sao Paulo School Learning Evaluation (SARESP), (c) Classes de Aceleracdo, (d) School Lunch Program (merenda escolar), (e) information technology and (f) school reorganization. An overview of each of these innovations, followed by respective lessons learned and future actions, is provided below. 91. The Programa de Educacao Continuada (PEC). The PEC is an unusual example of trying to radically change the character of teacher training by choosing to work with traditional training providers (local universities and training institutions) but shifting the locus of power and the traditional "rules of the game." The program was ambitious, in that it was developed in less than a year, implemented state-wide, and involved a large number of institutions, participants, training modalities and content areas. While flaws in evaluation make it impossible to know the degree to which desired attitudes, behaviors, and content were actually adopted in the classroom, a majority of participants reported that PEC courses had helped them better integrate new theories and practices into their ongoing work. The delegacias gained experience in identifying local training needs, and in negotiating with training providers to make sure that courses actually met the needs identified. Although not originally an objective of the PEC, a very important ancillary benefit was the realization by many universities and training institutions of just how little they actually knew of the realities and needs of public school teachers and administrators. As a result, several universities are currently re-thinking their pre-service teacher training and licensing programs. 92. Lessons learned from the PEC are as follows: (a) Local empowerment takes time. Delegacias were not equal in their ability to identify local training needs and negotiate with training institutions. There is general agreement, however, that the 1997-98 round of courses were better than those given in the first year. It is possible that more preparation of the delegacias regarding their new role might have helped avoid some problems, but as with any substantial change in policy or practice, there is likely to be a period of "working out the kinks" during which people learn from their mistakes. (b) There must be flexibility in matching training supply and demand. The programn design limited competition in the supply of training services. Delegacias and training suppliers were pre-matched, and if a particular delegacia 's determination of training needs 28 did not correspond with what the provider was able to offer, the delegacia was left to either accept the offer or opt out of the program. While the SEE did allow greater flexibility in the second year of the program (authorizing some delegacias to negotiate with providers serving neighboring regions), some intra-regional competition among providers from the outset might have led to a better selection of courses. It should be noted that the training providers contracted through competition (as opposed to sole-source) generally exhibited more flexibility in trying to meet delegacia needs. (c) Planning needs to be inclusive. PEC planners could have included interested universities and delegacias in compiling the list of cross-cutting themes and desired outcomes of courses, thus ensuring a better understanding and ownership of the SEE's overall training goals.60 At the university level, the very fast turnaround time for submitting course proposals meant that the proposals were not in some cases integrated within the university, and different groups ended up putting together and implementing the proposal. At the delegacia level, an equally short time for putting together training needs meant that those delegacias with strong connections to local schools assembled strong proposals, while others did not have time to undertake a more in-depth consultation. Particularly worth noting is that the role of the school principal in the development of the training request and accompaniment of the training was left loosely defined, and many principals had no more involvement in the PEC than as participants in management-directed courses. (d) Decentralized training requires particular attention to issues of monitoring and evaluation. The designers of the PEC had the commendable foresight of including an outside evaluation of the program. Less attention, unfortunately, was given to monitoring. While the heterogeneity of training experiences under the PEC is to be expected with the decentralization of demand and supply, the lack of a uniform monitoring system seriously impeded evaluation efforts. Nor was the very heterogeneity of experience always used to best advantage: it would be very interesting, for example, to do a focussed comparison of successful and unsuccessful experiences to try to draw some more generalized conclusions about what works and why. Despite drawbacks, however, the evaluation of PEC offers important information for the future development of the program. 93. Future Actions for PEC. The SEE is currently conducting its own internal evaluation of the PEC to determine whether the program will continue along similar or different lines. As a model for other countries or other Brazilian states, the PEC is most directly relevant for those which have a well developed system of universities and/or other training institutions which can reach the level of the school. Certain aspects of the program, however, such as its decentralized implementation, the identification of training demand closer to the school, a concern with reaching all members of the school team, and a concern for evaluation, are relevant to a much wider array of training designs. 94. Sistema deAvaliaicoo do Rendimento Escolar - Sat Paulo (SARESP). Sao Paulo was among the first states to develop its own student assessment system. The SARESP is technically 60 These were themes and goals that ran across course offerings, and included: (a) Participatory school management; (b) school and classroom organization; (c) creative use of curriculum; (d) use of library and laboratory; (e) continuous evaluation of students; and (f) use of educational statistics and information in the preparation of work plans. 29 sound, with a number of innovative features. Unlike the Brazilian national assessment test (SAEB) which is conducted on a sample basis, SARESP is a census-based test; all students in state-managed schools in designated grades are expected to take it.61 This allows for student- level, classroom-level, and school-level analysis for all schools. SARESP has also followed a longitudinal design, testing the same cohort of students from year to year, which provides a strong measurement of school effectiveness. Most importantly, SARESP is both a summative and a formative evaluation. Like most large assessment tests, SARESP is summative in that provides information on the accumulated learning of students at a given point. It has played a formative role within the education system, however, because through it teachers, parents and school leaders have learned to apply, utilize, and value assessment tests and the statistical information that their analyses can provide. 95. Lessons Learned from SARESP are as follows: (a) There are strong benefits to parental and school involvement in the testing process. Parental involvement was encouraged both through a mass-media camnpaign and the forrnation of testing oversight committees in each participating school. These committees helped ensure the integrity of the testing process and encouraged broad participation in and acceptance of the testing program. Schools were involved in both the application and immediate correction of the test, giving immediate feedback on performance and pinpointing areas for improvement.62 These measures have fostered a growing culture of evaluation, results, and accountability which is rare in many educational systems. (b) Outside expertise is critical, but there are advantages to technology transfer over time. At the outset of the SARESP program, technical expertise in assessment was sought from the Fundagao Carlos Chagas. Over the years, however, the FCC has transferred capacity to the SEE's implementation arm, the FDE. The FDE is now capable of undertaking increasing numbers of tasks related to SARESP implementation - an exemplary example of institution building. The transfer of knowledge to the FDE, moreover, makes it less likely that a change of administration would jeopardize the prograrn as a whole by, for example, letting a contract with an outside testing agency lapse. (c) Development of an assessment system involves political will and leadership. Given the technical and logistical complexity of assessment testing, to say nothing of cost, the development of an assessment system is unlikely to succeed without clear conviction among top education policy makers that such a system is imperative. They in turn must find creative ways to convince parents, teachers, and the community at large of the value of testing.63 (d) Longitudinal design has both benefits and weaknesses. As mentioned above, the longitudinal design allows for a stronger and more equitable measurement of school 61 Municipal and private schools have also participated on a voluntary basis. 62 Students answer questions on the test itself, then transfer their answers to an answer sheet. The answer sheet is sent to the SEE for optical scanning, while the tests are corrected by the schools and results forwarded to the SEE. Results based on school reports have not differed significantly from "official" results obtained from optical scanning of answer sheets, and the schools have not had to wait for "official" results to come in. 63 In addition to promoting the involvement of school and parents, in Sao Paulo, the names of the top 100 schools in the SARESP test are published in the media. 30 effectiveness than year-by-year testing of a single grade. On the other hand, it creates reactions in schools (why always the same kids?) and raises questions about test-taking ability development biases (students learn how to take these kinds of tests) and the need for evaluating how other students are doing. Yearly application of tests also takes a heavy financial and administrative toll on the system. (e) Non-linkage to national testing efforts risks making SARESP a parallel effort, and perhaps more politically vulnerable in a change of administration. SAEB and SARESP are in fact complementary tests, designed to do different things. Linkage of the two, however, is technically possible, and would bring important benefits to both. SAEB would benefit from SARESP's strong school-level experience and implementation strategy, while SARESP would benefit from continuous exposure to state-of-the art testing development and a connection to a wider framework for national accountability. 96. Future Actions/ SARESP. Several Brazilian states currently designing assessment systems would do well to consider incorporating elements of SARESP's strategy for school and community-level participation. Overall, the downsides of the longitudinal strategy seem to outweigh its advantages - an alternative would be to focus on testing only one or two grades on a consistent basis (for example, 4th and 8th grades). Cost, administrative burden, and the danger of information overload would also suggest that testing every two years, rather than every year, might be a better strategy. This spacing would also allow for better pacing of national (SAEB) and state-level testing efforts. Finally, linking of state-level testing efforts with national ones is highly recommended. 97. Classes de Aceleracdo (Accelerated Classes). The Classes de Aceleraqdo program is an innovative example of addressing repetition and dropout rates by focussing on children with a history of repetition. Over the three years of its implementation, some 1,840 schools with significant age-grade distortion participated in the program, reaching a total of 133,000 children in grades 1-4, and another 53,000 in grades 5-8. The Classes de Acelera'ao is a "pull-out" program: children who have been held back and are older than their peers are streamed into special classes with different materials and specially trained teachers, and then mainstreamed back into the system after having "caught up" with their peers. Existing program statistics indicate that the program is working. Of children who started in AceleraVdo I over the last three years (children in grades 1 and 2 - the ciclo bdsico), 37.8 percent were tracked back into 4th grade, 19.6 percent were tracked into 5th grade, and 1.4 percent were tracked into 3rd grade. About one-fifth of children (21.3 percent) moved on to Aceleraqdo II and the overall dropout rate was 8.4 percent - higher than the statewide average, but not so high when considering that this is a population at particularly high risk for dropout. Of children in Aceleraqdo II, 2.5 percent went on to 4th grade, 76.3 percent went on to 5th grade, 2.8 percent stayed in the Acelera do II or had some other remedial work, and 6.3 percent dropped out. 98. Lessons Learned from the Accelerated Classes are as follows: 64 (a) School-wide familiarity and acceptance of the program is crucial. There have been some early problems with student adjustment when tracked back into "regular" classes. 64 Several of these "lessons learned" are gleaned from a joint evaluation done by the Ministry of Education, the Secretariat of Education for the State of Sao Paulo, and UNICEF. See MEC/SEE, "Ensinando e Aprendendo pra Valer!" Brasilia, 1998. 31 School-wide familiarity with program goals, teaching methods and materials employed in the Classes de Acelera,co helps to diminish the "shock" of re-entry for students and possible discrimination by teachers and students. (b) Good materials make a difference. The materials developed for the program have been wildly popular among teachers and students alike. In many schools, teachers of "regular" classes have opted to use materials from the Classes de Aceleraqdo because they consider them better than others available. Teacher training for the Classes de Acelera,cao is focussed on the use of materials. Given the low socioeconomic profile of students in these classes, the provision of free books has also been an important element in program success. (c) Teacher choice and incentives are key. All too often, more difficult, "problem classes" are assigned to less experienced teachers. The Classes de Aceleraqao program emphasizes the importance of assigning experienced, motivated teachers to teach children having difficulties. Moreover, the inclusion of specialized training and materials in the program has proven an incentive for more experienced teachers to become involved. (d) Students need to build self-esteem along with content knowledge and skills. Children with multiple repetitions tend to have very low self-esteem, which in turn negatively affects their willingness and ability to learn. While teacher attitudes are critical in addressing this issue, the program has a number of built-in factors conducive to improving self-esteem. These include: (a) lower student-teacher ratios (a maximum of 25 students to a class) allowing for more individualized teacher support, (b) the modular curriculum, which allows students to progress at their own pace; and (c) a process of continuous evaluation, rather than high-stakes tests. (e) A focus on absenteeism can reduce it. Many students in the Classes de Acelera,cao also work. Competing responsibilities or simply lack of interest in school can lead to absenteeism, which in turn is strongly linked to lower learning achievement and repetition. Strategies found to work in reducing absenteeism among Classes de Aceleraf do students have included: (a) more communication with parents (including home visits); (b) getting students to keep track of their own absences and think about the reasons for them; and (c) making going to class more enticing through game playing, tying lessons to student interests, and the use of "cliff-hanger" questions, problems, or stories at the end of each class, with the answer deferred to the next one. 99. Future Actions/ Accelerated Classes. The Classes de Acelerafao model and CENPEC materials are currently in use in several other Brazilian states besides SRo Paulo (including Parand and Goias). The program is also being promoted nationally by the federal Ministry of Education, and was awarded a UNICEF national prize for innovation. As the program is still quite new, there is as yet little information available regarding the educational outcomes of Classes de Acelera,cao students once they are tracked back into the "regular" system. Evaluation of these longer term outcomes of the program will be important in gauging its overall effectiveness; initial indications, however, are promising. 100. School lunch Program. The volatile dynamics of school lunch - and the potential consequences for perceived and actual failures in adequately addressing school lunch - are well known in Brazil and particularly in Sao Paulo. DSE has been proactive over the past three years 32 in constructing a transparent and analytical foundation for improving the nutritional content and customer acceptance of the merenda. Nearly three-fourths of the students participating in the program are today served through a decentralized approach, passing financial resources to the local level, empowering municipalities and schools to take charge of implementation. However, this delegation of implementation to the local level is made stronger by information systems that track financial flows and outputs, providing essential feedback to the central and local levels on school lunch performance, thereby significantly improving efficiency. The program took a critical look at menus, revising and enhancing offerings in order to expand student acceptance and reduce wastage. Extensive field visits orchestrated in partnership with university-level nutrition interns demonstrated effective and symbiotic linkages between academic institutions and local needs. Attention given to the training of school lunch personnel addressed service provision "at the front line" and operationalized the findings and recommendation of the field visits. 101. Lessons learned from the School Lunch Program are as follows: (a) Decentralization is only as effective as the information which drives it. The DSE rightly identified the comparative advantage of shifting implementation of the program to the local level. Funds transferred to municipalities (R$0.13 per capita) were based on cost analyses and are updated regularly. Annual inventories of kitchen equipment and continual updates on municipal level performance (e.g., students served, inputs procured, cost parameters) aid the DSE in future budgeting and allocation decisions. A nutritional assessment further augmented this information base.65 (b) A strong sense of accountability raises the probability of success. DSE has taken a fully transparent approach to disseminating the ongoing activities of the school lunch program. Menus are now widely publicized and even made available on the SEE's website, which also posts technical specifications related to procurement. Such transparency is rooted in an explicit desire to sustain the gains achieved to date in enhancing nutritional content and student acceptance. (c) Partnerships can be cost-saving and mutually beneficial. Nutritionist interns gained valuable experience in the field while the DSE obtained timely and critical assessments of the school lunch experience and how to improve it. Other Bank projects would do well to imitate these types of "symbiotic relationships", especially as public budgets grow more restricted. 102. Future Actions for the School Lunch Program. The revised technical specifications for centrally purchased foodstuffs have already had an impact in the school lunch programs of other Brazilian states. In loco supervision of the centralized program contributed to quality enhancement; it remains to be seen how the decentralized program can benefit from similar supervision efforts. 103. Information Technology. Sao Paulo is one among a number of states in Brazil introducing information technology (IT) into education administration and management. Investments in IT have facilitated several objectives, including: (a) improved access to and use of accurate, up-to-date planning informnation (including data from the school census and student 65 Nutritional Study, Antonio Campos Campino 33 assessment results); (b) more agile communication between the SEE and its local offices (delegacias); and (c) reduction in time spent on administrative tasks at the level of the school and delegacia. Although the attempt to create an independent school-based management system under the project was not successful, Sao Paulo is currently piloting a federally developed system in 200 schools (SAE-Ministry of Education).66 Innovative features of Sao Paulo's IT experience include plans to connect schools to delegacias and the SEE via the Internet, and also a large measure of intra-governmental collaboration. As part of a state goverm-nental initiative in information technology, government agencies with strong IT infrastructures (notably Fazenda and the Secretary of Public Security) are currently providing technical support to schools and local education offices, relieving the SEE of the immediate need to provide such support itself. 104. Lessons learned from Information Technology are as follows: (a) Sophisticated management of IT is fundamental. Problems in government contracting of IT are myriad, and usually result from lack of sufficient technical knowledge within government agencies to: (i) adequately specify their needs in technical terms; and (ii) adequately manage the work of contractors. Identifying an IT team and recruiting a core of skilled technical personnel is a crucial first step to any IT investment. This was not done sufficiently early in Sao Paulo, and contributed to delay in the development of the school- based management system. (b) You must have a plan. Prior to making any purchases, the SEE produced a comprehensive IT plan. Although the plan suffered some modifications along the way, it served as a crucial roadmap for implementation. A major area of weakness, as indicated earlier, was the lack of sufficient IT personnel to manage the implementation plan. Given the rapid pace of changes in hardware and software, the plan should cover issues of replacement, as well as maintenance, technical assistance, and training. (c) Where the market is well developed, there are advantages to the decentralization of equipment purchases. As part of the implementation plan, all state schools bought computers, printers, and accessories locally with funds transferred by the SEE. Although a centralized purchase would have undoubtedly resulted in some cost savings, these were outweighed by the logistical advantages of having each school able to depend on a local provider for installation, replacement of faulty equipment, etc. (d) IT is an area of potential public-private partnership. Sao Paulo is currently exploring the possibility of having private institutions assume the costs of connecting schools to the Internet, and recruiting other private entities to help with training of personnel and maintenance of equipment at the school level. This is a strategy that could be used in other states, though with more difficulty where the business community is less developed. 105. Future Actions/ Information Technology. The SEE is moving ahead with the full implementation of SAE-MEC, probably within the next year. Schools will be connected to corporate systems via the Internet, and the SEE has already developed a website (www.educacao.sp.gov.br) with information on its various programs and public access to several databases (notably the student census data and SARESP evaluation data). Although in the 66 SAE-MEC is partly financed by the Bank through the Northeast Basic Education II and III projects and the Fundescola projects 34 Brazilian context Sao Paulo is not necessarily in the forefront of use of IT in education, given the long-term trend toward increased use of IT worldwide, Sao Paulo's experience may provide some useful ideas and lessons for others.67 67 All nine Northeast states invested heavily in IT under the Bank-financed Northeast Basic Education II and II projects, and many of their systems are currently more developed than Sao Paulo's. Sao Paulo is more advanced, however, than many states in the North and Center-West of the country. IMPLEMENTATION COMPLETION REPORT BRAZIL INNOVATIONS IN BASIC EDUCATION PROJECT (Loan 3375-BR) PART II: STATISTICAL INFORMATION TABLE 1. SUMMARY OF ASSESSMENTS Pre-Reformulation Post-Reformulation and Summary Assessment A. Achievement of Obiectives Not Not Substantial Partial Negligible applicable Substantial Partial Negligible applicable Macro Policies El[ El [Z E0] Sector Policies l] ] I E] 0 E] [ 1 Financial Obiectives2 07[ [ 0 El El Institutional Development3 0 E W E E] a E [ Physical Obiectives4 EJ [51 El EZl [ ] E Poverty Reduction4 4 [k Z E [ ] ] ] Gender Issues E1 E ] W E Z ] ] Environmental Objectives [m Z [1 E E L [] Public Sector Management5 I El] E E] a Private Sector Development6 [71 F F-1 Fm1 [71 F1 Post-reformulation ratings equate to summary assessments. While the IEB was not formally reformulated, this ICR considers the actions taken during the post Midterm Review period (e.g, establishment of quarterly benchmarks) to constitute, in practical terms, a reformulation of the Project. 2 Quarterly Benchmarks established in the Post-Reformulation period. 3 Institutional Development and Assessment Component 4 In School Quality Improvement and Preschool Components as related to the poverty focus of the IEB. 5 Information Technology as financed under the Institutional Development Component. 6 NGO participation under the Preschool Component. 36 TABLE 1. SUMMARY OF ASSESSMENTS (CONTINUED) Pre-Reformulation Post-Reformulation and Summary Assessments B. Proaect Sustainability Likely Uncertain Unlikely Likely Uncertain Unlikely (/) ) (VI (0)( (V) (VI) -1 1 X-1 [::] r---1 E l C. Bank Performance Highly Highly Satisfactory Satisfactory Deficient Satisfactory Satisfactory Deficient (/) (/ (V/) (/) (/)VI) Identification [ EO [1 hI [ Preparation assistance ] 1] [ ] [ ] 0 Appraisal2 [ Cl [l] 0 [ L Supervision3 L O O ] [ [ D. Borrower Performance Highly Highly Satisfactory Satisfactory Deficient Satisfactory Satisfactory Deficient Preparation [ [ [ [ [z [ Implementation 1 ] ] [S 7 ] Covenant Compliance ] EC [ L [7 L E. Assessment of Outcome Highly Highly Highly Highly Satisfactory Satisfactory Unsatisfactory Unsatisfactory Satisfactory Satisfactory Unsatisfactory Unsatisfactory () V) V) V) (") V) V) (V) Post-reformulation ratings equate to summary assessments of the entire project. 2 Namely in regard to assessment of SEE procurement capacity under the Project. 3 Based on the Bank's failure to move earlier toward Loan cancellation. 37 TABLE 2. RELATED BANK LOANS LoAN TITLE Purpose Year of Approval Status Preceding operations Urban Basic Education Improve educational 1984 Repaid (Ln 2412-BR) opportunities for children in the urban North and Center West of Brazil Northeast Basic Improve educational 1980 Repaid Education opportunities for children (EDURURAL) in rural Northeast Brazil (Ln 1867-BR) Following operations Second Northeast Increase learning; reduce 1993 Closed Basic Education repetition and dropouts; (12/31/98) (NEBE II) and increase graduation (Ln 3604-BR) rates Third Northeast Basic Increase learning; reduce 1994 Active Education (NEBE III) repetition and dropouts; (Ln 3663-BR) and increase graduation rates Parand Basic Improve learning 1994 Active Education achievement and (Ln 3766-BR) educational attainment among basic education school students Minas Gerais Basic Improve educational 1995 Active Education achievement in (Ln 3733-BR) elementary grades of state schools School Improvement Strengthen primary 1998 Active Project (Fundescola I) schools and related (Ln 4311-BR) public institutions 38 TABLE 3. PROJECT TIMETABLE Date Actual/ Steps in Project Cycle Date Planned Latest Estimate Identification (Executive Project Summary) 02/95/88 Preparation - 11/24/88 Appraisal 02/08/89 02/08/89 Negotiations 03/01/90 05/01/91 Letter of development policy (if applicable) - - Board Presentation 06/16/89 06/26/91 Loan Signing 10/04/91 10/04/91 Loan Effectiveness 01/07/92 01/07/92 Midterm review 06/01/93 06/13/94 Project Reformulation - 08/01/95 Loan Closing 06/30/98 04/30/99 Project Completion 09/30/97 12/31/98 I 39 TABLE 4. LOAN DISBURSEMENTS: CUMULATIVE ESTIMATED AND ACTUAL (US$ Million) Fiscal Year 1" Appraisal Estimate Z' Actual Actual as % of Estimate 1992 56.8 - September 30, 1991 0 - December 31, 1991 20.3 March 31, 1992 36.8 June 30, 1992 56.8 20.0 35.2 1993 127.8 31.5 24.6 September 30, 1992 77.8 23.9 30.7 December 31, 1992 96.8 23.9 24.7 March 31, 1993 110.8 23.9 21.6 June 30, 1993 127.8 31.5 24.6 1994 180.4 38.0 21.1 September 30, 1993 144.4 31.5 21.8 December 31, 1993 159.4 34.9 21.9 March 31, 1994 169.4 38.0 22.4 June 30, 1994 180.4 38.0 21.1 1995 214.1 48.4 22.6 September 30, 1994 190.4 38.0 20.0 December 31, 1994 199.4 41.9 21.0 March 31, 1995 207.4 48.4 23.3 June 30, 1995 214.1 48.4 22.6 1996 230.1 62.3 27.1 September 30, 1995 220.1 48.4 22.0 December 31, 1995 223.6 48.4 21.6 March31, 1996 226.6 52.4 23.1 June 30, 1996 230.1 62.3 27.1 1997 244.0 141.3 57.9 September 30, 1996 234.1 91.4 39.0 December 31, 1996 237.6 104.1 43.8 March 31, 1997 240.8 123.1 51.1 June 30, 1997 244.0 141.3 57.9 40 1998 245.0 220.8 90.1 September30, 1997 245.0 182.9 74.6 December31, 1997 - 200.8 82.0 March 31, 1998 - 212.9 86.9 June 30, 1998 - 220.8 90.1 1999 September30, 1998 - 234.9 95.9 December 31, 1998 - 234.9 95.9 March 30, 1999 - 242.4 98.9 June30, 19993 243.1 99.2 Final Disbursement: May 14, 1999 ' July 1 - June 30 2 including US$ 20.25 million deposited in the Special Accounts 3 US$1.9 million of the original Loan balance was cancelled at the request of the Borrower. 41 TABLE 5. KEY INDICATORS FOR PROJECT IMPLEMENTATION No performance indicators were established at appraisal. Project monitoring was deficient due to the implementation conditions, aggravated by economic turmoil, changing priorities in education and health sectors, wholesale changes in project-related staff, and procurement problems and bottlenecks. At reformulation (1996) a set of financial benchmarks was established and monitored as shown below, based exclusively on negotiated implementation goals and expenditures to assist line agencies adhere to a schedule specifying timing and amounts. The loan closed on December 31, 1998, after a six-montL extension, having disbursed 99 percent of the original $245 million. Benchmarks Achievement through July 1996 1 Decentralized transfer of funds 103% 2 Publication of bidding documents 153% 3 Signing of contracts 102% 4 Total expenditures of project 123% 5 Disbursement request from Bank 133% 42 TABLE 6. STUDIES INCLUDED IN PROJECT Study Purpose as Defined at Status Impact of Study Appraisal/Redefined Project Implementation 12 studies were conducted on Completed Limited, given early conclusion. Process study: different components and/or aspects 1996 Nonetheless, UGP and SEE staff NEPP of the implementation process found studies helpful in project restructuring. Impact Evaluation Study designed to measure impact of Completed Due to the timing of this study Study: Fundacao project innovations and inputs on 1995 (before much of project Carlos Chagas learning achievement implementation got underway) it was used primarily to evaluate the effectiveness of then current governmental programs and policy directions. Cost Study - FIPE Originally designed as basis for cost- Completed Study contributed to per-student cost benefit analysis of project in Phas I 1996 discussions in the establishment of conjunction with impact evaluation. e - the new federal financing law Evolved into development of Phase II - 1997 (FUTNDEF); SEE staff trained in computerized model for ongoing operation and up-keep of monitoring of educational costs. computerized tracking system, used for planning purposes. Nutrition Study - Diagnostic study of school feeding Completed - revised technical specifications; ANtrition Study Campos progragnostic study ofschoolfeeding Compl- official menus introduced quarterly; Antonio Campos program 1997 - direct transfer of funds to schools; Campino - field supervision; - purchase of kitchen equipment; - computerized cost monitoring - poverty/nutritional profile of students "A Importdncia da Assess beneficiary perspectives on Completed Helped to inform ongoing Participa9do education, and evaluate the 1995 educational reforms in Sao Paulo, as Comunitaria na importance of community well as wider beneficiary assessment Questdo da Educaqao e participation in school quality on education which included the da Pobreza" - improvement Northeast states. Fundaqdo Carlos Chagas ___ _ Study on Public-Private To identify NGOs within project area Completed Limited or negative - informal nature Partnerships. and clarify possibilities and barriers 1996 of many NGOs depicted convinced Converted to Study for NGO participation in pre-school state and municipalities already fairly on NGOs - program uninterested in working with NGOs CEBRAP that it would be difficult to work with them except on a grant basis. No new pre-school places were provided by NGOs through the project. Poverty Targeting - Post project poverty targeting Completed Only construction has an explicit Luiz Paulino evaluation 1999 poverty targeting goal and was shown to disproportionately benefit the poor. 43 "Avaliacao do Evaluation of the continuing Completed Helped guide implementation Programa de Educa,ao education program. 1998 process, particularly for second year; Continuada" - currently an instrument in the SEE's Fundacao Carlos own internal evaluation of the Chagas, Fundacao Joao program. Pinheiros, Cesgranrio, and Universidade Federal do Rio Grande do Sul Oral Health and Completed Hygiene Study on Vitamin A and Identify principal causes of morbidity Not done Study was planned to be sole Iron Deficiencies - among school-age children, and sourced, but proposal was Changed to study on propose strategy for improving prohibitively expensive. Price was school-age child prevention and treatment. negotiated down, but not before morbidity in 1994 project closing date made study nonviable. Municipalization Study Training of SEE and municipal Comprehensive Helped prepare the way for a jump in - Changed to a series of personnel on the implications of the study of policy municipalization of basic education didactic manuals new federal financing law for basic options for in Sao Paulo. produced by FUNDAP education (FUNDAP), organization municipaliza- of state and municipal systems in tion never done. assuming responsibility for basic Manuals education, and transfer of state goods completed in and personnel to municipalities. 1997 "0 Sistema de Ensino Statistical re-examination of past, Completed Contributed to reorganization of Regular no Estado de current and project student flows in 1995 school system, increased Sao Paulo" - Ruben Sao Paulo. prioritization of the higher levels of Klein basic education (grades 5-8), and the development of programs (Classes de Aceleracao) specifically aimed at correcting student flows. 44 TABLE 7. PROJECT COSTS AND FINANCING A. Project Costs (US$ million) Item Appraisal Estimate Actual/Latest Estimate Local Foreign Total Local Foreign Total Costs Costs Costs Costs A. In-school Quality 365.3 54.6 419.9 507.0 18.3 525.9 Improvement B. Pre-school 53.7 8.8 62.5 14.6 - 14.6 C. School Health 27.0 2.7 29.7 11.3 0.6 11.9 D. Institutional 7.2 0.4 7.6 19.6 0.7 20.3 Development E. Evaluation and 3.2 0.2 3.4 21.3 - 21.3 Dissemination F. Administration 1.7 0.1 1.8 7.9 - 7.9 Total Baseline Cost 458.0 66.8 524.8 581.6 19.6 601.3 Physical Contingencies 34.5 5.7 40.2 - - Price Contingencies 31.2 3.8 35.0 - - Total Project Cost 523.7 76.3 600.0 581.6 19.6 601.3 B. Project Financing (US$ million) Source Appraisal Estimate Actual/Latest Estimate Local Foreign Costs Local Foreign Costs Total Costs Costs IBRD - 245.0 223.6 19.5 243.1 Cofinancing Institutions 324.1 - - - - Domestic Contribution 30.9 - 358.1 358.1 Total 355.0 245.0 581.7 19.6 601.3 45 TABLE 8. STATUS OF LEGAL COVENANTS Original Revised Covenant Present Fulfillment Fulfillment Description Of Covenant Agreement Section Type Status Date Date Comments Loan 3.01a 10 CP 4/24/96 The Borrower declares its commitment to the Weak commitment in period objectives of the Project. .and to this end shall from Effectiveness until carry out the Project with due diligence... 1995; 84 percent of Loan disbursed in the final three years of the Project. Loan 3.Olb 10 C 4/24/96 Establish and maintain a Project unit and staff the Complied with Unit with a Director, two Coordinators and a financial administrator, whose qualifications and experience are satisfactory to the Bank and the Borrower Loan 3.01c 10 CP 4/24/96 Ensure that Borrower's agencies assist SEE with Principal compliance from respect to the following activities: FDE, for 1995 through Project physical facilities under Part A(l) and B(l); CENP Closure. for training under Parts A and B; DSE for feeding under Part A and B; SES for activities under Part C. Loan 3.01d 10 C 3/20/92 Through SEE, enter into an agreement with FDE Complied with on terms satisfactory to the Bank... Loan 3.0ld 10 C 3/30/92 For purposes of Part B of the Project, enter, with Complied with each Municipality that has or is ready to have an approved Municipal plan, into an agreement. Covenant Tvve: I Accounts/audit; 2 Financial performance/gencratc revenue from beneficiaries: 3 Flow and utilization of Project funds: 4 Counterpart funding; 5 Management aspects of the Project or of its executing agency; 6 Environmental covenants; 7 Involuntary resettlement: 8 Indigenous people; 9 Monitoring, review anid reporting; 10 Implementation: I1 Sectoral or cross-sectoral budgetary or other resource allocation; 12 Sectoral or cross-sectoral regulatory/institutional action; 13 Other Status: C = covenant complied with NC = not complied with CD = complied with after delay NYD = not yet due CP = complied with partially SOON = compliance expected in reasonably short time 46 TABLE 8. STATUS OF LEGAL COVENANTS (CONTINUED) Original Revised Covenant Present Fulfillment Fulfillment Description Of Covenant Agreement Section Type Status Date Date Comments Loan 3.01f 10 NC 3/30/92 4/24/96 Cause each municipality, under the agreement Complied with referred to in para (f) of this section, to enter into an agreement with PNPOs included in the Municipal Plan Loan 3.02a 10 C 4/24/96 Procurement of goods, works and consultants' Complied with services required for the Project and to be financed out of the proceeds of the loan shall be governed by the provisions of Schedule 4 Loan 3.02b 10 C 4/24/96 Establish single registries of contractors and Complied with suppliers for purposes of the Project Loan 3.02c 10 C 1/3/93 Not later than 01/31 of each year, starting 0 1/92 Complied with publish ads in two newspapers of wide national circulation in Brazil, in the Borrower's Diario Official and in Development Business. Loan 3.03 C N/A N/A Ensure that Basic Cycle and Extended School Day Complied with are fully implemented on a timely basis in the schools which have appropriate facilities for the implementation thereof. Covenant Type: I Accounts/audit; 2 Financial performance/generate revenue from beneficiaries: 3 Flow and utilization of Project funds: 4 Counterpart funding; 5 Management aspects of the Project or of its exectiting agency; 6 Environmental covenants; 7 Involuntary resettlement: 8 Indigenous people; 9 Monitoring, review and reporting; 10 Implementation: 11 Sectoral or cross-sectoral budgetary or other resource allocation; 12 Sectoral or cross-sectoral regulatorv/institutional action; 13 Other Status: C = covenant complied with NC = not complied with CD = complied with after delay NYD = not yet due CP = complied with partially SOON = compliance expected in reasonably short time 47 TABLE 8. STATUS OF LEGAL COVENANTS (CONTINUED) Original Revised Covenant Present Fulfillment Fulfillment Description Of Covenant Agreement Section Type Status Date Date Comments Loan 3 .04a CD 5/31/93 6/13/94 WB and Borrower shall at the request of either of Midterm Review was them meeting sometime between May and August delayed by one year to due 1993 or when an amount of the Loan equivalent to slow project implementation. US$70.0 million shall have been committed by the Borrower... .to review aspects of the project's execution Loan 3.04b 10 C N/A N/A For the purposes of the review, furmish to the Complied with Bank: (1) updated plans on execution of remaining civil works and (2) a report on the implementation of the Basic Cycle. Loan 3.04c 10 CP N/A N/A The Borrower shall make such modifications in Complied with the plans... put into effect such plans... requested as a result of the review. Loan 3.05a 10 CD N/A N/A Through the SEE, issue the Rules for Municipal Complied with participation in a timely manner but not later than three months after the Effective date Loan 3.05b 10 CP N/A N/A Through the Project Unit, review before the end of Municipal Plans were each year the performance of each Municipality initially reviewed but not during the year. annually after their approval Covenant Tvoe: I Acotmnts/audit; 2 Financial perforinance/generale revenue irom beneficiaries: 3 Flow and utilization olProjeet fiunds: 4 Counterpart lunding; 5 Managemiient aspects of' thc Project or of its executing ageicy; 6 Environmental covenants; 7 Involuntary resettlement: 8 Indigenous people; 9 Monitoring, review and reporting; 10 Implemenitation: 11 Sectoral oi cross-sectoral budgetary or other resource allocation; 12 Sectoral or cross-sectoral regulatorv/institutional action; 13 Other Status: C = covenant complied with NC = not complied with CD = complied with after delay NYD = not yet due CP = complied with partially SOON = compliance expected in reasonably short time 48 TABLE 8. STATUS OF LEGAL COVENANTS (CONTINUED) Original Revised Covenant Present Fulfillment Fulfillment Description Of Covenant Agreement Section Type Status Date Date Comments Loan 3.05c 10 NC Borrower shall not present Bank withdrawal applications under Category 4 in respect to expenditures by a given Municipality in question unless Borrower has undertaken corresponding review. Loan 3.06 5 C N/A N/A Through the SEE, shall not later than September 1 Complied with of each year during the execution of the project, starting 9/1/92, prepare and send to the Bank a report on the sources and uses of funds Loan 3.07 10 C N/A N/A Not later than December 31, 1993 employ Complied with consultants that shall assist in carrying out Part D(2) of the Project. Loan 4.01a 10 N/A N/A N/A Through Project Unit, maintain or cause to be maintained separate records and accounts to reflect in accordance with sound accounting practices Loan 4.0lb 10 C 6/30/92 Through Project Unit, shall have records and Complied with accounts audited .., furnish to WB not later than six months after end of Fiscal Year. Loan 4.01c 10 N/A N/A N/A For all expenditures with respect to which withdrawals from loan account were made on the basis of statements of expenditure. . . maintain records reflecting such expenditure. Covenant Tvle: I Accounts/audit; 2 fiancial per lbrmance/genite revenoc from beneficiaries: 3 Flow and utilizationl ot T'roject hlinds: 4 Counterpart tiinding; 5 Managemienit aspects ol the Projector otitsCexCeCtingagenicy: 6 Environmental covenants; 7 Involuntaryresettlement: 8 Iidigenouspeople; 9 Monitoring, review andireporting; 10 Implementatiotn: 11 Sectoral or cross-sectoral budgetary or other resource allocation; 12 Scetoral or cross-sectoral regipatorv/institutional action. 13 Other Status: C = covenant complied with NC = not complied with CD = complied with after delay NYD = not yet due CP = complied with partially SOON = compliance expected in reasonably short time 49 TABLE 8. STATUS OF LEGAL COVENANTS (CONTINUED) Original Revised Covenant Present Fulfillment Fulfillment Description Of Covenant Agreement Section Type Status Date Date Comments Loan 4.02 10 N/A N/A N/A (a) Cause all primary education facilities ... to be efficiently operated and maintained ... (b) provide funds, facilities, services and other resources .. . (c) make adequate provision for maintenance in annual recurrent budgets. Loan 3b 3 CP N/A N/A No payments for expenditures under Part B(l) of Complied with partially the project unless: (1) approved Municipal plan, Sched. 1 (2) legal opinion on agreement between state and municipality, (3) evidence that the Municipality has included as many PNPOs as possible. Loan 3c 3 CP N/A N/A No payments under Part C of Project unless (1) Complied with SEE appoints coordinator satisfactory to the Bank, Sched. 1 (2) submits implementation plan, (3) for expenditures within Sao Paulo, evidence that all steps necessary shall have been taken to ensure Borrower, Municipality and PNPOs can perform duties. Loan 3d 3 C N/A N/A No payments for expenditures under Part A(2)(b) Complied with of Project unless Borrower has submitted to Bank Sched. 1 a plan of specific activities. Loan 3e 3 CD N/A N/A No payments for expenditures under Part E(2) of Complied with the Project unless the Borrower has presented to Sched. 1 the Bank the study referred to in Part E(1) of the Project and a plan for activities under Part E(2) of the Project. Covenant Troe: I Accounts/audit; 2 Financial performance/gcnerate revenue from beneficiaries: 3 Flow and utilizationi of Project funds, 4 Counterpart funding; 5 NManagement aspects of the Project or olfits executing agency; 6 Environmental covenants; 7 Involuntan' resettlement: 8 Indigenous people; 9 Monitoring, review and reporting; 10 Implemenitation: 11 Sectoral or cross-sectoral budgetary or other resource allocation; 12 Sectoral or cross-sectoral reguwatory'institutional action; 13 Other Status: C = covenant complied with NC = not complied with CD = complied with after delay NYD = not yet due CP = complied with partially SOON = compliance expected in reasonably short time 50 TABLE 9. PROJECT COSTS AND BENEFITS No Internal Economic Rate of Return calculated at Project appraisal, nor recalculated at Project Closing. TABLE 10. BANK RESOURCES: STAFF INPUTS Planned Revised Actual Stage of Project Cycle Staff US$ Staff US$ Staff US$ Weeks Weeks Weeks Through Appraisal 0.0 197.5 n/a Appraisal-Board 0.0 63.2 n/a Board - Effectiveness 0.0 71.2 n/a Supervision 309.8 587.6 n/a Completion 15.0 19.8 n/a TOTAL 324.8 = = 939.3 n/a 51 TABLE 11. BANK RESOURCES: MISSIONS Days Specialized Perfornance Rating Stage of Month/ No. of in staff skills Impl. Develop. Types of Project Cycle Year persons field Represented Status Objectives Problems Through 12/1987 4 11 Op,Ec,Ph n/a n/a Appraisal 03/1988 4 15 Ec,Ph n/a n/a 07/1988 6 16 Ed,Ec,Ph,Op n/a n/a 08/1988 2 n/a n/a n/a w S........ ..... ...... ........ ....... .... .. _.._...... ... . --- - -- - _--.---_---._--..... . .....____ Appraisal 02/1989 8 14 Ed,Ph,Ec,Op, n/a n/a through A, Effectiveness 10/1989 2 1 Op,Ed n/a n/a 02/1990 2 5 Op,A n/a n/a 05/1990 2 7 Ed n/a n/a 02/1991 2 1 Op,A n/a n/a 05/1991 5 4 Op,Ec,Ed,A n/a n/a 05/1991 Initial Summary NR NR 05/1991 Portfolio Status Update 1 I 11/1991 4 3 Op,Ph,Ec n/a n/a ___ Supervision 12/1991 7 4 Ed,Op,Ds,A, 1 1 Au,Lg,Pr 04/1992 2 10 Op,A 1 1 07/1992 2 4 Op,Ev 2 1 08/1992 3 5 Ed. n/a n/a 10/1992 2 4 Op,A 2 1 02/1993 1 3 Op 2 2 06/1993 Portfolio Status Update 2 2 10/1993 6 8 Pr,Tm,A,Tt,Ph 2 2 02/1994 6 8 Pr,Tm,Ec,A,Tt 2 2 05/1994 2 1 Ed,Ec n/a n/a 06/1994 7 12 Pr,Op/Tm,Ec, S S A,Ph,Tt,Nu 09/1994 Portfolio Status Update U S 05/1995 Portfolio Status Update U S 06/1995 Portfolio Status Update U U 01/1996 Portfolio Status Update U U 03/1996 1 n/a Ed U U 12/1996 Portfolio Status Update U U 06/1997 Portfolio Status Update S S 01/1998 3 Ed/Tm,Pr,Ed S S 11 Specialization: A: Architect, Au: Audits, Ds: Disbursements, Ec: Economist, Ed: Education, Ev: Evaluation, Lg: Legal, Nu: Nutrition, Op: Operations, Ph: Public Health, Pr: Procurement, Tm: Task Manager, Tt: Teacher Training 52 Projeto Inova$oes no Ensino Bisico I Missio de Relat6rio Final (ICR) Ajuda Men6i6a 13 29 de Marpo, 1999 - Uma missao do Banco Mundial veio a Sao Paulo para levantar inforrna,o para o relat6rio final de avalia,co do projeto entre o dia 15 de Margo e o dia 10 de Abril. A missao foi feita em etapas, e era composta pelos seguintes integrantes: Helena Jubany 15-19 de Marco - rede fisica),Wilsa Ramos (15-25 de Margo - PEC e clases de acelera,co), Edward Bresnyan (22 a 26 de Margo - relat6rio geral), Kate Hovde (22-29 de Mar,o- relatorio geral), Robin Horn (dia 26 de Margo - task manager), Alberto Rodrigues (dia 24 de Marco - SARESP), e Claudio Rosemberg (dia 5-10 de Abril - Informatica). Os membros da missao fizeram varias visitas de camnpo para ver escolas, delegacias e prefeituras, e conversaram com tecnicos da Secretaria do Estado da Educaao nas areas de planejamento (ATPCE), estatisticas (CIE), merenda (DSE), educacao continuada (PEC), pedagogia (CENP), avaliaco de rendimento escolar (SARESP), comunicacao social, rede fisica (FDE) e materiais pedag6gicos. A missao teve tamb6m a oportunidade de conversar com a Sra. Secretaria da Educa,co, Profa. Rose Neubauer, e entrevistar membros da equipe da UGP e integrantes do projeto na area de saude escolar. A mnissao deseja agradecer todas as pessoas envolvidas nos contatos por ela realizados pela sua hospitalidade e colaboraao. 0 relat6rio final de avaliacao e um requisito do Banco Mundial para qualquer projeto. E composto por duas partes: (a) uma avalia,ao feito pelo pr6prio Banco; e (b) uma avaliac,o feito pela agencia executando o projeto. 0 relatorio feito pela SEE ja se encontra em estado avancado, com a espectativa de finalizac,o antes do dia 31 Maio. E expectativa da missao ter uma primeira versao do relat6rio final do Banco ate o dia 20 de Abril. Uma vez terminada, a primeira versao do relat6rio do Banco sera mandada a SEE (atraves da UGP) para comentaios e correg6es. A SEE tera trinta (30) dias para mandar comentarios, mas a missio notou que seria uma grande ajuda se os comentarios fossem mandados tao pronto quanto possivel. Foi acordado que a UGP tentari mandar os comentarios antes do dia 20 de Maio (sempre se nao houver atrasos da parte do Banco em mandar o seu relat6rio), e que em seguida o Sr. Robin Horn faria uma breve missao para conversar sobre os relat6rios e acertar mudangas para a versao final. Essa missao seri a i!ltima missao para o projeto, e com o entrega da versio final do relat6rio de avalia9io, o projeto sera considerado como oficialmente fechado. Kate Hovde Dr. Lauro de Almeida Carneiro Fil Consultora, Banco Mundial Coordenador do Projeto LEB o~~~~~~~~~~~~~~~~~~~~~~~~~C -~~~~~~~~~~~~ Oa ¢ 0 t %2~~~~~~~ .2iy tO F a a = ' 's~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~i g R tZ W r S w tS t1~~~~~~~~~~~~ 'B^'.--.~~~~~~~~~~~~~~~ C z ^ i '. x~~~~~~~~~~~~~~~~~I 0~~~~~ 5O Ntv t Me9-e:&$~- II O .¢ X i,O 0 ..0 0,q, .ti E t'C:ts) 54 - SECRETARIA DE ESTADO DA EDUCAQAO '01,1111 17 OBJETIVOS O objetivo do Projeto e melhorar a aprendizagem no ensino fundamental e como consequencia diminuir os indices de reten,cao e evasao escolar, das crian9as de familias de baixa renda, inicialmente da regiao Metropolitana de Sao Paulo, e posteriormente expandindo-se pelas areas carentes do Vale do Ribeira e da regiao de Sorocaba. Para atingir esses objetivos foi necessaria a elabora,ao de uma nova abordagem curricular no ensino de 10 grau, que inclui o aumento da j omada dos alunos e dos professores; a disponibiliza,cao de materiais e equipamentos pedag6gicos; a melhoria da qualifica,cao profissional de professores e de educadores; a amplia,co de oferta da pre-escola as crian,as de baixa renda; preve-se tambem, como metas essenciais para se atingir os objetivos, o refor,o dos programas de sauide escolar baseado no novo sistema descentralizado de sauide puiblica; o refor,o institucional dos 6rgaos responsaveis pela administra,ao do ensino de 1° grau a nivel estadual e municipal; assim como o desenvolvimento de um meio de colaboracao entre o setor puiblico e as ONGs na presta,cao de servi,os educacionais. Finalmente, e ainda objetivo, a avalia,ao, no desempenho escolar, dos efeitos da abordagem proposta, e a dissemina,ao dos resultados das pesquisas e experiencias decorrentes da implementa,co do Projeto. Eventualmente a experiencia podera ser disseminada para outros Estados da Uniao. 55 AVALIACAO DO PROJETO INTRODUCAO O Projeto apresentou um baixo desempenho na sua implementa,co do periodo inicial ate o final do ano de 1995, ano em que foi empreendida a retomada das a,ces visando o cumprimento das diretrizes emanadas da nova gestdo que se iniciara em janeiro de 1995. A partir do ano de 1996, evidenciou-se um elevado desempenho no cumprimento das metas entao acordadas com o Banco Mundial quando foram estabelecidos Bench Marks fisicos e financeiros a serem atingidos ate o final do Projeto. Tais metas foram integralmente cumpridas pelo Projeto. Na retomada das a,ces, ficou acordado com o Banco Mundial que as metas iniciais de alguns componentes e subcomponentes seriam reorientadas e, em alguns casos, a sua execuqao seria descentralizada. As novas diretrizes da politica educacional , resultantes ate mesmo da avalia,ao do impacto, realizada no arnbito do componente D do Projeto, estabeleciam que para se atingir ao objetivo maximo da proposta, isto e melhorar a aprendizagem no ensino fundamental, as inova,ces deveriam abranger a escola de primeiro grau como um todo e nao apenas o Ciclo Basico. Para tanto era necessario: - oferecer aos alunos das la a 4a series: (i) maior carga horAria de aulas (5 horas); (ii) salas de aula com arranjo pedag6gico orientado exclusivamente para essa faixa etaria aliada a um mobiliario dimensionado adequadarnente para a estatura media das crian,as dessa faixa etaria e (iii) a separa,ao de alunos menores dos maiores para evitar a inconveniencia da convivencia num mesmo espa,o; - que o treinamento deixasse de ser somente orientado para professores do Ciclo Basico passando a contemplar professores de todo o ensino de 10 grau (Pa a 8a s6rie) e gestores de todo o Estado; - ampliar a rela,ao de materiais educacionais que constavam do Projeto inicial com a inclusao de outro materiais pedag6gicos e equipamento de informatica educacional; - instalar um Sistema de Informa,ces Gerenciais para a SEE; - ampliar a abrangencia das a,6es de Divulga,ao do Projeto redirecionando-as nao s6 para o puiblico especifico como inicialmente proposto, mas tambem para comunica,co de massa; - ampliar a area de abrangencia do Projeto incluindo-se outras areas de pobreza do Estado como as Regioes de Sorocaba e Vale do Ribeira; - ampliar a tipologia de interven,ces em constru,oes com a inclusao das obras de reformas gerais, pequenos reparos e adequa,ces de constru,oes existentes; 56 - descentralizar as avoes de contrata9do e obras de pequeno vulto e de aquisig6es de bens atrav6s das Associag6es de Pais e Mestres das escolas da area programAtica. Com a redistribui9ao dos alunos do ensino fundamental em escolas destinadas somente a alunos de lI a 4a series e outras para alunos da 5a a 8a series, houve um saldo excedente de aproximadamente 100 edificios escolares o que determinou uma revisao na previsao de novas salas de aula a construir, assim como altera96es nas metas iniciais, como ja foi dito. A seguir acham-se elencados os resultados alcan9ados na implementa9ao de cada Componente do Projeto relativos ao periodo compreendido desde a data de efetividade do Contrato de Empr6stimo em 7 de janeiro de 1992 a 30 de abril de 1999, data de t6rnino do Periodo de Graga do Projeto. RESUMO DO SUCESSO DO PROJETO Melhoria da qualidade na escola Construcao de escolas - No total o Projeto IEB executou 1.119 obras civis dentre as quais 424 obras de novas escolas e ampliaq6es que resultaram em 3.387 novas salas de aula com a criagdo de 355.635 novas vagas. As demais 695 obras concluidas com recursos do Projeto se referem a adequaqoes e reformas gerais e de pequeno porte que beneficiaram adicionalmente 565.985 alunos. uadro 1- Consolidado das Obras Civis executada's Obras 350 155 216 61,7 139,4 3.000 1.840 2.447 81,6 133,0 Novas Ampliades 340 171 208 61,2 121,6 1.150 753 940 81,7 124,8 Adequagbes N.A. 12 1 2 N.A. 100,0 N.A. N.A. N.A N.A. N.A. Reformas N.A. 319 319 N.A. 100,0 N.A. N.A. N.A N.A. N.A. Gerais Pequenas N.A. 364 364 N.A. 100,0 N.A. N.A. N.A N.A. N.A. Reformas Fonte: Fundacdo para o Desenvolvimento da EducaCJo - FDE 57 Os indicadores de custo resultantes da implementa,ao das obras civis acham-se comparados no Quadro 2 com os previstos no SAR. Quadro 2 - Comparativo de Custo das Obras Civis e Mobiliario de Escolas de 1° Grau Custo de construao de USD 44.939 N.A. USD 80.323 N.A. USD 86.776 N.A. 93,10 8,03| uma nova sala de aula Custo de construgAo de USD 49.748 N.A. USD 65.745 N.A. USD 70.746 N.A. 42,21 7,61 uma sala ampliada __ _1______ Custo do mobiliario de USD 3.916 8,71 USD N.D N.D. USD 5.050 5,82 11,24 N.D uma nova sala de aula Custo do mobiliario de USD 3.357 6,75|USD N.D. N.D. USD 2.5113,55 5,05 N.D uma sala ampliada _ _ _ _ _ _ _ _ ________ _______ Fonte: Funda(Wo para o Desenvolvimento da Educa,do - FDE N.D = Ndo Definido N.A.= Ndo se Aplica Com exceqao do mobiliario escolar adquirido em 1994, relativo as salas de aula construidas ate entao, o qual foi adquirido com recursos do Projeto, o mobiliario para as demais salas de aula construidas, reformadas, adequadas ou recuperadas posteriormente a esta data foi totalmente adquirido com recursos pr6prios da SEE. Treinamento de professores - 0 treinamento de professoresfoi previsto para ser realizado em 02fases: Fase I - 1992-1994 com cursos de iniciafao para toda a rede de Ensino na Regido Metropolitana de Sdo Paulo e Fase II - 1995-1998 com cursos avanqados apenaspara escolas que ainda nao tivessem implantado a Jornada Unica (JU) para o Ciclo Bdsico (CB). Embora a fase I tenha ocorrido na forma prevista, porem com baixo indice de execu,cao, a fase II foi totalmente reformulada em 1996. Neste ano, o treinamento passou a ter novo enfoque e abrangencia. A demanda de treinamento parte dos professores e atinge professores de todo o Estado. Instituiu-se o Projeto de Educaqao Continuada que foi executado em dois niveis: (i) Nivel Central - sao priorizadas as atividades voltadas para as liderancas educacionais ou seja Delegados de ensino, Supervisores de ensino, Assistentes Tecnicos Pedag6gicos, Diretores de escola e Professores Coordenadores e (ii) Nivel Descentralizado - sao priorizadas as atividades voltadas para o conjunto de professores. As agces, nos dois niveis, foram executadas em todo o Estado de Sao Paulo, abrangendo todas as Delegacias de Ensino que foram agrupadas em 19 p6los de capacita,co. Estas agoes foram avaliadas por Instituic6es especializadas, que concluiram que o Programa de 58 Educa9Ao Continuada, em que pesem sugestoes de algumas alteracoes no escopo geral, foi um sucesso, tendo atingido maior nuimero de professores que o previsto. Neste ano tambem foi criado o Projeto Classes de Acelerasao, implantado inicialmente em 160 escolas que teve como objetivo reverter o quadro de repetencia e evasao das escolas. Sua meta para 3 anos (1996-1998) foi o treinamnento de professores para trabalhar na reorganizacao da trajet6ria escolar de 3.500 escolas do ensino fundamental. Com o objetivo de fortalecer o PEC e apoiar as lideran,as educacionais da SEE na correcao dos rumnos da educa9do promovendo, planejando, apoiando e avaliando mudan9as educacionais nas escolas e em especial nas salas de aula foi realizado o Seminario Oficina "Facilitando Mudancas Educacionais", para 800 educadores. 0 Quadro 3 mostra o niimero de profissionais treinados atraves dos distintos projetos e seminarios de capacitacao mencionados: Quadro 3 - Nuimero de Profissionais treinados Workshops e Seminarios Professores 11.974 Face a Face Profissionais em educa,cao 8.728 Profissionais p/ cursos de Profissionais em educagdo 20.000 Formacao A Distancia Projeto de Educaao Profissionais em educaao 102.528 Continuada Classes de Aceleracao Professores 4.347 Coordenadores 1.093 Supervisores 890 Assistentes Tecnicos 190 Diretores 100 Seminario Facilitando as Delegados e Supervisores de ensino, Mudancas Educacionais Assistentes Tecnicos Pedag6gicos, 800 Dirigentes da SEE, Diretores de escolas e representantes das I Universidades e outros arceiros Fonte: UGP/SEE Materiais pedag6gicos e informdtica educacional - Para melhorar a capacidade das escolas na oferta de material diddtico-pedag6gico, o Projeto previa a aquisi,do, no inicio, de materiais pedag6gicos visando o CB e 3aS e 4as series. Ate' 1995 estes materiais foram adquiridos somentepor meio de compras centralizadas. A partir de 1995 as compras passaram a ser tambem descentralizadas, atraves de recursos repassados cis Associa,ces de Pais e Mestres - APMs - entidadesjuridicas das escolas. 59 Entre 1993 e 1995, foi prevista a distribuicao de livros, alfabeto plastico, material dourado, escala Cuisinaire para 2.268 escolas com CB a 4a serie. Este material foi distribuido na seguinte propor,ao: - 2.800 escolas receberam livros - 100% do previsto - 1.698 escolas receberam alfabeto plastico - 75% do previsto - 500 escolas receberam material dourado - 22% do previsto - 500 escolas receberam escala Cuisinaire - 22% do previsto Esta distribuicao indica desempenho pouco satisfat6rio, a nao ser na area de lingua portuguesa. A partir de 1995 Projeto ampliou a oferta para todo o Ensino Fundamental, as compras passaram a ser centralizadas e descentralizadas conforme segue: Compra Centralizada - Entre 1995 e 1998 foi previsto e adquirido livros paradidaticos e fitas de video para aproximadamente 3.000 escolas de CB a 4a serie e 5a a 8a s6rie do ensino fundamental. Foram adquiridos 1.695.170 livros paradidaticos ( para escolas de CB a 4' serie) e 466.830 ( para escolas de 5' a 8a serie); distribuidas 77.580 fitas de video para escolas de CB a 4a serie e 16.400 para escolas de 5a a 8a s6rie; Foram feitas tarnbem assinaturas anuais de jornais e revistas para 886 escolas. Alem disso foram adquiridos, atraves de licitacao internacional materiais para 3.000 escolas do Projeto. Para as escolas com CB a 4a serie: - 3.267 conjuntos de Instrumentos Musicais - 3.267 m6dulos rurais e urbanos - 3.267 conjuntos de instrumentos de laborat6rio Para escolas de CB a 4a e de 5a a 8a series; - 5.720 reproducoes de obras de arte - 4.026 conjuntos de modelos anat6micos de torso e arcada dentaria para escolas de 5a a ga serie: - 2.300 conjuntos de paineis cientificos - 6.710 estereo microsc6pios; - 994 conjuntos de 05 computadores para uso pedag6gico, acompanhados de 01 camera de video; 01 scanner; 01 fax modem; 02 impressoras coloridas; 02 No Breaks; 05 monitores. - 42 titulos de softwares educacionais totalizando 4.972 exemplares. Compra Descentralizada -Para a aquisic,ao de material e mobiliario pedag6gico foram feitos repasses as APMs de 3.000 escolas, segundo o seguinte criterio: escolas com menos de 08 salas receberam R$6.400,00 (seis mil e quatrocentos reais), e acima de 08 60 salas receberam R$9.000,00 (nove mil reais), para material pedag6gico. Para aquisicao do kit mobiliario para cada escola a proposta foi de R$390,00 (trezentos e noventa reais). O repasse foi realizado entre 1995 e 1996 quando foram repassados recursos para 2.346 escolas de CB a 4' serie e 1.717 escolas de 5a a 8' serie para aquisi,ao do kit pedag6gico. Para aquisi,co do kit mobiliario foram repassados recursos para 2.187 escolas de CB a 4a serie e 806 escolas de 5a a 8' serie. Adicionalmente, foi feito um refor,o para a aquisi,ao de material pedag6gico variando de R$1.000,00 (hum mil reais) a R$4.000,00 (quatro mil reais), de acordo com o nurmero de salas com base em R$610,00 (seiscentos e dez reais) para cada sala dedicada ao ensino de CB a 4a serie. Escolas com menos de 04 salas nao receberam este recurso. Visando transformar algumas escolas em escolas-p6los de capacita,ao e dissemina,co de experiencias pedag6gicas inovadoras foram repassados recursos para 235 escolas e 147 Oficinas Pedag6gicas, que receberam R$25.000 (vinte e cinco mil reais) cada uma para a aquisi,ao de equipamentos e materiais pedag6gicos. Do mesmo modo as escolas de formacao de professores que atuam como Centros de Forma,co e Aperfei,oamento do Magisterio - CEFAM, tarnbem receberam recursos, no valor de R$ 11.500,00. Merenda escolar Enriquecimento da merenda escolar - A implementa,ao do componente se efetivou atrav6s dos seguintes projetos: (i) Mudan9a do CardApio; (ii) Supervisao de Campo e (iii) Treinamento de Preparadores de Merenda. 0 projeto de MudanVa de Cardapio e Implantavao do PEME resultou em: (i) revisao das especificac,9oes t6cnicas dos alimentos; (ii) implantado o sistema de controle de qualidade de alimentos nas escolas; (iii) fixacao de cardapios trimestrais atendendo as exigencias minimas do PNAE - Programa Nacional de Alimentacao Escolar; (iv) implanta,co gradativa do PEME Projeto de Enriquecimento da Merenda Escolar pelo qual as escolas recebem mensalmente recursos para aquisi,co de alimentos pereciveis e (v) acompanharnento direto na prepara,co das merendas nas escolas orientando-as na organizac,o dos estoques, preparagco de receitas e atendimento aos cardapios e aos."per capitas" fixados pela OMS para esse fim. O Projeto de Supervisao de Campo visou visitas as Unidades Escolares por de um grupo de estagiarios do 40 ano de Nutricao das Universidades/Faculdades do Estado de Sao Paulo. Em 1997 foran estabelecidos nessas escolas cardapios trimestrais, adequados as caracteristicas e necessidades dos alunos, Entre setembro de 1996 a outubro de 1998 foram efetuadas as seguintes visitas de supervisao de campo a escolas da rede publica estadual de ensino de 10 grau. 61 Quadro 4. Visitas de Supervisao de Campo Realizadas 1996 286 673 983 983 1997 1.817 1.179 2.996 2.936 1998 698 50 74890 Fonte: DSE/SEE Aquisicao e utilizacao de equipamentos adicionais de cozinha - Ap6s a elabora9ao de um Inventario Geral de Necessidades de Equipamentos e Utensilios de Cozinhas das Escolas da Rede Estadual de Ensino de 10 Grau foram contratados consultores especializados para a elaboracao de um conjunto de especificacoes tecnicas para os 50 itens de reposicao das cozinhas das escolas, tais como geladeiras, freezers, fogoes, liquidificadores e utensilios diversos a serem adquiridos pelo Projeto. DSE realizou, com recursos pr6prios, diversas licitacoes para aquisicao de utensilios e equiparnentos nos exercicios de 97 e 98, tendo obtido bons resultados conseguindo renovar i 00% dos utensilios e 100% dos equipamentos das escolas centralizadas bem como 20% das escolas descentralizadas dentre as 6.000 escolas de 10 grau do Estado de Sao Paulo. A partir de 1998 o Inventario Geral de Necessidades de Equipamentos e Utensilios de Cozinhas das Escolas da Rede Estadual de Ensino de 10 Grau foi informatizado e passou a ser atualizado anualmente permitindo a proposicao de Planos de Reposicao Equiparnentos e Utensilios de Cozinhas mais realistas. Treinamento de pessoal de alimentacao nas escolas - Entre outubro de 1997 e maio de 1998 foram treinadas 1380 merendeiras pertencentes as escolas da Capital do Estado e 1834 merendeiras de escolas do interior do Estado na area de abrangencia do Projeto. Em 1998 para que todos os municipios do Estado pudessem ser beneficiados com a Capacitacao assegurando assim a continuidade do Projeto em regioes (DEs) nao abrangidas pelo Projeto IEB, foram treinados 113 multiplicadores dentre os quais Supervisores de Ensino, Professores, Nutricionistas, Encarregados da Merenda Escolar e Visitadores Sanitarios e, posteriormente 8.000 merendeiras. lntroducao no DSE de um sistema de informacoes e acompanhamento de custos - foi implantado um Sistema de Gerenciamento Informatizado da Merenda Escolar, que proporciona: planejamento de todas as compras; registro de compras e plano de distribui9ao; controle de estoque central (on line) e dos estoques remotos das escolas; analise dos custos "per capita" de alimentos, de cardapios, etc e reformulacao de compras em razao das informnaces apuradas em relat6rios gerenciais de consurno, estoque, aceitabilidade de cada produto, entre outros. 62 A introdu,co desse sistema gerou um enorme impacto em toda a estrutura e funcionamento do DSE, impondo a necessidade de revisao de competencias e atribui,ces das equipes tecnicas, treinamento e acompanhamento em servi,o, reorganiza,co e moderniza,co dos Armazens, introdu,co de novos sofiwares e controles informatizados em outros setores, tais como as areas de financas e compras, economia nas compras (melhor qualidade na quantidade necessaria com melhor pre9o); redu,co de perdas por desperdicio, em nivel central e local; redu,ao/ elimina,cao de alimentos com prazo de validade vencido nas escolas e racionaliza,ao dos criterios de distribui,co de alimentos, com controles mais efetivos que garantem a eficiencia da transportadora e da escota. Ensino na Pre-escola Expansdo dapre-escola - Dos 39 municfpios da GSP, 26 assinaram em 1994 Convenio com o Projeto. Destes, apenas 21 renovaram o convenio em 1996, e 11 ofizeram em 1997 tendo os outros 10 desistido porfalta de recursos para a contrapartida conforme mostra o Quadro 5 abaixo. Quadro 5 - SituaVio dos Conv8nios com as Prefeituras Previsao inicial (1992) 3 9 ---0 120.00 Protocolo de inten,co (1993) 32 1.227 92.000 272 Assinatura de Convenio 26 1.062 79.650 (1994) 234 Renova,co de Convenio 21 99 429 (1996) 32.175 Renova,cao de Convenio I11 3 4 9.152 I (1997) 143 Do quadro depreende-se que, em fun,ao das desistencias, houve uma revisao da previsao de obras ate o final do Projeto, caindo a previsao inicial de 120 mil vagas adicionais para aproximadamente 32 mil vagas, ou seja, a construgao de 99 escolas totalizando 429 novas salas de aula. 63 At6 a conclusao do Projeto forarn construidas 53 pre-escolas em financiadas em parte com recursos do Projeto resultando em 241 novas salas de aula o que possibilitou a cria,ao de 15.779 novas vagas de ensino pre- escolar nos municipios conveniados. 0 nu6mero m6dio de alunos por sala de aula nova 6 de 32,7 alunos por tumo e por nova pr6- escola construida 6 de 297 crian,as. 0 numero de m2 por aluno por sala 6 de 1,45 m2 e por escola 6 de 1,9 m2. 0 custo do m2 construido, incluindo-se obra civil, mobiliario e material pedag6gico foi de USD 468,77 e o custo por aluno, incluindo-se obra civil, mobiliario e material pedag6gico foi de USD 934,10. Treinamento de professores de ensino pri-escolar - Contratados consultores especialistas em educa,cao para procederem a revisao e atualiza,do da Proposta Pedag6gicapara a Pre-escola elaborada em 1987, resultando o livro "Proposta Pedag6gica para a Pri-escola para distribui,do aos professores das pre-escolas dos Municipios conveniados eposteriormentepara servir de referenciapara todo o Estado. Realizados os seguintes seminarios: (i) Seminario de Lan,camento do Modelo de Plano Municipal de Expansao e Melhoria da Educacao Pre-escolar; (ii) Seminario sobre o tema "Livro - Proposta Pedag6gica para a Pr6-escola"; (iii)Seminario na PUC-SP, orientado para as 439 ONG's que atuam na RMSP com crian,as na faixa de o a 6 anos, identificadas atrav6s de urn levantamento cadastral promovido pela Secretaria de Estado da Promo,ao Social e FDE, destinado a divulgar o Projeto IEB e o Programa Pre-escolar, do qual participaram de 200 pessoas. Neste seminario foi apresentado um questionario que preparou as ONG's no campo da pesquisa que a CEBRAP executou para a qualificacao das mesmas; (iv) Seminario de Capacitagco para Presta,ao de Contas; e (v) I Encontro das Pre-escolas do Projeto IEB. a) Consultorias - Contratados consultores para: (i) preparar os Termos de Referenciapara orientar os municipios conveniados ao Projeto na elabora,do dos seus Planos Municipais de Ensino; (ii) assessorar os municipios na elabora,do e implementa,do dos seus Planos Municipais de Expansao e Melhoria da Educa,do Pri- escolar, bem como na criaJcdo de modelos de planta,do e implementa,cdo desses Planos Municipais aprovados e na elabora,do dos instrumentos para Avalia,do e Aprova,do final desses planos em seus aspectosfinanceiros, fisicos e educacionais; (iii) elaborar Forarn criados os: (i) o Manual de "Qualidades do Espa,o e dos Equipamentos na Pre- escola - Recomenda,ces" destinado a orientar os projetos dos edificios de pr6-escolas em termos da qualidade esperada dos espa,os a serem construidos e dos equipamentos a adquirir; (ii) modelos de: Plano Municipal de Expansao e Melhoria da Educa,co Pre- escolar; (iii) Parametros de Custos de Constru,co, equipamentos e mobiliario; Programa Arquitet6nico e Projeto Padrao de Arquitetura; (iv) livro contendo a Proposta Pedag6gica e de Capacita,co; (v) Manual de Presta,ao de Contas; (vi) Programa de Capacita,ao do Quadro de Magist6rio; (vii) Metodologia de Analise Demografica; (viii) Metodologia de Avalia,ao dos Planos Municipais de Expansao e Melhoria da Educa,co Pr6-escolar; (ix) Cartilha com os passos do Pre-escolar, do Convenio a Inaugura,ao das Escolas e (x) 64 minuta da documenta,ao de licita,ao de obras e aquisi,co de bens, bem como de selecao de servicos de consultoria. 2. Saude na escola Com base no PGI cada um dos municipios definiu o que seria realizado a partir das suas propostas de aten9ao integral A sauide da crianca e do adolescente, destacando as acoes financiadas pelo BIRD e a respectiva popula,ao alvo, a serem implementadas e monitoradas separadamente, elaborando assim Pianos Municipais de Implanta,co do Projeto IEB , para o periodo de 1993-1997. Inicialmente, os recursos do Projeto foram destinados a RMGSP e posteriormente, com a reformula,a9o do projeto, ao final de 1995, foram expandidos para os 96 municipios das Regioes de Sorocaba e Registro. IntegraVdo sauide nutriV4o - Nado executado pelo Projeto uma vez que, a Secretaria de Saukde concluiu ser mais l6gico uma iniciativa da propria Secretaria da Educa do, definindo temas, carga horaria e recursos humanos necessarios para, entdo, ser contratada uma consultoriapara adequaqdo de conteuido programdtico apropriado. a) Examesfisicos - Ndo executado pelo Projeto. b) Testes de Visdo e Audip7o e encaminhamento ds UBSs para tratamento Sauide ocular - Foram realizados treinamentos envolvendo 540 profissionais da sauide - divididos 27 turmas; foram adquiridas 8.000 Tabelas de Snellen (instrumento para a triagem), porem a implementacao dos testes de acuidade visual nAio ocorreu, em virtude da nao disponibilidade de t6cnicos especializados - no caso o oftalmologista - em todos os municipios participantes, para proceder As consultas, e tambem A inexistencia de urn sistema de referencia institucional. Alguns municipios contudo manifestaram interesse e condicoes de proceder ao teste de acuidade visual, porem como um edital para o fornecimento de servicos 6pticos (para aviar receita de 6culos) nao havia sido definido pelo BIRD e SES, estes municipios realizaram a atividade autonomamente, sem respaldo do IEB. Em setembro de 1998 o edital e a Short List de institui,ces especializadas em Sauide Ocular foi aprovado pelo Banco, com previsao de reembolso exclusivamente para os servi,os a serem executados ate dezembro de 98. Contudo, em virtude da complexidade das atividades e das varias institui,ces a serem convidadas, tornou-se inviavel a implementa,ao dos servi,os previstos at6 o final do Projeto IEB. 65 Saude auditiva - Elaborado o " Estudo de Viabilidade de Assistencia Integral em Sauide Auditiva na Rede Puiblica", que constatou a ausencia de recursos humanos suficientes para a implementagao das metas do sub-componente, ausencia de um sistema de referencia regionalizado e especializado de servi9os que desse conta da integralidade da agao, e falta de experiencia acumulada na rede de sauide puiblica. Como resultado, optou- se pela contratagao do Contrata9ao Direta da Funda9ao para o Estudo e Tratamento das Defornidades Cranio - Faciais (FUNCRAF) para desenvolver essas atividades dado tratar-se de um Institui9ao de not6ria especializagao, com os requisitos apropriados e necessanos para testar a implementa9ao, criando, a partir de resultados, um modelo exequivel e que garantisse a institucionalizagao e a continuidade das a9oes. Foram triadas 19.637 crian9as; realizados treinamentos, envolvendo 6.991 pessoas (196 turmas) entre profissionais da educagao, pais e comunidades em geral. c) Conclus4o do programa de imuniza(ao - Nado executada pelo Projeto. Realiza,co de prdticas preventivas de Satide Bucal - Executados procedimentos coletivos de saude bucal em 1. 692.626 crian,as. Adquiridas 6.456.844 escovas dentais e 5.463.027 tubos de pasta dental entre outros materiais de consumo, equipamentos para 104 consult6rios dentdrios e 26 clinicas modulares, 1.000.000 de exemplares da revista "Turma da M6nica e a Sazude Bucal" e um video com 165 c6pias para treinamento. Treinados 6.589 profissionais. d) Estudo piloto para medir a deflciMncia de vitamina A eferro - FUNCAMP desenhou apesquisa e sobre as condi,ces de vida e saude das crian,as de 5 a 9 anos na RMGSP. 0 trabalho ndo foi realizado dado o elevado custo da pesquisa e o prazo para sua execuVdo. e) Observa,ico da sauide dos estudantes de 1°grau - Ndo executado pelo Projeto. f) Inspe(Wo ambiental das escolas - Ndo executado pelo Projeto. i) Registro de a,ces coletivas desenvolvidas nas escolas e nas UBSs - Criado os procedimentos para o registro, adquiridos 101 equipamentos, software e mobilidrio de informatica e adquiridas 1. 400.600 fichas de encaminhamento entre outros materiais de consumo para apoio as a,ces de registro de a,ces de sazude. 66 j) Treinamento de pessoal das UBSs - Foram adquiridos e repassados aos municipios materiais de consumo para realizaqdo de seminarios locais entre os profissionais da saude e da educaqdo, para 11.489 pessoas. g) Recrutamento de pessoal - Contratados os servi os de consultoria da FUNDAP para prestar o apoio tcnico e administrativo i Coordenaqdo e executar serviqos de treinamento. 3. Avaliag&o e divulgaVAo 0 componente implementou tres tipos de estudos: analise de custos, processo de implementa,co e impacto e a avalia,ao do Programa de Educa,ao Continuada. A analise de custo mediu os reais custos das medidas introduzidas pelo projeto, o custo - aluno e o custo-eficacia das medidas; a pesquisa sobre o processo de implementa,co examinou as condicoes e determinantes institucionais da implementa,ao dos varios elementos dos programas. A avalia,ao do impacto analisou os resultados do rendimento dos alunos em escolas que receberam os insumos do projeto. Quanto a avalia,cao do PEC, esta procurou avaliar a eficiencia e a eficacia do programa. Para desenvolver estes estudos previa-se a contrata,co de institui,ces especializadas o que foi feito tendo sido contratadas institui,oes externas, por not6ria especializa,ao . Desenho e execuVdo de estudos de avaliapdo Anilise de custos - A Funda,ao Instituto de Pesquisas Economicas-FIPE e FUNDAP apresentaram propostas de projeto, que foram analisadas por consultores contratados pela UGP. Aprovada a proposta da FIPE , esta executou o estudo com o objetivo de se estimar os custos anuais por alunos, em uma amostra, e produzir uma analise empirica dos custos dos recursos educacionais fornecendo instrumnentos de apoio ao planejamento, a avalia,ao e a decisao da politica educacional da SEE. A metodologia utilizada para a pesquisa custo - aluno foi a de Numero - Indices e para o custo eficacia a dos ingredientes de Levin. Esta pesquisa desenvolveu uma metodologia para a apura,ao do custo - aluno, um indicador necessario na avaliac,ao das diversas intervenc,es adotadas na rede puiblica de ensino e para garantir a qualidade da gestao dos recursos educacionais. Como extensao de seu trabalho, a FIPE desenvolveu um software e respectivo manual de utiliza,ao para um sistema informatizado que visou instrumentalizar a SEE para acompanhar e atualizar os calculos de custo - aluno. Durante todo o ano de 97 a FIPE juntamente com a ATPCE, efetuou adapta,ces e altera,ces, sempre que necessario. A FIPE se encarregou tambem do treinamento de 67 pessoal para a operavAo do sistema. Em 1998 o sistema em sua nova versao, passou a ser utilizado pelos tecnicos da ATPCE, encarregados dos planejamentos na SEE. Anailise do processo - 0 Nuicleo de Estudos de Politicas Puiblicas-NEPP da UNICAMP, elaborou e executou o projeto para a avalia,ao do processo de implementac,o das inova,ces do Projeto IEB, com o objetivo de acompanhar apenas as medidas realmente implantadas, contemplando as seguintes interven,ces: Macro politicas da SEE, Capacita,ao, Material Pedag6gico, Plano Diretor e Contrato de Gestao, Horas de Trabalho Pedag6gico (HTP) e Coordenagao Pedag6gica, e Saiude Escolar. 0 NEPP organizou e realizou tambem um workshop para os tecnicos da SEE, destinado a discutir os resultados alcan,ados e receber sugestoes no sentido de aperfei,oar os trabalho. Algumas sugestoes foram apresentadas sem contudo alterar a linha de trabalho proposta no projeto executivo. Os resultados finais do trabalho foram apresentados em reuniao final, com a participa,co de especialistas em avaliasAo da SEE e de outras institui,ces. Foram produzidos 12 relat6rios desta pesquisa que vem sendo bastante utilizados por varios 6rgaos da SEE. Anilise do impacto - Em 1990 a Funda,cao Carlos Chagas-FCC foi contratada para desenvolver o desenho da avalia,ao do impacto. 0 objetivo da pesquisa foi o de se verificar o impacto nao s6 das inova,oes como tambem das macro politicas educacionais sobre os niveis de aprendizagem dos alunos, matriculados nos diferentes tipos de escolas, nao apenas nas escolas com Jornada Unica mas tambem nos demais tipos de escolas: escolas com Jomada llnica; escolas sem Jornada UJnica e Escolas Padrao. A seguir a mesma instituicao foi contratada para iniciar a pesquisa avaliativa Avaliacao do processo de Inova,ces no Ciclo Basico e seu impacto sobre o ensino- aprendizagem na Regiao Metropolitana de Sao Paulo", durante 3 anos. A pesquisa envolveu dois estudos: (i) de natureza longitudinal e abordagem quantitativa, abrangendo cerca de 3.600 alunos matriculados no CBI em 92, e (ii) de natureza antropol6gica, em urm nuimero reduzido de alunos, 140, com abordagem mais qualitativa. Foi definida a amostra e aplicados questionarios em professores e dirigentes, bem como aplicadas as provas nos alunos e a seguir analisadas. Para a apresenta,ao do relat6rio final da avalia,co foi organizado um workshop para se discutir os resultados alcan,ados, do qual participaram especialistas nacionais e pesquisadores da FCC encarregados da pesquisa. A UGP e a FCC realizaram tambem um Seminanrio Intemacional para a disseminacao dos resultados, que recebeu o nome de "Modelos Avaliativos" com o duplo objetivo: de disseminar as conclusoes da pesquisa de impacto e discutir um modelo para a implanta,co de um sistema de avalia,co permanente para o Estado de Sao Paulo. Em 1997 a conclusao do trabalho foi sintetizada em uma publica,Ao - " A escola que faz diferen,a", para divulgar os resultados aos profissionais da area da educa,ao. Esta publica,ao foi amplamente utilizada nas Horas de Trabalho Pedag6gico das escolas. AvaliaVio de Treinamento - A UGP juntamente com o auxilio de consultoria externa e ajuda de especialistas do Banco Mundial, delineou a pesquisa avaliativa do programa Educa,co Continuada. Considerando a grande extensao do programa, quer em 68 abrangencia geografica, quer em niimero de participantes, optou-se por dividir o estado em 4 regioes (abrangendo cada uma varios dos p6los organizados pelo PEC) e contratar 4 institui,ces extemas para desenvolverem a pesquisa e uma 5a instituicao para coordenar os trabalhos afim de que houvesse compatibilidade nos procedimentos de pesquisa e de que se pudesse estender as conclusoes para o Estado como um todo. Visando a avaliagao administrativa do PEC foi contratado um consultor para a montagem de um Sistema Gerencial que abrangeu todas as informag6es sobre as va6rias a,ces desenvolvidas no ambito do PEC e devera servir de catalogo de oferta das a,ces para o Estado de Sao Paulo. Para a dissemina,co dos resultados foi organizado um workshop. As conclusaes finais serao enviadas As DEs como subsIdios para a continuidade do programa. Disseminapio - A partir de 1995foi iniciada a divulgacdo das experiencias e resultados do projeto nao so no dmbito da educa,do mas tamberm a nivel estadual visando informar, mobilizar e obter o apoio da popula,do e, em particular dos alunos, para as diretrizes educacionais implantadas pela SEE no dmbito do projeto IEB. Assim, foram implementadas as seguintes a,oes: (i) elabora,cao e distribui,do do jornal "Escola Agora, aprendendo sempre ", com uma tiragem de 300.000 exemplares por nulmero, visando manter um canal de comunicaqao entre a SEE, as DEs e as UEs; (ii) produ,co e distribuiq ao do Boletim "Fazendo Escola " com a tiragem de 20.000 exemplares por nuimero; (iii) producdo e emissdo, atraves da TV Cultura, de programa em canal aberto com video sobre a Reorganiza,co Escolar e (iv) elabora,co e divulga,cao de uma cartilha, entilulada " 0 que mudou na escola piiblica paulista?" que traz respostas ias perguntas sobre a atual gestdo do sistema educacional e informa, a comunidade escolar, as diretrizes educacionais implantadaspela SEE e (v) publica,ao do boletim da Secretaria de Estado da Educacao "Fazendo Escola" com uma tiragem de 20.000 exemplares cada uma. 0 cons6rcio RINO/DPZ foi contratado para desenvolver eventos previstos e implementar o Programa de Comunica,co Social. Para tanto, foram desenvolvidas as seguintes atividades: Planejamento Geral do Programa; Cria,ao do Material a ser divulgado; Produ,ao deste material. Contratou- se tamb6m consultores especializados em Comunica,ao. Varias campanhas educacionais sobre os projetos da SEE foram realizadas no armbito desse subcomponente a saber: (i) "Escola nas Ferias" que proporciona uma nova oportunidade de recupera,co e promo,co para aqueles alunos que nao tiveram aproveitamento escolar durante o ano escolar a qual obteve, em 1996, a participa,ao de 225.545 alunos, dos quais 74.832 eram alunos de CB a 4a e os demais de 50 a 80 serie do ensino fundamental e resultou na aprova,ao de 53,3% do total de inscritos; (ii) "Campanha Recursos repassados As APMs" que informou sobre os recursos repassados diretamente as Associa,ces de Pais e Mestres, no sentido de mobilizar seus membros a participarem das a,oes; (iii) " Campanha SARESP 97" com anuincios em jornais, revistas, comerciais em TV, chamadas em radio, que visavam obter alto indice de adesao e participa,ao dos alunos nas avalia,oes que seriam aplicadas nos alunos de 4a e 8a series; 69 (iv) "Campanha Municipaliza,co" que visou obter o apoio dos Govemos Municipais e da comunidade para a municipaliza,co do ensino de 1a a 4a serie, veiculando amplo material informativo nos principais jornais; (v) " Campanha Carreira do Professor" que procurou motivar, mobilizar e estimular a adesao e participa,cao do corpo docente estadual ao plano de reforma e melhoria do ensino basico, por meio da produ,cao e veicula,cao de comerciais para TV, chamadas de radio, anuncios de joomal, videos informativos; (vi) " Campanha Dia do Professor", " Campanha Treinamento de Merendeiras". Desenvolvimento institucional e avaliacao Estudos independentes e MunicipalizaC4o - Em 1989 o Estado de Sao Paulo havia dado inicio a um esforf o de municipalizaVdo do ensino fundamental, especialmente a pre- escola, como meio de dar resposta ai maior preocupavao da SEE, no que diz respeito ai questdo organizacional e gerencial, representada na necessidade do envolvimento das autoridades mais proximas do processo ensino-aprendizagem, afim de garantir respostas mais rapidas e efetivas as necessidades das crianqas. A meta do Projeto era municipalizar todas as escolas dos 38 municipios da regiao de abrangencia do projeto exceto o municipio de Sao Paulo. Para tanto foram realizados: (i) o levantamento de experiencias anteriores, visando a realiza,ao de estudos para a municipalizac,ao de todas as escolas; (ii) o modelo do Plano de Municipaliza9cao; (iii) levantamentos da capacidade financeira dos municipios para a absor,ao do ensino de 10 grau; (iv) um Seminanrio, onde os secretarios municipais de educa,ao discutiram o modelo do Plano de Municipalizacao apresentado. Foi tambem elaborada a proposta pedag6gica para a pre-escola, constituindo uma publica,ao. Para discutir o conteuido da proposta, foi realizado um Seminario. Em 1995 a participa,cao municipal na pre-escola atingiu 100% da meta e passa a ser incentivada a promo,co da participa,ao dos municipios na solu,cao dos problemas da Educa,ao Fundamental. A meta passa a ser a municipaliza,ao das escolas que oferecem o Ensino Fundamental. Em 1996 foi assinado o Decreto n° 40.673, instituindo o Programa da Municipaliza,co. Para auxiliar esse processo a Secretaria da Educac,o contratou os servi,os da FUNDAP, como assistencia tecnica especializada para o Programa de Municipaliza,ao e do SEADE, que realizou estudos analiticos identificando desigualdades entre os municipios. Ate dezembro de 1998, atraves do Programa de Municipaliza,ao, 283 municipios firmaram convenio com a SEE. Com isto, a porcentagem de atendimento do Ensino Fundamental pelo municipio que em 1990 era de 10% subiu para 18%, representando praticamente o dobro do atendimento, entretanto, considerando apenas o ensino de CB a 70 4a o percentual passou de 16% para 31%. Em marco de 1999 novos convenios foram assinados, elevando para 366 o nuimero de municipios conveniados . Asimm, o Estado de Sao Paulo conta hoje com 471 prefeituras assumindo o Ensino Fundanental por convenio ou por iniciativa pr6pria. A SEE manteve assistencia tecnica aos municipios tendo realizado transferencias de recursos financeiros SEE/FUNDEP, cessoes de uso de bens patrimoniais do Estado para municipios e permissao de uso de predios estaduais pela Rede Municipal. Em 1997 a FUNDAP apresentou produtos que servirarn de subsidios para o desenvolvimento dos trabalhos da Politica de Descentralizacao do Ensino Fundamental, entre eles 3 publica,oes " Aspectos Financeiros", "Instrumentos Juridicos e Administrativos" e "Instrumentos de Gestao Municipal". Para divulgacao, analise e discussao das publicacoes, a FUJNDAP, o SEADE, e a SEE realizararn urn Seminario e organizaram um curso para orienta,co aos tecnicos da SE. A SEE reputa imprescindivel, num pr6ximo momento, a necessidade de monitoramento e avaliacao dessas mudancas no interior do sistema puiblico do estado para possiveis correcoes e ou continuidade do processo Refor,o institucional - Este sub-componente objetivou o "Refor,o das Institui,ces responsaveis pela gerencia de educa,do: melhoria da capacidade de planejamento e avalia,do da SEE e dos munic'pios para a administra,cao de um sistema escolar de ensino fundamental. Outro objetivo visava o refor,o da capacidade das ONGs em oferecer serviqos educacionais de qualidade ". Para atingir os primeiros objetivos foram realizados estudos sobre os seguintes temas: (i) "Financiamento da Educacao no Estado de Sao Paulo", que permitiu elaborar posteriormente, documentos sobre financiamento na area de educacao; (ii) a Assessoria Tecnica de Planejamento e Controle Educacional - ATPCE - da SEE elaborou uma base primaria de dados da Educacao que foram coletados e processados por ela e pelo CIE - Centro de Informacoes Educacionais; iii) implantacao de ums sistema desenvolvido pela PRODESP visando a reorganizacao de rede; iv) analise do fluxo escolar do ensino regular; v) estudos visando identificar as fontes e usos dos recursos destinados a educacao com a apresentacao de relat6rios sobre o financiarnento da educacao no Estado de Sao Paulo. Para agilizar a execucao das metas deste sub-componente foram contratados: (i) a Fundacao de Desenvolvimento Administrativo - FUNDAP - com o objetivo de assessorar na area administrativo-econ6mica e financeira a SEE na implantacao da politica de descentralizacao do ensino basico; (ii) a Fundacao Sistema e Analise de Dados - SEADE - para realizacao de estudos, pesquisas, analises, sistematizacao, geo-referencial e disseminacao de indicadores s6cio-educacionais para o estado de Sao Paulo. 71 Visando aumentar a capacidade institucional da SEE quanto ao seu nivel de automa,ao e gerenciamento de sistemas de informa,ao foram adquiridos da Unisys do Brasil, centralizadamente, equipamentos para um sistema de informatiza,ao da rede da SEE e de 143 Delegacias de Ensino e 6 6rgaos centrais e descentralizados recursos para 6.000 escolas para compra de equipamentos de informdtica. Adicionalmente, foi contratada a Unisys do Brasil para desenvolver um software aplicativo para a administra,co escolar. Afim de fortalecer a competencia tecnica dos profissionais da educa,co nas areas administrativas e pedag6gicas foram produzidos e distribuidos as escolas: (i) 23 videos . abordando temas como: APM; Conselho de Escola; Informatica na gestao escolar etc. e (ii) Jornal:Escola-Agora Sistema de Avaliafdo de Rendimento do Estado de Silo Paulo - SARESP - Em 1992, a CENP executou uma Avaliaqdo Diagn6stica das escolas integrantes do Projeto de Reforma de Ensino no Estado de Sdo Paulo que permitiu estabelecer um perfil inicial do rendimento escolar dos alunos das 306 escolas-padrao. Foram avaliadas 306 escolas, envolvendo 27.609 alunos de todas as 8as series, em conteudos de Portugues, Matematica, Hist6ria, Geografia, Ciencias, referentes as vdrias series de 1°grau. Posteriormente, segundo nova diretriz, o programa de avalia,a-o passou a considerar o rendimento das escolas de toda a rede e nao s6 das escolas padrao. Foi efetuada uma revisao do universo das escolas, determinando-se nova amostragem para a avalia,ao de 818 escolas representativas das caracteristicas gerais do sistema de ensino e instituindo-se o Programa de Avalia,cao Educacional da Rede Estadual de Ensino. Foi constituido um grupo de trabalho com assessoria extema indicada pelo Banco Mundial que procurou aperfei9oar o delineamento da avalia,cao, que passou a ser: "Sistema de Avalia,ao de Rendimento Escolar do Estado de Sao Paulo" - SARESP- cujo projeto foi elaborado pela Coordenadoria de Ensino e Normas Pedag6gica - CENP e pela FDE, com consultoria do grupo de trabalho. Em 1994 a avalia,co abrangeu 152.279 alunos de toda a Rede Publica Estadual das 4as e 8as series, que realizaram provas de Portugues, Matematica e Reda,ao. Os alunos das 8a series tambem foram avaliados em Ciencias, Hist6ria e Geografia que resultou na constata,ao de que os dados gerados mais serviam para orientar os 6rgao gestores das politicas educacionais do que o conjunto das escolas da rede. Ap6s as discussoes do projeto e dos objetivos de Medi,ao do Rendimento concluiu-se o projeto a ser implementado. Na seque8ncia, a Funda,ao Carlos Chagas capacitou te6rica e administrativamente o grupo de tecnicos da SEE /FDE encarregados da execuyao do programa. Iniciou-se a implantagco do Sistema ap6s o referendo do projeto logrado por ocasiao do Seminario Intemacional de Modelos Avaliativos no qual, especialistas de renome internacional, apreciaram o projeto e expuseram teorias relativas a Sistemas de avalia,ao, concluindo pela boa delinea,ao do projeto elaborado, embora reputado como bastante arrojado. A SELECT treinou as DEs e UEs para-a aplica,ao e corre,ao das provas. 72 Em 1996 iniciou-se a Ia aplica,ao de provas na universalidade dos alunos das 3a e 7a series do ensino fundamental de Sao Paulo bem como em alguns alunos das escolas municipais e particulares, embora estas nao fossem incluidas na analise dos resultados realizada pela Funda,ao Carlos Chagas. Em abril de 97 foi realizada a 2a aplicac, o porem em alunos das 4a e 8a sries do ensino Fundamental. Em 1998 foram avaliados alunos das 5 series do ensino Fundamental e lI serie do ensino Medio. Com esta terceira aplica,co a SEE ja tem elementos para iniciar, paralelamente a avalia,co de desempenho, a avalia,ao com cunho longitudinal, onde se pretende analisar o ganho real no aproveitamento dos alunos. Para esta analise, a FCC, contratada para a analise, juntamente com os tecnicos da SEE estarao utilizando a Teoria de Resposta ao Item -TRI. d) UtilizaVAo de parcerias publico/privadas: ONGs Com o objetivo de refor,ar a capacidade das ONGs em oferecer servi,os educacionais de qualidade foram realizados estudos de avaliacao do funcionamento das ONGs na Regiao Metropolitana de Sao Paulo, quanto a educa,ao pre-escolar. Estes estudos foram realizados pelo Centro Brasileiro de Analise e Planejamento - CEBRAP que apresentou relat6rio sobre a distribui,ao espacial da atendimento as ONGs e avaliacao dos aspectos formais de funcionamento das ONGs e sua capacidade financeira; Para agilizar a execu,ao das metas deste sub-componente foram contratados alem do CEBRAP: a Cooperativa T&enica Educacional - CTE - para estudar as formas legais e institucionais de parceria, como efetiva-las e apresentar propostas sobre politica de atua9ao da SEE com empresas e entidades e a Funda,ao Carlos Chagas - FCC para realizar pesquisa sobre a importancia da participacao comunitaria na questao da educa,cao e da probreza. iiA DI 9! & I.;& APLICACAO DOS RECURSOS Os Quadros 6 a 8 mostram a Aplica9ao Consolidada Total dos Recursos do Projeto por Componente comparando-as com as previsoes do SAR no periodo 1991 a 1996 e com as metas dos Bench Marks de 1996 ate o encerramento do periodo de Gra,a do Projeto em 30/04/99. Deste se depreende que o valor total do Projeto estimado em US$ 600,0 milh6es foi superado atingindo US$ 601,8 milh6es, apesar da reduzida adesao dos municipios na implementa9ao do sub-componente BI no qual foram repassados somente 38,5 % dos recursos para eles previstos . 73 Quadro 6 - Contribuigies por Quantidade e InstituiVcao Valores em USD Milhoes BIRD 24 0 40,8 243,0 99, 1 40,4 Estado 331, 55,2 349,5 105,4 58,1 Municipios 24,0 4,0 9,3 3 8,51, Fonte: UGP/SEE Quadro 7 - Aplica,cao total dos recursos por componente e categoria de despesas Valores em USD |Componente A -Mehoria da gauaLidade na escola Construcio - Obras I 3428 532,59 Equipamentos c materiais Mobilias e Equipamentos 2a 7 366.03829 Equipamentos e Utensilios 2b 170 210,40 Bens 3 120 500.003,31 Treinamento Primeiraa fae - parte A(2) Sa 3 043.717,04 Segunda fase - Parse A(2) 5b 30 561.914,22 Cnmpnnenze B - Ensso na pre-escola Construcio/Equipamentos e mater6sis 4 14 083.824,81 Treinamento 6 13.804,96 Servigos de especialistas 9 430 465,0i Cemponenwe C - Saude no eacioa Equipamentos e materiais/gastos 10 11 918.622,27 Servios de especialistas 11 12.115,01 Comnonente D - Avaliacao e divusleaco Servisos de especialistas 7c 20.341.105,69 Ccn,ssenme E - DesemonlJmens o Inshfyc,nnal e s,alsac Servisos de especialistas Servigos de consultoa 7a 21.062.794,88 Servios de consuatona 7b 158.918,34|| A dministracdo do p-ie t. Servigos de espeeiaiistasIgastos operacionais de acrescimo 8 7 881,003,03 Fontesi UGPISEE recorenie Fonte: UGI>/SEE 74 No Quadro 8 a seguir se observa tambem que ate 1995 somente 18,9% do valor do acumulado previsto no SAR ate aquele ano havia sido executado e que a partir de 1996, com a retomada do andamento do Projeto, ocorreu um desenvolvimento linear e crescente na aplicacao dos recursos superando as metas previstas nos Bench Marks. Quadro 8 - Aplicafao dos Recursos CComparaSao Anual 630- 68616.7- 600 _ s 570-;5r 601.6 540- 510 stc~~~~~~~~~~~~~~~~~~~~~~~~~~~~4 .32e6 X 450 40 ~o 420- oc -/ 406.2 E 360 6330 E 300 1 2702 E 240 1 1210 1- 160 100 120 106.4 60 60 1991 1992 1993 1994 1995 1996 1997 1998 1999 Ano Civil jan a dez) 4-PREVISTO SAR -PREVISTO BENCHMARK -AREALIZDO A sustentabilidade do processo depende basicamente do Estado e Municipios envolvidos, quando for o caso, assegurarem os recursos financeiros para o prosseguimento dos seguintes linhas de acao, uma vez que ja existe capacidade instalada nesses orgaos para seu planejamento, implementacao, monitoramento e avaliacao, adquiridas com a experiencia vivenciada no Projeto IEB: (i) programnas de construcao de novas escolas e de ampliacao, adequa,cao e recupera,co das escolas existentes bem como de sua manutencao preventiva e corretiva pela FDE; (ii) programa de complementa,ao e substitui,cao de mobiliario e equipamento escolar pela FDE; (iii) programas de capacita,co de profissionais de ensino; (iv) programa de dota,co de material pedag6gico; (v) programa da Merenda Escolar (vi) municipalizacao do ensino Fundamental e da Pre-scola; (vii) 75 aplica,co do Sistema de Avaliac,o do Rendimento Escolar; (viii) programas de Sauide na Escola pela SES e (ix) Estudos sistematicos de avalia,ao de custos, processo, impacto e Educa,ao Continuada aliados ao continuo esforco voltado ao desenvolvimento institucional. A recente reelei,ao da atual gestao do Governo Estadual constitui uma garantia da continuidade, aprimoramento e financiamento da atual politica educacional amplamente apoiada, desde a fase inicial de sua implanta,co, nas linhas de a,co preconizadas pelo Projeto. A nivel municipal, o prosseguimento das a,ces de municipaliza,ao do ensino pre-escolar e fundamental tem obtido a sucessiva adesao das municipalidades ainda nao conveniadas o que demonstra o acerto da politica adotada e assegura condi,6es para o seu progresso . 76 O Projeto de Inovac,es no Ensino Basico de Sao Paulo executado pela Governo do Estado de Sao Paulo atraves da Secretaria de Estado da Educa,ao no periodo de 7 de janeiro de 1992 a 31 de abril de 1999, com recursos advindos em parte do financiamento do Banco Intemacional de Reconstrugao e Desenvolvimento - BIRD, parte do Tesouro do Estado de Sao Paulo e parte da Prefeituras Municipais conveniadas ao Projeto, apresentou resultados expressivos para a melhoria do ensino puiblico estadual de 10 grau na Regiao Metropolitana de Sao Paulo e Regioes de Sorocaba e Vale do Ribeira e para o ensino pAblico municipal pre-escolar na Regiao Metropolitana de Sao Paulo diminuindo a reten,co escolar das crian,as das familias pobres e migrantes das areas programaticas, descongestionando as escolas existentes. Os resultados alcan,ados pelo Projeto no cumprimento dos objetivos de diminuigao dos indices de retengao e evasao das escolas bem como das demais metas compromissadas no Acordo de Emprestimo acham-se espelhados de forma sintetica nos Quadros 9 e 10 a seguir. Quadro 9 - Rede Estadual - Indice de Retensao do CB a 4a serie do Ensino Fundamental RMGSP 14,6 13,6 14,8 13,1 9,2 3,5 1,07 Area de abrangencia do Projeto 14,11 11,23 12,33 10,82 7,55 2,95 () Fonte: Centro de Infarmacdes Educacionais (I) dado preliminar; 2) inclue as Regioes: GSP, Sorocaba e Registro e (3) dada nado disponivel Ouadro O - Rede Estad al - Indice de Evasao do CB a 4a serie do Ensino Fundamental RMGSP 6,24 5,63 5,16 5,03 4,12 2,69 2,24 Area de abrangEncia do Projeto 6,77 6,24 5,6 5,4 4,29 2,73 (3) Fonte: Centro de Informacoes Educacionais (1) dado preliminar; 2) inclue as Regioes: GSP, Sorocaba e Registro e (3) dado nbo disponivei IBRD 21533 i _ 5 _ ~~~~ a, 47 00B R A Z I L >-6t,+ SOUTH AMERICA INNOVATIONS IN BASIC EDUCATION PROJECT >- - MINAS -. ItA>ll , SCHOOL DISTRICTS AND MUNICIPALITIES .x-,s-tGRl RRAIAZ, AV- INGESOAPUL \_ BrARA5i, A$ ' 5':\ IN GREATER SAFO PAULO s O''wPAUL 0 S y R A.C ., S A ANAi, .1 AL. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~2IGARATA An~ s FRANCISCQ ;S;'~~~~~t -~~, PARAN CFA S 7%~~~~~~~~~~~~~~~~~~~~~~~7 A \ r-RANCISo 0 / {< ] , t~~~~~~~~~~~~FACO DA SA N TA ISABEL *_tu0 'I ~~~~~ROrHA, i MAIRIPORA AE CAJAM4R r--CAIEcRA$ RAM ,R54F r 7Al 9 . SANTA BRANCA u. SANTANA DE PARNAIBA D C P -23930 5\9 /--JOW 9 D - .n-2 vQ;t ; (_ rGLARA REMA R/ 23' 30 E j7 ROsQU i Fv v !jfaS- 'S ,'!! MO J I DAS r; _, : - jr1 n };Ej,j,r N e S IZ W SARe z g tm- f CRUZES1 > \sA SALESCGPOLIS ,$CRUZES SAc RCA~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~_ A I ITAPEV }> / - . . ]' ' CununlOM irAI 55 RIa z 8F1;\ t | t>E>ASIb! P 21 _ |1l / ; \ / 1 BRITlbA MIRIM ,QA'R .AO!, VRPAU (C A TIi- COTIA - E MA 5UZANO * | N ; OK: r CTV ITAPECERICA - 5 GUACU SAO B 1Re,, 'na-- -- eianol School Division Boundories V-x~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~~~~~~~~School Distict Boundaries .1 ,/ a - t ,, /: ,. ),' .1' ' t 'I . - School District Offices outside the QJUUITSBA ' Municipoli7y of Sao Paulo A AAA%NYOA 1 _ 5,S 18 /] - r A i N / < 5 !'12 School Districts insid the Municipsality V SAC VlC~~~~~~~~~~~~~~~~~~~~~~~~NTB - ~~~~~~~~~~~~~of SBo Paulo' Built-up Areas _ ......... _ ... . " r.s-/e % { GRPAANIDAE =' I' irf - -= Resorvoirs 24-0 0 V 9 - - : / Su ~ - ~~ ~ - Rivers ..... ... Municipolity of Sao Paulo Limits of Greater Sao Paulo Area Municipal Boundaries State Bounduries (inset) 0 5 10 I5 20 KILOMETERS --- Internationol Boundaries (inset) See Name List. A47O00 46-30A 43300 APRIL 1980