Doing Development Differently (DDD): A Pilot for Politically Savvy, Locally Tailored and Adaptive Delivery in Nigeria DELIVERY CASE STUDY Claudio Santibanez/World Bank THE LAGOS EKO SECONDARY EDUCATION SECTOR PROJECT: TAILORING INTERNATIONAL BEST PRACTICES TO IMPROVE EDUCATIONAL OUTCOMES AT THE STATE LEVEL MARCH 2016 The Lagos Eko Secondary Education Sector Project: Tailoring International Best Practices to Improve Educational Outcomes at the State Level Cover: Claudio Santibanez. Students at an Eko Secondary School. This case study was written by Sabrina Roshan, Roland Lomme, Halimatou Hima and Claudio Santibanez as part of the Delivery Case Studies series produced by the World Bank’s Nigeria Country Team under the guidance of Olatunde Adetoyese Adekola and Katherine Bain. The Delivery Case Studies series—part of the Doing Development Differently initiative—aims to generate knowledge on what works in Nigeria and why. These cases are among a number of instruments being piloted to help the World Bank continually improve its effectiveness as a partner to Nigeria. The Doing Development Differently pilot is task managed by Katherine Bain in the Governance Global Practice. The authors are grateful to Indira Konjhodzic for comments and support on earlier drafts. They also acknowledge the invaluable feedback provided by national stakeholders, including the former Project Coordinator and Special Assistant to the Governor, Ronke Azeez as well as editorial support from Amanda Green. The paper does not represent the views of the World Bank’s Board of Directors, and any errors are those of the authors alone. Contents   Contents Executive Summary v Tracing the Process v Lessons Learned vi Introduction 1 The Development Challenge: Nigeria’s High Demand for Education, but Low Quality 2 The Structure of Nigeria’s Education Sector 2 Challenges of Lagos State’s Education Sector 3 The Education Sector Politics in Lagos State 4 Launching the Lagos Eko Secondary Education Project to Improve the Quality of Education 5 Positive Project Improve, 2009–13, then Decline in 2014 5 Results Setback in 2014 6 Test Scores Bounce Back in 2015 6 The Delivery Challenge: Tailoring International Best Practice to Leverage Impact in the Nigerian Context 9 Targeting Interventions in Public Secondary Education to Maximize Impact in Lagos State 9 Why Target Secondary Education in Lagos State? 9 Why Focus on Education in Lagos State? 9 Identifying Best Practice Principles in Education Reform and Tailoring Them to the Local Context in Lagos State 9 Nurturing a Multilevel Culture of Performance in Lagos State 11 Tracing the Process of the Eko Project: From Design to Implementation and Beyond 12 Challenges in Designing the Eko Project 12 Key Actors Trigger and Enable Change in the Education Sector 12 Alignment of Political and Institutional Leadership in Lagos State 12 Leaders with the Sector Experience Are Needed to Unpack Systemic, Root Problems 13 A Small but High-Powered PMU That Combined Internationally Competitive Technical Skills with Local System Knowledge and Effective Leveraging of State Institutions 13 A World Bank TTL Who Fostered a Partnership Based on Trust, Continuity, and Hands-on Supervision 13 Designing a Participatory Design Process Led by the State 14 Government-Driven Consultations Around a Clear Vision Leading to Strong Ownership of the Project’s Design 14 Role of State-Level Constituencies: Teachers, Principals, and Unions 14 Role of Nongovernmental Constituencies in Education Reform: Parents and Community Members 14 Implications of the Participatory Design Process in the Eko Project 15 iii Eko Project Implementation Arrangements Designed to Span Government Levels with Clear Roles and Responsibilities 15 Design of Eko Project Systems for Data Collection and Analysis and Their Impact on Perceived Education Performance 15 Eko Project Implementation Challenges 17 Even in a Challenging Data Environment, a Data-Driven Approach Facilitates the Effective Use of Performance Incentives 17 Using Data Enables Accountability through Multistakeholder Monitoring of Program Performance 18 Adaptive Implementation Allows Flexibility to Seize Windows of Opportunity and Adequately React to Setbacks 18 Performance Data Can Help Identify Gaps and Provide Evidence in Support of Exit Strategies 19 The Status and Sustainability of Project Interventions at the Eko Project Closing 20 Sustainability Is Enhanced by Proving the Value Added of a High-Performing System 20 Lessons Learned 22 Eko Project’s Highly Consultative and Participative Design Process Leads to Achievements 22 Smooth Project Implementation Thanks to Participatory Design and International Best Practices 22 Building Project Sustainability from the Start 23 The Science of Delivery 23 Bibliography 25 Annexes Annex 1: Project Summary 26 Annex 2: Roles and Responsibilities for Project Implementation 27 Annex 3: Eko Project Implementation Arrangements 33 Annex 4: Basic Education Certificate Examination Results for Lagos State 38 LIST OF FIGURES Figure 1: The Dynamics of Change in the Eko Project 6 Figure 2: Implementation Arrangements under the Eko Project 16 LIST OF Tables Table 1:  Primary and Junior Secondary Gross Enrollment Rates by Household Consumption Quintile, Lagos, 2006 3 Table 2: WASSCE Scores at Public Secondary Schools, Lagos State 7 Table 3: Lagos Eko Secondary Education Project, Intermediate Results Indicators 7 Table A2.1: Component Management and Institutional Responsibilities 29 Table A2.2: Roles and Responsibilities for Implementation—State Level 31 Table A2.3: Roles and Responsibilities for Implementation of Additional Financing— Local Levels 31 Table A4.1: Basic Education Certificate Examination Scores, Lagos State 38 iv Executive Summary   Executive Summary This case study seeks to understand how the Lagos Eko Tracing the Process Secondary Education Project (Eko Project) in Nigeria tailored international best practices to leverage impact During the Eko Project’s design phase, a small set of through education sector reforms in Lagos State’s public state actors served as essential change agents, leading a secondary school system. As the economic center participatory and thorough design process that resulted of Nigeria and a financial powerhouse in West Africa, in clear and effectual implementation arrangements to Lagos State has benefited from significant education facilitate project execution. Among the most effective sector reforms initiated by reform-minded state officials. enabling actors were the governor and his deputy, who Demand for education has always been high in the state, also served as commissioner of education; and, on the but for many years the quality of secondary schools World Bank side, a task team leader (TTL) who fostered lagged behind. Infrastructure deficiencies, shortages in a partnership with the state based on trust, continuity, learning materials, and scarce opportunities for teachers’ and close supervision. The yearlong design process made professional development compounded these problems. a conscious investment in local ownership, seeking and incorporating feedback from stakeholders to build on In this context, a new governor took office in 2007, the clear vision presented by the governor and his team. on a platform that placed education sector reforms at During implementation, the project mobilized change the top of the agenda. The governor recruited a top- agents along the entire delivery chain. The project notch program coordinator and sought the World Bank’s deepened ties with the community over time through support to design and launch the Lagos Eko Secondary targeted and continuous engagement with a wide range Education Project in 2009. Drawing on international of stakeholders. The project team was also able to forge a best practices, the project set out to support improved close and cooperative relationship with the teachers’ and learning outcomes through school development grants, principals’ unions by sequencing more sensitive project performance-based incentives for schools, public-private components. partnerships for technical colleges, teacher training and mentoring, and more reliable performance measurement In implementation, the availability of data—and through improved standardized testing of learning training in its analysis—allowed the Eko Project achievement. The Eko Project, which is set to close in team to monitor overall project outcomes and hold June 2016, has worked steadfastly toward achieving its stakeholders accountable, thereby improving the development objective of improving the quality of public performance of all actors involved in school operations. junior and senior secondary education in Lagos State, School performance information was posted publicly so despite a significant drop in test scores in 2014—a setback all stakeholders were aware of how their schools fare in that accentuated a national-level trend. comparison to others and can argue for further change. This data-driven approach helped link incentives to actual This case study explores how the Eko Project tailored performance, motivating key stakeholders to perform international best practices to leverage impact in Lagos better and strengthening the project’s credibility. The State’s public secondary education system and assesses use of data allowed for adaptive implementation, how the project resolutely responded to the challenges helping document performance gaps and guiding course posed by the drop in test scores. Using a qualitative corrections, including in response to the 2014 drop in methodology based on semi-structured interviews test scores. Nonetheless, the project’s overall success and focus group discussions, the case study concludes was somewhat undermined by a weak monitoring and that a proactive approach in a moment of committed evaluation (M&E) framework, which made it difficult to political leadership from top to bottom—together consistently rate project performance and highlights with targeted program design, thoughtful adaptation of the need for project teams to pay more attention to international experience, and efforts to foster a culture establishing baselines, defining meaningful indicators, and of performance—created the conditions for meaningful identifying weaknesses in official data systems early on in and sustainable reform, despite the challenges posed by project preparation. The Project Management Unit (PMU) demographic pressures and funding constraints. reported a “cultural change” in “embedding” data-based performance monitoring among those who are involved in implementing secondary education in Lagos State. v Lessons Learned • Hire the right mix of skills in the PMU so that it has both political clout and managerial leverage. Many of the Eko Project’s achievements are a direct result • Empower a PMU to be flexible and aim at internalizing of its participatory design process, which adapted best innovation. practices to the Nigerian context. In the case of the Eko • Ensure that the PMU interacts regularly with all Project, political, managerial, and professional leadership; stakeholders and helps build collective buy-in. an internalized delivery chain leveraged by a small but • For the TTL to be present and make sure the task team effective PMU; and close but flexible supervision by the is available to help address implementation challenges. World Bank encouraged improvements and amendments • Use data and close supervision to identify problems as implementation progressed and challenges arose. Key and make course corrections in real time. design lessons include the need to: Key sustainability lessons include: • Take advantage of political buy-in to ensure that leadership percolates down to the frontline delivery • Identify the sustainers early on. level. • Use capacity building as a performance incentive. • Ensure multilevel stakeholder involvement. • Sequence interventions carefully to ensure that • Carefully articulate the respective roles of ad hoc reforms are not derailed early on. implementing agencies (PMU) and established • Gain exposure to other contexts where interventions institutions (line ministry) to foster close interaction in the sector have worked and apply them locally, in and ensure sustainability and ownership. a strategic and nuanced manner. • Establish a robust M&E framework from the outset that takes into account constraints in the availability Finally, an emerging framework on the science of of relevant data. delivery identifies key lessons from the Project such as: the importance of aligning political, managerial, and Eko Project implementation has been relatively smooth. professional agendas; the need to engage leaders with With a highly skilled, albeit small, PMU at its center and significant political power to enhance the efficacy of strong collective leadership down the delivery chain in delivery; the role of training as both an incentive and the state bureaucracy, innovative interventions have a means to develop skills for project implementation; been introduced and followed through, resulting in and the significance of innovative and responsive quick positive outcomes. Key implementation lessons include thinking in establishing nontraditional solutions and the need to: relationships. vi Introduction   Introduction This case study seeks to understand how the Lagos which seeks to support the World Bank’s development Eko Secondary Education Project (Eko Project) tailored effectiveness by putting institutions and governance international best practices to leverage impact through at the center of what it does, tailoring interventions education sector reforms in Lagos State’s public to the local context and building in adaptive learning secondary school system. These best practices include techniques to projects. The Lagos Eko Project was an intensive utilization of evidence-based policy making, selected as a case study in a series of case studies the granting of autonomy to a variety of stakeholders that aim to help the World Bank better learn from together with the expectation of accountability for practice. This case study is also part of the Science results, community participation, incentives for reform, of Delivery program that is contributing to the Global support for public-private partnerships (PPPs), and Delivery Initiative’s Library of Delivery Case Studies. The adaptive implementation. Global Delivery Initiative is a collaboration across the international development community to forge a new Research methods focus on the Science of Delivery1 case frontier in development efforts worldwide. study guidelines, drawing on semi-structured interviews and focus group discussions with key stakeholders to Understanding how this project effectively tailored better understand the implementation process and international best practices, while drawing on analyze the causal mechanisms for results achieved. participatory, innovative, and adaptive interventions, First, the case study reviews contextual factors and the offers practical lessons for other states in Nigeria and political setting in place when the project was launched, other countries facing similar challenges. The case study as well as project results. Second, the case study explores highlights four key contributors to the Eko Project’s the ways in which the Eko Project tailored international politically savvy, locally tailored and adaptive delivery best practices to leverage impact in the Nigerian context. approach: (i) it managed to build on collective leadership Next, the case study traces the design process, which along the entire delivery chain, from the political involved key enabling actors and a participatory approach, leadership of the governor down to the professional followed by an adaptive implementation process that leadership of classroom teachers; (ii) it bounced back used data to enforce accountability and build incentives, from a setback in outcomes by strengthening data with a view to the sustainability of project interventions. collection and analysis of results; (iii) it struck the right Finally, a few key lessons that emerge from the Eko balance between external and internal implementation Project’s experience are discussed. mechanisms to make the most of established institutions; and (iv) it aimed at macro-level impact by reaching down The case study is part of a Doing Development to individual learning achievements. Differently Initiative within the Nigeria Country Team, 1 It is important to note that even well-intentioned efforts at evidence-based policy making struggle in the Nigerian context, where data availability is limited and the political will for rigorous data collection and analysis can be low. 1 The Development Challenge: The Development Challenge: Nigeria’s High Demand for Education, but Low Quality Nigeria’s High Demand for Education, but Low Quality The Structure of Nigeria’s under the purview of SUBEBs) and senior secondary Education Sector education. Nigerian federalism also applies differently to basic and senior secondary education, with fiscal Nigeria’s education system comprises nine years of transfers from the federation available only for basic basic education, including six years of primary school education. The National Council of Education, an and three years of junior secondary school, as well interstate coordination body, plays an important role as three years of senior secondary school and four in horizontal policy alignment; for example, it decides years of tertiary education. The Constitution of the on the national curriculum and has been instrumental in Federal Republic of Nigeria distributes responsibility for the institutionalization of School-Based Management education across the three tiers of government: secondary Committees (SBMCs) across Nigeria’s states. education falls within the concurrent jurisdiction of the federal and state governments, and primary, adult, and On the whole, Nigeria allocates few resources to vocational education are the joint responsibility of state education. Total public expenditure on education and local governments. The constitution also ensures that amounts to less than 2 percent of gross domestic part of the government revenue collected at the federal product. The fiscal burden of education rests largely level is transferred to state and local governments. In with state and local governments. Close to half of public effect, state governments are responsible for primary expenditure on education is allocated to basic education. and secondary education while the federal government plays a dominant role in providing tertiary education. Lagos State—located in the southwest of Nigeria and Local-level agencies called Local Government Education home to the country’s largest and most economically Authorities (LGEAs) are in charge of basic education, but dominant city, Lagos—offers tuition-free public they report to the state government rather than to local education at the basic and secondary levels. Sitting for governments. The private sector is also active at all levels the West Africa Senior Secondary Certificate Examination of the education system, especially at the primary level (WASSCE) is also free of charge. As of 2007/08, when the (Härmä 2013). Its role has expanded quickly, including in Lagos Eko Secondary Education Project was prepared, Lagos State, where more than one-third of students are 11,387 schools (public and private) enrolled an estimated enrolled in private schools (Lagos State Government 2.3 million students. Of these students, approximately 2013). 1 million (40 percent) were enrolled in public schools. In the tertiary sector, Lagos State had five public technical The enactment of the Universal Basic Education Act colleges, five universities (two public and three private), in 2004 at the federal level, and in 2005 by the Lagos two polytechnics, one monotechnic, and four Colleges of State Government, bifurcated secondary education and Education (one federal and three private). Public education schools into junior secondary (which falls into basic is as likely to be underfunded in Lagos as anywhere else education along with pre-primary and primary) and in Nigeria, as it accounts for approximately 16 percent senior secondary. Ad hoc executive agencies, such as the of total appropriations at the state level (that is, less Universal Basic Education Commission at the federal level than US$500 million). However, the state government is and State Universal Basic Education Boards (SUBEBs) at committed to safeguarding social expenditures, even in the state level, have been established to implement the the context of fiscal pressures. universal basic education legislation and take over the management of earmarked budget resources. Demand for education is high in Nigeria and in Lagos State in particular. Rapid population growth, at The legal, policy, and institutional framework differs for 2.8  percent per year nationwide and 3.2 percent per junior secondary (which, as part of basic education, falls 2 year in Lagos State, has placed constant pressure on the of education was evident in low student performance education system to make room for more students. In on examinations and inadequate skills development general, Lagos State enjoys a high student attendance among graduates seeking employment. Infrastructure   levels, with primary gross attendance at 103 percent deficiencies, shortages in learning materials, and scarce and net attendance at 69 percent. The secondary gross opportunities for teachers’ professional development attendance rate is almost 117 percent and net attendance compounded these problems. is 69 percent (NPC DHS 2014).2 Below is a brief review of the key challenges in Lagos The high demand for education has spawned a dramatic State’s education sector: growth in the number of private schools in Lagos State over the past two decades. This increase is especially Poor people had limited access to secondary seen at the primary level, where most poor children are education. At the time of project inception, primary enrolled in private schools, at a high cost to their parents, gross enrollment rates were quite high, reaching nearly due to the lack of public schools in poor neighborhoods 100 percent for children aged 6–14. Gross enrollment and slums (Härmä 2013). As a result, the education sector dropped to 65 percent by senior secondary school and has split along income lines. Inequalities persist at the reached only 40 percent among the lowest income secondary level as well, although with the opposite quintiles (Table 1; NBS 2005 and 2006). Even with an result with regard to private (more socially selective) overwhelming demand for public secondary education versus public schools. Private schools in Lagos can often in Lagos, as evidenced by a high transition rate of offer a more conducive learning environment through 48  percent from primary to junior secondary school smaller classes and lower student-teacher ratios,3 better (NPC 2008), dropout rates before and after completion infrastructure,4 wider distribution of learning materials, of legally compulsory basic education were concerning. and motivated and accountable teachers, as well as In the first year of junior secondary school, 9 percent of convenience factors such as additional childcare for those enrolled left school prior to completion. Similarly, working parents. although 96 percent of junior secondary school graduates continued to senior secondary school, the dropout rate was over 16 percent during the first year and 25 percent Challenges of Lagos State’s in the last year of senior secondary school. Education Sector This phenomenon reflected inadequacies in the At the time of the Eko Project’s design, the education learning environment (including poor infrastructure, sector in Lagos State faced several critical challenges. overcrowding, and scarce learning materials), together Despite high adult literacy rates in Lagos, at 93 percent with the insufficient availability of schools (public and in 2005/06 (Lagos State Government 2009), poor quality private) to meet growing demand. The indirect costs Table 1:  Primary and Junior Secondary Gross Enrollment Rates by Household Consumption Quintile, Lagos, 2006 Primary by Income Quintile Junior Secondary by Income Quintile Gender 1st 2nd 3rd 4th 5th 1st 2nd 3rd 4th 5th Female 13 104 130 109 104 45 71 85 104 117 Male 10 67 100 101 113 57 82 78 94 89 Source: NBS 2005 and 2006. Note: 1 = lowest income quintile and 5 = highest income quintile. 2 According to the Nigeria Demographic and Health Survey (DHS), the net attendance ratio (NAR) indicates participation in schooling among those of official school age (6–11 for primary and 12–17 for secondary). The gross attendance ratio (GAR) indicates school attendance among youth of any age, from 5 to 24, and is expressed as a percentage of the school-age population for that schooling level, although technically the GAR is not a percentage. The GAR is nearly always higher than the NAR for the same level, because the GAR includes participation by youth who are older or younger than the official age range for that level. 3 The number of students per classroom in public junior secondary schools deteriorated from 85 in 2011/12 to 101 in 2012/13 (Lagos State Government 2013, iii). 4 Only two-thirds of public junior secondary school classrooms are usable, and fewer than two-thirds of public junior secondary schools have access to safe water (Lagos State Government 2013, iv). 3 of education were also a factor; despite an official Service providers were disenfranchised. Evidence policy of free education, parents were often required pointed to inefficiencies in the delivery of education to buy textbooks and teaching materials. Transportation services—in teacher deployment, for example, through difficulties brought about by urban expansion, including a lack of qualified teachers in the areas most in need of traffic congestion and growing distances to school, them. With a top-down management structure, limited exacerbated problems in accessing education. Finally, the funds, and insufficient accountability mechanisms in high opportunity costs at the secondary level, combined place, underpowered education service providers on the with the low employment rate of secondary school ground struggled to address quality constraints. graduates, acted as an additional deterrent. State capacity for policy making, management, and Gender parity in secondary school attendance was M&E was lacking. With limited state-level political only slightly higher than in Nigeria as a whole. Lagos commitment came limited capacity for managing the State ranked 13th among Nigerian states, with a gender education sector. From a policy perspective, there was parity index of 0.98 as compared to 0.93 at the national an inadequate understanding of how to target education level (NPC and RTI International 2011).5 quality improvements. Traditionally, the state turned to constructing school buildings rather than strengthening The quality and relevance of education were the capacity of teachers and administrators. A history of inadequate for skills development. The quality of problematic infrastructure development, brimming with primary and secondary education deteriorated markedly accusations of overspending and potential corruption, in the decade prior to the Eko Project. For example, also tainted the state’s education improvement despite a few high-performing schools, only 18 percent interventions. Management constraints ranged from of candidates taking the WASSCE in Lagos State in 2009 inefficient budget spending to scarce use of standardized obtained five credit passes and above (a score considered testing data to improve performance and hold relevant academically adequate for entry into university) in at stakeholders accountable. least five subjects, including in English and mathematics. Skills acquired in school were also inadequate. As many as 60 percent of senior secondary school graduates were The Education Sector Politics unemployable in 2007 (FMoE 2007), primarily because in Lagos State they did not have the skills needed to adapt to Nigeria’s dynamic business environment. It is difficult to replicate the combination of political factors and sequencing in the political economy of School curriculum was not well aligned with labor education sector reform in Lagos State when the Eko market needs. Low test scores reflected the lack of Project was designed, but it is critical to understanding foundational skills, including basic literacy and numeracy, how the project identified and seized opportunities which are of critical importance in supporting the to achieve meaningful change and, ultimately, to needs of the (formal and informal) labor market in deliver results. The politics behind how interventions Lagos. Technical colleges were in a shambles, and most are introduced and managed matters in reforming secondary schools included little practical or technical schools and systems, and the success of policy reforms content in the curriculum. At the same time, the poor depends a great deal on the capacity of their promoters reputation of Nigeria’s technical colleges as “schools of to gather support within a specific political economy last resort” for dropouts or less capable learners had context. How and when decisions are made along the driven down enrollment in these schools and further reform continuum and who is involved in decision making eroded their quality. make a difference to the outcome of reforms that entail potential benefits—or losses—for various stakeholders. Public spending on education was inadequate, of low quality, and subject to limited accountability. Data The political economy literature stresses that successful available at the time of Eko Project preparation suggested education reforms necessitate longstanding support that education funding had been declining in real terms in from a broad range of stakeholders, given their effect Lagos State. The bulk of budget resources were allocated on power structures and the requirement that they be to teacher salaries, leaving few funds for learning materials, taken up by service providers down the delivery chain. maintenance, and other critical expenditures. Weak accountability of capital expenditures was reflected in “Reforms like those in education required long the quality of school facilities. chains of implementation activities and decisions. 5 The gender parity index measures the ratio of girls to boys, with 1 indicating complete parity. A ratio of less than 9 indicates a smaller number of girls than boys. 4 Ultimately, education changes had to be adopted The teacher training component focused on at the classroom level if they were to improve identifying training and capacity development needs the extent to which children learned critical skills among secondary school teachers and developing   and abilities; this meant that multiple layers of or linking them with interventions to address those implementers needed to be on board for new needs. Standardized testing supported the program’s initiatives to succeed . . . These changes meant that performance incentives by establishing a consistent, governors, mayors, bureaucrats, teachers, students, comparable method of measuring learning achievements. parents, and communities needed to adopt new The Eko Project also supported project coordination and ways of thinking and behaving and learn to be more management, as well as efforts to strengthen the federal accountable in multiple ways” (Grindle 2004). post-basic education strategy. According to the project coordinator, at the time of Additional financing of US$42.3 million was approved the project’s preparation and implementation, there in March 2014 to: (i) add two additional rounds of was a demand for change from the public—and this grants to schools under the School Development demand was complemented by a desire for reform Grant subcomponent, based on the same criteria and among state leaders. Lagos State has a history of reform- procedures as in the original project; (ii) continue teacher minded leadership over the past 17 years, embarking on professional development; (iii) continue standardized substantive economic and governance reforms beginning testing of student learning achievement for 2014 and 2015; in 1999. Building on the groundwork laid by the previous and (iv) continue collaboration with the private sector on administration, a new government was elected in April technical education. 2007 on a platform that placed education reforms at the top of the agenda. The Eko Project’s objectives Positive Project Improvements from and activities aligned closely with the incoming state 2009–2013, then Decline in 2014 governor’s agenda and benefited from his political capital and personal interest and involvement. Implementation was relatively seamless, with the exception of a delay in the disbursement of additional financing until the end of 2014. According to the Launching the Lagos Eko Secondary project’s Implementation Status and Results Reports (ISRs) and other project implementation documents Education Project to Improve the such as Financial Management Supervision Reports and Quality of Education Procurement Post Reviews, the few minor procurement With the goal of improving the quality of public and financial management issues that have arisen have junior and senior secondary education, the Lagos been remedied or are being rectified. Results show State Ministry of Education launched the Lagos Eko significant improvement over baseline indicators until Secondary Education Project in 2009, supported by 2013 (Tables 2–3). a World Bank credit in the amount of US$95 million. Drawing on international best practices in the education In 2014, however, Lagos State’s WASSCE test scores sector and in-service delivery more broadly, the project dropped considerably. The state’s WASSCE performance set out to support improved learning outcomes for ranking dropped from first to sixth of 36 states and the more than half a million public school students and Federal Capital Territory. The share of students obtaining 7,000 teachers and school administrators in 637 schools five credit passes, including in English and mathematics, through school grants, performance-based incentives, dropped from 46 percent in 2013 to 28 percent in 2014. In teacher training, and standardized testing. response, the governor asked senior education executives to urgently develop a strategy to address the decline in School improvement grants were made available to scores. The project coordinator noted that “nobody was all public junior and senior secondary schools in Lagos talking about test scores before [the] Eko Project. For State as an additional funding source to improve the the first time, we started to feel embarrassed” by poor quality of education services on the basis of an approved results. school improvement plan. Additional performance grants were awarded to the best-performing 40 percent As a result of the setback in test scores, together of schools based on criteria such as test scores and with insufficient progress toward some of the teacher attendance. All five technical and vocational project’s intermediate results indicators (Table 3), the colleges in the state were included in the project and World Bank team has downgraded the Eko Project’s received grant funding to strengthen partnerships with project development objective rating to “moderately the private sector, so as to improve the linkages between satisfactory.” skills training and labor market demand. 5 Figure 1:  The Dynamics of Change in the Eko Project • Key actors trigger and enable change • Participatory design process led by the state • Implementation arrangements designed to span levels of government, with clear roles and responsibilities Design • Design of systems for data collection and analysis and their impact on perceived performance • Even in a challenging data environment, a data-driven approach facilitates the effective use of performance incentives • Use of data enables accountability through multistakeholder monitoring of program performance Implementation • Adaptive implementation allows flexibility to seize windows of opportunity and adequately react to setbacks • Performance data can help identify gaps and provide evidence in support to exit strategies • Sustainability is enhanced by proving the value added of a high-performing system, including: • Giving autonomy to and encouraging performance in schools through school grants • Close monitoring of outcomes through student assessments • Deconcentrated generation of data and the role of the zonal project administrators (ZoPAs) in monitoring and management Closing • Building collective leadership along the delivery chain down to service providers • Investing in mentoring and training teachers and school principals while holding them accountable for performance Results Setback in 2014 better mechanisms for data collection and analysis and to strengthen M&E capacity at the district and school Although not yet fully understood, the slippage in levels. The project team forged a partnership with test scores may have resulted from the funding gap Google for Education to create resource centers that between closure of the original project and the delayed could help strengthen the data management capacity effectiveness of additional financing. Owing to this gap, of administrative staff; to mobilize online resources to most activities were frozen for 10 months—including train teachers; and to help localize content (including by school improvement grants, performance awards, ensuring that good teaching results were recorded). The teacher training, provision of essential materials, and the team also developed strategies with regard to building activities of volunteer teachers. In addition, there was a classrooms and deploying teachers, and ZoPAs were reduction in afternoon lessons, Saturday coaching, and instructed to monitor after-school lessons. monitoring and supervision activities by ZoPAs. Other potential reasons for the setback include the curriculum The drop in test scores sparked a policy debate in reform being undertaken at the time and the shortage Lagos State. In response to a proposal that students be of qualified and experienced teachers following a spate selected to participate in the WASSCE, the governor of retirements (500 of 3,000 teachers retired in District refused, arguing that passing the exam was a right not IV alone during this period), leading to a higher student- to be denied to any student. teacher ratio (World Bank 2015a). In addition, the setback in results took place in the context of a system-wide decline in 2014 WASSCE test scores across Nigeria, Test Scores Bounce Back in 2015 although those in Lagos State declined by a greater Test scores for 2015 reveal significant improvement degree. over the previous year, even meeting or surpassing the performance recorded in 2013 (Table 2). These Although the decline in results was disappointing, improvements are demonstrative of the systematic, the project coordinator felt that this “dip” in project institutionalized cultural change embedded in the Eko implementation was “really what brought the project Project’s delivery chain, which affects all public secondary together,” as it encouraged the project team to develop schools in the state. 6 Table 2:  WASSCE Scores (%) at Public Secondary Schools, Lagos State 2009 2013 2014 2015 Subject   Target Actual Target Actual Target Actual English 71 50 81 82 85 84 92 Mathematics 57 49 83 82 84 83 87 Biology 52 37 84 84 84 85 79 Subject average 60 45 83 83 84 84 86 Five credits, including 18 41 46 43 28 45 46 English and mathematics Source: World Bank 2015a. Table 3:  Lagos Eko Secondary Education Project, Intermediate Results Indicators Baseline and Progress to Date End Target Baseline Datea 11/4/2015 12/31/2015 School principals and vice principals trained on 0 4,091 4,609 performance improvement and grant operations (9/1/2008) School principals and vice principals trained on 0 4,255 4,609 performance improvement and grant operations (12/31/2009) (cumulative) Teachers trained in core subjects (cumulative) 0 9,869 9,576 (12/31/2009) District staff trained in project monitoring, 0 1,994 2,143 administration, and financial management (12/31/2009) (cumulative) Functional Education Management Information 0 25 12 Management (EMIS) centers developed in the (12/31/2009) Education Districts Districts reporting against agreed data using the EMIS 0 0 6 to produce reports (12/31/2009) (9/30/2015) Teachers trained in leadership and management 0 12,088 12,606 (cumulative) (12/31/2009) Technical college staff trained in project monitoring, 10 13 40 administration, and financial management (12/31/2013) (cumulative) Source: World Bank task team implementation documents. Note: The project was set to close on December 31, 2015. The government requested a six-month extension in additional financing to June 30, 2016. The rationale, as captured in the September 2015 Aide Memoire, was the delay in project effectiveness and implementation as a result of school closures due to the Ebola Virus Disease, the recent change in government, the retention of a new project coordinator, and the need to disburse the remaining/committed funds for the following activities: disaggregated analysis of 2015 test results to justify the increase in performance compared to 2014, monitoring and evaluation of school grants, and training of more district staff, school principals, and vice principals. a. Baseline dates vary due to the lack of available data during project design. 7 The state government could not substitute for the While there is no evidence of a single, definitive factor delayed availability of funds, but it did react promptly behind the 2015 increase in scores, the Eko project has to the setback in WASSCE test scores by strengthening clearly institutionalized a data-driven culture in Lagos delivery mechanisms, including information State’s public secondary education system. Regardless management to diagnose the underlying factors. of the ebb and flow of student scores, the fact that The project team is still investigating the causes. There stakeholders at both the highest and lowest levels of could be cases where, despite the governor’s objection, the delivery chain turned immediately to the data to upstream selection of WASSCE participants by principals determine what went wrong and how to make things wary of low scores may have led to a biased increase right again, demonstrates that the Eko Project has in overall test scores. This would be attributed to a changed the game in the state’s secondary education reduction in the number of candidates—which would be sector. Commissioners, district officials, principals, likely to exclude low performers—sitting for the exam. teachers, parents, and community members all looked to strengthen data collection, increase the specificity of According to the project team, other potential causes information gathered, and expedite responses to changes for the 2015 improvement in scores include:(i) increased in results with a view to improving student performance. monitoring and supervision by ZoPAs (from three times In the long run, this could help alter the broader approach per month to four) to address the retirement of teachers; to data in Nigeria, where political will and capacity for (ii) the conversion of a number of qualified volunteer data collection and analysis are limited. teachers to permanent and regular staff positions; (iii) the reinstatement of afternoon lessons and Saturday This case is illustrative of a shift in demand for the coaching; (iv) the intensification and increased frequency state to collect and provide constituents with reliable of early morning lessons; (v) efforts to urge stakeholders, data. Stakeholders at all levels in Lagos State turned a including SBMCs, to focus on teachers’ and pupils’ negative into a positive: because of the lack of specificity attendance and punctuality; and (vi) the inclusion of in information on why the scores dropped, schools have more teachers in training and retraining activities to boost ramped up strategic data collection and analysis. morale and motivate them to boost their productivity (World Bank 2015a). 8 The Delivery Challenge:   The Delivery Challenge: Tailoring International Best Practice to Leverage Impact in the Nigerian Context Tailoring International Best Practice to Leverage Impact in the Nigerian Context Targeting Interventions in Public junior secondary) as well as resources through the Second Secondary Education to Maximize Primary Education Project, which provided federal and state counterpart funding to targeted primary schools Impact in Lagos State across Nigeria. During this period, donor agencies— In an effort to support the government’s priority of including the World Bank—were focusing their education targeting needier students and schools, while also sector investments at the primary level, providing little seeking to maximize program effectiveness, the attention to secondary education. Eko Project focused narrowly on public secondary education in Lagos State. These design decisions Why Focus on Education in Lagos State? strategically concentrated program investments on issues identified as priorities by the state government, where In a country with significant disparity in socioeconomic there was reform traction and local ownership. The move development across 36 states and the Federal Capital toward more purposeful targeting was based in part on Territory, it may seem unexpected for the Eko Project lessons learned in the implementation of past projects in to have focused on relatively well-off Lagos rather than Nigeria, including the Second Primary Education Project, on other states with more severe challenges in ensuring implemented at the federal level.6 educational access and quality, especially in the north of Nigeria. According to the Project Appraisal Document (World Bank 2009), Lagos State was also selected on Why Target Secondary Education the basis of: (i) its significant role in Nigeria’s economic in Lagos State? growth; (ii) the poor quality of public education, especially High primary school attendance, varied access to in urban slum areas (which shelter over two-thirds of funding, and the significant level of private schooling Lagos’s population); (iii) the high quality of its Education guided the decision to concentrate on addressing Sector Plan; and (iv) its demonstrated commitment to and quality deficits at the secondary level. As noted ownership in the development of the education sector. above, despite automatic progression of students from primary to junior secondary school, completion rates in secondary schools were considerably lower. As reported Identifying Best Practice Principles by the project’s task team leader (TTL), failure rates in Education Reform and Tailoring were so high that citizens began questioning why they Them to the Local Context in Lagos were sending their children to public school at all. This concern was exacerbated by the low employment rate State of secondary school graduates. Moreover, at the time Before launching public consultations on program of project preparation, primary schools had an edge in design, the governor, the PMU, and the World Bank’s funding, as they received earmarked funding from the project team looked to international best practices to government for universal basic education (primary and understand what had worked to improve secondary 6 The Second Primary Education Project was approved in May 2000, became effective in August 2000, and closed in December 2004. One of the main project lessons was that, at the design stage, resources needed to be targeted through a variety of quality-enhancing inputs and allocated in sufficient quantity to be able to achieve the intended impact (World Bank 2005). 9 education quality in other countries and why. The team (Vegas and Umansky 2006; Vegas 2005). Experience found that successful reforms in other countries had showed that incentives were most effective when they several factors in common, including meaningful attention reached beyond monetary approaches, especially in cases to and use of data, mechanisms for autonomy and where it would be difficult to enforce disincentives—for accountability, community participation, incentives for example, removing underperforming teachers from their reform, support for PPPs, and adaptive implementation. positions (Vegas and Umansky 2006). These intervention areas were complemented by an increase in infrastructure investment in secondary Support for PPPs. The role of PPPs in strengthening schools. The team applied international best practices education sector outcomes was of significant interest to to the local context, designing a blueprint around specific the project team, given the project’s focus on improving intervention areas and implementation approaches that the employability of secondary school graduates in played up Lagos State’s strengths and took account of Lagos State and the importance of strengthening the its limitations. links between the state’s technical colleges and the labor market for which they were to prepare their Evidence-based policy making. Prior to the Eko Project, students. International experience suggested that PPPs student performance in Lagos State was being measured could increase access to education (Patrinos, Barrera- through a system of standardized exams. What the project Osorio, and Guáqueta 2009). PPPs have also improved could contribute in this regard were clear and consistent the quality of education service delivery when coupled procedures for data collection and utilization within the with strong oversight mechanisms for quality control state’s public secondary education system to improve and other interventions to improve schools (Patrinos, data accuracy and help ensure that this information was Barrera-Osorio, and Guáqueta 2009). being used effectively to measure and monitor school performance based on criteria set either by individual Adaptive implementation. The Eko Project team’s schools, districts, states, or the federal government investment in an iterative design process resulted in (World Bank 2008). It was also important to ensure that successful adaptive implementation. The utility of this data were analyzed and presented in a manner that would approach is supported by evidence across international support informed decision making and policy responses development sectors (Glennerster and Takavarasha 2013; at each relevant level of policy implementation, including Duflo and Kremer 2005; World Bank 2015b). Using this schools, districts, and the state. approach, the Eko Project was able to adapt to windows of opportunity during implementation—for example, Autonomy and accountability. The role of greater by taking advantage of training opportunities and stakeholder autonomy in supporting mechanisms for partnerships with global information and communications accountability—upward to higher levels of government, technology (ICT) service providers. The project also downward to schools, and outward to the community— demonstrated effective adaptation to challenges and has provided an impetus for improved performance and a willingness to respond to the evolving context, as clearer feedback on what is and is not working in the evidenced by its flexible response to the drop in test sector and how it might be strengthened. Together, these scores in 2014. factors lead to higher-quality service delivery (World Bank 2008; World Bank Social Accountability Sourcebook). With these building blocks in place, the World Bank’s education specialists led a brainstorming session with Community participation. As a complement to all stakeholders to discuss how international best stronger accountability tools, meaningful community practices could be tailored to the Lagos State context to participation improves oversight of the education sector. most effectively address the governor’s twin priorities: Community participation is an important element in low learning outcomes in secondary education and the providing effective oversight, including monitoring employability of graduates. The consultations confirmed student and teacher attendance, tracking the results of the need for effective accountability mechanisms to school spending, and providing anecdotal evidence to nurture meaningful demand for change, the importance support assessments of learning outcomes. Community of incentives that moved beyond financial resources, and participation can also be mobilized to provide financial the need to link resources to results. or other material and nonmaterial support to schools (Uemura 1999; Jimenez and Sawada 1998; Heneveld and In the search for context-appropriate responses Craig 1996). to these systemic problems, the project team led multistakeholder study tours to Chile, Colombia, Reform Incentives. The design of incentives proved Mexico, and the United States—countries that had critical to the Eko Project’s successful implementation. achieved success in secondary education sector reform. The project team looked at how to build incentives into Through these study tours, the joint government–bank the project that aligned with key stakeholders’ interests team defined a set of principles for designing a program 10 to improve secondary school dropout rates and increase strengthened by devolving significant responsibilities to student employability after graduation within the Lagos ZoPAs, teachers, principals, and SBMCs, and by establishing State context. lines of accountability between the PMU, the district, and   the schools via implementation arrangements. Together with transparency mechanisms built into the project Nurturing a Multilevel Culture (including posting school and student performance of Performance in Lagos State scores on school notice boards), these implementation arrangements ensured that any achievement failures The Eko Project’s performance orientation operated could be addressed. Benchmarking student learning at four levels of implementation. First, at the state achievements at the school and classroom levels level, the governor made clear his high expectations incentivized good performance among teachers and for performance and was personally keen to seek school administrators. The alignment between political, improvements. Second, PMU level performance was managerial, and professional performance and incentives spurred by appointing a project coordinator with whom to perform was instrumental to the results achieved the governor enjoyed a close advisory relationship. under the Eko Project. Third, district- and school-level accountability were 11 Tracing the Process of the Tracing the Process of the Eko Project: From Design to Implementation and Beyond Eko Project: From Design to Implementation and Beyond During the Eko Project design stage, a small set of The Eko Project was designed to help the government actors served as change agents, leading a participatory translate this policy framework into tangible design process that involved decision makers across improvements in secondary education in Lagos State. various levels of government and resulted in clear The central elements of the program’s design are and effectual implementation arrangements. Once highlighted below. implementation began, incentives were used in an effective way, linked to data-driven monitoring of Key Actors Trigger and Enable Change school performance while remaining aligned with the in the Education Sector preferences of key stakeholders. With a strong PMU in place and a flexible TTL, the project shifted its approach Alignment of Political and Institutional toward responding to needs on the ground, maintaining Leadership in Lagos State relevance, and avoiding potential pitfalls. Finally, the Given his experience as deputy governor, the incoming PMU engaged government counterparts early on so as Lagos State governor benefited from a smooth to train and develop the skills of state and district officials transition into office and was able to dive quickly into who would eventually be responsible for sustaining Eko the substance of his reform agenda. He understood reforms after project closure. Over the Eko Project’s life the education sector issues and had insider knowledge span, the dynamics of change worked to leverage the on how to implement political reforms. The governor lessons learned in other secondary education reform was well acquainted with budget processes and the experiments into results on the ground in Nigeria. (See state bureaucracy. He was ready to start delivering Figure 1 for a mapping of the dynamics of change in the on his agenda early on and was determined to do so project, each of which is explored below). differently by focusing on specific sectors with a targeted approach. He used unconventional methods to improve outcomes, including setting up a delivery unit inside the Challenges in Designing State Ministry of Education, bringing in an outsider to the Eko Project manage this project, and using evidence to inform policy When the Eko Project was designed, the Government making and incentives, among other tools. of Lagos State was aware of the significant challenges it faced in improving its provision of education and Combined with the political clout that came with had put in place a high-quality strategic plan for the holding the highest political seat in the state, this sector. Developed in 2007, the Lagos State Economic background provided the governor with the bandwidth and Empowerment Development Strategy recognized he needed to support the Eko Project implementation. that enhancing citizens’ access to education could help He established a reporting structure through which the reduce extreme poverty and empower the population. project coordinator updated him on project progress The resulting Education Sector Plan, developed by the every two weeks and convened a monthly meeting of state government with support from the World Bank senior education executives to discuss sector issues. and the United Kingdom’s Department for International Development (DFID) and presented in 2008, aimed Because of his focus on education sector reform, the to provide accessible, equitable, quantitative, and governor appointed his deputy to serve simultaneously qualitative education for all, fostering self-reliance and as commissioner of education. The commissioner’s socioeconomic development in Lagos. background and expertise were in the education sector, as a former teacher and education administrator. While 12 this arrangement no longer holds, at the time it was The rest of the PMU’s small team was hired on the basis emulated by other state governments. of their experience with World Bank projects, and it was made clear that performance deficits could lead to   Leaders with Sector Experience Are Needed dismissal. The project coordinator held staff accountable to Unpack Systemic, Root Problems and delegated authority, thereby empowering an effective, adaptive, flexible, and independent team. Owing to their strong sector knowledge and focus, As an advisor to the governor, the project coordinator the state’s leadership team made a clear diagnosis of had his ear, allowing for efficient resolution of any systemic problems in the education sector. According problems faced by the project. In addition, there has to the project’s TTL, the leadership team recognized first been consistency in the PMU throughout the life of the that, without an effective accountability system there project, with no changes in staff until the most recent would be no meaningful demand for change—even if elections, when the project coordinator was replaced. sector services were not working. Second, the team More information on the PMU structure is provided noted that the motivation to perform was inhibited below, in the section on implementation arrangements. in the sector by the fact that high performers did not attract greater resources. Finally, the team understood The project coordinator kept in close contact with the that results needed to be linked to resources to establish bureaucracy—including each district’s tutor general/ accountability, strengthen the quality of teaching and permanent secretary (TG/PS), zonal officers, and school school facilities, and improve students’ test scores, principals—whom she convened every month. She met learning outcomes, and employment opportunities. separately with school principals every term and with SBMCs and student representatives every year. To keep The state’s vision was clear. Rather than focusing on direct contact with officials in charge of implementation, inputs with the support of a traditional investment she decided up front that the PMU would resist the project, it sought to reform how the entire system secondment of officials from the state government. functioned, changing the rules of the game. As the project coordinator pointed out, there was a “yearning A World Bank TTL Who Fostered a Partnership for change, whether it was social or political . . . there Based on Trust, Continuity, and Hands-on was a need for change in the education system and there was a leader at a high level who wanted that change, Supervision with highly skilled people in Lagos State to execute that The project has benefited significantly from two change.” consecutive TTLs with a hands-on approach to project management and supervision, the first based A Small but High-Powered PMU That Combined in Washington, DC, and the current TTL based in Abuja, Internationally Competitive Technical Skills Nigeria. The DC-based TTL encouraged and enabled with Local System Knowledge and Effective strong involvement on the part of Abuja-based staff, with Leveraging of State Institutions strong substantive support from the education specialist in the country office. By the time the education specialist Eko Project implementation has reaped the benefits took over as the project’s TTL in 2012, he had considerable of a well-functioning PMU, managed by a project experience with the operation and a clear understanding coordinator who had significant experience with of the supervision model. The DC-based TTL, who international donor agencies, relevant technical training, oversaw the project’s design and early implementation, and strong leadership skills. The PMU’s strength and its continues to be engaged today. leadership provided effective oversight, coordination, and communication with other stakeholders as part of The staffing continuity allowed for management the implementation process. The project coordinator also consistency, and the new TTL’s location in the country demonstrated political savvy in her project management office has enabled frequent, hands-on implementation approach. From the beginning she was aware that, as an support and fluid communications. Regular visits to outsider, she needed to build an internal coalition for Lagos have encouraged the development of a strong reform in the sector. When engaging stakeholders from partnership built on trust and respect and a flexible the National Union of Teachers, for example, she focused working arrangement with the PMU, allowing for real- selectively on influential pro-reform counterparts— time problem solving and the ability to seize reform namely the Association of Nigeria Conference of opportunities as they arise. As a result of this close Principals subgroup—resulting in efficient collaboration working relationship, the PMU has space for a healthy and bypassing bureaucratic barriers at the broader degree of autonomy. union level. The Association has actively supported Eko Project implementation by organizing peer exchanges of experience on best practices. 13 Designing a Participatory Design Process Led Every participant was given a voice. In fact, the National by the State Union of Teachers, for example, successfully pushed back on a proposal to test teachers regularly on Government-Driven Consultations Around their performance based on agreed-upon criteria. To a Clear Vision Leading to Strong Ownership “get the ball rolling,” the project team agreed to steer of the Project’s Design clear of teacher performance assessments during the project’s initial phases.7 The decision to work closely and The highly consultative process of designing the cooperatively with the unions was a deliberate design Eko Project’s interventions and implementation choice. This type of empowerment demonstrated the structure sought input on all project elements, project team’s commitment to avoiding a top-down including potential reforms, incentives, and oversight approach to project design. Areas of agreement also mechanisms. The approach was led by the government shaped the project’s design. For example, all state and involved stakeholders at the state and district levels, and nongovernmental constituencies advocated for all of whom shared a clear vision of the project’s goals transferring funds directly to schools—and this element and how it should be implemented. According to the was quickly incorporated into the project’s design. project coordinator, this shared perspective arose from stakeholders’ common interest in improving the quality The Eko Project’s approach demonstrates the feasibility of schools and channeling resources directly to schools, of effective engagement and partnership with unions supported by the governor’s clear vision from the start. if those working with the unions understand their The close attention paid to ownership in the project’s interests and adapt to them in a manner that does not design fed into a similar approach to effective project directly or immediately compromise the efficacy of implementation through the alignment of incentives. The the interventions. One TG/PS in Lagos State reported time and effort invested in creating a truly participatory that engagement with unions promoted transparency design process paid off. about school performance. As a result, even members who were doubtful initially became “less skeptical” over Following an initial brainstorming session, design- time as they saw a positive shift in test scores. phase consultations with the World Bank and its education specialists took place at three levels: Role of Nongovernmental Constituencies (i)  between the governor and his policy team; (ii) between the commissioner of education and the in Education Reform: Parents and Community PMU; and (iii) between teachers, parents, community Members members, and other school administrators. The project Targeted and continuous engagement of coordinator attended each consultation session and nongovernmental constituencies was a priority, ensured that findings were communicated with each both at the design phase and in the early days of set of stakeholders. Owing to the early identification implementation, when most of the building blocks for of two priority issues in secondary education (learning the project were set in place. The project team consulted outcomes and employability) and three key mechanisms regularly with parents and other community members for improving them (accountability, incentives, and a on the project’s design components, implementation results orientation), the project’s clear vision was carried plans, and expected results. During the design phase, this through during the design phase consultations. engagement occurred through three large stakeholder consultations. Role of State-Level Constituencies: Teachers, Principals, and Unions Once implementation began, the project team met on a quarterly basis with representatives of the education State-level consultations took place four times district, including SBMCs and parents, to showcase during the one-year design period, bringing teachers, progress toward planned reforms and build confidence principals, and union representatives together to discuss in the project. They held similar meetings once per term planned reforms in the education sector. Most of the at the community level, with a view to building a critical groups agreed on the need to improve poor student mass of people who had seen and understood the results performance on state standardized exams. To initiate the of the Eko Project. The participation of parents and conversation with these stakeholders, the World Bank community members helped tailor project components and the project coordinator relayed a message from the to the operating environment, including, for example, governor: the state had access to resources through this how school performance was assessed. project, and guidance was being sought on how best to spend them. 7 It only introduced teachers’ competency testing in 2014. 14 Implications of the Participatory Design Process project documentation and coordinated effectively by in the Eko Project the PMU. (See Annex 3 for a more detailed description of implementation arrangements at the federal, state, As a result of this design methodology, the PMU has   district, and school levels.) been able to ensure multistakeholder ownership across all levels of government and among nongovernmental The deconcentrated approach adopted in the design stakeholders to support the achievement of results of the Eko Project’s implementation arrangements, during implementation. The project coordinator supported by a highly empowered PMU, established considered the role of the PMU to be that of a facilitator, the kinds of accountability mechanisms (upward, supporting the education districts that would ultimately downward, and outward) that enable autonomy and be responsible for sustaining the changes undertaken effectiveness. In the words of one teacher: through the project. This is especially pertinent today, as the project moves toward closure. The PMU does “I have a large role now  .  .  .  there has been a not substitute for the established bureaucracy; the decentralization of power since Eko; it used to be Eko approach leverages the established bureaucracy to from the State Ministry.  .  .  . There is even more initiate, implement, and sustain the reforms. information flow and teachers can complain to principals; ZoPAs oversee things including PICs The project coordinator’s approach, which from the [Project Implementation Committees], of which I design phase involved high levels of engagement with am a member.” the education districts to ensure both effectiveness and sustainability, was complemented by efforts to ensure buy-in from parents and teachers. This inclusive Design of Eko Project Systems for Data approach is demonstrated in all elements of the project’s Collection and Analysis and Their Impact implementation processes, including, for example, the on Perceived Education Performance monitoring of school performance conducted by ZoPAs, Generally, the results framework designed for the district project advisory committees (DPACs), SBMCs, Eko Project has struggled to tell a consistent story parents, community members, and external consultants about performance. This challenge stems in large part who specialize in monitoring and quality assurance. from the limited availability of baseline data during the design phase, the addition and removal of several results Eko Project Implementation Arrangements indicators during implementation, and the definition of Designed to Span Government Levels overly ambitious targets during the additional financing with Clear Roles and Responsibilities period, coinciding with a dip in test scores. The Eko Project is essentially a state-level project, as In the design phase, the team faced difficulties in opposed to a countrywide or federal intervention. The gathering reliable baseline data because the West State Ministry of Education is responsible for project African Examinations Council—the entity that execution, in close coordination with the education administers the WASSCE—had little experience in districts, LGEAs, SUBEB, and federal agencies. The Federal systematically collecting and aggregating data for Ministry of Finance (FMoF) acts as the Recipient for each state. Student scores were recorded and used project activities that relate to the government’s financial by students to apply for additional schooling, but the and legal obligations; Lagos State has signed a subsidiary scores were not being aggregated at the national, state, financial agreement with the FMoF. The Federal Ministry district, or even school level to provide an understanding of Education (FMoE) plays a coordinating and monitoring of education sector performance more broadly. In some role in project implementation, although it lends a cases, therefore, the team used modeled estimates as significant portion of this responsibility to ZoPAs, DPACs, baselines against which to measure project results. SBMCs, parents and community members, and external monitoring consultants/quality assurance officers. The During the implementation phase, the Eko Project’s FMoE is also responsible for implementing the project innovative focus on encouraging the analysis of component on strengthening the federal post-basic test scores by districts, zones, and schools yielded education strategy. Provisional agencies established to a growing appreciation among key stakeholders for support the project are closely interconnected with the capturing progress over time. By 2011, 80 percent of permanent agencies of the bureaucracy, thus externalizing schools had begun requesting analyses of trends in test part of the delivery system rather than establishing a scores, including by subject, by test section, and by the parallel one. This structure is intended to ensure that psychometrics of test questions that helped standardize changes brought about by the project will remain after the degree of difficulty. However, this increased interest the project closes. The roles and responsibilities of each generated data that had not been foreseen during the entity (Figure 2; Annex 2) are spelled out clearly in the 15 Figure 2:  Implementation Arrangements under the Eko Project FMOF & FMOE Oversight Governor State Project Advisory State Committee (SPAC) Project Management Unit (PMU) Delivery Unit District Project Advisory Committee (DPAC) Districts Education District Zonal Project Administrators (ZoPAs) Project Implementation Committees (PICs)/Schools Schools project’s design, and in 2013, several intermediate results on modeled estimates that may have underestimated indicators were added to the project’s results framework the situation at the start of the original project, thus in an effort to reflect progress being made on a number leading to an overestimation of possible progress, and of fronts. By 2015, however, the project team had come second, in some cases, targets were set on the basis of to the conclusion that the added indicators were internal World Bank estimates of student performance burdensome to track and, in some cases, less meaningful rather than the scores aggregated and presented by the because they did not link back to baselines set out at state. Because performance against these indicators was the start of the project. Moreover, the team realized then measured according to the state’s numbers, which that some of the more recently added intermediate have tended to be lower than the World Bank education results indicators measured activities and inputs rather team’s estimates, this mismatch in data artificially lowered than project outcomes, as intended. In response, the perceptions of progress. team removed some of the newer indicators from the results matrix, creating a degree of instability in the M&E There are risks in both overestimating and under­ framework and complicating efforts to delineate trends estimating results targets. Given that success is over time. not invariable and task teams are eager to capture progress, increasing and adjusting results indicators may In the design of the additional financing operation, be the natural response to reporting and building on new targets were set to capture the later closing achievements appreciated on the ground. However, date and additional resources being devoted to Eko this experience demonstrates that even when cultural Project interventions. At the same time, two new core changes are embedded and improvements are being outcome indicators were added. Within a year, however, made in processes and outcomes, inconsistent reporting it became clear that the team had overestimated targets on results and the fragility of some achievements for the additional financing operation. This overreach may have the potential to undermine a project’s occurred for two reasons: first, baselines were based accomplishments by eliciting caution and skepticism 16 about results, thus potentially preventing the replication open and candid discussions and interactive question and of useful techniques and interventions. answer sessions, to ensure that the project responded to stakeholder preferences. The result has been an effective   set of incentives that motivate key project stakeholders Eko Project Implementation to perform well and strengthen the project’s credibility. Challenges The incentives built into the Eko Project were in Even in a Challenging Data Environment, large part transformational, in that they encouraged a Data-Driven Approach Facilitates the positive behavioral changes and motivated higher Effective Use of Performance Incentives performance. According to interviews, these incentives work in part because teachers and principals place a high The Eko Project introduced detailed performance value on recognition. Below is a list of those incentives measurement at the school level, disaggregated by that stakeholders most commonly reported as having subject to pinpoint specific improvement needs. The a positive influence on their performance. Because accuracy of school performance reports varied with the there has not yet been a rigorous impact evaluation of quality of data used. As project implementation began, the operation, the evidence linking these incentives to Lagos State experienced some difficulty in gathering performance is based on qualitative methods, including baseline data for these assessments, and the resulting semi-structured interviews and focus group discussions. low baseline measurements may have played a role in overestimating progress within the project’s M&E • School grants and performance-based funding framework (along with discrepancies in test score data rewards for schools. In addition to the school grants sources, as discussed above). The state’s data collection distributed to all schools covered by the project, high- mechanisms have improved since the project’s inception, performing schools were provided with awards in the however, which bodes well for the efficacy of related form of school office items or funds to purchase such incentive systems. items. Teachers and principals demonstrated pride in purchasing goods through these awards by openly School performance reports drive the performance- labeling these items. based incentives embedded in the Eko Project’s • The Governor’s Education Award for schools, design, and these are based on state-submitted test teachers, and principals. Schools have demonstrated scores for each school. In math, for example, scoring pride in winning these awards by marking the years is conducted for five subcategories: algebra, geometry, in which they have won them on the walls of the numeration, probability and statistics, and basic facilities. Several teachers and principals reported operations; for the English language, scoring is conducted that the Governor’s Education Award was the separately for composition, comprehension, grammar most effective motivator of performance, owing and syntax, literary skills, and speech development. to the level of prestige, appreciation, legitimacy, School performance reports benchmark each school’s and credibility it conferred upon recipients. In the performance at the level of the local government area words of one teacher, “the most effective incentive and the district. Susceptibility to potential data collection is the Governor’s Award, because it is prestigious weaknesses would thus affect this portion of a school’s and everyone benefits since the whole school gets overall performance score. it . . . everyone who has worked for it . . . my school has won it twice . . . even students benefit.” School scorecards capture other quantitative • High-quality training opportunities as a staff dimensions such as student and teacher attendance, performance incentive. Teachers, principals, and even as well as a wide array of qualitative factors such as ZoPAs repeatedly reported being motivated by the the availability of extracurricular activities, school opportunity to receive training in various issue areas, beautification efforts, staff training, student counseling, including leadership, budgeting, ICT, and M&E. They communication, student reading habits, and student indicated that these opportunities have helped them interest in and availability for learning (for example, understand how to do their jobs better and boosted whether they are busy with menial jobs during school their performance. This feature has the added benefit hours). Teachers are provided with records of their of strengthening skills in an area that is beneficial to students’ achievements during the previous term to help project implementation. them improve their teaching effectiveness. • The use of data to bolster competition between schools and facilitate peer learning. The competitive The consistent, transparent, and data-driven nature of data-based performance monitoring is approach to monitoring program performance helps evident in the visual prevalence of test scores and link incentives to performance. The incentives were other ranking factors in school buildings. In some cases, determined through a participatory approach involving an idea implemented by one school to strengthen 17 learning outcomes has had a demonstration effect on actors involved in school operations. Data collected other schools, driving students, teachers, principals, on school performance are posted publicly so that and community members to achieve what other students, teachers, principals, district representatives, and project participants have achieved. community members have access. One SBMC chairman claimed that students themselves make use of school- Using Data Enables Accountability through level WASSCE test scores when deciding in which school Multistakeholder Monitoring of Program to enroll. In addition, all purchases made for schools using award or grant funds—whether they are desks, printers, Performance or other supplies—are marked with a label that provides The establishment and operationalization of an overall anyone entering the school with information on the year results-focused approach is one of the critical outcomes of purchase and funding source. of the Eko Project, which has introduced a Secondary School Information Management System (SSIMS) that This transparent approach allows various stakeholder captures data at the district, zone, and school levels. groups responsible for monitoring school performance The SSIMS goes further than the national Education (ZoPAs, DPACs, SBMCs, parents and community Management Information System (EMIS) by capturing members, and external monitoring consultants/quality information on outcomes such as learning achievements. assurance officers) to hold each other accountable for results, with limited top-down pressure. The result has SSMIS data fall under the following categories: been an environment of self-reliance, legitimacy, trust, and motivation at the local level. 1.  Personnel records (such as deployment of staff by subject, including volunteer teachers, and participation Adaptive Implementation Allows Flexibility in training and professional development); to Seize Windows of Opportunity and 2.  Information on students (including total number, state Adequately React to Setbacks of origin, and gender distribution); 3.  Educational support to students (provided after On the relationship between project design and school hours); implementation, the former project coordinator 4.  Grant management; observed that “a project document is rather bland; one 5.  School information (including infrastructure); and cannot just read it, one must bring it to life.”9 The PMU 6.  Student performance assessment results, disaggre­ and TTL’s flexible approaches have allowed for adaptive gated by gender, subject, grade, and so on. implementation, with good results. Examples include the engagement of private sector actors, a flexible The SSIMS monitors and assesses the performance of all approach to training, the adaptation of a more holistic stakeholders, including district administrations, ZoPAs, index for measuring school performance, and collective school-level Project Implementation Committees mobilization in response to the drop in test scores in 2014. (PICs), the Parents Forum, and the SBMCs. In addition, it monitors the management of school grants. SSIMS The PMU and TTL have been effective in seizing centers have been established at the district level, windows of opportunity. In the case of PPPs for but information is collected at the school level and technical colleges,10 for example, the initial partnership aggregated at zone and district levels. SSIMS officers have between one technical college and a private sector been deployed to the districts and schools. Education provider was initiated when the TTL encountered one district heads (TG/PS) have access to and make effective of his former students. This student had transitioned to use of the data collected. An experienced TG/PS claimed, work for Samsung, and their initial conversation led to as a result, that the project made her “computer savvy.”8 formal talks with Samsung and eventually to the launch of a partnership. In another example, the TTL sought out The availability of data, training in its analysis, and its potential partnerships in Silicon Valley and identified an effective use have allowed the Eko Project team to entry point with Google. He immediately communicated monitor overall project outcomes and hold stakeholders this information to the project coordinator, who acted accountable, thereby improving the performance of all quickly on the opportunity. As a result, teachers now 8 Interview with a TG/PS in Lagos State, June 16, 2015. 9 Interview with project coordinator, June 15, 2015. 10 The PPP schemes at work in the four state technical colleges (which involve a grant of land and a building from the state government, together with teacher payments and initial equipment) have resulted in the establishment of industry academies (including Samsung for electrical appliances and Fesko for automation) where students, as well as industry employees, clients, and franchisees, are trained. For example, at the Government Technical College, Agidingbi, Samsung has trained close to 4,000 students and over 4,000 salesmen since 2012. 18 receive training on Google Plus, through which they can concern was that every school could not be measured download teaching materials and exchange best practices. against a single standard because every school faced Teachers have found this training opportunity to have a different constraints.   significant impact on their skills development; many wore Google Plus pins during interviews. In both cases, project In response to this concern, elicited during a consultation activities tailored to the local context, where PPPs have meeting with all stakeholders, the project established been seen as a mechanism to deliver a shared vision, a composite score that took into account test scores, have supported improved learning outcomes in schools the school environment, the participation level from while promoting investors’ brands and developing future parents and community members, sanitation at the human capital for their operations. school, and student opinions of school performance. The same template was put to use across all of the Eko School report cards were introduced as a benchmark. Project secondary schools, thus holding all stakeholders During an initial project assessment, schools wanted accountable—from teachers, students, and principals to know where their biggest performance gaps were. who are involved in improving test scores and the In response, the project team developed report cards parents and community members whose participation summarizing each school’s benchmark performance and is so valuable, to the school administrators and other identifying their most significant overall performance gaps. faculty members who are responsible for school grounds beautification, and staff who are responsible The governor developed a Delivery Unit to oversee and for sanitation. hold leadership accountable for delivering results. This unit ensured that representatives from the governor’s Performance Data Can Help Identify Gaps leadership team were present at the monthly meetings and Provide Evidence in Support of Exit that tracked deliverables and timelines. The governor realized that the only way to bring the project’s most Strategies senior decision makers together was to set up a forum Although the Eko Project’s mandate does not extend for discussing education in Lagos more broadly, as well as so far as to implement explicit exit strategies for the Eko Project specifically. This change created a team low-performing schools, the performance data it that kept leadership on track and speaks to the adaptive collects may be used by stakeholders to document nature of the project at all levels of government. The performance gaps and provide evidence in support of Delivery Unit was in operation for three years, after which sanctions. For example, the public placement of school the governor felt the leadership team was strong enough performance information inside school buildings has the to continue on its own, under the ongoing management effect of penalizing low performance. One interviewee of the education commissioner. reported that nonperforming principals are sometimes moved from senior to junior secondary schools or The role of the ZoPAs changed during implementation. demoted to classroom teaching positions. Only the During the initial implementation stages, the project team education commissioner and the education district identified a significant communication gap between have the authority to remove teachers, principals, and schools and education districts. This disconnect disabled administrators, and in many instances, performance issues the districts’ accountability with regard to grievances take place without proper documentation. or inputs from the schools, including members of the PICs. In response, the PMU assigned the ZoPAs as liaisons Overall, however, there is an emphasis on improving the between the two parties to enhance communication performance of those lagging behind, whether they are between schools and the districts, thus increasing teachers, students, principals, or other stakeholders upward, downward, and outward accountability, as well involved in school management. For example, students as enhancing project monitoring and supporting the who are struggling are given extra coaching. Teachers who improvement of results. are not performing well are engaged and offered cluster- based professional development support within the How schools were scored changed during school, zone, and district. Incentives are awarded based implementation. The participative and adaptive not on absolute performance but on the magnitude of environment that the Eko Project fostered has allowed incremental progress. it to overcome a “learning by doing” hurdle encountered in the implementation of school performance scores. The Eko Project is currently supporting the state’s Initially, the project foresaw the use of test scores alone thinking on creating a Teachers’ Competency Framework to measure performance. Teachers, administrators, and (TCF) to document the achievements of performers and other stakeholders, however, did not feel that test scores the deficits of nonperformers. The TCF idea was first alone would reflect the holistic progress of schools based introduced in early 2014. The framework sets standards on project interventions. As described by one TG/PS, the for teachers based on experience, marking the skills these 19 professionals should have to achieve the next level of in Lagos State. Teachers, ZoPAs, principals, and other promotion. While teachers’ base salaries would remain stakeholders reported that the data-driven performance unchanged, high performers would be eligible for greater management system would continue to be part of their increases in income. The TCF takes the different education operational approach moving forward. In response to levels into account—that is, primary school teachers a question on how the districts and schools expected will be tested on different skills areas than secondary to maintain the project’s approaches after the project school teachers—and it emphasizes development of closes, one ZoPA member reported having “internalized, and proficiency in ICT skills. This proposal does not go as adopted the Eko approach.” Below are the elements far as competency tests introduced in other states such of the Eko Project, and of the “cultural change” is has as Ekiti or Edo, however. The TCF has been designed to engendered, that have been cited by stakeholders as support consistent training and related capacity building most likely to be sustained: of educators. The next step will be to engage the National Union of Teachers and solicit their views; progress in Giving autonomy to and encouraging performance this regard has been slowed by the recent attention to in schools through school grants. The PMU and other national elections. While TCF implementation would not stakeholders indicated that because school grants have address the problem of nonperforming employees in been so successful and critical to implementation, they the civil service over the long term, it would encourage planned to continue within the system, albeit on a smaller performance within the project and could possibly have scale. The grants have served as a driving incentive and a a demonstration effect through its implementation. It mechanism to demonstrate school pride across the state. should be emphasized, however, that this proposal is As such, key stakeholders were highly positive about the likely to arise as a sensitive area for the unions and a continuation of these grants, believing that any incoming matter of necessary compromise. politician who saw how critical these grants have been to successful project implementation would gladly incur what they viewed as marginal costs in the state education The Status and Sustainability of budget for the grants. Project Interventions at the Eko Close monitoring of outcomes through student Project Closing assessments. Various stakeholders at the school and Sustainability Is Enhanced by Proving the district levels reported that they would continue to use Value Added of a High-Performing System student assessments to make decisions on how to move forward in developing school improvement plans, even Ensuring the sustainability of project interventions after project closure. was a key part of the Eko Project’s overall vision from the start. Unfortunately, the Eko Project did not put in Deconcentrated data generation and the role of place a plan for impact evaluation, and it is too early ZoPAs in monitoring and management. The role of to judge the overall sustainability of the efforts. The ZoPAs in monitoring and management is outlined clearly project’s design phase focused on fostering change in the project’s implementation arrangements, and they through incentives, data-driven accountability, and a will remain in place when the project closes. It is with broad community of support. The project coordinator, this perspective that the PMU has maintained close recognizing her transitory role, aimed to ensure the communication with district-level stakeholders. Through sustainability of a well-functioning system: “From the the capacity building supported by the Eko Project, beginning, in the back of my mind, I knew I was not ZoPAs have had significant exposure to training on data here permanently . . . I knew that this was a short-term collection and analysis. In an interview, one reported that project, that we were facilitators of change and had to the use of data is among the most powerful instruments work closely with the education districts because they for sustainability, and that ZoPAs have learned to are the ones responsible for sustaining the changes.” The organize, advocate, and work without depending on the project implementation framework rests essentially on government. This combination of technical training and established state institutions, apart from a small PMU. In leadership understanding signals a strong likelihood that her view, and according to several other key stakeholders, ZoPA members will continue to implement data-driven, the project has achieved a functioning system from performance-based school planning. which it would be difficult to turn back and, as such, is likely to be sustainable. Building collective leadership along the delivery chain, down to service providers. The Eko Project has promoted The PMU reported that a “cultural change” had taken collective leadership at the school level. School principals place, and that the mindset of data-based performance no longer hold a monopoly on managerial responsibility; monitoring is “embedded” in the understanding of although they remain the chief accounting officers in those involved in implementing secondary education their school, they have become primus inter pares. In 20 some schools, all PIC members sign off on bid evaluations this past academic school year with a transfer of 28 new and contract awards, checks bear two or three signatures, teachers who had not yet been exposed to Eko Project and so on. interventions. With little guidance from the district, she   was able to train them quickly, transferring the skills she Investing in mentoring and training teachers while built as an Eko principal. One teacher reported that holding them accountable for performance. According “principals will be handing over mantles, even teachers to interviews with ZoPAs, teachers, principals, TG/ are in Project Implementation Committees so [skills] will PSs, and other stakeholders, the training of all parties be passed along, everyone is involved so it is easy to involved in project implementation has been so robust pass on the baton.” The likelihood of skills transfer was that skills will continue to be transferred via training of emphasized repeatedly during interviews in response new principals, teachers, and others by those who have to concerns about principals who have undergone been exposed to the Eko Project. One senior secondary significant capacity building as a result of the project, school principal reported that she had been surprised but are nearing retirement age. 21 Lessons Learned Lessons Learned Eko Project’s Highly Consultative replicated to some extent in multistate or national and Participative Design Process projects. The project also benefited from upstream sector reforms at the national and state levels and Leads to Achievements parallel donor support. For example, SBMCs have been Many of the Eko Project’s achievements are a direct introduced in every school, by law, to give schools more result of its highly consultative and participative design independence. The SBMCs have been actively supported period, a strong political champion, an effective PMU by other donors (including DFID). that worked both inside and outside the government, a significant level of managerial and professional capacity Carefully articulate the role of ad hoc implementing along the delivery chain, and an accessible and flexible agencies vis-à-vis established institutions. Positioning TTL that supported adaptive implementation. At the a PMU so that it can act both inside and outside same time, weaknesses in the design of the project’s the government system can enable a more flexible, M&E framework have complicated efforts to accurately empowered, and effective semi-autonomous understand and consistently track performance. Several management unit. Its capacity to leverage established lessons emerge from the Eko Project’s participatory institutions is paramount. While the proximity and and flexible approach to program design, as well as direct reporting between the project coordinator and its challenges in accessing reliable data at the project’s the governor may be unique in this case, establishing a inception, as follows: mainstreamed PMU with a leader who has the credibility and authority of an “insider,” but with less politically Take advantage of political buy-in by ensuring that motivated incentives as an “outsider,” can be a successful leadership percolates down to the frontline delivery operating model. level. If considerable political buy-in exists at the project design stage, political leadership can help build strong Establish robust M&E from the outset that takes ownership and participation around a clear vision for into account the availability of relevant data. The the design of project interventions and implementation credibility of a project’s performance rests on its ability arrangements. This combination of strong leadership and to demonstrate achievement of outcomes. The results meaningful participation can foster an operating culture frameworks for World Bank operations influence their that is based on a high level of accountability and, as a performance ratings, and low performance limits the result, is highly efficient. Positive impact can be traced to potential for project scale-up, follow-on operations, the complementarity between political leadership and and application of lessons learned to other contexts. institutional, managerial, and professional capacity. In Careful consideration of indicators used, reporting the context of the Eko Project, the Lagos State governor consistency, and diligent monitoring of results indicators was highly interested in the education sector, and the should be meticulously managed from project inception technically strong project coordinator was supported throughout implementation. by a top-notch implementation team. The culture of accountability and ownership, combined with various training initiatives, has developed and strengthened Smooth Project Implementation professional capacity at all levels, including down to Thanks to Participatory Design and individual schools. International Best Practices Ensure multilevel stakeholder involvement. Although Based on a desk review of all fiduciary and nonfiduciary it is important for the project design process to be led project documents, and on interviews with the TTL, by the government, with World Bank support, pushing the PMU, and project beneficiaries, Eko Project for the involvement of a wide range of stakeholders implementation was relatively smooth, thanks in at various levels of government can help promote part to the careful attention paid to a participatory ownership during design and implementation and thus design process that aimed to adapt international best strengthen project effectiveness. Although this project practices to on-the-ground realities in Lagos State. had the advantage of single-state implementation— With a rigorously recruited PMU serving as the project’s with fewer complexities in administration, management, nucleus, innovative interventions were introduced and and the political economy of reform—the Eko Project’s kept up, resulting in positive project outcomes. multistakeholder approach to project design can be 22 Key lessons learned in implementing the Eko Project be used to spur competition, determine how rewards design are as follows: are distributed, and help redress low performance. Transparent data availability can help build a system of   Hire the right mix of skills in the PMU so that it has both accountability across those implementing the project political clout and managerial leverage. Recruiting civil and its beneficiaries. The Eko Project benefited greatly servants recommended for their experience on donor from the identification of data that were already being projects helps support efficient project management. collected and could be internalized to help improve the When hiring PMU staff, it is important to ensure that quality of secondary education. there is sufficient capacity for both policy initiative and day-to-day project management. Building Project Sustainability Empower a PMU to be flexible and aim at internalizing from the Start innovation. It is important to give the PMU the flexibility and authority to alter project details in response to Because of the project coordinator’s “insider-outsider” opportunities that may arise, and to correct for design role, sustainability beyond World Bank support has features that are not working well. Once innovations been at the core of the project from the start. Although are successfully tested, project implementation should it is too early to draw lessons from this project as it promote their institutionalization and mainstreaming nears closing, some tentative suggestions on improving beyond the scope and timeline of the project. The Eko the likelihood of sustainability beyond project closure Project TTL not only supported the PMU in doing this, but include the following: also encouraged adaptation by identifying opportunities for their consideration. • Identify the sustainers early on. Government stakeholders who will play a crucial role in sustaining Ensure that the PMU interacts regularly with all project-related reforms after closing should be stakeholders and helps build collective buy-in. Flexible identified at the design stage and granted the and adaptive project execution depends on updating and capacity building they need to conduct this role receiving feedback from nongovernmental stakeholders effectively. This process requires building managerial on a regular and consistent basis. Empowering a PMU to and professional leadership within public institutions. be flexible during implementation will not on its own • Use capacity building as a performance incentive. allow for the execution of new ideas and implementation Using training opportunities as an incentive can drive changes. Nongovernmental stakeholders who are part improved performance. In the long term, training and of the design process should receive regular, consistent on-the-job learning can have significant impacts on communication from the PMU, so as to support flexible results and their sustainability. and adaptive project execution that allows for quick • Sequence interventions carefully to ensure that adjustments based on elements that are not working or reforms are not derailed early on. Project design opportunities that arise during implementation. and implementation should be built on a transitory outlook that brings the project’s sustainability after Be present and make sure the task team is available to closure to the forefront of its planning and approach. help address implementation challenges. A TTL who is • Gain exposure to other contexts where interventions in country, consistently present on the project site, and in the sector have worked and apply them locally, able to stay with the project over time can establish a in a strategic and nuanced manner. Local realities high level of trust with the PMU, thus allowing the project inform how sectoral interventions are designed and team to more effectively overcome obstacles and what their outcomes are. Focusing on how other respond to opportunities. Eko Project implementation countries have applied local knowledge, technical was well served by such an arrangement. experience, stakeholder consultation, and adaptive implementation can be critical in ensuring that Use data and close supervision to identify problems international experiences can integrate successfully and make course corrections in real time. Progress is into the local context over the medium to long term. not linear and should not be expected to be; part of adaptation is to collectively regroup and reassess options when results do not meet expectations. Data, at the right The Science of Delivery level of disaggregation, should be used not only to monitor During the fall of 2013, the World Bank (in collaboration project performance, but also to help build a system of with academics) analyzed case study work on the accountability among those implementing the project science of delivery. The emerging framework identifies and its beneficiaries and to support an effective reaction five elements, listed below, that are seen as important to unexpected or disappointing results. Moreover, in a for enabling science-of-delivery approaches. The delivery project that relies on performance incentives, data can 23 lessons from the Eko Project are generally applicable holding teachers and parents accountable for students’ to other education projects and other sectors. These educational outcomes. This accountability system has include the importance of aligning political, managerial, set a strong basis for performance, spurring overall gains and professional agendas. In addition, there are lessons in students’ educational outcomes. Third, the inclusive on engaging leaders with significant political power to nature of the project’s design has created space for enhance the efficacy of delivery; the role of training greater collaboration during implementation. For as both an incentive and a means to develop skills instance, parents have felt invested and informed enough for project implementation; and the significance of to help track progress by working closely with the SBMCs. innovative, responsive, quick thinking in establishing nontraditional solutions and relationships. Evidence to achieve results. The case study shows how access to data can help unlock performance by Relentless focus on citizen outcomes. From design fostering a spirit of competitiveness among teachers and to implementation, the Eko Project focus has been on principals. Data help identify weak points and monitor improving the quality of educational outcomes. This trends, while also highlighting high performance areas. In case study shows that international best practices, when this regard, students’ results are displayed publicly and tailored to the local context, can provide implementers the best teachers are rewarded. Similarly, schools that with a powerful set of “tested” toolkits to achieve results have shown the greatest improvement in educational faster. Instilling a culture of excellence and accountability outcomes are rewarded. Even with nonlinear progress in a system that has gradually degraded is one of the throughout its lifespan, the project has instilled in school project team’s most significant achievements. This shift administrators a sense that data collection is a means to has caused previously discouraged parents to regain chart a better direction for their schools. interest and hope that a public education system of good quality is possible in Lagos State. The PMU understood Collective leadership for change. Even with the focus that this was not a merely technical task, but one that on delivering high-quality services, the results achieved would require regaining trust after years of deficient under the Eko Project may have been difficult to sustain service delivery: “We needed to improve the confidence without the leadership provided by the governor from of parents. Parents were losing hope in public education. the outset. In addition to identifying education as a We needed to showcase what we were doing.” priority sector, the governor recruited a highly skilled project coordinator who had worked in the sector and Multidimensional response. The Eko Project has understood its complexities but could also function as demonstrated that improving targeted educational an “outsider” with respect to the government. To lead outcomes demands a multidimensional approach. For the change process, the project team staffed the PMU the first time in Nigeria’s education sector, the public with people who had experience working on World Bank and private sector have successfully come together as projects specifically. According to the project coordinator, partners. PPPs between technical schools and private this approach sent a signal that underperformance would entities such as Samsung have brought a new set of not be tolerated. Individual leadership that inspires eyes through which to measure success. For example, collective action, particularly through communities’ in addition to measuring graduation rates, these schools involvement in school management committees, has look at employability and strive to provide their been at the heart of the Eko Project’s implementation students with a relevant, hands-on learning experience. progress. This approach may sound intuitive, in retrospect, but in a context that sees professional schools as “schools Adaptive implementation makes participation of last resort,” as stated earlier, it is innovative. Private meaningful. At various stages of its implementation, the partners are invested in developing a workforce with project has had to adjust to account for changing realities the best technical skills, while schools show noticeable and for the multiplicity of ideas generated by stakeholder improvement in the quality of their programs and the consultations. The project’s adaptive approach has lent employability of their graduates. Second, the project greater legitimacy to stakeholder consultations; if people team has chosen to tackle the direct causes of poor see some of their ideas taken into account, they are more students’ performance by: (i) changing the incentives likely to be open to a participatory process. The adaptive of key stakeholders, such as teachers and principals; nature of the project’s design and implementation process (ii) empowering students to run small gardening and may be one of the reasons why a truly participative laboratory projects in their schools; and (iii) keeping process has taken root at various levels in the delivery principals accountable for teachers’ performance, and of secondary education in Lagos State. 24 Bibliography   Bibliography Duflo, Esther, and Michael Kremer. 2005. “Use of Randomization in the Evaluation of Development Effectiveness.” In Osvaldo N. Feinstein, George Keith Pitman, and Gregory K. Ingram, eds. Evaluating Development Effectiveness, Vol. 7 of World Bank Series on Evaluation and Development. New Brunswick, NJ: Transaction Publishers. FMoE (Federal Ministry of Education), Federal Republic of Nigeria. 2007. “Ed Tap Survey.” Glennerster, Rachel, and Kudzai Takavarasha. 2013. Running Randomized Evaluations: A Practical Guide. Princeton, NJ: Princeton University Press. Grindle, Merilee. 2004. Despite the Odds: The Contentious Politics of Education Reform. Princeton, NJ: Princeton University Press. Härmä, Joanna. 2013. “Access or Quality? Why Do Families Living in Slums Choose Low-Cost Private Schools in Lagos, Nigeria?” Oxford Review of Education 39 (4): 548–66. Heneveld, Ward, and Helen Craig. 1996. Schools Count: World Bank Project Designs and the Quality of Primary Education in Sub-Saharan Africa. Washington, DC: World Bank. Jimenez, Emmanuel, and Yasuyuki Sawada. 1998. “Do Community-Managed Schools Work? An Evaluation of El Salvador’s EDUCO Program.” Working Paper Series on Impact Evaluation of Education Reforms Paper No. 8. Development Research Group, World Bank, Washington, DC. Lagos State Government, Federal Republic of Nigeria. 2013.“2012–2013 Lagos State School Census.” Lagos, Nigeria. ———. 2009. “Nigeria: Lagos Household Survey 2005–2006, First Round.” Central Office of Statistics, Lagos State Government, Lagos, Nigeria. NBS (National Bureau of Statistics). 2005 and 2006. “National Core Welfare Indicators Survey 2006, Third Edition.” NBS, Federal Republic of Nigeria. NPC (National Population Commission). 2008. “Demographic and Health Survey Ed Data Profile 2008.” NPC, Federal Republic of Nigeria. NPC of Nigeria and RTI International. 2014. “2013 Nigeria Demographic and Health Survey (DHS).” NPC (Federal Republic of Nigeria) and ICF International. ———. 2011. “2010 Nigeria Demographic and Health Survey (DHS) EdData Survey: Education Data for Decision-Making.” NPC (Federal Republic of Nigeria) and RTI International, Lagos, Nigeria, and Washington, DC. Patrinos, Harry Anthony, Felipe Barrera-Osorio, and Juliana Guáqueta. 2009. The Role and Impact of Public-Private Partnerships in Education. Washington, DC: The World Bank. Uemura, Mitsue. 1999. “Community Participation in Education: What Do We Know? Effective Schools and Teachers and the Knowledge Management System HDNED.” World Bank, Washington, DC. Vegas, Emiliana, ed. 2005. Incentives to Improve Teaching: Lessons from Latin America. Washington, DC: The World Bank. Vegas, Emiliana, and Ilana Umansky. 2006. Improving Teaching and Learning through Effective Incentives: What Can We Learn from Education Reforms in Latin America? Washington, DC: The World Bank. World Bank. 2015a. “Lagos Eko Secondary Education Project: Implementation Support Mission Aide Memoire.” September. World Bank, Washington, DC. ———. 2015b. World Development Report 2015: Mind, Society, and Behavior. Washington, DC: World Bank. 2003. World Development Report 2004: Making Services Work for Poor People. Washington, DC: The World Bank. ———. 2009. “Project Appraisal Document on a Proposed IDA Credit in the Amount of SDR 63.6 Million (US$95 Million Equivalent) to the Federal Republic of Nigeria for a Lagos Eko Secondary Education Project.” Human Development 3, Country Department AFCW2, Africa Region (May 18). Washington DC: World Bank. ———. 2008. “Governance, Management, and Accountability in Secondary Education in Sub-Saharan Africa.” World Bank Working Paper No. 127. Africa Human Development Series. Washington, DC: The World Bank. ———. 2005. “Implementation Completion Report (IDA-33460) on a Credit in the Amount of US$55 Million (SDR40.1 Million) to the Federal Republic of Nigeria for a Second Primary Education Project.” Human Development AFTH3, Africa Region (June 21). Washington DC: World Bank. ———. N.d. “Social Accountability Sourcebook.” http://www.worldbank.org/socialaccountability_ sourcebook/. 25 Annex 1: Project Summary Annex 1: Project Summary Project Components Component 4: Strengthening of the Federal Post- Basic Education Strategy (estimated base cost: US$5 Component 1: Promoting Secondary School million):The objective of this component is to provide Effectiveness through School Development Grants capacity building and technical assistance to the Federal (estimated base cost: US$62.6 million): The objective Ministry of Education to enhance the development of of the school development grants is to raise education the national post-basic education strategy and support outcomes in junior and senior secondary education by policy dialogue on the strategy with States. providing public secondary schools in Lagos access to yearly discretionary resources with an explicit focus on improving the quality of education services as priority needs are defined at the school level. School progress Restructuring and Additional will be rewarded through additional performance grants Financing for the top 40 percent of schools based on criteria for The project was restructured twice, once in November measuring quality improvements (such as test scores 2013 and again in June 2014. The initial restructuring led to and teacher attendance). Grants based on private sector a recommitment of funds from the Lagos Metropolitan partnerships are also provided for the five technical Development and Governance Project in the amount of colleges. US$36.3 million. The second restructuring allowed for an extension of the project closing date, as there was a need Component 2: Enhancing Quality Assurance for Junior to complete requisite procedures to gain effectiveness and Senior Secondary Schools (estimated base cost: of the additional financing. US$13.7 million): The objective of this component is to establish a standardized system for measuring Finally, while the project was set to close on December 31, students’ learning achievements in core subject areas 2015, the government requested a six-month extension and support teachers to develop the skills needed to in additional financing to June 30, 2016. The rationale, as better teach these areas. This component would support captured in the September 2015 Aide Memoire, was the the improvement of the quality assurance systems in delay in project effectiveness and implementation as a Lagos State to measure progress, identify deficits, provide result of school closures due to the Ebola Virus Disease, feedback to service providers, and establish appropriate the recent change in government, the retention of a benchmarks/standards. new project coordinator, and the need to disburse the remaining/committed funds for several activities. These Component 3: Project Coordination and Management activities include the disaggregated analysis of 2015 test (estimated base cost: US$4.6 million): Effective results to justify the increase in performance compared implementation of the project will depend upon efficient to 2014, monitoring and evaluation of school grants, and coordination mechanisms, proper financial management training of more district staff, school principals, and vice and procurement practices, timely implementation, principals. and effective M&E of project outcomes in Lagos. This component will provide the necessary resources for effective coordination and M&E, and the implementation of an information and communications strategy. 26 Annex 2: Roles and   Annex 2: Roles and Responsibilities for Project Implementation Responsibilities for Project Implementation Key Stakeholders PMU was formally created under the direct supervision of the commissioner of education—at the time also the State Project Advisory Committee (SPAC): The SPAC deputy governor—and later the Executive Governor reviews progress reports, approves annual work programs himself. During project implementation, the PMU reports and budgets, advises on key implementation issues, and directly to the Governor and formally to the State ensures that the agreed performance targets and timelines Ministry of Education. This semi-autonomous, hybrid for activities under the different project components arrangement allows for the PMU to simultaneously work are met. The SPAC is chaired by the Commissioner of “inside” and “outside” the State Ministry of Education Education of Lagos State, and members include the at the same time. It also ensures that the PMU is Permanent Secretary of the State Ministry of Education aligned with and has the backing of the most powerful (Vice Chair), Chair of the State Universal Basic Education person in the state, allowing for efficient resolution of Board, six Tutors-General/Permanent Secretaries of implementation bottlenecks, while also ensuring that Education (one from each education district), the it has buy-in from the public sector. The Federal PMU Permanent Secretary from the Teachers Establishment implementing the Science and Technology Education and Pensions Office, the Chair of the House Committee in Post-Basic Education (STEP B) project manages the on Education, the Chair of the State Central Parents and federal activities under Component 4. One of the key Teachers Association, the Senior Special Assistant to responsibilities of the PMU is to interface meaningfully the Governor on Technical-Vocational Education, two with the technical departments at the state, district, and Representatives—one from the community (SBMCs) zonal levels to ensure efficient implementation. and the other from civil society/nongovernmental organizations, and the Project Coordinator of the PMU, These implementation arrangements enable the PMU to who is the nonvoting executive secretary of the SPAC. maintain consistent contact with the schools, allowing The SPAC meets twice a month and is assisted by the for extremely robust oversight and for sustainability PMU. purposes. It does so by meeting with all stakeholders regularly (including the SBMC and parents) but with School-Based Management Committees (SBMC): a specific emphasis on the education districts. The School-Based Management Committees have been Education District Team and the Tutor General/Permanent established for all school clusters to provide a fundraising, Secretary are charged with leading resource management. community outreach and oversight role while supporting The PMU informs this leadership team whenever funds school planning and development. Members include are being disbursed, and this team monitors activities, those from the community that the schools are a part outcomes of activities, school development plans, and so of—these could include parents and members of the on. The PMU holds meetings with all stakeholders about community who want to serve as interlocutors between what needs to be done at each school and then meets schools and the relevant communities. They participate with each school’s education districts. The education to ensure that community inputs, grievances, and needs districts then inform the PMU, which sends the funds to regarding their schools are addressed. schools electronically. As such, the school administration districts are never out of touch with their schools, and Project Management Unit (PMU): The PMU was funds are disbursed based on tangible real-time needs established in the office of the deputy governor/ while the PMU stays informed on the status of project commissioner of education to support the coordination implementation. As facilitators of change, the PMU and fiduciary management of activities of the works closely with the education districts that will be implementing directorates/units and the reporting of responsible for sustaining the changes following project project activities under the first three components. The closure. 27 The PMU team is composed of a project coordinator who ZoPAs are key players in the M&E system, as they are serves as an advisor to the governor. She is paid by the responsible for collecting school-level data, verifying the state government as a public servant, with a consolidated data, and aggregating the data at the district level. salary including all allowances, accommodations, and so on. While not comparable to the private sector, this salary Project Implementation Committees (PICs): PICs are is similar to that of a permanent secretary in government. established in each public secondary school and include The PMU team includes an education specialist (with the school principal as chairman, the vice principal expertise in teacher training), a procurement officer, a (academic) as secretary, the vice principal (administrative) project accountant, an administrator, and a M&E specialist. as treasurer, three classroom teachers (with a preference They are all civil servants paid on the civil servant salary for core subject teachers), a representative from the scale. Based on a request by the project coordinator and Parent Teachers Association, and a representative from their applicable experience/expertise, the core team is the School-Based Management Committee. The PICs are redeployed from relevant departments/units within responsible for the preparation of school improvement government. This is an open and transparent process plans for strengthening student performance in core through direct transfer and assumes that PMU staff subjects. In addition, they support the ZoPAs in data would return to their respective positions in government collection, collaborate with local community members after the project closes, without loss of seniority or on M&E activities, manage grant procurement, mobilize entitlements. While the project coordinator did not community support for the school, and oversee the personally handpick the team members, she did and financial management of grant implementation. The PICs continues to institute a working system through which receive requisite training for school grant implementation she ensures that nonperformers are removed from the and school improvement planning. PMU. The team members were selected on the basis of their experience working on other World Bank projects. Delivery Unit: The governor introduced a tailored Such experience was emphasized, in particular, for the delivery unit to ensure that representatives from the procurement and finance officers who were suggested by leadership team (including the tutor general/permanent their respective ministries/agencies to facilitate project secretary, commissioner of education, permanent implementation. secretary at the State Ministry of Education, chairman of the State Universal Basic Education Board, and the District Project Advisory Committee (DPAC): The PMU) were present at a monthly meeting that traced and DPAC primarily oversees Component 1 by providing tracked deliverables and timelines. This practice allowed policy oversight, evaluating and approving school grant the governor to hold leadership accountable for results. proposals, and monitoring performance indicators. The DPAC Chair is the tutor general/permanent secretary of Project Implementation Manual: Institutional each district; that is, he ranks as the highest nonelected Arrangements official within the State Ministry of Education. The DPAC meets twice a month. Besides the Chair, other members 1.  Effective and successful implementation of the Lagos include representatives of School-Based Management Eko Secondary Education Project will be based on the Committees, representatives of the Teachers following governance principles: Establishment and Pensions Office, representatives of • Intensive coordination across tiers and branches the State Universal Basic Education Board, and District of government; Directors for Junior and Senior Secondary Schools who • Compliance with clearly defined project oversee Zonal Project Administrators. The DPAC also implementation rules (for example, for efficient oversees the Lagos State Technical and Vocational management and allocation of resources according Education State Board, which governs the public private to agreed work plans); and partnership grants for technical and vocational colleges • Transparency and information sharing, especially and manages the capacity-building activities to help the on matters of performance within the public colleges create their proposals and provide support to sector and with the public at large. the DPAC for approving the public-private partnerships. 2.  Implementation arrangements have been developed with the objective of ensuring that effective Zonal Project Administrators (ZoPAs):ZoPAs are made collaboration can occur within the established up of senior education officers assigned to help a cluster institutional structures and personnel. In order to of schools implement the school grants. Their role support the coordination of activities across these includes technical support to schools in general planning, structures and different levels of government, a Project implementation, and grant management; collection and Management Unit (PMU) has been established headed dissemination of key school data; liaising between the by the Special Adviser to the Lagos State Governor schools and district; and general troubleshooting to to report directly to the Lagos State Governor. The ensure that issues are resolved in a timely manner. The 28 PMU will receive support through the project with 3.  Institutional Framework. Table A2.1 lists project the following objectives in mind: components and the responsible implementing • Effective collaboration between the institutions agencies. Figure 1 shows the institutional framework   responsible for project implementation; for planning, approval and execution of the project. • Clearinghouse for overall financial management 4.  A National Education Sector Steering Committee and large procurement packages; (NESSC) will provide a forum where project • Promotion of mainstreaming of procedures within performance and implementation issues can be the current government structures; discussed and knowledge can be shared at the • Provision of project oversight, M&E of activities; State and Federal level. Lagos State will join the • Provision of an appropriate link between NESSC. Although the NESSC will not have any direct government and development partners; project implementation role since this function will • Promotion of adequate stakeholder consultations; be the primary responsibility of each State Ministry and of Education (SMoE), it will become a particularly • Promotion of government leadership as a basis for important forum for the following: (a) providing ensuring program sustainability. oversight and coordination of all donor education Table A2.1:  Component Management and Institutional Responsibilities Component Responsible Institution Implementation Responsibilities 1.0 Promoting Secondary School Effectiveness 1.1 Sub-component 1.1: Secondary Education District TG/PS chairs the DPAC School Development Grants; • TG/PS of the district and • DPAC Ensure effectiveness and efficiency of the implementation of sub- 1.2 Sub-component 1.2: components 1.1 and 1.2 Secondary School Performance Awards Supervision of grant and training operations 1.3 Sub-component 1.3: Public- Technical Committee on Review the grant proposal, Private Partnership in Partnerships (TCP) and PMU recommend the proposal for grant Technical Education award and ensure high standard of probity in discharging its responsibilities, and perform other tasks as may be assigned to it by the TCP 2.0 Enhancing Quality Assurance for Junior and Senior Secondary Schools 2.1 Sub-component 2.1: • PMU Improve process and system for Standardized Testing for • Lagos State Examinations conducting examinations and Secondary Schools Board carrying out students’ assessment 2.2 Sub-component 2.2: • PMU with training consultant Develop guidelines and coordination Strengthening the Capacity of support and organize training Teachers and Principals • Education District 3.0 Project Management At the state level State Project Advisory Broad oversight Committee (SPAC) project implementation At the district level District Project Advisory Responsibility of the district headed Committees (DPAC) by TG/PS 29 programs (for example, Lagos Eko, SESP, and ESSPIN); 6.  State Level. At the state level, the governor chairs (b) ensuring compliance with ongoing federal reforms; the SPAC. To ensure coordination of all education (c) monitoring performance targets and timelines for sectorwide policies and strategies, SPAC meets on a activities and addressing cross-cutting issues; and monthly basis covering all aspects of the education (d) sharing of information among stakeholders about sector including primary, secondary, technical and effective project implementation. broader education sector plans such as MTSS and 5.  The Committee would be managed and chaired activities supported by other donor agencies. To by the Federal Ministry of Education (FMoE) guarantee that the project has clear leadership and or his/her representative and includes key direction, the governor has appointed a Special stakeholders responsible for project monitoring and Adviser, reporting directly to the governor ensuring implementation. This is the main policy body for all progress, annual work programs, budgets, and key state-based education projects at the federal level. implementation issues are agreed with the governor The NESSC will include Commissioners of Education, and SPAC. The SPAC members would include the Permanent Secretaries, participating states, relevant following directors of the FMoE, Permanent Secretary, Federal 1.  CHAIR: Governor, Lagos State Ministry of Finance (FMoF), Director General of the 2.  Commissioner of Education Lagos State, Vice Chair Debt Management Office and National Planning 3.  Special Adviser to the Governor, Lagos Eko Project Commission, and additional members selected on 4.  Chair of SUBEB (the agency responsible for junior the basis of their potential to add value to the work of secondary education) the committee (such as heads of federal agencies and 5.  Chairman, LASTVEB professional associations, private sector, and so on). 6.  Permanent Secretary, SMoE 7.  Six Tutors-General/Permanent Secretaries of Roles and Implementation Responsibilities Education (TG/PSEs), one from each education district The Project will be implemented in all public junior and 8.  Permanent Secretary, Teachers Establishment and senior secondary schools (including the five technical Pensions Office colleges) in Lagos State. To support implementation at the school level, capacity strengthening, management District Level and oversight would be organized at the state and district levels. The Technical Committee on Partnerships 7.  At the district level, the project would be governed (TCP) would be responsible for reviewing the proposal by District Project Advisory Committees (DPAC). This submitted for support. The Technical committee will program management committee would be chaired be made up of representatives of Lagos State Technical by the TG/PS of the district. The committee would and Vocational Education Board (LASTVEB) and the meet at least twice yearly or more frequently as may Technical Colleges and PMU. The TCP would review the be ordered by the TG/PS. The committee members grant proposal, recommend the proposal for grant awards would include the Education Secretaries, District and ensure high standards of probity in discharging its Directors, Zonal Project Administrators (ZoPA), and responsibilities, and perform other tasks as may be SBMC representatives of the following Districts: assigned to it by the TCP. The TCP would consist of up District I—LGAs of Agege, Alimosho, Ifako/Ijaye to four LASTVEB representatives with technical expertise District II—LGAs of Ikorodu, Kosofe, Somolu and experience in technical education matters. Other District III—Epe, Etiosa, Ibeju/Lekki, Lagos Island members will include nominated Government College District IV—Apapa, Lagos Mainland, Surulere Principals. District V—Ajeromi Ojo/Ifelodun, Amuwo- Odofinn, Badagry District VI—Ikeja, Mushin, Oshodi/Isolo 30 Table A2.2:  Roles and Responsibilities for Implementation—State Level State Level   State Project Advisory Committee a. Reviews the Eko Secondary Education Project Progress Report; (SPAC) b. Supports the achievement of the Project work plan program by the PMU; c. Assists in attainment of agreed performance targets and timelines for activities under different components. Project Management Unit (PMU) a. Monitors grant implementation progress to ensure that the project focuses toward its desired outcomes and provides continuous feedback to improve performance of project implementation; b. Interfaces meaningfully with the technical departments at districts and zonal levels to ensure efficient implementation at the school level; c. Supports the technical departments or other implementing units as facilitator of the flow of credit funds and project resources; d. Acts as the clearinghouse for securing Bank action on procurement and disbursement of funds on the activities, goods, works and services contract financed from the IDA credit; e. Handles the regular management of the fiduciary aspects (procurement, financial management, M&E, etc.) of the project; f. Establishes and maintains regular communication and maintains liaison with the World Bank, DFID, SMoE and relevant authorities in all matters that concern the project; and g. Provides oversight functions in all respects and is an arbiter in matters needing clarification related to project implementation. Table A2.3:  Roles and Responsibilities for Implementation of Additional Financing—Local Levels District District Project Advisory Committee a. Provides strategic guidance for grant implementation at the (DPAC) district level; b. Ensures that agreed timelines for activities under the different components are met; c. Ensures effective grant implementation through monitorable performance indicators; d. Addresses and resolves critical issues that could hinder grant implementation; e. Verifies the eligibility of the grant proposals prepared by schools; f. Endorses the eligibility of school improvement plans for approval of the TG/PS of Education through the DPAC; g. Monitors project-supported teacher training activities for deployment of policy and other programmatic activities; h. Consolidates and reviews monitoring and other reports prepared by the Zonal Project Administrator for submission to the PMU. District Technical Review Committee a. Provides technical inputs to the school improvement plan (DTRC) submitted annually; Formed by DPAC ad hoc b. Reviews grant proposals and verifies the eligibility of the grant proposals prepared by schools; and c. Endorses the school improvement plan to the DPAC. 31 Table A2.3:  Continued Zone/School Zonal Project Administrators (ZoPAs) a. Provides technical support to schools in general planning and implementation and facilitates training; b. Interfaces between the schools and the Education District; c. Ensures that all required data are collected, compiled, and submitted to the District on grant-related matters (e.g., performance indicators, targets, etc.) and on other project-related matters (nongrant monitoring reports); and d. Represents the zones at DPAC meetings. Schools Project Implementation a. Prepares annual School Improvement Plan for effecting Committees (PICs) improvement in student performance in English, math and sciences; b. Assists in identifying teacher training needs for grant- and nongrant-assisted teacher development programs; c. Collects data on grant implementation as requested by ZoPA; d. Collaborates with local community members on matters related to performance reviews and M&E activities; e. Manages the day-to-day running of the project with the assistance of ZoPA; f. Provides information to the school community and a channel for community’s voice to be heard in the school; g. Mobilizes community support for the school; h. Identifies school needs and means of addressing these needs; i. Draws up and monitors School Improvement Plans in accordance with the guidelines set out in the manual; and j. Oversees the financial management of the school grant to ensure transparency and accountability. 32 Annex 3: Eko Project   Annex 3: Eko Project Implementation Arrangements Implementation Arrangements Implementation Arrangements— Finance (FMoF) is the representative of the recipient as Overall it relates to the financial and legal obligations of the government. Lagos State has signed a subsidiary financial The majority of project activities are implemented at the agreement with the FMoF. The FMoE and FMoF have state level. For the state activities, the State Ministry of oversight of the project and the FMoE plays a coordinating Education is the responsible agency for project execution and monitoring role in the implementation of the project. in close coordination with the Education Districts, LGEAs, The FMoE is also responsible for implementation of the SUBEB, and federal agencies. The Federal Ministry of activities under Component 4. FEDERAL STATE DISTRICT SCHOOL FMOF & FMOE Oversight SPAC PMU DPAC Education District ZoPAs/School Clusters 33 Implementation Arrangements—Federal Level • The federal component of the project is coordinated by the Federal Ministry of Education, Department of Basic and Secondary Education, but the project is mostly implemented at the state level by the Lagos Ministry of Education. • A technical officer in the STEP-B Project Management Unit (PMU) manages the implementation of this component and is responsible for day-to-day project implementation and serves as the main link between Federal Ministry the Federal Ministry of Education and the World Bank. The Federal PMU implementing the STEP B project of Education will undertake management of the federal activities under Component 4. (FMOE) • Members include the Commissioners of Education, Permanent Secretaries, participating states, relevant directors of the FMOE, Permanent Secretary, Federal Ministry of Finance, Director General of the Debt Management Office, and national Planning Commission, among others. • The NESSC is responsible for: (a) providing oversight and coordination of all donor education programs National Education (e.g., Lagos Eko, SESP and ESSPIN); (b) ensuring compliance with ongoing federal reforms; (c) monitoring Sector Steering performance targets and timelines for activities and addressing cross-cutting issues; and (d) sharing of Committee information among stakeholders about effective project implementation. (NESSC) 34   Implementation Arrangements – State Level Implementation Arrangements—State Level Chair: Commissioner of PMU Education of Lagos State Vice Chair: Permanent Secretary State Ministry of Education Chair of SUBEB (agency responsible for junior secondary education) Six tutors-General/Permanent Secretaries of Education (one for each education district) State Project Advisory Permanent Secretary, Teachers Committee (SPAC) Establishment and Pension Office The SPAC reviews progress reports, approves annual work programs and budgets, advises on key implementation issues, and ensures Chair, House Committee that the agreed performance targets on Education and timelines for activities under the different components are met. Chair, State Central PTA Senior Special Assistant to the Governor on Technical- Vocational Education Two Representatives, one from the community (SBMCs) and the other from civil society (NGOs) Project Coordinator of the PSU, non voting-executive secretary of the SPAC 35 Implementation Arrangements – District Level Implementation Arrangements—District Level Chair: TG/PSE of each district PICs District Directors for ZoPAs Direct Project JSS and SS Advisory Committee (DPAC) Schools Primarily oversees component 1 by Representatives of providing policy SUBEB oversight, evaluating and approving school grant proposals and monitoring performance indicators. Representatives of TEPO Representatives of SBMCs Lagos State Technical and Vocational Education Board (LASTVEB) The PPP grants for technical and vocational colleges are governed by the newly established LASTVEB. LASTVEB oversees the capacity building activities to help the colleges create their proposals and provide support to the DPAC for approving the PPS. 36   Implementation Arrangements – School Level Implementation Arrangements—School Level Chairman: School Principal Secretary: Vice Principal, Academic Project Implementation Committee (PIC) Established in each public secondary school and responsible for: preparation Treasurer: Vice Principal, Administrative of school improvement plan for impacting student performance in core subject areas, support to ZoPAs for collection of data, collaborating with local community members on M&E Three classroom teachers (preferable core subjects) activities,managing grant procurement, mobilizing community support for the school and overseeing financial management of grant implementation. Representative from PTA Representative from SBMC 37 Annex 4: Basic Education Annex 4: Basic Education Certificate Examination Results for Lagos State Certificate Examination Results for Lagos State The Basic Education Certificate Examination (BECE) Lagos State’s BECE results have not been used in this is conducted for candidates in their third year of case study because, for junior secondary schools, it plays junior secondary school. The BECE covers 22 subjects. the role of an intermediary exam. The following table Candidates are expected to sit for a minimum of presents the state’s BECE results for the three most recent 10 subjects and a maximum of 13. A candidate is deemed years, as compared to a baseline of 2009. to have passed the BECE if he or she earns a pass grade or above in at least six subjects, including English and mathematics. Table A4.1:  Basic Education Certificate Examination Scores, Lagos State 2009 2013 2014 2015 Subject Target Actual Target Actual Target Actual English 32 45 37 74 70 74 68 Mathematics 31 45 35 49 45 49 59 Basic science 27 45 51 69 65 69 50 Subject average 30 45 41 64 60 64 59 Public junior secondary Not Not Not 36 35 36 36 students at or above credit for tracked tracked tracked averaged subjects of English, mathematics, and basic/ integrated science Source: World Bank 2015a. 38