Operational Note 3 Iterating Implementation for Impact Starting Small and Building Up How do local governments fine tune their service delivery in the face of persistent or emerging challenges? What does a “change of mindset” in addressing service delivery challenges look like? The first operational note in this series showed how an initial desk review of data by the government MELAYANI team in Belu district revealed that teacher certification was not a key driver of student learning, as they had previously thought. This finding prompted the team to visit well- and poorly- performing schools in their own district to see if they could better identify what did seem to matter. A second operational note, “Digging Deeper: Iterating Understanding of a Problem” documents that process and what the team found. This operational note examines what the Belu MELAYANI team found in the field and how they used that information to develop, test and refine solutions to challenges in teaching and learning. The problem- solving approach is not terribly difficult—it does not require that local governments “start over” in understanding the problems that they face. However, it does require new approaches in gathering information, developing and implementing solutions, maintaining an open mind about whether a chosen solution works (rather than a ‘set and forget’ approach of just assuming it will), and retaining a problem-solving focus. This operational notes series aims to share experiences and practical lessons from MELAYANI – Untangling Problems in Improving Basic Services (Menguraikan Permasalahan Perbaikan Layanan Dasar di Indonesia). MELAYANI is a program that builds local government capacity to address service delivery problems at the district level. It does so through helping district governments identify meaningful problems, break them down, analyze their parts, and develop and refine solutions. The methodology for problem solving builds on the problem-driven iterative adaptation (PDIA) methodology developed by a team at Harvard University. It focuses on building team ownership of problems and solutions, empowering local staff to innovate and experiment, using data to understand problems and their causes, and iterating to sustainable solutions. The program emphasizes that staff themselves must do the work to understand the problem and identify and implement solutions. MELAYANI provides tools to support the process, which is guided by a trained coach, who is supported by a mentor with expertise in the PDIA methodology. MELAYANI receives support from Australia’s Department of Foreign Affairs and Trade and is managed by the World Bank. of implementing School Operational Funds Initial Challenges Pushed (Bantuan Operational Sekolah—BOS).3 the Team to Learn More About Their Own Schools This operational note focuses on the most “exploratory” of the three topics, teaching As described in previous operational notes, the and learning. To understand how teaching and government MELAYANI team in Belu district learning varies across schools, the team asked focused on improving the quality of primary only two questions of teachers and headmasters education. An initial break-down of the problem in the schools they visited: led the team to believe that low numbers of certified teachers was the main cause of poor • What factors support you / your teachers in learning outcomes. Testing that belief against teaching students well? available data (by correlating performance with • What obstacles do you face in teaching / in proportion of certified teachers in each school) supporting teachers to do well? led the team to discover not only that there was little correlation between the presence of The responses to these questions not only certified teachers and higher student test scores, demonstrated that there were good practices but also that some of their other assumptions being implemented in some of Belu’s primary about what drives better performance (e.g., schools. They also helped the team to better urban schools do better, rural schools do worse) articulate the differences between the schools were wrong as well.1 that were more and less successful at promoting student learning. These findings surprised the MELAYANI team and made them more willing to question their assumptions about the causes of problems and Positive Deviance about solutions. They decided to visit schools Provided Inspiration as to in their own district to see if they could better What Could Work identify drivers of improved student learning. In While it was important that the team identified total, the team visited ten schools, split evenly problems in implementation, they were also between those that were “well-performing” proud (and in some cases surprised) to also and those that were “poor performing.”2 The see examples of implementation working.4 team looked at three aspects of performance: Open ended visits from the district staff are (i) teaching and learning, (ii) the role of school not the norm in Indonesia, where ‘monitoring’ committees, and (iii) the administrative burden 1 See “Keeping it Simple: Supporting Government to Use Evidence to Solve Problems,” Operational Note 1 in this series. 2 Based primarily on average school test scores. For the last six schools (after the instruments were tested), the team also began to consider location/access, school management (based on input from active school monitors) and student population (notably the presence of internally displaced people). While different factors were not necessarily evenly split into two groups, the idea was to better understand additional factors in school performance. 3 See “Digging Deeper: Iterating Understanding of a Problem,” Operational Note 2 in this series for more details on this process. 4 For more on positive deviance, see Pascale, Richard, Jerry Sternin and Monique Sternin. 2010. The Power of Positive Deviance: How Unlikely Innovators Solve the World’s Toughest Problems, Brighton Mass: Harvard Business Review Press. 2 is often focused on ensuring compliance and issues. This left teachers feeling that they communicating guidance rather than better had no support if they faced challenges in understanding the situation on the ground. All the classroom. three members of the MELAYANI team that undertook the school visits were impressed • Support internal working groups and with the approach of listening to headmasters learning. All schools, in theory, have and teachers and allowing them to explain “mini teacher working groups” (kelompok what they thought was going well. Key findings kerja guru- mini – KKG mini) which are an included: opportunity for teachers to learn from each other. Where these were functioning, a Headmasters play an important role in senior or recently trained teacher6 would motivating teachers and encouraging them share their knowledge on a particular topic. to continue learning and solving problems. Limited funding for this activity is provided To support teachers in the classroom, it is by School Operational Funding (Bantuan important that they: Operasional Sekolah--BOS). In exceptional schools, the headmaster identified a need • Undertake routine supervision. This and hired an outside trainer once a year. means that headmasters come into the classroom, watch the teacher delivering a • Engage the school committee around both lesson and interacting with students. It also budget discussions and relations with means that they give feedback on areas parents. Budget transparency was important for improvement and provide support to for both teachers and parents. The timely develop a plan for improvement. In poorly arrival of books and materials was critical performing schools, headmasters did for teachers, who were further dispirited if not undertake supervision, or did it from they did not receive them and did not know outside of the class (looking at paperwork if they had been ordered by the headmaster. alone). In those situations, headmasters indicated that they did not want to create The team noted that teachers could also be tension by criticizing teachers, or were active in helping each other, even across schools, simply uncomfortable providing feedback.5 but their motivation to do so was often strongly influenced by their school environment. Currently, • Hold routine meetings focused on teachers are particularly confused about the classroom issues. Teachers benefitted from new curriculum, as it takes a different approach opportunities to discuss challenges that to teaching and learning, and has new lesson they have in the class with other teachers planning and reporting requirements. and the headmaster. In poorly performing locations, meetings did not happen at all While there were examples of innovation and or focused exclusively on administrative good practice coming from headmasters and 5 Headmasters are often promoted from the ranks of senior teachers. Some of them described discomfort providing feedback to individuals who were their peers for many years. 6 Due to funding constraints, often only a limited number of teachers from a school are trained on a given topic. The government is currently rolling out a new curriculum, and many teachers mentioned needing additional support to understand it. 3 teachers, this was less the case with respect to Using Existing Delivery school monitors.7 In the majority of locations, Mechanisms to Support the school monitors had not visited the school Impactful Change recently (in some cases for years) and where they had, they often just signed the guest book, Existing but dysfunctional working groups of but did not interact with teachers in any way. headmasters and teachers provided a potential Given that the MELAYANI team was composed opportunity to take a more systematic approach primarily of school monitors, this last finding to supporting positive interactions within led to some reflection about school monitor schools. The government MELAYANI team was attendance and performance. composed of a small group of school monitors interested in change. They realized that they Positive change happens despite, rather could use many of their routine activities than because of, district systems of support supporting headmasters and teachers to for educational management. The “islands improve school-level interactions. They looked of excellence” uncovered by the field visits to three existing forums: provided excellent examples of positive interactions between headmasters, teachers 1. the teachers working groups (kelompok kerja and school committees. They are consistent with guru – KKG), held across a group of schools; other findings indicating that engaging all school 2. headmaster working groups (kelompok kerja level actors, or “whole school change” is most kepala sekolah – K3S) held across the same effective in supporting innovation and learning. group of schools; and However, these practices were generally 3. the mini teacher working groups (KKG mini) being implemented individually, independent which are held within each school. from support or motivation from the broader educational system. These forums are meant to be supported by school monitors but are often not implemented The discovery of positive deviants reframed the for a variety of reasons: funding challenges, challenge for the team. With several examples school monitors not showing up, teachers/ of good practices in hand, the “solution” that the headmasters not joining as they don’t find the team sought changed from what to do to help activity useful, and/or only using the meeting to teachers in the classroom to how to systematize develop annual test materials. successes by sharing and refining practices that they could already see working in Belu. Focusing One member of the Belu MELAYANI team, Ibu on the positives helped the MELAYANI team feel Agus, was herself a school monitor. She decided empowered and motivated, while also looking at that she could use her schools and working groups the negatives showed them where they needed as a pilot to better understand how to make to focus. them more effective. Drawing on discussions 7 School monitors are employed by the district education department to provide technical support and oversight to groups of schools. In theory, they bring expertise both to headmasters and teachers and provide assistance through school visits and facilitating discussions across schools. 4 with teachers during the school visits to be well briefed on topics (and managed undertaken by the MELAYANI team, Ibu Agus if they got off point) and time needed to started focusing on themes that headmasters be scheduled for headmasters to grapple and teachers themselves had identified during together with how they would apply lessons those visits. She also set aside time after each to their schools and staff. working group to reflect on the experience with the MELAYANI coach and participants. Capturing Ibu Agus’ reflections in a way that can These explicit discussions helped identify how be communicated to other school monitors will the working groups themselves could be most be important if they are to also implement a impactful for teachers and headmasters. better working group approach. The pilot revealed three characteristics Expanding the important for effective working groups: Authorizing Space for Reform 1. The most critical aspect was ownership and relevance of the topics covered by the PDIA methodology emphasizes the importance working group participants. Rather than of an authorizing environment for reform— using the agenda planned by the education paying attention to generating support from department, Ibu Agus focused the working stakeholders inside an organization. Establishing groups on needs identified by teachers and an authorizing environment can be harder headmasters during the field visits. than it would first appear, even within a single department. 2. In line with findings about the importance of headmaster support for teachers, she While Ibu Agus was working hard to support realized that the content of the teachers’ change in her working groups, both she and and headmasters’ working groups needed the rest of the MELAYANI team were aware that to be linked. For example, in the school their ability to effect greater change was limited visits, teachers had raised challenges in by their positions as school monitors. School understanding the lesson planning approach monitors are staff of the education department, under the new curriculum. Ibu Agus worked but they report to the head of the department to ensure headmasters understood the and do not sit within any of the education approach, and then helped them talk sections. Monitors generally do not develop together about how to support staff lesson their own materials, so without support from planning. the sections that plan the working groups, it is difficult for them to scale up good approaches 3. Preparation was key to the effectiveness independently. While monitors are generally of groups. The working group meetings senior (many are former headmasters), not needed more planning and management everyone in the education department thinks than she had previously realized: topics that they work effectively. Indeed, budgets for needed to be sufficiently focused to allow for the Belu school monitors had been reduced in useful discussion, external speakers needed the last budget for that reason. 5 The government MELAYANI team needed to a political directive from the highest level. Ibu convince key staff in the education department Agus planned a headmaster’s working group to that the approach being piloted by Ibu Agus discuss key elements of leadership, including was useful and was worth expanding. The supervision and providing constructive feedback team had struggled to fully engage key staff in to teachers. She invited Ibu Oi, who was working the education department, such as the heads on the headmaster training, to the working of primary or secondary schools and staff of group. Ibu Oi is also in charge of planning the their teams. While the head of the education working groups for primary school teachers. She department was nominally supportive of the was impressed by the positive changes she saw MELAYANI team, providing them with platforms in Ibu Agus’ headmaster working group. Ibu Oi to discuss their findings, the level of his support brought the changes to the attention of her boss, for actual reform was not entirely clear. the head of the primary school section, which brought further opportunities for discussion of Establishing broad awareness of their work led to the MELYANI team’s reform ideas. an unexpected opportunity to generate broader support for their ideas in a politically salient While it is difficult to point to a single action way. Independent to the work of the MELAYANI that prompted change, at least three practices team, but consistent with the Bupati’s8 focus on contributed to a significant shift in the relationship education, the Bupati changed the process of between the MELAYANI government team and recruiting headmasters from one of appointment the primary school section of the education to a competitive process. In parallel, as part department: (i) continual sharing of information of a broader strategy to engage the education of the MELAYANI team’s work with the broader department, the MELAYANI team had been staff of the education department, (ii) support continually sharing results of their investigations for existing reform efforts (headmaster training) and analysis. The primary school section by the MELAYANI team and (iii) demonstrating of Belu’s education department, which was value through pilot activities. Throughout, the working on the recruitment process, used the MELAYANI team was consistent in emphasizing MELAYANI team’s findings about the importance their focus on improving the quality of education of the role of the headmaster to bolster their in Belu which meant that they started to be own argument to improve the training provided “known” for something. to new headmasters. They wanted to include “leadership” in the headmasters’ training, an Getting Started Through idea which found political support, but which Practical Iterations needed further work to operationalize. The initial discussion between the MELAYANI Translating the concept of headmaster leadership team and the primary school section on into practical and concrete behaviors became headmasters’ leadership developed into an entry point for linking the MELAYANI team’s discussions on a wider range of issues. This work to broader discussions happening within further engagement of the broader education the education department which responded to department staff empowered and invigorated 8 Elected district head. 6 the Melayani change process. Seeing the Finally, Ibu Agus and Ibu Oi reached out to the value from Ibu Agus’ pilot, Ibu Oi asked for head of the primary school section to consider her assistance in planning the primary school funding. There are two existing sources of funding teachers’ and headmasters’ working groups. As a headmasters’ and teachers’ working groups: result, Ibu Agus and the MELAYANI coach sat with one from the (national) operational funding for Ibu Oi to ensure that the working groups could schools (Bantuan Operasional Sekolah), which is be adequately planned for the remainder of the planned and managed by schools themselves, year. For the first time, the head of the primary and one from the local government budget. school section also joined the discussion and was Monies from both sources are often released more active in considering how the approach well after the start of the financial year, though might work elsewhere. Unfortunately, the local government funding tends to be more discussion also revealed that the headmasters’ delayed than national funding. However, schools working group was overseen by the workforce are often using monies that should be allocated section in the education department, while for these meetings for other things. While the primary school teachers’ working group by the head of the primary school section had been primary school section. Linking up the two would aware of this diversion of resources, she had not be more difficult than expected. been not concerned about it, since she did not feel the working group meetings were useful. As a start, Ibu Agus, Ibu Oi and the MELAYANI With the “proof” provided by Ibu Agus that the coach turned to planning the primary school working groups could be used more effectively, teachers’ working group, so that they could the head of primary school section was happy to bring a proposal to the other section regarding support a reminder to schools that they can (and headmasters’ working group. The normal process should) use existing budget to support working for planning the working group meetings is to set group meetings. the frequency (in this case every month), and the topics for all of the meetings at the beginning Improving Last Mile of the year. Based on her experience so far, Ibu Process Quality—Moving Agus knew several topics that the teachers and Beyond Form Without headmasters still wanted to cover, so she could Function develop a plan for the next few months. However, she proposed that once those topics had been Improving the delivery of basic services like covered, she should talk to the teachers again health and education which are provided or use the results of the meeting evaluations through thousands of decentralized delivery to determine new topics. Ibu Oi agreed to this points is particularly challenging. The high level approach. This is a simple but important change of transaction volume (more than a million which handed more ownership of working group teachers, teaching different lessons each day), forums to teachers themselves, giving working the high level of discretion involved in the group participants the ability to make the content teaching function, coupled with the low level of more relevant to their day to day challenges. monitorability make it particularly challenging to implement improvement. Even with a good 7 curriculum, sufficient school books and well- it became clear that she needed to spend maintained infrastructure, creating an effective more time working through the details of how learning environment in the classroom depends they would be implemented. For example, very much on how teachers are supported the working group on leadership, while well and supervised. It follows that the quality of received by the education department, proved the processes which are aimed at supporting somewhat problematic in implementation. and supervising teachers to make this kind of Although Ibu Agus and the MELAYANI coach felt improvement matters. Processes which match that they had provided enough clarity around the form but do not perform the function will the concept of “leadership,” the resource person not stimulate the desired change. that they had engaged (a headmaster himself) delivered a two-hour lecture on very general The work of the Belu MELAYANI team reveals types and principles of leadership (including a that for ‘last mile’ of district to school level, discussion of Machiavelli) rather than a practical two kinds of processes matter. First, much information that might help a headmaster in her service delivery improvement takes place in or his day-to-day school management. Because workshops and meetings. Their effectiveness the resource person was a highly respected varies dramatically, but is a crucial factor individual, no one wanted to ask him to be more in supporting effective change. Capacity concrete or shorten his presentation. However, development of teachers and headmasters his lecture was followed by a useful discussion through various working groups are a key between headmasters about the real challenges medium for influencing the quality of teaching— that they face in their schools. but there is a high risk that these workshops can themselves be almost completely ineffective in In debriefing discussions between Ibu Agus, achieving that objective if they are either poorly the Belu coach and the World Bank mentor,9 designed or implemented. The second key there was agreement that the use of “experts” process for influencing the quality of teaching needed to be more carefully guided: experts is the interaction between school monitors should be briefed ahead of time, and stopped if and teachers during school inspections. The they go too far off topic. Topics for the working challenges of service improvement therefore groups also needed to be narrow enough to revolve around two challenges: (i) how to deliver allow for a focused discussion. In addition, the a quality workshop, and (ii) how to achieve working group agenda needed to include more some level of consistency of implementation by time for headmasters to talk to each other school monitors who have very different levels and ask questions of resource people (when of motivation and educational philosophies. available). Reflection on what worked and did not work in this event allowed Ibu Agus and Even though Ibu Agus thought carefully about the coach to think more carefully about how the content of the working group meetings, to manage the meetings themselves. Going 9 MELAYANI coaches are trained in the MELAYANI/ PDIA approach. They live in the locations that they are supporting and provide day-to-day facilitation and assistance to the government-led MELAYANI teams working on their chosen problems. Coaches are supported by mentors (World Bank staff and senior consultant) with a deeper knowledge of PDIA, who provide assistance throughout the process, providing analytical guidance, trouble shooting and support in implementation of the process. 8 forward, these reflections will be important section has decided to focus on the bottom 50 to communicate to other school monitors if schools. They are using test scores to look more they are to also implement a better working carefully at what ‘poor performance’ means, group approach. Perhaps the most important focusing on schools with low scores across change to communicate is the importance of the board, rather than a mix of high and low adopting a stance of ongoing critical reflection clusters. In an initial meeting with the selected and continuous improvement about the way schools, they invited some better-performing innovations, initiatives and programs are schools to share good practices with their lower- delivered. performing counterparts. Establishing a Virtuous Focusing on poor performers coupled with Cycle of Iteration and listening to the view of frontline workers led to Innovation another initiative. While initial discussions were focused on test preparation, they also asked Small successes became the foundation for more teachers what a better training approach would ambitious reform. Involvement of Belu education look like. In response to what the teachers department staff in the MELAYANI team said, the primary school section has expanded generated increased momentum and a greater the time for training for teachers in grades 1, awareness of opportunities for innovation and 3 and 6. To implement this decision to expand reform. The primary school section began to the duration of training, they plan to use expert develop a clearer vision of what they wanted to teachers, who have already received several achieve. Discovering examples of the types of rounds of training, to help. changes that the primary school section could make allowed them to see opportunities more Focus on how teachers teach, not easily in their daily work. These ranged from just what is taught small changes, such as asking certain schools to As the MELAYANI team engaged more closely share good practices at the routine all-school with the education department, it became meeting, to re-thinking the routine training possible to ask increasingly difficult questions approaches. The MELAYANI coach has noticed about drivers of educational quality in Belu. a huge difference, noting “now I don’t call them With an increased focus on training, the [the MELAYANI team], instead they are calling MELAYANI mentor was able to revisit questions me with ideas.” Some key approaches informed raised in initial discussions about how the their increased effectiveness in stimulating education department approached how to positive change. teach (approaches to developing literacy and numeracy) as opposed to what to teach (the Focusing on poor performers curriculum). The primary school section of the One set of changes worth highlighting is the shift education department was interested in this in focus to lower performing schools, selected question. She confirmed that current support for with more awareness of teachers’ needs and teachers focuses primarily on the content of the locally-available resources. The primary school curriculum, and noted that she was not entirely 10 Members of the MELAYANI team and Ibu Agus’ pilot headmaster working group, respectively. 9 sure who would be responsible for addressing for the implementation of the teachers’ “how” to teach, though speculated that it may working groups, which the school monitors are be included in certification trainings provided by required to support. Following changes to the the workforce division. practices of holding teachers’ working groups, school monitors are now required to sign the Discussions with selected school monitors and attendance register and fill in a reporting format headmasters10 indicated that the primary school on the working group discussion itself. Direct monitors could draw on materials from another sanctions may not be available to the team development program which focused on working which is promoting the reforms, but evidence of with teachers on how to teach literacy and attendance at working groups is still an important numeracy. This discussion supported the team first step toward accountability. to become more aware of the need to focus on early grades, working to ensure children have a What Have We Learned? foundation of literacy and numeracy, rather than focusing only on test preparation in grade six for The examination of both good and bad quick test results, which had been the approach school practices was important for the team, to date. The primary school section immediately as it simultaneously demonstrated what proposed using existing trainings as a place to was possible in Belu as well as highlighting start talking about how to teach and are starting gaps that still exist. The way that the team to think about how they can best target early approached understanding the problem “built grade teachers. in” an examination of positive deviance and thus demonstrated potential solutions. Seeing Instituting a culture of continuous positive examples motivated the team to think improvement about how they could contribute, made it easier With multiple change responses being for them to discuss solutions with others and implemented, it is useful to remember that not helped them have a vision to work toward. everything works. Within the Belu education department, a tension remains between the An iterative approach can help local primary school section that is supporting governments improve existing systems. The innovation and change, and the workforce 2018 World Development Report: Learning to section that is not engaged. Oversight of school Realize Education’s Promise11, advocates an monitors (who formally report to the head of adaptive approach that encourages innovation the education department) is still weak and and agility in developing local solutions to local politically fraught. However, the Belu case problems in education. The experience in Belu study shows what can be achieved by just one demonstrates key benefits of this approach: section, if it is truly engaged and creative. For example, although the primary school section • Iteration allows governments to start does not have any power to directly oversee working with what they have, rather than the school monitors, they are responsible “starting from scratch.” There is a tendency 11 World Bank. 2018. World Development Report 2018: Learning to Realize Education’s Promise. Washington DC: World Bank. 10 to jump to new program approaches when it for that purpose. There is a risk that local becomes clear that something is not working. government officials only think about changes to However, by taking a moment to determine their programs when they are in the (very busy!) specifically what was and was not working throes of developing yearly plans and budgets. in existing approaches, opportunities for This case shows that understanding key issues incremental improvements were revealed may require an investment of time, but may also which could have substantial impact. lead to a better use of existing funding without the need for increased or altered allocation. • Iteration is often cost-efficient, as many Improving existing monitoring and evaluation challenges are related to poor incentives approaches to respond to identified questions or weak implementation. In this case study, (and the use of the results that the produce) working groups were not seen as relevant may help local governments to more routinely for teachers and headmasters, since their take a similar approach. content was focused on what was important for district level administrators (developing Reforms often focus on overarching changes test material) rather than what was important to program structures, but details matter a lot to teachers and headmasters (improving in implementation, particularly where they classroom learning and teacher supervision). concern the interactions from the district to Clarifying the goal of the working groups and the facility level. This disconnect on the “last re-orienting them toward learning outcomes mile” is often a challenge for civil servants may already be a step forward. since they may need different skills to navigate their bureaucracies and to understand • A gradual approach rather than a ‘big bang’ implementation. A small example from this case approach to reform provides time for the is teacher and headmaster ownership of their development of momentum and support, working group content. District staff approached which in turn can expand opportunities for planning the working groups as a set of activities reform and allow the group to press into in a workplan that needed to be completed. more difficult problems. The cultivation They were not required to define the content in of support from the Belu primary school the plan but tend to do so in order to “finish” the section resulted in a much more dynamic plan so that all parts can be evaluated. However, approach to reform as staff of that part of seeing the value of teacher and headmaster the education department started to see engagement in choosing the topics, the opportunities across their portfolios. In education department modified their approach addition, it made them more receptive to to fix the time of all of the working groups (every discussions about additional challenges that month), but only selecting content for the first they might be able to address. four meetings, leaving the rest to be decided following discussions with participants. System improvements do not necessarily require changes to the plan and budget. In It requires a certain amount of tenacity to this case, money was available to finance the remain focused on implementation, and a working groups, but was not being utilized change of mindset to adopt a critical approach to 11 implementation realities. Within the Indonesian the school monitors, an issue that had been planning, budgeting and implementation cycle, an ‘elephant in the room’ for some time. For there is little space or support to look closely members of the Belu education department, it at implementation: monitoring and evaluation can be exciting to see progress, but also daunting are both underfunded and not particularly to deal with what may seem to be multiplying useful. Monitoring is generally understood challenges that need to be overcome to effect to be checking program implementation meaningful change. against design and evaluation is often focused on financial disbursement. Neither examine Continual communication of goals and results whether a program is working or not. Often, helped broaden support so others could it is assumed that if a policy is developed or identify opportunities for the team. There is program change made, it will be implemented often an assumption that local actors can identify as designed. There is little awareness of the all opportunities and act strategically. In reality, possibility that policies may look very different many actors (including those on the MELAYANI on the ground than they did on the pages of a team) have partial information or limited power. regulation or program design. If there is the Consistent communication from the team sense that an activity is not working, then the allowed external actors (the primary school assumption is often that it should be scrapped, section) to identify their own opportunities that and a new approach designed to replace it. The were aligned with the MELAYANI team’s work. headmasters’ and teachers’ working groups are a good example of this. They were already being Bringing These Lessons effectively de-funded in response to a perception to Scale that they were not working, but a closer look at how they could work better changed the mind of The process described in this note – digging education department officials and stimulated into the technical minutiae of a specific issue enthusiasm to work on improving them through and working through the details of reform – more focus on implementation. are less amenable than previous steps to the application of a generic tool to be used by It also requires resolve to work through that local governments. This is due to the fact refining and scaling up an improvement, even that solutions need to be tightly linked to the within a single district. The case demonstrates underlying problems uncovered in previous the potential of multiple small coordination stages of the analysis. and management challenges to improve the implementation of a routine activity. It shows However, the central government can play an both the power of iteration for improvement, important role in ensuring that there is space for and that every step forward has the potential to local governments to examine their performance raise new questions. For example, an improved and work on scaling up improvements in the approach in the working group finally brought following ways: more serious attention to the performance of 12 In general, the findings in Belu echo international discussions: World Bank. 2018. World Development Report 2018: Learning to Realize Education’s Promise. Washington DC: World Bank. They also are similar to those found elsewhere in Indonesia: See Arlianti, Rina and Sheldon Shaeffer. 2019. Education Innovations in East Java: A review and analysis of INOVASI’s stocktake study. Jakarta Indonesia: INOVASI. 12 Central government actors could reward Central government program managers could districts for concrete examples of scaling out think more carefully about how to create the improvements. There is currently a strong focus “preconditions for success” in the form of on innovation at the local government level. functioning school communities. The findings The central government is very interested in of the team in Belu echo the conclusions of documenting new practices. If it took a similar other work that demonstrate the value of interest in documenting and sharing not just “strong leadership, empowered teachers and the content of good practices, but how good a close collaboration within the whole school practices spread, they could provide some community”12 in supporting innovation and important motivation to districts to look at how growth. In addition to thinking about the processes for stimulating change. technical needs of teachers and headmasters, attention and support needs to be given to how they interact. This, in turn, may require a shift in mindset at the national level as to how change happens. This operational note was written by Karrie McLaughlin with input from Kathy Whimp. Thanks to reviewers Rachel Lemay Ort, Jumana Qamruddin, Michael Woolcock and Noah Yarrow for feedback, as well as for the support and insight of World Bank MELAYANI team members Ahmad Zaki Fahmi and Noriko Toyoda. Acknowledgement and appreciation is extended to the MELAYANI coach in Belu, Mikhael Leuape, and the team in Belu who are working to improve the quality of education in their district. 13