~ .. y CONNECT COLLECT 99013 t\. f~~~~A~~ FEBRUARY 2C·28, 2014, Wuhln11on, DC PROJECT EXPO Science of Delivery AT WORK http://sod Introduction What does it take to deliver results, measurable improvements in quality of life, every time in every country? We all know that sound technical knowledge is important. But we also know that to deliver outcomes for citizens we need to combine that technical expertise with on the ground delivery know-how. Relentlessly focusing on the outcomes, deliberately taking multi-stakeholder and multi- sectoral perspectives, systematically using and collecting evidence, developing a deep understanding of the political context and individual incentives, consciously building in feedback loops and course correction mechanisms - those are some of the approaches that teams use to get the delivery right. By capturing the learning from those approaches, we are building a collective and cumulative body of operational knowledge and know-how that can help development practitioners deliver better results. That is the science of delivery at work. Relentless focus on citizen outcomes a. Identify the nature of the problem based on a thorough understanding of citizens' demands and the local context. b. Design a context-sensitive strategy to relentlessly focus on outcomes, defined as measurable welfare gains of citizens . c. Pay attention to other factors , beyond the project, that influence citizen outcomes. Multi-dimensional response a. Facilitate multi-stakeholder coalitions and multisectoral perspectives to identify problems and solutions. b. Convene development partners and build on their comparative advantages. 2 Evidence to achieve results a. Use the best available evidence to identify the nature of problems and to develop solutions. b. Develop local evidence, through experimentation and other strategies, to refine solutions. c. Collect, throughout the project cycle, evidence of results to enable course correction. d. Contribute to the global body of knowledge with the evidence accumulated during implementation. Leadership for change a. Identify the incentives that motivate individual behavior change and integrate them in solution design . b. Understand the political economy and drivers of change to determine how and when to best engage with the client. c. Ensure leadership support and facilitate coalition building among different stakeholders. Adaptive implementing a. Develop an adaptive implementation strategy that allows for iterative experimentation, feedback loops, and course correction . b. Build a committed multidisciplinary team with the right skills, experience, and muscle memory. c. Maintain the capacity for reflection and a diagnostic mindset. Take action , step back, and assess the results of the action . The projects featured in this brochure represent a small sample of projects that illustrate the use of the approaches listed above. Some have drawn on one main approach , others on some, and others on all of them . 3 Argentina: Road Safety Contact: Veronica Raffo, E-mail: vraffo@worldbank.org What was the development challenge? Road crash fatalities are an epidemic in the developing world , the number one killer of the young, with huge economic and social costs. These fatalities can only be stabilized and then reduced if all actors work together to strengthen institutions, ensuring strong and sustainable multi-sectoral coordination and action . While past projects have attempted to lower the mortality rate in Argentina, these have historically been implemented as fragmented single-sector initiatives with outcomes too small to be measured, and were often one-off projects with no follow-up activities. The Road Safety Project supported the newly created Argentina Road Safety Lead Agency (ANSV) to unify safety efforts and combat road crash fatalities. What approaches did this project use to deliver solutions? To attack this epidemic of road crash fatalities, the Road Safety Project provided constant support to the ANSV, enabling it to deliver its institutional management functions and build partnerships to save lives. The project gathered evidence to continuously retrofit project design and implementation, using expertise, methodologies, and systems developed by countries that perform well at ensuring road safety. Since credible, accessible performance data and related crash analyses are crucial to lowering road crash fatalities, the Road Safety Project made connections across sectors to gather this information and ensure that all relevant analysis was being brought to bear on the problem. It also leveraged strong partnerships with police agencies, the health sector, and the OECD's International Road Traffic and Accident Database (IRTAD) Group to push forward life-saving initiatives, ranging from highway improvements in high-risk corridors, to innovative safety education campaigns. What lessons does this project offer for others? In a key lesson for how to generate better citizen outcomes, this project demonstrated that well-focused national leadership and supporting resources can catalyze effective 4 partnership engagement with provincial and local governments, NGOs and the private sector to achieve consensus on desired results and the measures to achieve them . Another crucial innovation has been the Incentive Fund (with output-based disbursements), which has enabled the ANSV to engage in a purposeful way with a wide body of partners, bringing them together to focus on road safety. An additional lesson is that South-South dialogue and action on a regional basis can be stimulated when best practice measures are taken and given high visibility. The Argentina Road Safety Project has created interest in other Latin American countries facing the challenges of creating a lead agency to manage their national road safety effort. It has also spurred action to create a Regional Road Safety Observatory. Argentina's work in partnership with the IRTAD Group and its Spanish counterparts is serving as a model to assist the related development of regional protocols - offering its neighbors access to a useful lens for relentless focus on citizens' outcomes. Source: World Ban k Group. 5 Tanzania: Productive Safety Net Project Contact: Ida Manjolo, Email : imanjolo@worldbank.org Manuel Salazar, Email: msalazar2@worldbank.org What was the development challenge? In Tanzania, despite compulsory education for children from 5-12 years, those from poor households often either drop out of school or are not enrolled in school because their parents cannot afford to pay for tuition fees and other required school materials and contributions. In such instances, children are forced to engage in petty trading or work as laborers so that they can bring food to their families. Those who continue schooling, do so in difficult conditions, without proper school uniforms or books, let alone proper nutrition. The Tanzania Social Action Fund (TASAF) sought to alleviate these conditions by piloting a safety net program through a community based conditional cash transfer project which supported the elderly and children from poor households. What approaches did this project use to deliver solutions? Given the substantial detrimental effects that low incomes had on children's education, the TASAF project focused on providing modest cash transfers to selected poor households with children of school age, and the elderly. In addressing the root of the education problem, the program transferred cash conditional on the pupils from cash- transfer-beneficiary-households actually attending school , as well as requiring that the elderly attended clinics for check-ups. Once a month, special sessions were organized on capacity building , health, and effective use of the cash received. After three years of the pilot, results showed that households initially identified as poor had improved their situations substantially, both in terms of increased asset ownership and empowerment. Impact assessment results showed that before the program, poor households could afford one meal per day at most, but after being enrolled in the program could afford three meals a day. Children from these households recorded good performance in school and truancy was reduced . Children who were going to school 6 barefoot with inadequate school materials and tattered uniforms are now proud to report that those days are long gone and they have a sense of belonging in the community. What lessons does this project offer for others? By addressing the root of the education problem in Tanzania, the project was able to achieve positive outcomes not only for children's education, but also in terms of income, health and community based safety nets, so that the CCT had a wide-ranging set of cross-cutting positive effects. The CCT, and community training sessions and expert facilitation enabled beneficiaries to become enterprising. Some used part of the transfer to pay for their community health fund insurance. Others started small businesses to generate more income so as to become self-sufficient and less dependent on the cash transfers. This project has provided input to the design of the current Productive Social Safety Net Project, and the cash transfers are being rolled out to all 161 districts nationwide to over 900,000 households. Source: Tanzan ia Social Action Fund (TASAF). 7 Bangladesh: Partnership for Clean Textile {PaCT) Contact: S. Akhtar Mahmood , Email : smahmood@ifc.org What was the development challenge? While Bangladesh 's textile industry is an important driver of the national economy, it is also a heavy consumer of water, and a major polluter of water bodies around it. Suboptimal use of chemicals and inadequate effluent treatment is contaminating surface waters, negatively affecting the lives of nearly 12 million people. Competitiveness and further growth in Bangladesh 's apparel industry will critically depend on achieving efficiencies in water use and reducing discharge . In the PaCT project, firm-level advice from I FC is complemented by multi-stakeholder engagement leading to policy and regulatory interventions , and collective action by industry. The target is to induce change in 200 factories and lay the ground for changes in the rest of the industry after the project closes. What strategies did this project use to deliver solutions? A multipronged approach is needed to induce adoption of good practices. In this project, the IFC used its convening power to bring together many actors around a shared agenda. PaCT embodies a multi-stakeholder approach by working to link buyers, factories, financial institutions, sector associations, and local communities and governments for coordinated action on water sustainability. Through PaCT, global apparel buyers develop and apply harmonized procurement guidelines that integrate water sustainability criteria. The leading industry association, Bangladesh Garment Manufacturers and Exporters Association (BGMEA), is partnering with PaCT to establish a Textile Technology Business Center, to support adoption of CP measures by sharing technology case studies and associated technical and financial information. A Textile Sustainability Platform is being formed to reach out to national and sector- level stakeholders to address water sustainability challenges in the sector. And in partnership with leading factories, PaCT will work with business, government, and 8 communities in selected textile manufacturing clusters, to develop and put in practice a vision for Clean Clusters. What lessons does this project offer for others? This project shows that in transforming a sector, relationships - among governments, communities, and business - must also be transformed . World Bank Group leadership brings together multiple stakeholders to change incentives and behavior of diverse actors to move forward a change agenda . PaCT faces major challenges in implementation , but in addressing these challenges is working on a number of different tools to address these, including survey instruments, awareness building materials, assessment methodologies and guidance notes for clients . These , and other knowledge products such as case studies, can also be shared with others. Source: PaCT Project Team , World Bank Group. 9 Indonesia: National Program for Community Empowerment in Rural Areas Contact: Sonya Woo, Email: swoo1@worldbank.org What was the development challenge? The government of Indonesia needed to cost-effectively deliver basic services and assistance to the rural poor across its vast territory. The government, with World Bank funding and technical assistance, put in place a community-driven development (COD) system, operating via direct transfer of grants from the national treasury to collective community bank accounts, to enable communities to undertake development activities. Government subsequently scaled up the program - first called Kecamatan Development Program (KDP) and later the National Program for Community Empowerment, or PNPM-Rural - nationally and adapted it to deliver a broader range of services in line with evolving local priorities. What approaches did this project use to deliver solutions? Following a series of mixed methods diagnostics, called the Local Level Institutions studies, which uncovered latent capacity possessed by communities to solve problems collectively, the government implemented the COD. This drew on community problem- solving capacity, with facilitation support, to cost-effectively deliver services in rural communities across Indonesia. The Bank led in establishing the PNPM Support Facility (PSF) to convene multiple government and international development stakeholders, enabling the Bank to maintain a strong, continuous presence locally. The Bank drew on deep relationships from its sustained presence on the ground, which was indispensable in supporting the implementation and scale-up of a national COD program in a country as large and diverse as Indonesia, and it allowed the Bank to engage deeply with reform-minded policymakers during windows of opportunity. The compilation of a large body of evidence support also helped to build crucial sustained commitment among World Bank management. 10 The use of rigorous evidence was vital throughout the program. Randomized control trials were used , such as for the differential impacts of incentivized versus non- incentivized versions of PNPM Generasi , a community cash transfer program delivered through PNPM-Rural aimed at improving access to priority health and education services. Findings led to the mainstreaming and scale-up of PNPM Generasi. A study of the participation of marginalized groups in PNPM led to the design and roll-out of a special program of support to national NGOs active in engaging sex workers, ethnic and religious minorities, prisoners, and other marginalized groups in development activities. What lessons does this project offer for others? This project shows the importance of drawing on rigorous evidence to inform program decisions. The KDP, now PNPM-Rural, prioritized the use of rigorous mixed-methods research to design the program and measure impacts, allowing for successful scale-up. Impact evaluations have provided rigorous evidence of PNPM-Rural's impacts on increases in consumption among the poorest, and improvements in access to basic services. Source: Curt Carnemark, World Bank Group Flickr. 11 Afghanistan: Rural Enterprise Development Program Contact: Kamran Akbar, Email: kakbar1@worldbank.org What was the development challenge? About 80 percent of Afghanistan's population depends on agriculture and livestock for a living. This project sought to improve the incomes of rural men and women working in the agricultural sector by helping them access financing and develop sustainable enterprises. However, the project's progress was initially delayed, as it had ambitious goals with limited capacity and experience. High levels of bureaucracy and slow disbursement also contributed to the delay. To address this, the project underwent a mid-term restructuring that enabled the project to reach goals more effectively. The result was increased access to village level financing , reaching more than 51 ,000 rural poor in 22 districts, surpassing $US1 .5 million. The project also helped establish nearly 200 small and medium enterprises across the country. Over half the savings group members are women , and the number of female borrowers surpasses that of men . What approaches did this project use to deliver solutions? In May 2013, the project was restructured through reduction of the project's scope and revision of targets, given Afghanistan's prior lack of experience in implementing such a livelihood project. The organizational structure and staffing was transferred from public institutions to private providers. In adapting to the local challenges, the project incorporating Pasto-speaking facilitators to work with local communities, and included migrant and disabled people in the enterprise development pilots. Local consultants were recruited and tra ined in order to build local capacity, as well as to have a pool of resources for future similar projects. 12 What lessons does this project offer for others? Since the restructuring mission of May 2013 , the project has shown considerable progress. The pace of activities has increased and the project is moving towards realizing most of its revised targets . Substantial progress in enterprise development among pastoral nomads has been made, transformational actions with regard to fixed- salaries staff have been undertaken , and training of facilitators - of whom about half are women - has been completed . The successful adaptation of this project was made possible by identifying problems early on and discussing them candidly, and making appropriate adjustments to the local context. It was also critical to put in place incentive mechanisms to ensure that performance in implementation met expectations, with improved monitoring systems playing a key role. Source: Kamran Akbar, World Bank Group. 13 Nigeria: ICT for Social Accountability Contact: Paula Rossiasco, Email : prossiasco@worldbank.org What was the development challenge? The development challenge was to ensure better service provision in health and agriculture by effectively collecting feedback from World Bank project beneficiaries in Nigeria. The project had to avoid a common pitfall : tools for collecting feedback from project beneficiaries often fail to represent citizens' voices and are disconnected from decision making processes. The system, called MyVoice, will enable decision makers to make accurate, timely, and efficient programmatic decisions based on real-time beneficiary feedback via SMS messages to a publicly advertised short code . Creating mechanisms for using ICT to transmit information in real time up the management stream - to both World Bank staff and government counterparts - allows for immediate and timely aggregation of voice and facilitates the use of this information to improve service delivery. What approaches did this project use to deliver solutions? The MyVoice system relied on evidence from user experiences to ensure that the desired result - citizen use of the ICT to enhance social accountability - occurred . Since many tools fail due to poor understanding of social context, a significant portion of project design time was dedicated to gathering evidence of user needs via an ethnographic approach. Citizens were interviewed and user experiences documented to inform program design. Other stakeholders were canvassed to understand their incentives for using information captured by the feedback mechanism, including local government and traditional authorities, local NGOs and CSOs, and service providers. The system was designed and implemented adaptively, iteratively, and experimentally, utilizing agile software development methodologies to allow for continuous prototyping and testing. The end product was a system that drew on extensive evidence of citizen needs to design the ICT. What lessons does this project offer for others? This project showed the importance of gathering extensive, multi-disciplinary forms of evidence to inform ICT design these tools often fail. The central role given to understanding the political economy of service delivery and the social context in which projects take place provides an insight into the importance of locally-gathered , multi- disciplinary evidence to unchain the potential of ICTs to aggregate voice meaningfully. At the same time, effective citizen engagement in development projects is a multisectoral issue. It requires a deep understanding of the technical design, the decision making mechanisms for service delivery, the incentives of the different stakeholders involved , the social context, and the potential that ICTs can offer to bring these different aspects together in a concise, user- friendly, and timely manner. The tool was conceived in a modular way, so it could be adapted to other operations in the Nigeria World Bank portfolio once the project context is fully understood . Source: Reboot. 15 Global Scaling Up Rural Sanitation Contact: Eduardo Perez, Email: eperez1@worldbank.org What was the development challenge? Worldwide, 2.5 billion people lack access to improved sanitation, and three quarters of these people live in rural areas. Seeking to respond to this development challenge , in 2007, the World Bank initiated a partnership with the governments of India, Tanzania, and Indonesia to start at scale projects that served as learning laboratories to provide lessons for policy changes; this was then scaled up in 10 additional countries. From base-line to 2013, WSP helped governments to scale up access to improved sanitation for 22 million and encouraged 19 million people to stop practicing open defecation in the 13 focus countries. What approaches did this project use to deliver solutions? The Scaling up Rural Sanitation program includes an iterative process to help governments develop at scale solutions, as shown in the figure. The initial countries served as learning laboratories, with lessons learned in India, Tanzania, and Indonesia being informed by real-time data collection and monitoring. This resulted in a theory of change that is now being replicated in 1O • Chent· and problem-driven engagement • Shated diagnostics additional countries. • Prog1ammatic, multi-seaor approach • Results focused deaslOfl makmg It allows TTLs in all 13 countries to apply • Capture and shanngof I Problem Identification I • MobiliZation of best global knowledge and operation talent lessons learned while ~wledge also adapting and • Synthesis of lessons learned, I Evaluation and Leaming I Help Governments • Best evidence based, globaHocal ~ Develop At Scale I Design I • Integrated rrulb-sector/ correcting course as leammgtrom multi-stakeholder atlure Solutlona nnovanve solutions they scale up, collecting • New products their own evidence and learning tools I Implementation and Iteration I locally to target effective • Technical aSSIStanee behavior changes and • Capaaty building • lteranve feedback loops, mid-course interventions. I 001Tecoons, operatK>nal learning 16 What lessons does this project offer for others? This project showed the key importance of learning from projects that serve as "laboratories," and using these lessons learned to inform both changes in policy choices , and changes to the project when it scales up. Data collection and monitoring should be designed so that they can be used by governments in real time to make programs more effective. Moreover, it is vital to make global knowledge connections through enabling South-South government exchanges, sharing state-of the art information with government partners and promoting global capacity building through e-learning and other training products. Source: WSP, World Bank Group. 17 Cambodia: Second Education Sector Support Project {SESSP) Contact: Simeth Beng, Email : sbeng@worldbank.org What was the development challenge? Drop-out rates due to economic reasons in upper primary and secondary schools have been very high in Cambodia. This program focused on the design, scaling-up and mainstreaming of scholarship programs. As a result, scholarship programs have contributed to significant increases in school enrollment and attendance (approximately 20 percentage points) and pointing the way to increased learning. What approaches did this project use to deliver solutions? The projects used systematic feedback of lessons from delivery to ensure that they were flexible, adapting to meet the needs of beneficiaries. Two rigorous impact evaluation programs shed light on the factors that maximize the impact of the scholarships on schooling and learning outcomes. The first impact evaluation program indicated that the program had large positive impacts on school enrolment and attendance rates. At the same time, the evaluation also showed that the provision of scholarships alone was insufficient to significantly improve the learning outcomes of students from poor households. The second impact evaluation studied not only program's impacts on school participation but also on achievement, a domain in which there had been a dearth of analysis. The evaluation then recommended scaling up the program by targeting students with high academic potential, while ensuring that the poorest students also continue to be included. In this way, continuous adaptation based on rigorous evidence allowed the project to address the real needs of students (and their families) . What lessons does this project offer for others? Close collaboration with the Ministry of Education from the design stage was crucial , as implementation status was regularly discussed as a result of joint field visits in order to 18 jointly identify bottlenecks and adjust implementation strategies in response to evaluation results . Regular implementation support activities also helped build the Ministry team's capacity. Source: World Bank Group Flickr. 19 Avian and Human Influenza (AHi) Contact: Olga Jonas, Email : ojonas@worldbank.org What was the development challenge? Avian and human flu raises the specter of a global pandemic that could kill hundreds of millions, make billions ill, and cost trillions of dollars. Uncoordinated actions in response to a future outbreak could be very costly. The World Bank committed substantial senior management and staff resources to develop and gain acceptance for an evidence-based, results-oriented framework with the standard-setting international organizations and other expert partners. The sectors engaged were animal health , human health, disaster risk management (whole-of-society preparedness), and communications . The Bank also provided financing of $1 .3 billion ($1 .2 billion from IBRD/IDA) for 72 operations in 60 countries, in the largest multisectoral global health program ever. What approaches did this project use to deliver solutions? The threat of pandemics is inherently multidimensional: it spans sectors while also being multisectoral as well as global and transborder, meaning that this represents a need for rapid engagement in all sectors, and for support to pandemic prevention and preparedness in developing countries. The Bank contributed pivotal knowledge on economic impact and compensation schemes , fostering alignment in the international partnership. The common framework included many stakeholders, including developed countries, developing countries, the United Nations and other international agencies, veterinary services , public health agencies, disaster risk management authorities, and others. Meanwhile, regular consultations at the national and international levels built partnerships across borders and among sectors, ensuring coherence of the global program . Because of its implementation experience in all countries, the Bank brokered pragmatic solutions, bringing together animal health , human health , and other sectors as well as donors and international agencies, increasing the overall effectiveness and efficiency of the global response . 20 What lessons does this project offer for others? Agreement on a common framework is vital to make complex endeavors succeed, especially in an emergency and where there are multiple partners, professions, and stakeholders. Also, coordination on multidimensional solutions is not spontaneous. It costs money, takes time, and requires high-level attention, but it can make the difference between success and failure. The Bank is well-placed to play the requisite integrating role by financing and supporting coordination and implementation of multisectoral programs. Source: Gerardo Bravo Garcia, Avian Flu Series, 2006, Oil & Gold Leaf on Canvas. 21 Morocco: National Initiative for Human Development Contact: Mohamed Medouar, Email: mmedouar@worldbank.org Andrea Liverani, Email: aliverani@worldbank.org What was the development challenge? The main development challenge was a lack of participatory mechanisms to identify peoples' needs for programs to alleviate poverty and social exclusion. The Moroccan government's approach to dealing with poverty and social exclusion has tended to be centrally-driven , with little room for participation , weak coordination among line ministries, and inadequate targeting , and although the Government of Morocco (GoM) dedicates 50 percent of public expenditures to social sectors, results have been below expectations. This project focused on strengthening participatory local governance and social accountability to ensure that basic infrastructure and socio-economic services meet the basic needs of excluded and vulnerable groups, particularly women and youth . Result indicators and activities of the Program Action Plan (PAP) were chosen to track this outcome, including percentage of the population involved in the participatory approach , overall satisfaction w ith the participatory process , percentage of women and youth in the local governance committees , increasing grassroots ownership of local development plans, and transparency in the identification and selection process. What approaches did this project use to deliver solutions? The project leveraged the power of participation to produce better evidence on how basic services were meeting , or not meeting, peoples' needs . This approach included activities to strengthen participatory local governance and social accountability, develop bottom up integrated strategic planning , and improve targeting. For each of these aspects, the program laid the groundwork 22 to gather rigorous evidence by developing result indicators and disbursement- linked indicators in order to measure results. In tandem with strategies to increase participation , this led to increased engagement of the target population in participatory programming and evaluation of benefits, reflecting the demand and community driven development objective of the program . In addition , the result of the impact evaluation will provide information on the impact of INDH . What lessons does this project offer for others? This project shows the importance of engaging the target population as part of the monitoring process, with beneficiaries providing feedback, together with more traditional methods of monitoring, such as impact evaluations. Thus, participatory and inclusive models of service delivery, together with capacity building and facilitation support for communities, can themselves provide the necessary feedback to inform adjustments. This, in turn , relies on flexibility in project design and implementation to allow course correction. This project also found that the use of poverty maps was very useful in enabling the targeting of resources to poor areas. Source: World Bank Group Flickr. 23 Latvia: Who is Unemployed, Inactive or Needy? Contact: Emily Sinnott, Email: esinnott@worldbank.org What was the development challenge? The central challenge was a lack of information about Latvia 's poor and unemployed, which was needed to develop anti-poverty programs and effectively target beneficiaries. While Latvia has among the highest rates of people at-risk of poverty in the European Union, stereotypes of benefit dependency and low work ethic of the poor have hindered discussions on how to tackle social exclusion in the country. As Latvia recovered from the 2009 crisis, the World Bank partnered with the government to investigate tax, benefit, and training policies to combat long-term unemployment and draw people into the workforce. The study provides background evidence to motivate government reforms to the tax and benefit system and for the development of labor market programs. The results of the study were widely reported in the media, and the Government of Latvia has stated publicly that they are being used as background to inform future policy reforms. What approaches did this project use to deliver solutions? The study fundamentally depended on gathering strong evidence to inform a sensitive debate; this effort involved numerous sectors, namely social protection, fiscal policy, education and training , and health. While the work was carried out at the request of the Ministry of Welfare and the State Employment Agency, the team worked closely with a steering committee made up of technical staff from other agencies, and working together to gather the widest possible datasets to support findings and inform policy recommendations . Indeed , big data-gathering it and sharing it for better policy-was at the heart of this effort. To examine the role of taxes, benefits and employment programs, a large panel database was created for individuals linking various government administrative datasets. This linked data on individuals in the social security, social assistance, employment I\_ 24 services, population registry and health insurance databases. The data was for 91 monthly waves from 1/2006 to 7/2012 and covered 43 percent of the total population . The rich data allowed the team to investigate benefit dependency and look at the impact of employment programs with a strong evidence base. The study contributed with strong evidence on the lack of benefit dependency in Latvia , which was widely reported in the media and used by the Government of Latvia to argue for the need for anti-poverty programs (a new development). What lessons does this project offer for others? This project showed the possibility of using big data to inform the very difficult policy debate on benefit dependency. In addition, by involving a large number of stakeholders in the study from inception to completion (with numerous presentations of initial results etc.), and making all the analysis available online and at public conferences, the project was seen as an objective study of the situation in Latvia without a political agenda . Source: iStockphoto. 25 Ghana: Disaster Preparedness and Watershed Management Contact: Carl Dingel, E-mail: cdingel@worldbank.org What was the development challenge? Recurring floods in the north of Ghana called for an improved flood forecasting system to better manage the situation, and find long-term solutions for mitigation . However, in working towards this there was no clear "institutional home" to take the lead in establishing such a system , nor was there reliable hydrological data set up to support it. This project focused on building a robust forecasting system that would be easy to operate and flexible enough to incorporate new information as it was made available. This required building trust among all relevant agencies, including developing an understanding and agreement of shared responsibilities . What approaches did this project use to deliver solutions? This project focused on a rigorous analysis and validation of the existing data and information . To adapt to changes in the program, it was crucial that the technical design of the White Volta Flood Hazard Assessment was flexible enough to accommodate a broad range of data and information to produce the most accurate flood forecasting information possible. With a flexible technical design that can constantly be re-evaluated , the available information allowed the project to adapt to new information, be more accurate, and remain state of the art. The project steering committee was anchored in the involved national organizations and allowed the project to have a forum and to address upcoming challenges and changes in the system design. What lessons does this project offer for others? This project offers multiple lessons for delivering solutions in situations characterized by a need for adaptation and coordination among multiple agencies. Anchoring the project locally, and the establishment of a well-functioning steering committee to address upcoming challenges and adapt the project design as necessary, was vital. The project 26 showed that technical designs should be built on a rigorous analysis of the available information, and should allow enough time for validation of information. It was crucial to get technical experts on the task team who could stay up to date on the state of the art technology while also focusing on knowledge transfer. Source: Water Resources Comm ission , Ghana; HKV Consultants, The Netherlands. 27 Nicaragua: The ECOM Monitoring Pilot Contact: Miguel Angel Rebolledo Dellepiane, Email: mrebolledo@ifc.org What was the development challenge? The development challenge was low productivity, and therefore low income, among coffee farmers. A significant demand existed for services to improve productivity, such as market information, financing, and agricultural extension (visits to farms and agricultural inputs). This project focused on trying to understand which services were appropriate to the local context. The IFC identified services that company ECOM could contribute that would lead to increasing income by increasing productivity. The project also found that certain tools could increase farmer yield , share of crop sold , and income; bring new farmers into the supply chain; and strengthen the feedback that farmers provide to its staff in the field across regions. All this contributed to increased income and self-reported empowerment by these farmers. What approaches did this project use to deliver solutions? In looking to identify what specific services could be appropriate to helping improve Nicaraguan ECOM farmers ' incomes, the project tested for the first time a combination of tools including the Simple Poverty Scorecard , "Constituent Voice ," and a database of farmers collected and managed by the IFC client. The combination of tools targeted the local context on several different levels. The scorecard estimates the likelihood that a household has an income lower than a reference value. The Constituency Voice is an adaptation of consumer-insight research to farmers that looks into service relevance and quality; as well as into the client's relationship characteristics with farmers , both tangible (e.g. , increased income) and intangible (e.g ., confidence, empowerment). What lessons does this project offer for others? This project suggests that there can be a set of indicators that align incentives of farmers, the IFC client, and the IFC. These should focus on actual and perceived 28 impacts on farmers' income, and help the client to strategically deliver services with the potential to expand its business and improve quality of reporting to investors with a development focus . They should also provide IFC better insight on progress towards achieving expected impacts. This is a trend that companies in the agribusiness space are currently exploring both individually and through their associations. Source: Miguel Dellepiane, World Bank Group Flickr. 29 India: DSCL Sugar - Increasing Productivity Contact: Bradford Roberts, Email: broberts@ifc.org Harsh Vivek, Email : hvivek@ifc.org What was the development challenge? DCM Shriram Consolidated Limited (DSCL), an IFC investment client, is one of the major sugar producers in the state of Uttar Pradesh , India, providing the 150,000 sugar farmers in its catchment area with vital sources of income. However, both farmer productivity and plant productivity were low (67 percent of sugarcane crushing capacity was used at one plant, for example), which in turn lowered farmer incomes. The company tried in-house measures to enhance farm productivity, but had limited success as its programs were isolated , piece-meal , and not well managed. To strengthen farmer outreach , IFC partnered with DSCL to increase productivity, and incomes, of sugarcane farmers via an integrated advisory services project. IFC worked to help the company and farmers through three tracks. First, it worked to create a more user friendly and practical agricultural extension manual, giving extension agents detailed understanding of training needs in a practical format with extensive use of photos and examples. Second, IFC worked to improve the farmer training materials, producing a farmer flipchart, a 12 month calendar of farm level activities with photos to demonstrate the practice and the content in simple "directive" statements. Finally, IFC helped DSCL improve extension agents' training. All this contributed to increasing farmer productivity and income. What approaches did this project use to deliver solutions? On the training delivery side, IFC noted during the 1st year evaluation that while farmers were changing their practices and improving their productivity, they were not satisfied with the training delivery from the extension agents. Taking into consideration the evaluation results, the IFC sugar expert then worked with the extension agents to improve their training delivery, conducting mock training exercises to evaluate the 30 \ agents' approach and give guidance on how to improve. Additionally, early on in the M&E, one issue noted was that during the baseline collection of data, productivity estimates as provided by farmers seemed higher than the company believed . To more accurately measure productivity at farm level , the project used in-field crop cutting to measure the "true" productivity of the farmer's output. The project then used crop cutting for each of the follow-on surveys, to measure the production of the surveyed farmers, thereby ensuring accurate measures of productivity. What lessons does this project offer for others? This project identified key lessons. First, project managers may need to adjust during the course of a project to ensure impact, and should assume that things won't go as planned at project start-up. Moreover, rigorous monitoring and evaluation throughout the course of a project is needed to allow the IFC to credibly attribute good results to project impact. Source: Bradford Roberts , World Bank Group. 31 Morocco: Ouarzazate Concentrated Solar Power Project Phase I Contact: Sameh Mobarek, Email: smobarek@worldbank.org Fanny Missfeldt-Ringius, Email : fmissfeldt@worldbank .org What was the development challenge? Morocco is the largest energy importer in the MENA region , but it is also seeking to reduce its dependence on imported fossil fuels , as well as its carbon footprint. In 2010, Morocco adopted a 'National Plan Against Climate Change' that sought to decouple its economic growth and its greenhouse gas emissions by developing a progressive renewable energy agenda. Morocco thus recognized the twin-benefits of adopting climate- friendly policies to enhance its energy security and adapt the country's development trajectory to reduce its global carbon footprint. Ouarzazate I Concentrated Solar Power (CSP) Project was the first step in realizing these objectives by developing and implementing the first 160- MW phase of the 500-MW Ouarzazate (now called 'Noor') Complex. What approaches did this project use to deliver solutions? There were five key elements to the project's successful development, which involved a wide range of actors forming connections and partnerships to move forward a mu lti- dimensional response . First, the project enjoyed strong public support from the highest levels of government. The government of Morocco established and funded the Moroccan Agency for Solar Energy (MASEN) and staffed it with skilled , highly motivated personnel. Second, the Project enjoyed extensive support from the donor community because of its potential to model new, clean technology. Third , early coordination and agreements with donors demonstrated the commitment of the donor community behind the project, stimulating private sector interest and contributing to the success of its procurement process. Fourth , MASEN's choice to use a well-designed public-private partnership (PPP) model and sophisticated international financial , legal, and technical experts to advise it during the design and procurement process allowed for optimal alignment of 32 risks between public and private players. Fifth, the project built on past lessons learned from other CSP developments worldwide, which helped reduce the project's costs and increase its efficiency. What lessons does this project offer for others? At the country level , national support and government backing for Bank engagement across sectors was key. The capacity and motivations of government counterparts, as well as a clear implementation timeline that is tied to a well-defined public deliverable, are critical. Engagement of well-qualified, well-funded technical , legal, and financial advisors to guide the design and implementation of the project are an essential element of the development process. At the Bank level, it was important to secure strong internal engagement from a Bank with the right skills mix. Good coordination and communication between the Bank team and different internal stakeholders should ensure that information is shared in the most efficient and clear manner, with issues discussed and resolved quickly. Staff should focus on the fact that the Bank exists to serve the client's needs and objectives - not the other way around. Source: Sameh Mobarek, World Bank Group. 33 Uganda: Reproductive Health Voucher Contact: Peter Okwero, Email: pokwero@worldbank.org Leslie Villegas, Email : lvillegas@worldbank.org What was the development challenge? In Uganda, poor, rural women often lack professional medical attendance during childbirth , leading to elevated risk of maternal mortality. The project created a system to ensure that private partners effectively delivered professional care during childbirth . The project used a voucher scheme embedded within an Output-Based Aid (OBA) structure, allowing for verification of outputs and payments for services against outputs. The project exceeded its initial target (136,000 women), resulting in 137,964 women receiving skilled birth assistance. This was a first for the majority of them , and 92 percent reported being satisfied with the treatment. What approaches did this project use to deliver solutions? The Bank worked to change incentive structures in order to improve the provision of services by private partners. The voucher scheme provided potent incentives to change partner behavior, as subsidy disbursements can easily be linked to output verification. The feedback provided by this process tells project managers not only what is working , but also what is not working , and encourages adaptation when measures are not working . The resulting acceleration of service utilization over time allows for informed decision making and illustrates behavioral shifts toward more adaptive learning and efficient provision of services. What lessons does this project offer for others? The experience gained from the Uganda Reproductive Healthcare Project-both as a pilot project and at national scale up-has entered the project management literature as a powerful example of the flexibility and precision of Output-Based Aid as a tool to change behaviors and enable adaptive management. Output and outcome indicators, being both the results metrics and the check of contractual quality, allow managers to 34 adapt and learn what is working and not working based on verifiable evidence. The use of outputs not only incentivizes partners, but in addition burnishes the financial integrity of a project - if stakeholders are confident that a project is efficient, transparent, and free of corruption , project benefits can spill over into civil society as well. Overall , one of the major advantages of Output-Based Aid is that project managers adapt and learn what is working and not working based on objective and verifiable evidence across sectors. Source: GBOPA. 35 Indonesia: Partnership for Knowledge-Based Poverty Reduction Contact: Vivi Alatas, Email: valatas@worldbank.org What was the development challenge? Partners in the Government of Indonesia face challenges in producing rigorous evidence and applying it to develop policy and program responses to reduce poverty and vulnerability, tackle inequality, and stimulate job creation. The local supply of research and analytical services in Indonesia is extremely limited and can be difficult for policymakers to access. This program was established in response to a request in 201 O from the Vice- President of the Republic of Indonesia and his Deputy for Social Welfare for technical assistance and capacity building to support national efforts to coordinate and integrate national poverty reduction policies and programs. The initial request focused on supporting reforms for the country's array of household-based poverty reduction programs that target poor households. Increasingly, they are seeking analytical service and technical assistance . The program is directed toward fulfilling the immediate needs of government partners that are shifting towards an evidence-based approach to policy making. What approaches did this project use to deliver solutions? The program supports the Government of Indonesia by improving evidence-based policy making for poverty reduction programs and policies. To carry this out, the program adopts a three-pronged strategy, providing assistance at each stage of the knowledge- to-policy cycle . First, the program works on providing poverty analytics and building analytical capacity to inform poverty reduction policies, programs (with a focus on household-based poverty reduction programs) and poverty reduction strategies. It also supports the government in the design , implementation , and evaluation of key poverty reduction programs. Finally, it works to improve the quality and accessibility of data required for poverty analysis and policymaking. 36 What lessons does this project offer for others? This program has contributed to the shift towards evidence-based policy making in the poverty and social protection sectors in Indonesia. In 2010, the President elevated oversight of the poverty strategy to the National Team for the Acceleration of Poverty Reduction (TNP2K), which is chaired by the Vice-President. The National Team is supported by a Secretariat that is responsible for drafting policies (with a priority on social assistance reform), establishing a national targeting system, and integrating monitoring and evaluation activities. The Secretariat includes six working groups that function as internal "think tanks" to develop evidence-based policies. TNP2K and the World Bank have contributed to creating an environment where evidence is the basis for policy recommendations and government decisions on poverty and social protection sectors. Source: Anne Cecil Esteve, Matahati Productions, World Bank. 37 Kuwait: Enhancing Labor Market Information Systems Contact: Ghassan N. Alkhoja , Email : galkhoja@worldbank.org What was the development challenge? Mismatches in the labor market negatively impacted employment opportunities, and policymakers in Kuwait were unable to make evidence-based decisions to correct these distortions. Information crucial for human resource development, and ultimately for enabling the public to make good decisions on job-seeking choices , was fragmented and scattered across agencies. The World Bank supported the development of the Labor Market Information System (LMIS) to consolidate labor market information and make it accessible and available What approaches did this project use to deliver solutions? This project focused on ensuring that policymakers and citizens had the information that they needed to make better decisions. During the course of implementation , the World Bank team utilized international experience in labor market information development, while contextualizing the solutions provided to the Government of Kuwait (GoK). The use of local language was extensive; everything was localized to Arabic, leading to better traction and acceptance by the client. All this was coupled with a process of sustained engagement and continuous focus on accomplishing goals, with Bank staff and resources on the ground to assist the client every step of the way. The World Bank team reached out to stakeholders extensively through a series of focus groups, where it heard firsthand from government officials, members of the private sector, students, and civil society groups about their needs, the challenges they face , and the solutions and ideas they offered . In addition , the World Bank team sought out partnerships, as much-needed labor market information is scattered across many agencies. The Bank brought these agencies to one table, explained the challenges, inquired about solutions, and sought buy-in from all. This process of extensive consultation has ensured that LMIS incorporates information needed by multiple 38 stakeholders. Now, accurate statistics can help policy makers formulate effective policy, while projections of future labor market needs can help students make the school-to- work transition. What lessons does this project offer for others? Localization in terms of language, local presence, and utilizing local resources were direct contributors to success; to ensure positive impact on citizens on the ground, this localization should be a main feature of engagements. Undertaking a rigorous diagnostic to identify problems, and verify what citizens needed, enabled a structured approach to address the needs of the client. Sustained engagement, through almost continuous presence of Bank staff and resources on the ground to assist the client every step of the way, was also crucial. Moreover, since lack of information had engendered some suspicion toward institutions, the World Bank focused on rebuilding trust for improved decision making ; it accomplished this by being context sensitive and used its convening power, as well as leveraging Open Data principles. Source: Ghalia Al Refaei , Kuwait Central Statistical Bureau. 39 India: Improving Rural Livelihoods in Bihar Contact: Parmesh Shah, Email : pshah@worldbank.org What was the development challenge? The state of Bihar has some of the worst social and economic indicators in India, with significant challenges of social exclusion, lack of access to finance, landlessness, low food security, and indebtedness of rural households. The project facilitated a unique institutional model for the development of rural livelihoods in the state, using a professional support architecture of service delivery to develop commun ity institutions of the poor, build social , financial and economic capital , and support the poor in their interactions with state and market. In seven years of operation, the project has mobilized 1.4 million poor households into Self Help Groups (SHGs) to access finance for farm modernization , training , and to start enterprises; some 70 percent of these households have retired their high-value debts. A recent independent impact evaluation has also observed that the participant households have higher savings, lower high-cost debts, higher food security, more assets, and empowered women . What approaches did this project use to deliver solutions? The World Bank, returning to Bihar after a gap of 25 years , used four main approaches to improve outcomes in a difficult environment. First, the task team engaged intensively with a broad range of stakeholders to understand political and social realities on the ground before determining the project design. The team stayed in Bihar for three months at a stretch , holding 12 major consultations; in addition , potential beneficiaries in over 1,000 villages were consulted to ascertain their needs. Second, the task team made concerted efforts to get buy-in from the major political players at the highest levels of government. Furthermore, senior bureaucrats and officials were taken on study tours of successful projects in Andhra Pradesh . This peer-to peer learning proved to be extremely effective. 40 Third, the project invested in a dedicated cadre of 160 skilled young professionals, almost half women , who could support village communities, and regularly engage with the task team to keep them updated about the situation on the ground. Fourth, an experienced project consultant was placed on the ground in Bihar, making Bank expertise constantly available to project officials. What lessons does this project offer for others? This project showcases four key lessons that helped staff meet their goals in Bihar. First, to ensure better outcomes for citizens, it is important to spend time on the ground to understand social and political realities. Second, the project should ensure presence of quality human resources on the ground. Third, building trust with political leadership and wider circles of influence is irreplaceable. Last, the Bihar experience suggests that, to deliver solutions for citizens in difficult socio-economic contexts, it may be advisable to emphasize geographical expertise rather than sector specialization. Source: Vinay Vutukuru , World Bank Group. 41 FYR Macedonia: Public Expenditure Policy Based Guarantee Contact: Birgit Hansl , Email : bhansl@worldbank.org What was the development challenge? The Euro Zone turmoil during 2011-2012 created strong headwinds for Macedonia's economy, leaving it facing a complex tangle of problems with negative impact on citizens: low revenues, unpaid government obligations, and a weakening private sector, while access to financing from virtually closed international financial markets exacerbated the issue. The World Bank worked with the government to obtain a large commercial loan leveraged by a World Bank guarantee, creating a fiscal buffer which allowed for the fast repayment of arrears and installing measures to prevent the reoccurrence of such an economical harmful problem . As a result, a tremendous pick-up in economic activity was observed in Macedonia, resulting in one of the best growth outcomes in 2013 among the Western Balkan countries . The policy-program also supported measures which helped realize major public expenditure savings by introducing public procurement of drugs and other medical devices, and better targeted public social expenditures to the poor and vulnerable . Most of all, the extended maturity of the guaranteed loan significantly reduced the costs of borrowing , saving Macedonia over EUR50 million in interest costs over a 7 year period . What approaches did this project use to deliver solutions? The reform program was multi-dimensional in nature, spanning several sectors, and ! included World Bank team members from macroeconomic, fiscal and public financia l management, health and social protection , mirrored by a diverse team of ministry counterparts. The World Bank also initiated a multi-stakeholder process, as the team helped to explain to other stakeholders, such as the opposition and the press, why there was a short-term need for higher borrowing and a higher deficit in order to address the arrears problem . The World Bank team was also able to strengthen essential 42 partnerships with other multilaterals and commercial lenders, and provide authorities with the opportunity to gain experience working with a sovereign lender. What lessons does this project offer for others? This project offers a key lesson for the construction of multi-dimensional spaces and responses: it is important to remember that the World Bank can play an important role in creating for counterparts the space to debate and identify jointly with other country stakeholders solutions to problems which are everybody's concern. The crucial opportunity lies in reaching out to various stakeholders and helping to facilitate needed dialogue on a difficult topic, such as increasing the debt level or short-term fiscal deficit, as the World Bank is often regarded as a neutral mediator. As an already trusted counterpart, the Bank also has the responsibility to create new partnerships for them with other multilaterals or bilaterals, private international grant-making foundations or the private sector. Source: Bojan Shimbov, World Bank Group Flickr. 43 Yemen: Social Fund for Development Phase IV Contact: Mira Hong, Email: mhong@worldbank.org What was the development challenge? In Yemen, a high proportion of the population lives in poverty, with low access to health, education , water, and other basic services. This falls particularly heavily upon vulnerable populations , such as women, youth , children , and special needs groups. The Social Fund for Development (SFD) , currently in its fourth phase, aims to enhance economic opportunities and reduce the vulnerability of the poor. It targets villages in the poorest sub-districts, with the aim of improving living conditions. The program is implemented through SFD Branch Offices and has proved successful in responding to urgent development needs as well as improving access to basic services. What approaches did this project use to deliver solutions? To improve living conditions, SFD focuses on multiple sectors including economic and agricultural activities, water, roads , health and education, according to community plans based on priority needs. One representative example of SFD's activities, and its approach , is its Rural Girls' Education program. To increase the number of girls in school , the program draws on a menu of complementary, multi-dimensional interventions to strengthen the overall outcome, including water supply, school infrastructure and equipment, awareness campaigns, training and capacity building at the school , community and local government levels, and non-formal community classes for girls and women . The required combination of intersecting interventions for a targeted community is based on prior identification of the particular factors constraining girls' enrollment in that community. For example, SFD has found that, to change deeply engrained traditional attitudes, the engagement of stakeholders over a period of several years is needed. In each targeted community, an initial community assessment is carried out to determine the particular constraints and the types of engagement that are needed . Often, the key is to engage religious and other community leaders and , when they are resistant to promoting girls' schooling, to enlist a respected peer to encourage a change of view. 44 Additionally, SFD is adaptable: during crisis in Yemen in 2011-2012, project activities were threatened by national disruption of fuel supplies and severe security constraints. The SFD responded quickly and innovatively, switching to the use of local materials for construction , and using camels to transport them. In addition, the SFD quickly scaled up and adapted its Labor-intensive Works program to help meet the increased needs of the poor and vulnerable. What lessons does this project offer for others? Four factors are identified as being crucial for SFD to produce innovative multi- dimensional solutions while also remaining flexible. First, operational autonomy is key - SFD is an autonomous institution under the Prime Minister's Office, with a Board of Directors drawn from government, civil society, the private sector, and the financial sector, bringing together representatives of many sectors. Second , SFD adopts a decentralized structure, with strong capacity in Branch Offices. Third , SFD focuses strongly on partnerships and capacity building at local government level. And finally, SFD is committed to transparent procedures with emphasis on monitoring and evaluation. Source: Yemen Social Fund for Development. 45 Colombia: Cartagena Water Supply and Sewerage Contact: Greg Browder, Email : gbrowder@worldbank.org Carmen Yee, Email : cyeebatista@worldbank.org What was the development challenge? Cartagena is a coastal city in Colombia, and tourism is a main pillar of the economy. In 2000 , pollution from domestic wastewater was contaminating tourist beaches, degrading the environment, and producing a public health crisis. Sewerage coverage in 2000 was only about 60 percent, and domestic wastewater flowed untreated through the streets and into Cartagena Bay and an ecologically sensitive estuary. Cartagena entered into a public private partnership with a Spanish water company to provide professional management to expand water and sewerage services , and to adopt a low-cost submarine outfall rather than an expensive conventional wastewater treatment plant. In 2013, Cartagena has achieved 90 percent sewerage coverage and dramatically improved water quality in its bay, coastal waters , and estuary. The World Bank provided US$90 million to the Cartagena water company (ACUACAR) to help finance the wastewater investments, and also provided technical assistance and politica l support for the public-private water utility model. What approaches did this project use to deliver solutions? The starting point for the submarine outfall , wh ich discharges partially treated sewage, was located in an isolated Afro-Caribbean fish ing village, whose inhabitants were concerned that the outfall would undermine their fishing livelihood. The community originally opposed the submarine outfall solution and in 2004 filed a complaint with the Inspection Panel, which ultimately was satisfactorily concluded. After a long process of socialization with the community, including field visits to other outfall projects, education on the nature and impacts of the outfall, and infrastructure investments in the community, the Cartagena water company was able to obtain the trust and confidence of the village. 46 Today, many of the villagers work in the treatment plant, the village water supply, sewerage, and road infrastructure has been upgraded , and there are no discernible environmental impacts from the submarine outfall. What lessons does this project offer for others? This project exhibits some key lessons for sustainable sanitation solutions. For example, the type of outfall solution employed here requires extensive consultations with local communities, and extra attention must be paid to poor, indigenous, or otherwise disadvantaged groups. While this low-cost system outfall is an attractive option, it must be based on extensive water quality monitoring , planning , design, and execution to ensure that it is an appropriate environmental solution. It also shows that a public-private water utility model, where the municipal government provides financing and oversight, and a professional private operator ensures efficiency and continuity, may be a viable alternative in situations with sufficient public sector financial capacity, but weak municipal governance. Source: ACUACAR. 47 The World Bank Sciehce of Delivery AT WORK Introduction What does it take to deliver results, measurable improvements in quality of life, every time in every country? We all know that sound technical knowledge is important. But we also know that to deliver outcomes for citizens we need to combine that technical expertise with on the ground delivery know-how. Relentlessly focusing on the outcomes, deliberately taking multi-stakeholder and multi- sectoral perspectives, systematically using and collecting evidence, developing a deep understanding of the political context and individual incentives, consciously building in feedback loops and course correction mechanisms - those are some of the approaches that teams use to get the delivery right. By capturing the learning from those approaches, we are building a collective and cumulative body of operational knowledge and know-how that can help development practitioners deliver better results. That is the science of delivery at work. Relentless focus on citizen outcomes a. Identify the nature of the problem based on a thorough understanding of citizens' demands and the local context. b. Design a con~ext-sensitive strategy to relentlessly focus on outcomes, defined as measurable welfare gains of citizens. c. Pay attention to other factors, beyond the project, that influence citizen outcomes. Multi-dimensional response a. Facilitate multi-stakeholder coalitions and multisectoral perspectives to identify problems and solutions. b. Convene development partners and build on their comparative advantages. 2 Evidence to achieve results a. Use the best available evidence to identify the nature of problems and to develop solutions. b. Develop local evidence, through experimentation and other strategies, to refine solutions. c. Collect, throughout the project cycle, evidence of results to enable course correction . d. Contribute to the global body of knowledge with the evidence accumulated during implementation . Leadership for change a. Identify the incentives that motivate individual behavior change and integrate them in solution design . b. Understand the political economy and drivers of change to determine how and when to best engage with the client. c. Ensure leadership support and facilitate coalition building among different stakeholders. Adaptive implementing a. Develop an adaptive implementation strategy that allows for iterative experimentation, feedback loops, and course correction . b. Build a committed multidisciplinary team with the right skills, experience, and muscle memory. c. Maintain the capacity for reflection and a diagnostic mindset. Take action, step back, and assess the results of the action. The projects featured in this brochure represent a small sample of projects that illustrate the use of the approaches listed above. Some have drawn on one main approach , others on some, and others on all of them . For more information, go to intranet: http:\\sod 3 Argentina: Road Safety Contact: Veronica Raffo, E-mail: vraffo@worldbank.org What was the development challenge? Road crash fatalities are an epidemic in the developing world , the number one killer of the young, with huge economic and social costs. These fatalities can only be stabilized and then reduced if all actors work together to strengthen institutions, ensuring strong and sustainable multi-sectoral coordination and action. While past projects have attempted to lower the mortality rate in Argentina, these have historically been implemented as fragmented single-sector initiatives with outcomes too small to be measured, and were often one-off projects with no follow-up activities. The Road Safety Project supported the newly created Argentina Road Safety Lead Agency (ANSV) to unify safety efforts and combat road crash fatalities. What approaches did this project use to deliver solutions? To attack this epidemic of road crash fatalities, the Road Safety Project provided constant support to the ANSV, enabling it to deliver its institutional management functions and build partnerships to save lives. The project gathered evidence to continuously retrofit project design and implementation, using expertise, methodologies, and systems developed by countries that perform well at ensuring road safety. Since credible, accessible performance data and related crash analyses are crucial to lowering road crash fatalities, the Road Safety Project made connections across sectors to gather this information and ensure that all relevant analysis was being brought to bear on the problem. It also leveraged strong partnerships with police agencies, the health sector, and the OECD's International Road Traffic and Accident Database (IRTAD) Group to push forward life-saving initiatives, ranging from highway improvements in high-risk corridors, to innovative safety education campaigns. What lessons does this project offer for others? In a key lesson for how to generate better citizen outcomes, this project demonstrated that well-focused national leadership and supporting resources can catalyze effective 4 partnership engagement with provincial and local governments, NGOs and the private sector to achieve consensus on desired results and the measures to achieve them. Another crucial innovation has been the Incentive Fund (with output-based disbursements), which has enabled the ANSV to engage in a purposeful way with a wide body of partners, bringing them together to focus on road safety. An additional lesson is that South-South dialogue and action on a regional basis can be stimulated when best practice measures are taken and given high visibility. The Argentina Road Safety Project has created interest in other Latin American countries facing the challenges of creating a lead agency to manage their national road safety effort. It has also spurred action to create a Regional Road Safety Observatory. Argentina's work in partnership with the IRTAD Group and its Spanish counterparts is serving as a model to assist the related development of regional protocols - offering its neighbors access to a useful lens for relentless focus on citizens' outcomes. Source: World Bank Group. 5 Tanzania: Productive Safety Net Project Contact: Ida Manjolo, Email: imanjolo@worldbank.org Manuel Salazar, Email: msalazar2@worldbank.org What was the development challenge? In Tanzania , despite compulsory education for children from 5-12 years, those from poor households often either drop out of school or are not enrolled in school because their parents cannot afford to pay for tuition fees and other required school materials and contributions. In such instances, children are forced to engage in petty trading or work as laborers so that they can bring food to their families. Those who continue schooling, do so in difficult conditions, without proper school uniforms or books, let alone proper nutrition. The Tanzania Social Action Fund (TASAF) sought to alleviate these conditions by piloting a safety net program through a community based conditional cash transfer project which supported the elderly and children from poor households. What approaches did this project use to deliver solutions? Given the substantial detrimental effects that low incomes had on children's education, the TASAF project focused on providing modest cash transfers to selected poor households with children of school age, and the elderly. In addressing the root of the education problem, the program transferred cash conditional on the pupils from cash- transfer-beneficiary-households actually attending school , as well as requiring that the elderly attended clinics for check-ups. Once a month, special sessions were organized on capacity building, health , and effective use of the cash received . After three years of the pilot, results showed that households initially identified as poor had improved their situations substantially, both in terms of increased asset ownership and empowerment. Impact assessment results showed that before the program, poor households could afford one meal per day at most, but after being enrolled in the program could afford three meals a day. Children from these households recorded good performance in school and truancy was reduced. Children who were going to school 6 barefoot with inadequate school materials and tattered uniforms are now proud to report that those days are long gone and they have a sense of belonging in the community. What lessons does this project offer for others? By addressing the root of the education problem in Tanzania , the project was able to achieve positive outcomes not only for children's education, but also in terms of income, health and commun ity based safety nets, so that the CCT had a wide-ranging set of cross-cutting positive effects. The CCT, and community training sessions and expert facilitation enabled beneficiaries to become enterprising . Some used part of the transfer to pay for their community health fund insurance. Others started small businesses to generate more income so as to become self-sufficient and less dependent on the cash transfers. This project has provided input to the design of the current Productive Social Safety Net Project, and the cash transfers are being rolled out to all 161 districts nationwide to over 900,000 households. Source: Tanzania Social Action Fund (TASAF). 7 Bangladesh: Partnership for Clean Textile (PaCT) Contact: S. Akhtar Mahmood , Email : smahmood@ifc.org What was the development challenge? While Bangladesh 's textile industry is an important driver of the national economy, it is also a heavy consumer of water, and a major polluter of water bodies around it. Suboptimal use of chemicals and inadequate effluent treatment is contaminating surface waters , negatively affecting the lives of nearly 12 million people. Competitiveness and further growth in Bangladesh's apparel industry will critically depend on achieving efficiencies in water use and reducing discharge. In the PaCT project, firm-level advice from IFC is complemented by multi-stakeholder engagement leading to policy and regulatory interventions, and collective action by industry. The target is to induce change in 200 factories and lay the ground for changes in the rest of the industry after the project closes. What strategies did this project use to deliver solutions? A multipronged approach is needed to induce adoption of good practices. In this project, the I FC used its convening power to bring together many actors around a shared agenda. PaCT embodies a multi-stakeholder approach by working to link buyers, factories, financial institutions, sector associations, and local communities and governments for coordinated action on water sustainability. Through PaCT, global apparel buyers develop and apply harmonized procurement guidelines that integrate water sustainability criteria . The leading industry association, Bangladesh Garment Manufacturers and Exporters Association (BGMEA), is partnering with PaCT to establish a Textile Technology Business Center, to support adoption of CP measures by sharing technology case studies and associated technical and financial information. A Textile Sustainability Platform is being formed to reach out to national and sector- level stakeholders to address water sustainability challenges in the sector. And in partnership with leading factories, PaCT will work with business, government, and 8 communities in selected textile manufacturing clusters, to develop and put in practice a vision for Clean Clusters. What lessons does this project offer for others? This project shows that in transforming a sector, relationships - among governments, communities, and business - must also be transformed . World Bank Group leadership brings together multiple stakeholders to change incentives and behavior of diverse actors to move forward a change agenda. PaCT faces major challenges in implementation, but in addressing these challenges is working on a number of different tools to address these , including survey instruments, awareness bu ilding materials, assessment methodologies and guidance notes for clients. These , and other knowledge products such as case studies, can also be shared with others. Source: PaCT Project Team, World Bank Group. 9 Indonesia: National Program for Community Empowerment in Rural Areas Contact: Sonya Woo, Email : swoo1@worldbank.org What was the development challenge? The government of Indonesia needed to cost-effectively deliver basic services and assistance to the rural poor across its vast territory. The government, with World Bank funding and technical assistance, put in place a community-driven development (COD) system, operating via direct transfer of grants from the national treasury to collective community bank accounts, to enable commun ities to undertake development activities. Government subsequently scaled up the prog ram - first called Kecamatan Development Program (KDP) and later the National Program for Community Empowerment, or PNPM-Rural - nationally and adapted it to deliver a oroader range of services in line with evolving local priorities. What approaches did this project use to deliver solutions? Following a series of mixed methods diagnostics, called the Local Level Institutions studies, which uncovered latent capacity possessed by communities to solve problems collectively, the government implemented the COD. This drew on community problem- solving capacity, with facilitation support, to cost-effectively deliver services in rural communities across Indonesia. The Bank led in establishing the PNPM Support Facility (PSF) to convene multiple government and international development stakeholders, enabling the Bank to maintain a strong, continuous presence locally. The Bank drew on deep relationships from its sustained presence on the ground, which was indispensable in supporting the implementation and scale-up of a national COD program in a country as large and diverse as Indonesia, and it allowed the Bank to engage deeply with reform-minded policymakers during windows of opportunity. The compilation of a large body of evidence support also helped to build crucial sustained commitment among World Bank management. 10 The use of rigorous evidence was vital throughout the program. Randomized control trials were used, such as for the differential impacts of incentivized versus non- incentivized versions of PNPM Generasi , a community cash transfer program delivered through PNPM-Rural aimed at improving access to priority health and education services. Findings led to the mainstreaming and scale-up of PNPM Generasi. A study of the participation of marginalized groups in PNPM led to the design and roll-out of a special program of support to national NGOs active in engaging sex workers, ethnic and religious minorities, prisoners, and other marginalized groups in development activities. What lessons does this project offer for others? This project shows the importance of drawing on rigorous evidence to inform program decisions. The KDP, now PNPM-Rural, prioritized the use of rigorous mixed-methods research to design the program and measure impacts, allowing for successful scale-up. Impact evaluations have provided rigorous evidence of PNPM-Rural's impacts on increases in consumption among the poorest, and improvements in access to basic services. Source: Curt Carnemark, World Bank Group Flickr. 11 Afghanistan: Rural Enterprise Development Program Contact: Kamran Akbar, Email: kakbar1@worldbank.org What was the development challenge? About 80 percent of Afghanistan's population depends on agriculture and livestock for a living. This project sought to improve the incomes of rural men and women working in the agricultural sector by helping them access financing and develop sustainable enterprises. However, the project's progress was initially delayed, as it had ambitious goals with limited capacity and experience. High levels of bureaucracy and slow disbursement also contributed to the delay. To address this , the project underwent a mid-term restructuring that enabled the project to reach goals more effectively. The result was increased access to village level financing , reaching more than 51 ,000 rural poor in 22 districts, surpassing $US1 .5 million . The project also helped establish nearly 200 small and medium enterprises across the country. Over half the savings group members are women , and the number of female borrowers surpasses that of men . What approaches did this project use to deliver solutions? In May 2013, the project was restructured through reduction of the project's scope and revision of targets , given Afghanistan's prior lack of experience in implementing such a livelihood project. The organizational structure and staffing was transferred from public institutions to private providers. In adapting to the local challenges, the project incorporating Pasto-speaking facilitators to work with local communities, and included migrant and disabled people in the enterprise development pilots. Local consultants were recruited and trained in order to build local capacity, as well as to have a pool of resources for future similar projects. 12 What lessons does this project offer for others? Since the restructuring mission of May 2013, the project has shown considerable progress. The pace of activities has increased and the project is moving towards realizing most of its revised targets. Substantial progress in enterprise development among pastoral nomads has been made, transformational actions with regard to fixed- salaries staff have been undertaken, and training of facilitators - of whom about half are women - has been completed . The successful adaptation of this project was made possible by identifying problems early on and discussing them candidly, and making appropriate adjustments to the local context. It was also critical to put in place incentive mechanisms to ensure that performance in implementation met expectations, with improved monitoring systems playing a key role. Source: Kamran Akbar, World Bank Group. 13 Nigeria: ICT for Social Accountability Contact: Paula Rossiasco, Email : prossiasco@worldbank.org What was the development challenge? The development challenge was to ensure better service provision in health and agriculture by effectively collecting feedback from World Bank project beneficiaries in Nigeria. The project had to avoid a common pitfall: tools for collecting feedback from project beneficiaries often fail to represent citizens' voices and are disconnected from decision making processes. The system, called MyVoice, will enable decision makers to make accurate, timely, and efficient programmatic decisions based on real-time beneficiary feedback via SMS messages to a publicly advertised short code . Creating mechanisms for using ICT to transmit information in real time up the management stream - to both World Bank staff and government counterparts - allows for immediate and timely aggregation of voice and facilitates the use of this information to improve service delivery. What approaches did this project use to deliver solutions? The MyVoice system relied on evidence from user experiences to ensure that the desired result - citizen use of the ICT to enhance social accountability- occurred. Since many tools fail due to poor understanding of social context, a significant portion of project design time was dedicated to gathering evidence of user needs via an ethnograph ic approach. Citizens were interviewed and user experiences documented to inform program design. Other stakeholders were canvassed to understand their incentives for using information captured by the feedback mechanism, including local government and traditional authorities, local NGOs and CSOs, and service providers. The system was designed and implemented adaptively, iteratively, and experimentally, utilizing agile software development methodologies to allow for continuous prototyping and testing . The end product was a system that drew on extensive evidence of citizen needs to design the ICT. 14 What lessons does this project offer for others? This project showed the importance of gathering extensive, multi-disciplinary forms of evidence to inform ICT design these tools often fail. The central role given to understanding the political economy of service delivery and the social context in which projects take place provides an insight into the importance of locally-gathered , multi- disciplinary evidence to unchain the potential of ICTs to aggregate voice meaningfully. At the same time , effective citizen engagement in development projects is a multisectoral issue. It requires a deep understanding of the technical design, the decision making mechanisms for service delivery, the incentives of the different stakeholders involved , the social context, and the potential that ICTs can offer to bring these different aspects together in a concise , user- friendly, and timely manner. The tool was conceived in a modular way, so it could be adapted to other operations in the Nigeria World Bank portfolio once the project context is fully understood. Source: Reboot. 15 Global Scaling Up Rural Sanitation Contact: Eduardo Perez, Email: eperez1@worldbank.org What was the development challenge? Worldwide, 2.5 billion people lack access to improved san itation, and three quarters of these people live in rural areas. Seeking to respond to this development challenge , in 2007, the World Bank initiated a partnership with the governments of India , Tanzania , and Indonesia to start at scale projects that served as learning laboratories to provide lessons for policy changes; this was then scaled up in 10 additional countries . From base-line to 2013, WSP helped governments to scale up access to improved sanitation for 22 million and encouraged 19 million people to stop practicing open defecation in the 13 focus countries . What approaches did this project use to deliver solutions? The Scaling up Rural Sanitation program includes an iterative process to help governments develop at scale solutions, as shown in the figure . The initial countries served as learning laboratories, with lessons learned in India, Tanzania, and Indonesia being informed by real-time data collection and monitoring . This resulted in a theory of change that is now - being replicated in 1O • Chem· and problenHlriven engagement • Sh81ed