72451 SIEF Mid-Year Monitoring Report 2010 August 31, 2010 About the SIEF program The Trust Fund for Impact Evaluation and Results-based Management in Human Development Sectors, or Spanish Impact Evaluation Fund (SIEF), was established in 2007 by the World Bank and the Government of Spain to support the Impact Evaluation (IE) of innovative programs to improve Human Development (HD) outcomes. Its main goal is to improve the effectiveness of development policies by expanding the evidence base on the impact of programs affecting HD outcomes. The SIEF program is managed by the Bank’s Human Development Network Office of the Chief Economist. SIEF resources support: (i) prospective, rigorous impact evaluations of programs in 11 eligible Human Development and Sustainable Development sectors and 72 eligible developing countries across all regions; (ii) intensive training programs for government counterparts, Bank staff, and staff of partner development agencies in impact evaluation methods; and (iii) publication and dissemination of evaluation results through articles, meta-studies, and other outreach events and efforts, including web- based materials. About this report This SIEF Mid-Year Monitoring report summarizes the main financial flows generated and progress made by the SIEF program in the first eight months of calendar year 2010. This report has been produced and will be submitted to the government of Spain as agreed in the last SIEF Steering Committee meeting, held in Madrid on December 3, 2009. Contact information For further information on this report, or any inquiries about the SIEF program, please, contact Laura Rawlings at Lrawlings@worldbank.org or Carlos Asenjo at Casenjoruiz@worldbank.org. 0 Spanish Impact Evaluation Trust Fund (SIEF) MID-YEAR MONITORING REPORT 2010 SIEF Progress Update Mid-2010: Main Trends The SIEF work program has shown substantial progress from January to August 2010, and is on track for meeting its goals. As of August 31, 2010, the main goals of the capacity building program have been achieved, the impact evaluation research program is fully underway, and the results dissemination program has been launched. As shown in Boxes B and C, most of the decisions agreed at the last SIEF Steering Committee meeting and of the activities mentioned at the “Moving Forward� chapter of the SIEF Annual Report 2009 either have been completed, or are making progress as planned. 1 Table 1.- Spanish Impact Evaluation Trust Fund. Financial Summary FY2008-FY2011 : 16 Millions 14 12 10 Allocated 8 Receipts 6 Committed 4 Disbursed 2 0 End FY2008 End FY2009 End FY2010 Financial End FY2008 End FY2009 End FY2010 Act. FY2011¹ Act. FY2011 ¹ Flows ² (Jun 30, 2008) (Jun 30, 2009) (Jun 30, 2010) (%) Allocated 13,556,214 15,104,793 15,289,368 15,139,718 - Receipts 6,647,125 11,183,960 14,352,270 14,901,260 98.42% Committed 1,053,252 3,578,075 8,291,515 8,957,554 59.17% Disbursed 708,295 2,711,872 6,200,325 6,952,897 45.92% 1 Updated as of August 31, 2010. Data only reflect SIEF support; other sources of financing (World Bank, DFID) are not included. 1 The progress made over the last months is illustrated by the increasing financial flows generated by SIEF activities. The positive evolution of all budget execution indicators2 confirms the increasing activity of the SIEF program since January 2010 (See Table 1). Although the state of progress differs significantly among the four SIEF areas of activity –impact evaluation research, capacity building, results dissemination, and advisory services & management– they all are progressing in terms of budget execution, as shown in Table 2. More specifically, as of August 31, 2010: - 98.42% of SIEF allocations are receipts –i.e. funds that have been transferred from the SIEF main account to specific grant accounts that can be accessed by task team leaders. The progress shown by this indicator –as compared to the 86.06% achieved at the end of 2009–reflects the transfer of $749,000 to the task teams of the new Early Childhood Development cluster, and the channeling of a $700,000 towards cluster leaders for results dissemination activities. - 59.17% of SIEF allocations have been committed to third parties by task teams or SIEF core team members, moving from 41.99% at the end of 2009. The growth of SIEF commitments mainly reflects the increasing activity of the research task teams; - 45.92% of SIEF allocations have been disbursed, moving from 31.92% at the end of 2009. This increasing trend is due to the delivery of products by research teams. Box A. Financial flows and budget execution indicators In this report four types of financial flows have been taken into consideration: Allocations are the amounts that have been addressed by the SIEF Steering and Technical Committees or by the SIEF Program Manager to specific activities or projects. -Strictu senso, these amounts are not flows, but formal decisions regarding what receipts transfers, commitments and disbursements will be made. Receipts are the amounts that are transferred from the SIEF main account to specific grant accounts so that impact evaluation teams (or SIEF core team members in case of activities delivered directly by them, such workshops or advisory services) can have access to the amounts that have been allocated to the projects or activities for which they are responsible. Commitments are the payments to third parties -such as firms or consultants- that impact evaluation teams (or SIEF core team members in cases as explained above) have formally agreed to make conditioned to their delivering of a specific product or service. Disbursements are the amounts that are paid to such third parties upon receipt of the agreed deliverables. Three budget execution indicators are used in this report: % Receipts / allocated reflects the extend until which SIEF allocations have been channeled towards the individuals (SIEF core team members, cluster leaders and impact evaluation task team leaders) that are directly accountable for delivering specific products or services. % Committed / allocated reflects the extend until which SIEF allocations are being used to hire firms and consultants, or to address staff time in order to ensure the delivery of products and/or services under a specific planned timeline. % Disbursed / allocated reflects the extend until which SIEF allocations have been used to pay for the products and/or services delivered. 2 For details regarding the types of financial flows and indicators used in this report, see Box A. 2 Table 2. – Spanish Impact Evaluation Trust Fund. Financial Summary FY2011 ¹ ² ³ % Receipts / % Committed % Disbursed / Allocated Receipts Committed Disbursed Activities Allocated / Allocated Allocated Act_FY2011 Act_FY2011 Act_FY2011 Act_FY2011 Act_FY2011 Act_FY2011 Act_FY2011 1. Impact Evaluation Studies (IEs) 9,920,377 9,681,919 6,346,449 4,451,083 97.60% 63.97% 44.87% 1.1. Quick Win IE Studies 879,851 879,851 879,851 879,851 100.00% 100.00% 100.00% 1.2. Innovation Fund IE Studies 2,046,025 2,042,125 1,320,867 838,136 99.81% 64.56% 40.96% 1.3. Cluster IE Studies � 6,994,501 6,759,943 4,145,730 2,733,096 96.65% 59.27% 39.07% 1.3.1 - Health Contracting / Pay 1,357,681 1,356,042 856,935 470,728 99.88% 63.12% 34.67% For Performance Reforms 1.3.2 - Conditional Cash 861,545 811,545 463,284 352,252 94.20% 53.77% 40.89% Transfers 1.3.3 - Malaria Control 752,735 753,075 519,206 418,484 100.05% 68.98% 55.60% 1.3.4 - Active Labor Market / 1,323,125 1,178,125 921,188 691,997 89.04% 69.62% 52.30% Youth Employment 1.3.5 - Basic Education 982,913 929,039 722,576 446,893 94.52% 73.51% 45.47% Accountability 1.3.6 - HIV/AIDS Prevention 966,501 982,217 607,846 324,382 101.63% 62.89% 33.56% 1.3.7 - Early Childhood 750,000 749,900 54,695 28,360 99.99% 7.29% 3.78% Development 2. Capacity Building 1,500,000 1,500,000 1,326,732 1,315,509 100.00% 88.45% 87.70% 3. Results Dissemination 1,500,000 1,500,000 260,580 203,671 100.00% 17.37% 13.58% 3.1. Program Outreach & Results 800,000 800,000 260,580 203,671 100.00% 32.57% 25.46% Dissemination 3.2. Cluster Results Dissemination 700,000 700,000 0 0 100.00% 0.00% 0.00% 4. Management / Advisory Services 2,219,341 2,219,341 1,023,794 982,634 100.00% 46.13% 44.28% 4.1. Program Management 1,088,714 1,056,923 144,428 143,291 97.08% 13.27% 13.16% 4.2. Cluster Coordination � 595,000 595,000 311,948 271,925 100.00% 52.43% 45.70% 4.3. Advisory Services 535,627 567,418 567,418 567,418 105.94% 105.94% 105.94% 5. Total (5=1+2+3+4) 15,139,718 14,901,260 8,957,554 6,952,897 98.42% 59.17% 45.92% NOTES: ¹ Updated as of August 31, 2010. ² For details regarding the types of financial flows and indicators used in this report, see Box C. ³ Data only reflect SIEF support; other sources of financing (World Bank, DFID…) are not included. � Excludes cluster coordination and results dissemination. � Excludes cluster results dissemination. 3 Box B. Monitoring of the decisions / intended results agreed at the last SIEF Steering Committee: Decisions / Intended results Achievements / State of progress ² 5th SIEF Steering Committee meeting (Dec. 3, 2009) As of June 30, 2010 I. SIEF Program Update: "The Impact Evaluation program is substantially advanced, but not on track for Completed. Clarifications regarding the construction of the Financial completion by the original November 2010 program deadline… SIEF team Summary table were provided at Annex A.1 of the SIEF Annual Report agreed to provide clarifications regarding the construction of the Financial 2009. √ Summary table". "The Training program will conclude early, with the completion of the four Completed. The SIEF workshop program has been completed. Since remaining scheduled workshops…" December 3, 2009, four impact evaluation workshops were delivered in Cape Town (December 7-11, 2009), Kathmandu (February 22-26, 2010), √ Rio de Janeiro (April 26-30, 2010), and Accra (May 24-28, 2010). II. SIEF Steering Committee Decisions: "the SIEF monitoring proposal that will be developed in early 2010" On track. A first draft version of the SIEF Monitoring & Evaluation plan was produced in April 2010, and it is at present being discussed within the SIEF team. A revised draft version of the plan will be submitted to the … donors for comments in September 2010. "Further clarifications regarding the use of resources allocated to Strategic Goal On track. As Goal 4 is a horizontal priority affecting all SIEF programs, # 4 were requested and will be provided in the next Annual Report (to be there is no specific line budget for this purpose. The SIEF team has received in January 2010)" provided examples on how SIEF supported activities are addressing this … goal, and is open to estimate the cost of activities directly addressed to this goal. "The SIEF World Bank team agreed to prepare a short note providing the Completed. A Memo For The Extension note was produced on February 2, justification for the extension which will be used to make the necessary 2010. The Amendment to the SIEF Administrative Agreement was signed √ modifications to the Administrative Agreement" on March 4, 2010. "The Steering Committee approved the establishment of a new SIEF impact Completed. On January 19, 2010, the SIEF Technical Committee selected evaluation cluster on Early Childhood Development" four impact evaluations on Early Childhood Development programs in Cambodia, Chile, Mozambique, and Nicaragua. The total SIEF/DFID √ allocation to the new cluster is $750,000. Other Topics: "The Steering Committee agreed that the 2009 SIEF Annual Report will be Completed. The SIEF Annual Report 2009 was submitted on January 29, submitted in January 2010…" 2010. A revised version, incorporating comments and suggestions made √ by the Government of Spain, was produced on May 10, 2010. "...and that program updates will be provided every six months." Completed. The present SIEF Mid-Year Report 2010 is the first of these √ half-year program updates. "The report will also include a breakdown on information on financing and Done. A breakdown information on financing and activities in the area of activities in the area of Management and Advisory Services." Management and Advisory Services was provided in the revised version of √ the SIEF Annual Report 2009 (page 8). "The World Bank’s SIEF team expressed their enthusiasm and support for On track. The Government of Spain and the World Bank have expressed conducting a dissemination event for Spain to be held in 2010, most likely in their interest in conducting a dissemination event on the SIEF program in Madrid… The DGPOLDE team agreed to share specific ideas with the World Spain, but no specific dates or format have been agreed yet. Other types ... Bank SIEF team in the early part of 2010." of dissemination events in Spain, such as the presentation of the Impact Evaluation handbook, or presentations of intermediate results of SIEF specific impact evaluations, are currently under discussion. "The World Bank SIEF team briefed the Steering Committee on the European Cancelled. No further interest in carrying out this event was shown by the Commission proposal for conducting an event related to nutrition and early concerned parties. childhood development with World Bank collaboration as part of Spain’s • leadership of the EC in 2010. The DGPOLDE team agreed to be in touch with the EC team and inform the Bank of any interest in this collaboration." 4 Box C. Monitoring of the intended results / challenges described at the last SIEF Annual Report: Intended results / Challenges ¹ Achievements / State of progress ² "Moving Forward" SIEF Annual Report chapter (Jan. 29, 2010) As of June 30, 2010 Capacity building: "In total, by December 31, 2010 the SIEF will have delivered 12 impact Completed. The SIEF workshop program has attained its expected goals. evaluation workshops in all developing regions in the world. Nearly 1,900 Since it was created in July 2007, 12 impact evaluation workshops have participants among evaluation experts, researchers and managers of programs been delivered in: Cairo (January 2008), Managua (March 2008), Madrid affecting Human Development outcomes will have been trained in the (June 2008), Manila (December 2008), Lima (January 2009), Amman application and management of IEs." (March 2009), Beijing (July 2009), Sarajevo (September 2009), Cape Town (December 2009), Kathmandu (January 2010), Brazil (April 2010), and Accra (2010). In total, 1,905 participants among evaluation experts, √ researchers and managers of programs affecting Human Development outcomes have been trained in the application and management of impact evaluations. In FY2011 the SIEF will continue supporting regional impact evaluation workshops in collaboration with other World Bank units and donors. "The SIEF will continue delivering specific, on-demand, advisory services to On track. The SIEF is delivering advisory services to client countries on client countries in order to support their applying impact evaluation methods impact evaluation in the following areas and countries: maternal-child to increase the effectiveness of their programs and policies. health (Argentina, Ghana, Rwanda, Bolivia); housing and health (El Salvador, Uruguay, and South Africa); hand-washing and sanitation (Tanzania); youth employment (Dominican Republic, Bolivia, and Uganda); early childhood development (Mexico and Nicaragua); income … support and employability (El Salvador); conditional cash transfers (Nigeria and Nicaragua). "Also, some type of on-demand training may also be given to evaluation On track. Some impact evaluation experts, researchers, and managers experts, researchers, and managers from SIEF donors." from Spain's AECID who attended the SIEF impact evaluation workshops in Rio de Janeiro and Accra expressed their interest in designing impact evaluations on their programs. On-demand training sessions on impact … evaluation methods were delivered in Washington, DC and Amsterdam. Research: "Most of the teams will be moving from baseline data collection towards On track. 20 of the 51 impact evaluation teams supported by the SIEF follow-up collection and preliminary impact evaluation analysis." have collected or are collecting follow-up data, and 8 impact evaluation studies have been produced: - Maharashtra and Orissa Rural Water Supply and Sanitation (Jun. 08); - Long-term Follow Up of a Simulation Program in Jamaica (Jun. 09); - NUSAF Youth Opportunities Program in Uganda (Jun. 09); - Chile Solidario (Jul. 09); - CCTs and HIV Risk in Malawi (2 papers in Oct. 09; 1 paper in Mar. 10); … - Performance-based Contracting for Rwanda Health Services (Jan. 10). "The main challenge for the SIEF core team will be to ensure that the new Completed. All concept notes have successfully been reviewed. 49 of the impact evaluations complete the review process of their concept notes and 51 teams supported by the SIEF (including all Early Childhood access their SIEF allocations, especially those of the new cluster on Early Development studies) are accessing their SIEF allocations. The two Childhood Development." remaining teams (working on the ALMP evaluations in Liberia and South √ Africa) are expected to access their SIEF allocations in September 2010. ² State of progress codes: √ Completed … On track ! Pending • Cancelled 5 Box C (Cont.). Monitoring of the intended results / challenges described at the last SIEF Annual Report: Intended results / Challenges ¹ Achievements / State of progress ² "Moving Forward" SIEF Annual Report chapter (Jan. 29, 2010) As of June 30, 2010 (Cont.) Dissemination: "In January 2010, dissemination resources will be allocated to cluster teams, Completed. A call for proposals for cluster dissemination was launched in and management arrangements will be made by the SIEF team in order to April 2010. The seven clusters submitted dissemination proposals, guarantee that cluster teams can access the resources allocated for this totaling $700,000 (47% of the SIEF results dissemination budget). All purpose." cluster dissemination proposals were approved in the summer of 2010, and at present cluster have access to 100% of their dissemination funds. √ "The Impact Evaluation Training Toolkit will be published in 2010." On track. A first draft version of the Impact Evaluation Training Toolkit was produced in May 2010. The document has been reviewed by impact evaluation experts from the World Bank and academic institutions. A final … draft will be completed in August 2010. The final version will be published later in 2010 and translated into Spanish in early 2011. "Also, the SIEF team will ensure in 2010 that the handbook and the other On track: training materials produced by the SIEF –impact evaluation presentations and - All workshop presentations are accessible at the SIEF website the day videos– are accessible to workshop participants and to any other interested after each workshop is concluded. persons through the SIEF website." - The training videos have already been produced and edited; they will be … posted on the SIEF website in September 2010. - The training toolkit will be accessible at the website in January 2011. "...a broad communication strategy will be implemented. This will include the On track. The SIEF communication plan is being implemented: dissemination of research documents, policy notes, training products, and SIEF - Conference presentations on the SIEF / HDNCE IE program were events, such as conferences." delivered at the Oversees Development Institute in London (May 4, 2010), and at the NONIE meeting in Bonn (March 29, 2010); and findings of SIEF- supported studies were presented at conferences in Washington DC, Atlanta, Jakarta and Rwanda. The SIEF / HDNCE impact evaluation program was also presented at the World Bank IEG Evaluation Week Fair (Apr 19-23, 2010) . … - The policy notes design has already been approved, and a consultant has been hired to support from July 2010 the implementation of the SIEF dissemination strategy, in particular, including the production of policy note drafts. "The second stage in the website design towards a result oriented layout will On track. The new website design is being developed in coordination with be performed. The training materials and the results of the evaluations will be the HDNCE and DIME. It is expected to be completed by the end of the the main contents of the new website, keeping the research-policy approach." year. … ¹ Intended results not specified already in the Minutes of the 5th SIEF Steering Committee meeting. ² State of progress codes: √ Completed … On track ! Pending • Cancelled 6 Research: As of August 31, 2010, 97.6% of the resources allocated to SIEF research windows are receipts transferred to impact evaluation teams. Only two teams –out of the 50 impact evaluations supported by the SIEF– are not using their allocations yet and, in both cases, the transfers will be made in September 20103. A total of 63.97% of the research budget has been committed by impact evaluation teams to consultants and firms in order to carry out the evaluations, and 44.87% of the funds have been disbursed to pay for delivered products and services. These figures show substantial progress compared to the 70.47% receipts transferred, 39.56% and 27,18% disbursed as of December 31, 2009. Yet, in FY2011 the main challenge of the SIEF remains to keep moving promptly towards the completion of the research work plan in order to be able to deliver the expected products by July 31,2011, as planned. Table 2 shows how the budget execution rate of the SIEF research windows ranks from the 100% of the Quick Wins window –that was closed at the end of 2009– to the 3.78% disbursed by the Early Childhood Development cluster –that was launched at the beginning of 2010, and of which all research teams have successfully submitted their Grant Funding Requests and have access to their their SIEF allocations. Apart from these two exceptional cases, from January to August 2010 all research windows have considerably increased their disbursements: - Cluster 1 (Health Contracting / Pay For Performance Reforms) from 9.2% to 34.67%; - Cluster 2 (Conditional Cash Transfers) from 22.24% to 40.89%; - Cluster 3 (Malaria Control) from 25.58% to 55.6%; - Cluster 4 (Active Labor Market / Youth Employment) from 27.52% to 52.3%; - Cluster 5 (Basic Education Accountability) from 11.28% to 45.47%; - Cluster 6 (HIV/AIDS Prevention) from 31.27% to 33.56%; - As for the Innovation Fund window, the disbursements increase is significant too, moving up to 39.07% of the window allocation from 20.54% on December 31, 2009. A more detailed view of the state of progress of each impact evaluation is shown in Table 3. 3 The two impact evaluation teams –Liberia and South Africa– that have not used their SIEF allocations yet are working on studies that were incorporated to the SIEF Active Labor Market Policies / Youth Employment cluster in the second half of 2009 with reallocation funds. The evaluation of the impact of the economic empowerment of adolescent girls in Liberia has been making progress with other sources of funding. They are planning to use their $70,000 SIEF allocation in the fall 2010, and will request the funds to be transferred this month. The team working on the impact evaluation of a wage subsidy program for youth in South Africa submitted a $75,000 Grant Funding Request (GFR) to the SIEF management in August. The request has been approved already, and the funds will be transferred in coming days. 7 Table 3.- Spanish Impact Evaluation Trust Fund FY2011: Impact Evaluation Research Program ¹ ² ³ Country / Allocated - Receipts - Committed - Allocated Receipts Committed Disbursed Impact Evaluation Studies Num. IE Task Team Leader TF No. Receipts Committed Disbursed Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Studies ? Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ 1.1. Quick Win IE Studies 4 - TF080237 608,661 608,661 608,661 608,661 0 0 0 Tanzania's Social Action Funds (TASAF II) Tanzania Berk Ozler TF090990 169,907 169,907 169,907 169,907 0 0 0 Long Term Follow-Up of Beneficiaries of a Jamaica Christel Vermeersch TF090992 57,065 57,065 57,065 57,065 0 0 0 Simulation Program Early Childhood Development (ECD) Brazil Pedro Olinto TF091145 119,599 119,599 119,599 119,599 0 0 0 Maharashtra and Orissa Rural Water Supply Priti Kumar, Rita Klees, India TF091146 125,000 125,000 125,000 125,000 0 0 0 and Sanitation Soma Ghosh Moulik 1.2. Innovation Fund IE Studies 11 - TF080238 2,046,025 2,042,125 1,320,867 838,136 3,900 721,258 482,731 1.2.1. Human Development 4 - - 746,275 746,075 706,849 549,505 200 39,226 157,344 TIVET Vouchers IE Project Kenya Shobhana Sosale TF095457 262,200 262,000 262,000 222,700 200 0 39,300 Citizen Report Card Project (Phase II) Uganda Damien de Walque TF093769 151,050 151,050 143,175 134,031 0 7,875 9,144 Ex-Combatant Reintegration and Liberia Mattias Lundberg TF094186 333,025 333,025 301,674 192,774 0 31,351 108,900 Peacebuilding in Liberia Effects of Home Based HIV Counseling and Kenya Markus P. Goldstein TF093925 0 0 0 0 0 0 0 Testing ? 1.2.2. Social Development 7 - - 1,299,750 1,296,050 614,018 288,631 3,700 682,032 325,387 Effect of Informal Settlement Upgrading on Arianna Legovini, South Africa TF094908 300,000 296,800 184,647 139,047 3,200 112,153 45,600 Health in S. Africa Nandini Krishnan Domestic Use of Irrigation Water Ethiopia Arianna Legovini TF094732 147,500 147,000 130,243 97,843 500 16,757 32,400 GPOBA/SIEF Uganda Water IE Uganda Lars Johannes TF094543 123,000 123,000 0 0 0 123,000 0 Innovative Treatment Solutions for Rural Kenya Arianna Legovini TF094805 200,000 200,000 167,610 33,522 0 32,390 134,088 Drinking Water Sources Access to Water Infrastructure and HD Paraguay Miguel Vargas-Ramirez TF096523 182,850 182,850 6,320 6,320 0 176,530 0 Outcomes in Rural Areas Sewerage Access and Health Outcomes for Uruguay Carlos E. Velez TF096575 151,400 151,400 120,300 7,001 0 31,100 113,299 Households in Urban Areas Water Rights Formalization for Agricultural Peru Marie-Laure Lajaunie TF096538 195,000 195,000 4,899 4,899 0 190,101 0 Use and HD Outcomes 8 Table 3.- Spanish Impact Evaluation Trust Fund FY2011: Impact Evaluation Research Program ¹ ² ³ (cont.) Country / Allocated - Receipts - Committed - Allocated Receipts Committed Disbursed Impact Evaluation Studies Num. IE Task Team Leader TF No. Receipts Committed Disbursed Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Studies ? Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ 1.3. Cluster IE Studies 36 - - 7,265,691 7,031,133 4,416,920 3,004,286 234,558 2,614,213 1,412,634 Sebastian Martinez, 1.3.1 - Health Contracting / Pay For 5 Adam Wagstaff, Pia TF080239 1,357,681 1,356,042 856,935 470,728 1,639 499,107 386,208 Performance Reforms Schneider Plan Nacer - Child-Mother Provincial Sebastian Martinez, Argentina TF094350 314,766 314,766 255,606 169,039 0 59,160 86,567 Investment Project Rafael Cortez Performance-Based Contracting for General Rwanda Christel Vermeersch TF092299 194,800 194,800 144,961 113,101 0 49,839 31,860 Health and HIV Services John Langenbrunner, Paying for Performance in China's Health China Magnus Lindelow, TF094336 400,000 400,000 35,944 35,944 0 364,056 0 Sector (Evaluation of Health XI) Adam Wagstaff Poverty Reduction Support Credits Impact Benin Christophe Lemiere TF094370 296,400 296,000 296,000 45,079 400 0 250,921 on Maternal Health Innovative Health Insurance for India's Poor India Robert Palacios TF096038 100,000 100,000 73,948 57,088 0 26,052 16,860 Margaret Grosh, 1.3.2 - Conditional Cash Transfers 6 Phillippe Leite, TF080240 1,002,736 952,736 604,475 493,443 50,000 348,261 111,032 Emmanuela Galasso Conditional and Unconditional Cash Burkina Faso Damien de Walque TF094734 224,600 224,600 210,543 110,405 0 14,057 100,138 Transfers in the Nahouri Province CCT program in Rural Education in Rebekka E. Grun, Najy Morocco TF093303 494,211 444,211 285,857 279,163 50,000 158,354 6,694 Morocco Benhassine Emanuela Galasso, Estimating the Dynamic Effects of Chile Pedro Carneiro, Theresa TF093505 108,120 108,120 108,074 103,874 0 46 4,200 Supportive Chile (Chile Solidario) Jones Evaluating and Improving the Indonesian Indonesia Susan Wong TF095573 15,805 15,805 0 0 0 15,805 0 Conditional Cash Transfer Anush Bezhanyan, Community Based CCT Pilot Program Tanzania TF097111 60,000 60,000 0 0 0 60,000 0 Manuel Salazar Contigo Vamos por Mas Oportunidades Mexico Theresa Jones TF096356 100,000 100,000 0 0 0 100,000 0 program in Guanajuato Jed Friedman, Arianna 1.3.3 - Malaria Control 5 TF080241 752,735 753,075 519,206 418,484 -340 233,869 100,722 Legovini The Role of Malaria Control in Improving Donald Bundy, Simon Kenya TF094562 71,587 71,587 71,587 71,587 0 0 0 Education Brooker The Role of Malaria Control in Improving Donald Bundy, Simon Senegal TF094564 71,247 71,587 71,587 42,952 -340 0 28,635 Education Brooker Improving Malaria Outcomes through Zambia Jed Friedman TF094147 202,687 202,687 129,236 129,236 0 73,451 0 Evidence-Based Program Design Improving Malaria Outcomes through India Jed Friedman TF093253 243,007 243,007 171,214 99,127 0 71,793 72,087 Evidence-Based Program Design Community Distributors and Patent Arianna Legovini, Edit Nigeria TF095606 164,207 164,207 75,582 75,582 0 88,625 0 Medicine Vendors for Malaria Control V. Velenyi 9 Table 3.- Spanish Impact Evaluation Trust Fund FY2011: Impact Evaluation Research Program ¹ ² ³ (cont.) Country / Allocated - Receipts - Committed - Allocated Receipts Committed Disbursed Impact Evaluation Studies Num. IE Task Team Leader TF No. Receipts Committed Disbursed Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Studies ? Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ 1.3. Cluster IE Studies 36 - - 7,265,691 7,031,133 4,416,920 3,004,286 234,558 2,614,213 1,412,634 Rita Almeida, Mattias Lundberg (from Jan 1.3.4 - Active Labor Market Programs / 8 2010), Christel TF080242 1,333,124 1,188,124 931,187 701,996 145,000 256,937 229,191 Youth Employment Vermeersch (until Dec 2009) India National Rural Employment India Rinku Murgai TF093639 290,000 290,000 251,755 217,021 0 38,245 34,734 Guarantee Evaluation IE - YDP, Youth Development Program Dom. Rep. Cornelia Tesliuc TF092383 199,625 199,625 185,582 105,215 0 14,043 80,367 NUSAF Youth Opportunities Program Uganda Sebastian Martinez TF092991 334,860 334,860 316,612 261,473 0 18,248 55,140 (YOP) Sangeeta Raja SIEF-Cluster 4-Malawi Malawi TF096079 200,000 200,000 159,015 100,065 0 40,985 58,950 Jobanputra Economic Empowerment of Adolescent Liberia Mattias Lundberg On track 70,000 0 0 0 70,000 0 0 Girls Turning Theses into Enterprises Tunisia Stefanie Brodmann TF096014 55,000 55,000 8,223 8,223 0 46,777 0 Cristobal Ridao-Cano, Effectiveness of Turkish National Turkey Sarojini Hirshleifer, Rita TF097379 98,640 98,640 0 0 0 98,640 0 Employment Agency Trainings Almeida A Youth Wage Subsidy Experiment for South Africa Milan Vodopivec TF097726 75,000 0 0 0 75,000 0 0 South Africa Harry A. Patrinos, Deon 1.3.5 - Basic Education Accountability 5 P. Filmer, Barbara TF080243 982,913 929,039 722,576 446,893 53,874 206,463 275,683 Bruns Pernambuco and Sao Paulo Performance Barbara Bruns, Claudio Brazil TF094053 228,874 175,000 80,757 80,757 53,874 94,243 0 Pay Reform for School Teachers Ferraz, Marcos Rangel Nepal Community Managed School project Nepal Nazmul Chaudhury TF093733 150,000 150,000 127,769 116,992 0 22,231 10,776 AGEs - Support to School Management Mexico Harry A. Patrinos TF094146 292,739 292,739 276,313 204,347 0 16,426 71,966 Parental Empowerment Role of Assessment to Provide Information Liberia Nathalie Lahire TF094418 211,300 211,300 189,180 10,200 0 22,120 178,980 for Accountability Randomized Evaluation of Andhra Pradesh Venkatesh India TF095778 100,000 100,000 48,556 34,596 0 51,444 13,960 Primary Education Sundararaman 10 Table 3.- Spanish Impact Evaluation Trust Fund FY2011: Impact Evaluation Research Program ¹ ² ³ (cont.) Country / Allocated - Receipts - Committed - Allocated Receipts Committed Disbursed Impact Evaluation Studies Num. IE Task Team Leader TF No. Receipts Committed Disbursed Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Studies ? Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ 1.3. Cluster IE Studies 36 - - 7,265,691 7,031,133 4,416,920 3,004,286 234,558 2,614,213 1,412,634 David Wilson, Damien 1.3.6 - HIV/AIDS Prevention 3 TF080244 1,086,501 1,102,217 727,846 444,382 -15,716 374,371 283,464 De Walque Evaluating CCT to Prevent HIV and Other Tanzania Damien De Walque TF092758 355,184 371,400 277,577 89,922 -16,216 93,823 187,655 STIs Evaluating the Impact of HIV Prevention Lesotho Damien De Walque TF091507 384,680 384,680 128,325 120,000 0 256,355 8,325 Campaign for the Youth CCT, Schooling, and HIV Risk Malawi Berk Ozler TF092384 346,637 346,137 321,944 234,460 500 24,193 87,484 Harold Alderman, 1.3.7 - Early Childhood Development 4 - 750,000 749,900 54,695 28,360 100 695,205 26,335 Sophie Naudeau Impact Evaluation and Cost-Benefit Analysis Cambodia Deon Filmer TF097107 211,000 211,000 0 0 0 211,000 0 of 3 types of FTI-funded ECD Interventions Nadie es Perfecto, IE of a Program to Chile Emanuela Galasso TF097130 200,000 200,000 17,833 10,458 0 182,167 7,375 Improve Parenting Skills IE of ECD in Mozambique Mozambique Sophie Naudeau TF096544 165,000 164,900 36,863 17,903 100 128,037 18,960 Atencion a Crisis Early Childhood Nicaragua Renos Vakis TF096907 174,000 174,000 0 0 0 174,000 0 Development & Conditional Cash Transfer Total SIEF Impact Evaluation program - - TF096307 9,920,377 9,681,919 6,346,449 4,451,083 238,458 3,335,471 1,895,365 Country / Allocated - Receipts - Committed - Allocated ² Receipts ² Committed ² Disbursed ² Impact Evaluation Studies Num. IE Task Team Leader TF No. Receipts Committed Disbursed Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ Studies ? Act_FY2011 ¹ Act_FY2011 ¹ Act_FY2011 ¹ S. Martinez, A. SIEF Cluster 1 Coordination World TF092886 185,000 185,000 19,290 19,290 0 165,710 0 Wagstaff, and P. E. Galasso, M. Grosh, SIEF Cluster 2 Coordination World TF096850 185,000 185,000 22,157 22,157 0 162,843 0 P. Leite J. Friedman, A. SIEF Cluster 3 Coordination World TF093306 185,000 185,000 82,057 69,661 0 102,943 12,396 Legovini R. Almeida, M. SIEF Cluster 4 Coordination World TF092151 185,000 185,000 80,833 56,833 0 104,167 24,000 Lundberg H. Patrinos, D. Filmer, SIEF Cluster 5 coordination World TF094050 185,000 185,000 49,341 45,714 0 135,659 3,627 B. Bruns D. De Walque, D. SIEF Cluster 6 coordination World TF094039 185,000 185,000 49,829 49,829 0 135,171 0 Wilson SIEF Cluster 7 coordination World H. H. Alderman TF096307 185,000 185,000 8,440 8,440 0 176,560 0 Total Cluster Coordination - - - 1,295,000 1,295,000 311,948 271,925 0 983,052 40,023 NOTES ¹ Fiscal Year Based. Updated as of August 31, 2010. ² For details regarding the types of financial flows and indicators used in this report, see Box C. ³ Data only reflect SIEF support; other sources of financing (World Bank, DFID…) are not included. ? Number of impact evaluation studies. Abandoned projects (in brackets) that did not generate any financial flows and have not been counted. Projects that have been funded through two different SIEF windows are shown in the same line. ? Impact evaluation study fully funded by the DFID IE TF. . 11 Capacity Building: In the first half of 2010 the SIEF team and faculty successfully delivered three impact evaluation regional workshops in Kathmandu, Nepal (February 22-26, 2010); Rio de Janeiro, Brazil (April 26-30, 2010); and Accra, Ghana (May 24-28, 2010). Since the SIEF program of workshops was launched in January 2008, 12 impact evaluation workshops have been delivered. In total, 1,905 participants among evaluation experts, researchers and managers of programs affecting Human Development outcomes have been trained in the design, implementation and management of impact evaluations (see Table 4). Table 4. - SIEF Impact Evaluation Workshops FY2009-FY2010: Main Figures Location Date Countries Attending Participants Project Teams Cairo, Egypt January 13-17, 2008 12 160 17 Managua, Nicaragua March 3-7, 2008 11 104 15 Madrid, Spain June 23-27, 2008 1 196 9 Manila, Philippines December 1-5, 2008 6 137 16 Lima, Peru January 26-30, 2009 9 182 18 Amman, Jordan March 8-12, 2009 9 206 17 Beijing, China July 20-24, 2009 1 212 12 Sarajevo, Bosnia September 21-25, 2009 17 115 12 Cape Town, South Africa December 7-11, 2009 10 108 11 Kathmandu, Nepal February 22-26, 2010 7 118 15 Rio de Janeiro, Brazil April 26-30, 2010 3 224 22 Accra, Ghana May 24-28, 2010 17 143 19 Total 103 1905 183 The workshops have received extremely high quality and usefulness ratings from participants, as shown in Table 5. The impact evaluation regional workshops program has raised much interest not only in client countries, but also within the Bank and in other agencies. This has allowed to leverage additional funds from other units and donors for these workshops4. As a result, 87% of the SIEF capacity building budget has been disbursed. (see Table 2). 4 The DFID supported the workshops in Kathmandu and Rio de Janeiro with $191,539 and $35,000, respectively. Other World Bank units contributed to financing and delivering the workshops in Rio de Janeiro (LAC and DIME, with $15,000 and $100,000, respectively) and Accra (HDNSP, with $95,000). 12 In FY2011 the SIEF will continue supporting regional impact evaluation workshops in collaboration with other World Bank units and donors. Until the end of calendar year 2010 the SIEF will support two impact evaluation regional workshops: in Tunisia (October 18-22, 2010) –mainly financed by the HRBTF–; and in South Korea (December 6-10, 2010) –mainly financed by the Government of South Korea. Table 5. - SIEF Impact Evaluation Workshops FY2009-FY2010: Participants feedback¹: How would you rate the How would you rate the Location Date overall quality of the overall usefulness of the activity? ¹ activity? ¹ Cairo, Egypt January 13-17, 2008 4.07 4.14 Managua, Nicaragua March 3-7, 2008 4.83 4.84 Madrid, Spain June 23-27, 2008 3.94 3.51 Manila, Philippines December 1-5, 2008 - - Lima, Peru January 26-30, 2009 4.43 4.54 Amman, Jordan March 8-12, 2009 3.93 3.88 Beijing, China July 20-24, 2009 4.38 4.10 Sarajevo, Bosnia September 21-25, 2009 4.37 4.35 Cape Town, South Africa December 7-11, 2009 4.29 4.16 Kathmandu, Nepal February 22-26, 2010 4.33 4.22 Rio de Janeiro, Brazil April 26-30, 2010 4.37 4.38 Accra, Ghana May 24-28, 2010 - - Average 4.29 4.19 ¹ Participant rating on a scale of 1 to 5 (1=minimum, 5=maximum). 13 Results Dissemination: The SIEF results dissemination program was initiated in 20095, and has made substantial progress in the first eight months of 2010. Some of the key milestones achieved since January 2010 are: - Participation at the Network of Networks of Impact Evaluation (NONIE) meeting in Bonn (March 29-30, 2010); - Participation at the World Bank Independent Evaluation Group Week Fair (April 19-23, 2010); - Presentation of the SIEF program at the Overseas Development Institute in London (May 4, 2010). - The handbook Impact Evaluation in Practice was finalized in July, and sent for publication in August. - All the dissemination proposals from the Cluster Leaders were submitted, reviewed and approved, and a total of $700,000 were transferred (100% of the resources allocated to cluster dissemination activities). Until the end of the year the SIEF team will: - Keep participating and conducting sessions disseminating the results of the fund in international external and internal events, including the International Summit of the Millennium Development Goals Forum in New York (September 2010); and the World Bank Independent Evaluation Group Week in Washington, DC (September 2010); - Launch the new policy notes series From Evidence to Policy. The series will explain the policy questions addressed and the evidence generated by each cluster in an easy to read manner. - Publish the handbook Impact Evaluation in Practice, and present it in international academic institutions and development forums. Management / Advisory Services: As shown in Table 2, the share of management costs in total SIEF disbursements was 5.97% at the end of August 2010, including the payments made for program management (2.06%) and the payments made by cluster leaders for coordination activities (3.91%). These figures show a relative increase compared to the end of calendar year 2009, when the share of management costs in total SIEF disbursements was 4.66% (2.70% for program management and 1.96% for cluster coordination activities). Additionally, the time and costs associated to the delivery by SIEF core team members of advisory services to support the design and implementation of impact evaluations by task teams –a budget category more related to the impact evaluation research program than to management – was 8.16% of SIEF total disbursements. This share is similar to the 7.85% shown at the end of calendar year 2009. 5 The SIEF Dissemination Plan was approved by the Steering Committee on December 3, 2009. 14