The Independent Assessment of the Japan Social Development Fund ( JSDF) Vo l u m e I – F i n a l R e p o r t December 29, 2014 Volume I – Final Report Universalia 245 Victoria Avenue, Suite 200 Westmount, Quebec Canada H3Z 2M6 www.universalia.com Volume I – Final Report Executive Summary  1. Purpose and Background  Established in 2000, the Japan Social Development Fund (JSDF) is a partnership between the Government of Japan (GoJ) and the World Bank Group (WBG). It is designed to provide grant assistance to initiatives that promote innovative approaches to addressing human development challenges in low and lower‐middle income countries of the WBG. The Fund is targeted toward relatively small‐scale pilot interventions of up to US$ 3 million in overall value. It is administered by the Trust Funds and Partnerships Department (DFPTF) within the WBG’s Development Finance Vice Presidency (DFi). In 2007, a first evaluation of the JSDF was undertaken. During annual consultations held in December 2012, the GoJ and the then CFPTO (now DFPTF) agreed to undertake an independent assessment of the Fund since the last review in 2007. The overall objectives of the Assessment are to contribute to the on‐going efforts to enhance the results of the Fund and improve its operation. More specifically, the Assessment reviewed: How JSDF grants are contributing to poverty alleviation among beneficiaries and how grants are helping to sustain livelihoods of beneficiaries; and, What operational and administrative issues and problems stand in the way of the efforts to improve the Fund’s operations. The Statement of Work for the Assessment is presented in Volume I, Appendix VIII of the Final Report. 2. Methodologies  The Assessment was based largely on a review of 53 JSDF grants approved and implemented between 2007 and 2012. A standardized assessment tool (the Grant Assessment Tool, presented in Volume II of the Final Report) was developed to assess all 53 grants. The major approaches used to undertake the Assessment were: Portfolio Review: The desktop portfolio review (largely a document review of World Bank project files) focused on assessment of 39 JSDF grants through document review, which represents 31% of the grants that had been approved since 2008. Field Missions: The assessment involved field missions in 9 countries to undertake an in‐depth review of 14 JSDF grants. Each field mission required one week of in‐country activities and involved interviews with World Bank personnel and with the project Task Team Leader (TTL), government representatives, as well as with representatives of the Government of Japan. The implementing partner (either a government agency or a civil society organization) was engaged by means of interviews and small group meetings. Meetings with beneficiaries were held in their own communities. Electronic Survey: An electronic survey was administered to 190 TTLs. It focused on issues related to the five axes of analysis, Relevance, Sustainability, Effectiveness, Efficiency and Programme Management.1 1 The response rate to the survey was over 60% and as such was considered to be statistically significant. i Universalia Volume I – Final Report 3. Highlights of Findings  Two extensive chapters of Findings were developed for this Assessment, one focusing on the contribution of JSDF grants to poverty reduction and livelihood support and the second on issues related to the management of the JSDF granting mechanism. Some 26 specific Findings are presented, with this Executive Summary highlighting the major conclusions derived from those Findings. Before discussing these major conclusions, it is important to stress that the Assessment reviewed a selection of Grants that were approved and largely implemented between 2007 and 2012. Therefore, their assessment was based on the then‐existing guidelines as well as application/ implementation procedures in force at that time. As well, it is important to underscore that, unlike many subject‐specific trust funds, the accountability for the administration of the JSDF is divided between its program staff located in DFPTF who have responsibility for the application and grant approval process, and TTLs and their sector and country level managers who assume accountability for grant implementation, with DFPTF staff having only indirect oversight. This is most apparent in JSDF grant agreements which are between the World Bank represented at the country level and the recipients, and not involving the JSDF program management located in DFPTF. a. The Contribution and Overall Worth of the JSDF i. Fundamental Characteristics of the JSDF The JSDF is a granting mechanism that is unlike many other World Bank administered trust funds. More specifically, it is not focused on a specific sector or thematic area, but rather designed as a multi‐sectorial approach to combating poverty by filling gaps, testing innovative and pioneering ideas and promoting community‐based solutions. There is a strong body of evidence drawn from desk top portfolio reviews and field missions to demonstrate that the JSDF grants filled gaps or met previously unaddressed needs. The range of JSDF gap filling is widespread. It encompasses not only filling national priority gaps or gaps in the World Bank, national, bilateral or other multilateral financing. It also empowers beneficiary populations to take control of their lives. In many respects, the JSDF gives a voice to the poor and the disenfranchised. JSDF granting activity has been noteworthy in the degree to which it has promoted the introduction of innovative practices. Numerous examples have been found of grants that clearly introduced new ideas and new ways of working to address poverty reduction and/or livelihood support within the context of the particular country or situation. This conclusion is based on the portfolio review, the field missions and the analysis of the operational practices of the JSDF during the assessment period. This Assessment Team was also asked to expand on the nature of innovation. This Assessment concludes that innovation within the framework of the JSDF should not be considered in terms of whether the actions constitute a “global first”. Rather, what is more important is whether the activities will be new, first time practices within the country in question. Likewise, innovation should also be understood to constitute bringing to the World Bank new ways of reaching out to the neediest and also new ways of delivering projects. Indeed, many examples were found of grants that introduced new ideas and new ways of working to address poverty reduction and/or livelihood support within the context of the particular country or situation, thus confirming the processes undertaken by the JSDF Secretariat and the Steering Committee during the period under review. At the same time, JSDF grants have not only provided a mechanism for the World Bank to promote piloting the introduction of new techniques into a given country, they have contributed to Universalia ii Volume I – Final Report the strengthening and broadening of the World Bank’s overall capacity to respond to country level challenges in an innovative fashion. One of the generally held views about trust funds that provide small grants with a short time span is that the work that has been supported is not sustainable2. Such is not the case with the JSDF. Some 66% of individual JSDF grants acting as pilots have resulted in expansion of the pilot project and thus demonstrate leverage. More importantly, the leverage and sustainability promoted by the JSDF transcends simply the continuation of the activity in question. There are numerous examples of how the lessons learned from a JSDF pilot activity have been transplanted into on‐going national development programming, thus expanding the scope of the benefits; and also integrated into the larger work programs of the World Bank itself, again resulting in an expansion of the positive benefits pioneered by the JSDF. The JSDF is based on a premise that its activities should be in response to the needs that beneficiaries themselves articulate. This Assessment showed that beneficiaries were consulted. This consultation tended to be concentrated during the initial design phase and thus contributed to the design of JSDF‐supported initiatives. There was variation in the degree of detail regarding this consultation in the application documents. However, field missions clarified the extent of beneficiary participation in the design of JSDF‐supported activities. Given global commitments to the harmonization of objectives across development cooperation activities as laid out in the Paris and Busan Declarations, this Assessment reviewed the congruity of the JSDF with World Bank and GoJ priorities. It found that individual JSDF grants are aligned at the country level with World Bank strategies and priorities along with national development plans, and are also in line with the Priority Policy for International Cooperation of the Government of Japan. Akin to this latter consideration, the Assessment Team was also tasked with the review of the extent to which, during implementation, the contribution of Japan was highlighted. Visibility at both the level of individual grants and at the programmatic level is broadly in line with the existing guidelines. But grant implementation is affected by the dual lines of accountability that characterize the JSDF program model. That is, while the JSDF program staff has full accountability for the process up to and through the grant application and signing of the grant agreement with a World Bank country office, responsibility and accountability for implementation, including ensuring the visibility of the contributions made by Japan, rests with TTLs and their World Bank country level management. There was a degree of unevenness in regard to the visibility of Japan that will require clarification so as to better showcase its contributions. ii. The Overall Worth of the JSDF One of the most important questions facing this Assessment was to analyse the overall worth of the JSDF in terms of contributing to poverty reduction and providing livelihood support. Overall, individual JSDF grants were aimed at contributing to poverty reduction and providing livelihood support on an individual project basis. They did so by tackling development issues that prevent beneficiaries from overcoming poverty and from improving their livelihood. The review of 53 grants showed a high level of output and first level outcome attainment. Furthermore, the activities supported by JSDF grants have reflected the complexities inherent in combating poverty by introducing a diverse range of mechanisms to address deeply rooted multi‐dimensional challenges. 2 A number of the stakeholders interviewed for this assessment voices similar views. iii Universalia Volume I – Final Report This positive conclusion is tempered to some degree that while the JSDF along with many other World Bank programs did not have a formal results framework, it implicitly had the key elements of one. Specifically, the design of the JSDF required that individual grants demonstrate the degree of their contribution to country‐level objectives (CAS or CPF). These were in turn inherently linked to higher level World Bank objectives. b. Operational Considerations i. Planning and Reporting Instruments During the period under review, 2007‐2012, available standardized grant application documents were thorough. However, some minor variances in the level of detail provided in grant files made it difficult to assess the quality of a few aspects of the application process. This general conclusion reflects several specific Findings that address the adequacy of the application process. In general, the capacities of the implementing partners are analysed through a flexible assessment process that contributes to effective implementation of the grants. Portfolio Review and interviews indicated a lack of familiarity with World Bank fiduciary procedures on the part of implementing agencies (generally specific financial management, accounting and administrative requirements) , a fact which had consequences in terms of a delay in implementation. It bears noting that the recent introduction across the World Bank of a new end‐of‐grant reporting instrument, the Implementation Completion Report (ICR), will significantly enhance overall grant reporting by providing an increased emphasis on outcome performance. As well, the development of a JSDF program‐wide results framework, now underway as part of the World Bank's overall new focus on results, also will likely have a positive impact on the JSDF. ii. Timing Issues Two issues related to timeliness emerged during this Assessment. The first related to the amount of time required to review applications. Based on the data that was available from the grant sample examined by this Assessment and during the timeframe in question (2007‐2012), the average time taken by the GoJ to review an application approached twice the anticipated standard of some two months. The causes for this situation are complex, largely involving factors beyond the control of the World Bank such as turnover among staff at the Ministry of Finance in Japan, lack of common understanding about key issues such as the nature of innovation, and the need to conduct intra‐governmental consultations before final approval. The second timing issue related to the amount of time required from approval of the proposal by Japan to the signing of the grant agreement with implementing partners, an average of in excess of 200 days for the sample of grants reviewed by this Assessment. The reasons for this length of time are varied. They range from the implications of the mandatory requirement of engaging many parts of the World Bank through to the consequences of TTL turnover. As well, the requirement for legislative ratification by recipient governments in some instances, along with changing country‐ level contextual circumstances that result in need for changes to the original proposal can impact on this period of time. The recently established World Bank Small Grant Guidelines and Procedures require the development of a draft grant agreement in advance of formal proposal approval by GoJ. This has reduced the time between the approval by Japan and the signing of the grant agreement. iii. Resource Considerations During the period under review, the JSDF unit was staffed by one full‐time professional, who was assisted by three part‐time colleagues, for a total of 1.9 staff years (including fractional allocations for administrative purposes, but not including staff time of World Bank centralized units such as Universalia iv Volume I – Final Report Legal, Comptroller, etc.). This level of resourcing was and remains less than that for comparable trust funds highly similar in design and purpose that Japan has established at the Asian Development Bank, the Inter‐American Development Bank and the Office of the Secretary General of the United Nations. This Assessment pointed to the extent to which the JSDF mechanism is facing challenges to meet the implications of the administration of an on‐demand global initiative that promotes innovation and catalytic activities. More importantly however, this Assessment points toward an increased overall workload, with additional responsibilities evolving in some of the following areas: Coordinating the grant application process and playing an increased challenge function so as to improve the results focus of the JSDF; More active monitoring of a portfolio in excess of 110 grants, including Implementation Status Reports (ISR), Grant Reporting and Monitoring (GRM), the Implementation Completion Memorandum (ICM), and the Implementation Completion Report (ICR); Maintaining more active liaison with representatives of the GoJ; Conducting orientation and sensitization missions to raise the profile of the JSDF; Organizing workshops and seminars for GoJ stakeholders, practitioners, academia and beneficiaries; Increased mounting of oversight missions with respect to specific grants to ensure effective implementation as well as to facilitate the resolution of implementation issues, and linkage to the new program‐level results framework; Analysing and synthesizing individual grant results within a new program‐wide results framework so as to better demonstrate how the JSDF contributes to larger World Bank‐ wide objectives; and Developing more sensitive reporting on the visibility of Japan by monitoring and playing a challenge function with respect to the new Implementation Completion Reports. Recommendations  The following Recommendations are not designed to be narrowly prescriptive, but rather to capitalize on the strengths of the program. Recommendations are presented in two broad categories: Strategic ‐ These Recommendations are designed to reiterate the strengths of the JSDF, clarifying its role and helping it to further strengthen its alignment with the overall strategies of the World Bank. Operational ‐ These Recommendations speak to more operational matters and specifically address some of the issues identified in various Findings. a. Strategic Several recommendations are presented that are primarily strategic in nature. They address the need for the World Bank Group and the Government of Japan to reaffirm their overall commitment to the JSDF mechanism as a whole. Such an affirmation would provide a solid underpinning on which the JSDF can continue to evolve. Recently, the World Bank as a whole has begun to introduce program‐level results frameworks across all its various activities. In the context of the JSDF, this development is welcome. The introduction of a program‐wide results framework will enable the JSDF to better demonstrate the relationship between the community level and grass v Universalia Volume I – Final Report roots activities it supports and the overall strategic objectives of the World Bank, namely to reduce extreme poverty, and increase shared prosperity. During this assessment there were discussions about the most appropriate long‐term organizational fit for the JSDF so as to maximize its strategic relevance for the World bank as a whole. A range of suggestions were made, primarily concentrating on either retention within the elements of the World Bank that are transverse in nature like DFPTF or GPSOS, or moving the JSDF to the newly emerging Social , Urban, Rural and Resilience (SURR) Global Practice On balance, the JSDF is best located in an element of the World Bank that is cross‐cutting in nature like DFPTF or GPSOS, given the multidisciplinary nature of the JSDF. The JSDF, because it is not a subject‐specific trust fund like so many others, transcends what would normally be considered as social development or social protection‐related activities. Therefore, it should be located in an environment which in itself is more transverse and which will enable the JSDF to better engage all elements of the World Bank. b. Operational Several operational level recommendations are presented. They fall largely into two categories. The first category addresses issues related to requiring TTLs to provide additional information with respect to beneficiary participation in project design, or the nature of the relationship between the proposed grant and the new JSDF program‐wide results framework. The second category concentrates on more structural or policy‐related issues. Since its inception, the JSDF has utilized seed fund grants to provide advance funds to Task Team Leaders to conduct grassroots consultations with potential beneficiaries and local stakeholders. This assessment however demonstrated that the use of seed fund grants has not necessarily resulted in any difference in overall grant performance. As well, Findings also show that there were delays in the overall implementation of JSDF grants that were caused in part by capacity challenges that faced various implementing partners; and specifically, challenges related to World Bank fiduciary guidelines, for example, banking, accounting, internal reporting and procurement matters. Furthermore, it was noted that collecting baseline data at the onset of project implementation rather than at the project design stage caused delays in starting project activities. Therefore, it is recommended that the funds currently set aside for seed fund grants be re‐profiled to not only provide support for consultations and concept note preparation but also capacity building among implementing agencies and baseline data collection following clearing of concept note by Japan. In this way, some delays in grant implementation could be mitigated. Given the dual accountability for the JSDF, with the administration of the implementation of grants being the responsibility of individual TTLs, several recommendations have been made which would provide greater specificity with respect to periodic reporting by TTLs. Within that context, the recent introduction of the Implementation Completion Report (an end‐of‐grant evaluation‐like instrument) within the context of Small Grants across the World Bank as a whole, will help capture more results‐focused information. However, with this instrument comes the need to have sufficient resources to be able to analyse and synthesize their results‐oriented information so as to better demonstrate the linkages of JSDF activities with overall World Bank objectives. Ensuring the visibility of the Government of Japan throughout the implementation of JSDF grants is essential. However, as was noted above, there is some unevenness with respect to this matter that is largely caused by the dual lines of accountability where individual TTLs and not the JSDF, are responsible for grant implementation. To address this issue, it would be appropriate that Japan and DFPTF request Task Teams to provide an annual update on actions taken to promote Japan’s visibility. Universalia vi Volume I – Final Report Throughout the life of the JSDF, there have been some concerns expressed about the timeliness of the application process. While recent decisions to implement a rolling application process, as opposed to the prior fixed “round” approach, appear to have resulted in some modest time‐saving, more needs to be done to improve the timeliness of the application process. Fully applying the current “assumed consent” model is suggested, recognizing that the current two month review period should be seen a maximum needed by the Government of Japan to assess a grant proposal. Assumed consent implies that a grant would be considered to be approved unless specific direction to the contrary were given within an agreed upon period of time. Finally, the issue of the adequacy of the resources allocated to the administration of the JSDF needs to be reviewed. The JSDF operates with substantially fewer resources than analogous trust funds in the Asian Development Bank, the Inter‐American Development Bank or the Office of the Secretary General of the United Nations. With increased emphasis being placed on managing for results and improving the sensitivity of the JSDF to higher level World Bank objectives, early consideration should be given to discussions which would provide additional resources for the JSDF to meet new performance challenges. This Assessment presented two broad categories of Recommendations: Strategic and Operational. Recommendations however need to be categorized on the basis of the degree of their urgency. Two categories related to urgency are presented below, along with the identification of responsibility and overall timeliness. These two categories are: Urgent to implement – Recommendations which are central to the strategic direction of the JSDF mechanism as a whole and which underpin all subsequent recommendations. Important to implement – Recommendations that are largely operational in nature and constitute a range of activities that can be implemented by Development Finance and whose implementation would be highly important to improving the day‐to‐day operations of the JSDF mechanism and strengthening its contribution to the attainment of World Bank strategic objectives. All these Recommendations have a relatively short timeframe, a maximum of two years. As well, in line with the Statement of Work, responsibilities for implementation are suggested. Recommendation Matrix Recommendation Responsibility Timeliness Urgent to implement Immediate (within 6‐8 months) Recommendation 1: The World Bank Group needs to DFPTF/ the Senior Immediate reaffirm the value of the JSDF partnership with the Management of the Government of Japan to maintain the JSDF at, as a minimum, World Bank Group and the present funding level over the next three years. GoJ Recommendation 2: DFPTF managers should rapidly DFPTF Immediate finalize a strategic results framework for the JSDF so as to articulate how, as an innovative global program, it contributes to and is part of the hierarchy of evolving World Bank priorities and is aligned with the strategic goals of reducing extreme poverty and increasing shared prosperity. vii Universalia Volume I – Final Report Recommendation Responsibility Timeliness Recommendation 3: The JSDF mechanism should be located DFPTF and The Senior Immediate within an organization in the World Bank which possesses a Management of the transverse mandate, for example Development Finance or World Bank Group the Global Practices Strategies and Operations (GPSOS) unit. Recommendation 4: DFPTF and the GoJ should agree on a DFPTF and GoJ Immediate more explicit common understanding regarding the nature of innovation. Recommendation 5: DFPTF and the GoJ should revisit the DFPTF and GoJ Immediate current guidelines which address issues related to the visibility of Japan to more explicitly articulate what are minimum visibility criteria, what should be expected in different circumstances and what should be submitted to DFPTF to demonstrate visibility. Recommendation 6: In order to improve the results DFPTF Immediate orientation of the JSDF grant monitoring, DFPTF should provide additional guidance related to the presentation of periodic report writing and should revisit the application format and content of the application package. Recommendation 8: DFPTF and the GoJ need to revisit the DFPTF and GoJ Immediate current approach to formal approval of grants by the GoJ in an effort to reduce the current delays and to operate within existing agreements which are based on the principle of assumed consent. As well, the establishment of an Advisory Board to conduct a first level of review of proposals could be considered. Recommendation 10: DFPTF and the GoJ need to rapidly DFPTF and GoJ Immediate enter into negotiations to provide additional human and financial resources so that the JSDF can become more results‐oriented. Important to implement Recommendation 7: As a bridging mechanism, the GoJ DFPTF and GoJ Within 1 program should set aside a specific sum for up to three years to be year managed by DFPTF to accommodate carrying out a small number of ICRs per year. Recommendation 9: DFPTF and the GoJ need to amend their DFPTF and GoJ Within 1 program agreements to provide that the current seed fund grant be year transformed into a more flexible tool that would provide for a degree of advance spending so as to strengthen the capacity of implementing agencies to meet the mandated fiduciary requirements of the World Bank. Universalia viii Volume I – Final Report Acronyms  ADB Asian Development Bank AFR Africa CCT Conditional Cash Transfers CDD Community Driven Development CFPVP Concessional Finance and Global Partnership Vice Presidency CODEM Operations Center for Disaster Emergency CPS Country Partnership Strategy CSO Civil Society Organization DFPTF WBG’s Global Partnership & Trust Fund Operations ECCD Early Childhood Care and Development FY Fiscal Year GFRP Global Food Crisis Response Program GoJ Government of Japan GRM Grant Reporting and Monitoring ICM Implementation Completion Memorandum ICR Implementation Completion Report IDA International Development Association IDB Inter‐American Development Bank IDP Internally displaced persons IEG Independent Evaluation Group JICA Japan International Cooperation Agency JSDF Japan Social Development Fund LASED Land Allocation for Social and Economic Development LCR Latin America and the Caribbean MNA Middle East and North Africa Region MoF Ministry of Finance MoFA Ministry of Foreign Affairs NDP National Development Plan NGO Non‐Governmental Organization ODA Official Development Assistance OECD Organization for Economic Cooperation and Development PATH Program of Advancement through Health and Education ix Universalia Volume I – Final Report Acronyms  PRSP Poverty Reduction Strategy Papers RSR Rapid Social Response Program SAR South Asia Region TASAF Tanzania Social Action Fund TF Trust Fund TTL Task Team Leader UNTFHS United Nations Trust Fund for Human Security US United States WBG World Bank Group Universalia x Volume I – Final Report Contents  1  Introduction 1  1.1  Introduction 1  1.2  Background to the Assessment 1  2  Methodology 3  2.1  Data Collection and Analysis 3  2.2  Reporting 6  2.3  Limitations 6  3  The Japan Social Development Fund – An Overview 7  3.1  The Japan Social Development Fund at a Glance 7  3.2  JSDF Program Areas 8  3.3  JSDF Program Financial Contributions, Allocations and Disbursements 8  3.4  Regional and Sectorial Distribution of JSDF Regular Program Grants 9  3.5  Sister Poverty Reduction Funds 9  4  The Contribution of Individual JSDF Grants to Poverty Reduction and Livelihood Support 11  4.1  Individual Grant Performance Overview 12  4.2  Contribution to Longer Term Objectives – The JSDF as a Program 16  4.3  Grant Design 17  4.3.1  Beneficiary Consultation in Grant Design 17  4.3.2  Meeting Needs and Filling Gaps 18  4.3.3  Promotion of Innovation 19  4.3.4  Sustainability and Leverage 22  4.4  Implementation Capacity 24  4.4.1  Effectiveness of Implementing Partners 25  4.4.2  Beneficiary Participation in Implementation 26  4.5  The Complementarity of the JSDF 26  4.5.1  Complementarity with Country Partnership Strategies 27  4.5.2  Complementarity with Japan’s Overseas Development Objectives 27  5  The Management of the JSDF Program 29  5.1  The JSDF Grant Process 29  5.2  Timeliness 31  5.3  Visibility of the Government of Japan 34  5.4  The Program Management of the JSDF 35  xi Universalia Volume I – Final Report Contents  6  Highlights of the Achievements of the JSDF 39  6.1  Highlights of Achievements 39  6.1.1  Reducing Poverty and Supporting Livelihoods 39  6.1.2  Promoting Sustainability 43  6.1.3  Promoting Innovation 44  7  Recommendations 47  7.1  Strategic 47  7.1.1  Reaffirming Strategic Relevance 47  7.1.2  Increasing Recognition and Building New Partnerships 48  7.2  Operational 48  7.2.1  Clarifying and Strengthening Guidelines 48  7.2.2  Building a New Culture of Results 50  7.2.3  Strengthening World Bank Capacity to Manage the JSDF 51  7.3  Setting Priorities and Responsibilities 53  8  Summation 55  Exhibits  Exhibit 2.1  Total number of grants by region 4  Exhibit 2.2  Total grants value by region 4  Exhibit 2.3  Average grant value by window 5  Exhibit 2.4  Average grant value by type 5  Exhibit 3.1  Contributions, Allocations and Disbursements (US$ Million) 8  Exhibit 3.2  Regional Distributions of Regular Program Grants by Fiscal Year of Approval, FY09‐13 (US$ Million) 9  Exhibit 3.3  Sectorial Distribution of Regular Program Grants Approved in FY13 (US$ million) 9  Exhibit 4.1  The Diversity of JSDF Grants that on an individual basis contributed to alleviation of poverty 13  Exhibit 4.2  The Diversity of individual JSDF grants that contributed to Livelihood Support 14  Exhibit 4.3  View Of TTLs Regarding Whether There are Positive Outcomes in Relation to Poverty Reduction and/or Livelihood Support 14  Exhibit 4.4  Examples of Outcome Performance Identified by Field Missions 15  Exhibit 4.5  Examples of Outcome Performance Identified by Portfolio Review 15 Universalia xii Volume I – Final Report Exhibits  Exhibit 4.6  View of TTLs regarding whether beneficiaries were consulted as part of the design of the grant 18  Exhibit 4.7  Examples of Gap Filling by JSDF Grants 19  Exhibit 4.8  Examples of Contextually Innovative Practices 20  Exhibit 4.9  Ideas Tested by the JSDF Grant are being Applied in Various Other Contexts 22  Exhibit 4.10 Leverage of JSDF Grants 23  Exhibit 4.11 Sustainability Highlights Drawn From Selected Field Missions 23  Exhibit 4.12 Additional Examples of Leverage and Continuity Drawn from Selected Field Missions 24  Exhibit 4.13 Impact of Implementing Partners (53 grants in total) 25  Exhibit 4.14 Examples of Beneficiary Participation in Implementation 26  Exhibit 4.15 Examples of Congruity with Country Partnership Strategies 27  Exhibit 5.1  Time from Transmission to GoJ to Approval by Japan (2007‐2012) 31  Exhibit 5.2  Time from Approval by Japan to Grant Agreement Signed (2007‐2012) 33  Exhibit 5.3  Samples of Evaluation‐Like Instruments 34  Exhibit 5.4  Sectorial Distribution – All Grants 36  Exhibit 5.5  View of TTLs Regarding the Question task teams were given adequate budget to carry out the grant‐making process 37  Exhibit 7.1  Recommendation Matrix 53  Appendices  Appendix I List of Findings 57  Appendix II List of Recommendations 60  Appendix III Key Stakeholders Engaged 61  Appendix IV TTL Survey – An Overview 74  Appendix V Grants Subject to Assessment 76  Appendix VI Other Funds Supported By Japan 83  Appendix VII Rating of Individual Grants 85  Appendix VIII Statement of Work 90  Appendix IX Core Bibliography 94  xiii Universalia Volume I – Final Report 1 Introduction  1.1 Introduction  The Universalia Management Group Limited (Universalia) is pleased to submit this Final Report to the World Bank for the Assessment of the Japan Social Development Fund (JSDF). The overall objectives of the Assessment are to contribute to the on‐going efforts to enhance the results of the Fund and improve its operation. More specifically, the Assessment is expected to review: How and how much JSDF grants are contributing to poverty alleviation among beneficiaries; How grants are helping to sustain livelihood of beneficiaries; and What operational and administrative issues and problems stand in the way of the efforts to improve the Fund’s operations. The full Statement of Work for the Assessment (similar to a Terms of Reference) is presented in Appendix VIII of this report. This Assessment is presented in three (3) Volumes, organised as follows: Volume I is the main assessment report, and includes the following sections and chapters: an Introduction and Context section; a Methodology section; a section that gives an overview of the Japan Social Development Fund; a section whose Findings focus on the Contribution of Individual JSDF Grants to Poverty Alleviation and Livelihood Support; a section whose Findings focus on the Management of the JSDF; a section that focuses on Highlights of the Achievements of the JSDF; General Conclusions; Recommendations; and finally, a brief Summation. There are nine (9) appendices attached at the end of Volume I. We will refer to them throughout the report. Volume II presents all tools and instruments used to collect and analyse data, consolidated in the Launch Briefing Overview. Volume III includes all fourteen (14) Country Mission Reports. 1.2 Background to the Assessment  The Japan Social Development Fund (JSDF) is a partnership between the Government of Japan (GoJ) and the World Bank Group (WBG). This Fund, established in the immediate wake of the Asian financial crisis of the late 1990s, is designed to provide grant assistance to initiatives that promote innovative approaches to addressing human development challenges. The Fund is targeted toward relatively small‐scale interventions, normally less than US$ 3 Million in overall value. JSDF grant‐ financed projects are administered by the WBG’s Development Finance Vice Presidency (DFi), Trust Funds and Partnerships Department (DFPTF). Subsequent elements of this Final Report provide a more detailed overview of the nature of the JSDF. In 2007, an evaluation of the JSDF was undertaken. Since then, important contributions to poverty alleviation and sustainable livelihood have been achieved through the large number of diverse grants financed from the regular, special and emergency windows of the Fund; but there has not been a systematic assessment since 2007, nor have the JSDF and its various achievements been fully explained to the general public and people interested in development. 1 Universalia Volume I – Final Report During annual consultations held in December 2012, the GoJ and DFPTF agreed to undertake an independent assessment of the achievements and operation of the Fund since the last strategic evaluation in 2007. Before proceeding with this independent assessment, two important contextual considerations need to be highlighted. First, this Assessment is reviewing a selection of grants that were approved and largely implemented between 2007 and 2012. Therefore, their assessment was based on then existing guidelines; and, application/implementation procedures. For example, during this period of time, the JSDF application process operated on a time‐fixed call for proposals and subsequent approval in what was named a “round”. From 2012 onwards, a rolling approach has been in use. Second, unlike many subject‐specific trust funds, the accountability for the administration of the JSDF is divided between its program staff located in DFPTF, having responsibility for the application and grant approval process; and TTLs and their country level managers assuming accountability for grant implementation, with DFPTF staff having only indirect oversight. This is more apparent in JSDF grant contractual arrangements which are between the World Bank represented at the country level and the implementing partner; and not with the JSDF program management located in DFPTF. Universalia 2 Volume I – Final Report 2 Methodology  2.1 Data Collection and Analysis  The Assessment team adopted a mixed methods approach to collect and analyse data. These methodologies include: Document Review  Two primary sets of documents were reviewed during the course of this Assessment. The first set of documents consists of the data sets for all 53 grants that were reviewed either as part of the Desktop Portfolio Review or a Field Mission. These data sets consisted of grant applications and some supporting documents and grant agreements, as well as standardized World Bank periodic reports, end‐of‐grant reporting, and documentation related to any extensions to the duration of the grant or any changes in the nature of the outputs or budgets. Notwithstanding this level of documentation, as will be described in further detail in the Limitations section of this Final Report (section 2.3), there were gaps in documentation that were common across all 53 data sets. The second set was comprised of corporate documents from the World Bank Group such as Annual Reports of the JSDF, Annual Reports of other elements of the World Bank, as well as policy and strategy documents, and specific guidelines and agreements that govern the implementation of the JSDF. A list of these documents can be found in Appendix IX, Core Bibliography, of this Volume of the Final Report. Interviews  The range of stakeholders to be engaged through this Assessment was broad. Aside from individual beneficiaries engaged during Field Missions, respondents included: World Bank Headquarters managers, World Bank Country Directors and Country Managers, Task Team Leaders (TTLs), representatives of GoJ, implementing partners, and representatives of national and sub‐national (state‐level) governments. The full list of people interviewed is presented in Appendix III of this Volume of the Final Report. Interview protocols used to collect data with stakeholders as well as the generic survey used to collect data from TTLs are included in Volume II of this Final Report. Electronic Survey  An electronic survey was administered to 190 Task Team Leaders (TTLs) and other identified World Bank personnel who directly participated in the management or supervision of an individual JSDF grant between 2007 and the present. The survey used the Likert scale rating3 and focused on issues related to the five axes of analysis, Relevance, Sustainability, Effectiveness, Efficiency and Programme Management. A response rate of approximately 60% was secured for the survey, which meant that the results are statistically significant and reliable. 3 A Likert scale is commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research. When responding to a Likert questionnaire item, respondents specify their level of agreement or disagreement on a symmetric agree‐disagree scale for a series of statements. Often five ordered response levels are used. This assignment used such a five point scale: Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree; and, Strongly Agree, with the addition of a sixth – Do Not Know / Not Relevant so as to ensure a higher degree of accuracy. 3 Universalia Volume I – Final Report Portfolio Review  An important component of the evaluation methodology consisted of a Grant Portfolio Review of specific JSDF projects. The Portfolio Review focused on 39 individual JSDF grants, which represents 31% of the approximately 125 grants that had been approved since 2007. In addition to the 39 grants that were subject to a desktop review, 14 were selected for an in‐depth assessment through a Field Mission to the country in question. The selection criteria for the Portfolio Review grants were as follows: (a) geographic location of the grant; (b) completion rate (all grants reviewed were at least 70% completed); (c) grant window; (d) type of grant; and, (f) type of Implementing Partner. The same selection criteria were used for the 14 Field Mission grants, with additional weighting given to grants that had closed within the last year since this would make it more likely that stakeholders would still be available to engage and thus stakeholder participation would be ensured. Appendix V of this Volume presents the list of the 39 grants that were selected for desktop review and the 14 additional grants that were selected for field mission review. Full details including a detailed methodological discussion of the selection process can be found in the Launch Briefing Overview contained in Volume II of this Final Report. The following Exhibits present some of the basic characteristics of the sample of 53 grants. Exhibit 2.1 shows geographic distribution. Exhibit 2.2 shows the total value of all 53 grants, again on the basis of geographic region. Here, the 12 sampled grants from South Asia represent $37 million of the total value of the 53 grants of $102 million, which is itself about 15% of the total allocation of the JSDF since its inception in 2000. Exhibit 2.1 Total number of grants by region Exhibit 2.2 Total grants value by region Africa Africa 18% 18% 24% East Asia / Pacific East Asia / Pacific 37% Europe / Central  11% Europe / Central  15% Asia Asia 15% Latin America /  4% Latin America /  Caribbean Caribbean 15% 24% MENA 15% MENA 6% Exhibit 2.3 shows the average value of the grants on the basis of Window‐ Emergency versus Regular. While there were only 12 grants of the 53 grants sampled from the Emergency window, the average grant size was nearly twice that of the 41 from the regular window Exhibit 2.4 presents the average value grants by type – Capacity Building or Project. The 17 capacity building grants had an average value of less than half that for the Project type grants. Universalia 4 Volume I – Final Report Exhibit 2.3 Average grant value by window Exhibit 2.4 Average grant value by type Emergency $3,111,676.86 Project $2,305,017.25 Regular $1,582,787.32 Capacity $1,132,575.35 $0.00 $2,000,000.00 $4,000,000.00 $0.00 $2,000,000.00 $4,000,000.00 A standardized assessment tool (the Grant Assessment Tool, presented in Volume II of the Final Report) was developed to assess the 53 grants. The tool allowed the review of each grant on its Relevance, Sustainability, Effectiveness and Efficiency. It also served to rate the grant supervision. Each question required the development of a narrative, paragraph‐long answer, referenced to the documents upon which the answer was based. In order to promote cross‐comparison and ensure to the greatest degree possible quantification of results, each of the five factors was given a cumulative numerical rating, ranging from High (1) through to Low (4). Each of those ratings was then elaborated and justified through a narrative description paragraph. Overall, a cumulative rating for each grant was also instituted, again using the High (1) through to Low (4) scale. The overall performance ratings for each of the 53 grants are found in Appendix VII of Volume I of this Final Report. Field Missions  The Assessment involved Field Missions in 9 countries and in‐depth review of 14 JSDF grants. In a number of instances, two grants in one given country were identified so as to promote cross‐ comparison. In five of the countries, namely Cambodia, India, Kenya, Sierra Leone, and Colombia, a two person team was assembled, one being a senior assessor and the other being a regional or national assessment consultant. For the remaining countries of Uzbekistan, Nepal, Moldova, and Nicaragua, a single senior assessor was assigned. Each Field Mission required one week of in‐country activities and involved interviews with World Bank personnel and with the project TTL if present in the country (many TTLs are not physically present in the country in question). Interviews were also held with Government representatives, as well as with representatives of the Government of Japan. The implementing partner (either a government agency or a civil society organization) was engaged by means of interviews and small group meetings. Meetings with beneficiaries were held in their own communities, which necessitated a degree of in‐country travel so as to directly engage with people in the implementation context and thus to make possible observation of the JSDF‐supported initiative first hand. The final step of each Field Mission was a half day debriefing session with senior managers of the World Bank Country Office in question and with the TTL. The fourteen Country Mission Reports can be found in Volume III of this Final Report. 5 Universalia Volume I – Final Report 2.2 Reporting  The Statement of Work for this assignment laid out two primary deliverables, a Final Report and an Illustrated Report designed to highlight the successes, best practices and lessons learned from the JSDF process. The Final Report ‐ The present document is the Final Report for the Assessment. The Statement of Work for the assignment laid out some general indications as to the content of the Final Report. During the data synthesis exercise which occurred after the last field mission (between mid‐April and late May, 2014), a more detailed content plan evolved and was agreed upon by Universalia and the client, largely influenced by the positive set of evolving Findings and the need to most effectively capture the emerging data and analysis. This Final Report will be translated into Japanese. Country Mission Reports ‐ The Assessment Team prepared an end‐of‐mission debriefing in PowerPoint format, as well as a narrative four page Country Mission Report. The latter are presented in Volume III. The Illustrated Report ‐ The Statement of Work for this assignment specified that in addition to a traditional evidence‐based and objective assessment of the JSDF, an additional report that would be more public information/communications‐related in style was required. This document, which is presented in both English and Japanese, is designed to highlight the success stories and positive lessons learned. 2.3 Limitations  There was one major set of limitations encountered during this Assessment. These relate to the documents used for the Desktop Portfolio Review as well as the Field Missions. The format of a number of these documents, standardized for use across all World Bank small grants, made it difficult to assess subtle factors such as the degree of participation of beneficiary communities or the nature of outcomes that may have been achieved. They also did not lend themselves to analysis of elements such as the articulation of benefits in the context of “before and after” assessment. This made it difficult to fully understand the benefits and consequences of a JSDF grant as well as to conduct an in‐depth value‐for‐money assessment. Specifically, the requests made in the Statement of Work to attempt to quantify the cumulative contributions of the JSDF in terms of characteristics such as economic value, specific impact on women and girls, number of participants, or impact on those below the poverty line to name only four such characteristics, could not be undertaken due to the due to data limitations. Periodic reporting and end of grant reporting were inconsistent in their reporting with respect to these characteristics and others. Therefore, it was not methodologically possible to aggregate information in a consistent and reliable fashion so as to illustrate the cumulative quantitative effect of the JSDF during the period under review. It should also be noted that the accountability for the JSDF as a program is divided, with the JSDF program staff located in DFPTF having direct accountability for the application process while Task Team Leaders (TTLs) implementing grants at the country level hold responsibility and accountability for all aspects of implementation, including the formulation of all grant reporting documentation. These documentary limitations were to a significant degree mitigated for the 14 grants subject to Field Mission review since it was possible to directly engage implementing partners, TTLs and beneficiaries and thus to fill in gaps. It was not always possible to fully mitigate them for those grants subject only to Desktop Review however, despite best efforts being made to carefully comb through available documents. Universalia 6 Volume I – Final Report 3 The Japan Social Development Fund – An Overview  3.1 The Japan Social Development Fund at a Glance  The JSDF is a partnership between the Government of Japan (GoJ) and the World Bank, which was established in 2000 in response to the Asian financial crisis of the late 1990s. During initial negotiations, it was mutually agreed that the partnership would be expanded to have global scope, supporting innovative activities to benefit the most disadvantaged groups in eligible developing countries. The specific goal of the JSDF Program is to fund activities that support community‐driven development and poverty reduction programs that serve to enhance productivity, increase access to social and community services and infrastructure, and improve the living conditions of poor and vulnerable groups. It thus finances (a) projects supporting improved service delivery, especially innovative new approaches that can be scaled up; or (b) capacity‐building grants that target local governments and communities, NGOs, or civil society organizations (CSOs). Thematic areas of focus for the JSDF range across all the range of the World Bank’s list of priority themes, for example, from empowering female victims of violence through capacity building activities to programs aimed at mitigating the effects of the global economic crisis on at‐risk youth. JSDF grants are delivered by implementing partners who may be NGOs/CSOs, elements of local or national governments and community associations. They thus provide a platform for cooperation between these key actors in the development process. All low‐income and lower middle income countries, as defined in the current World Economic Indicators, are eligible to receive JSDF grants, which normally range in value from US$ 200,000 to US$ 3 million. The total amount allocated to the JSDF by the GoJ through the end of June 2013 is US$ 660.5 million. The JSDF has provided 330 grants in 85 countries, of which 165 were completed and closed prior to FY 2013 while a further 28 were closed by June 30, 2013. JSDF grant financed projects are administered by the WBG’s Global Partnership & Trust Fund Operations (DFPTF) of the Concessional Finance and Global Partnership Vice Presidency (CFPVP) as the Trustee of the Fund. 7 Universalia Volume I – Final Report 3.2 JSDF Program Areas  There are three main JSDF program grant areas: 1) Regular Program Grants, which finance activities that provide direct benefits to the disadvantaged. The two types of regular program grants are Project Grants and Capacity Building Grants. Project Grants are for innovative programs targeted at poor and under‐ served groups not reached by mainstream programs that deliver quick results, while Capacity Building Grants are aimed at empowering and strengthening local communities, NGOs, and stakeholders by engaging the target groups in their development through learning and doing. 2) Special Program Grants finance Project and Capacity Building Grants that respond to catastrophic natural disasters or post conflict needs. For example, the Afghanistan Special Program, initiated in 2001, has supported the country’s reconstruction through support for the delivery of social services. 3) The Emergency Window was introduced in 2010 in response to the food, fuel, and global financial crises. It provided support through Project and Capacity Building grants to contribute to protecting the poorest and helping those driven into extreme poverty as a result of the global economic crisis. The window worked in collaboration with two of the World Bank’s vulnerability programs, the Global Food Crisis Response Program (GFRP) and the Rapid Social Response Program (RSR). This window closed in June 2012. 3.3 JSDF Program Financial Contributions, Allocations and  Disbursements  The GoJ has been the sole financier of the JSDF program. The average annual contribution to JSDF is about US$ 51 million, an amount that includes the GoJ’s additional contributions to the Emergency Window of US$ 50 million in FY 2010 and FY 2012 respectively. Starting with FY 2012, allocations to the JSDF program trust fund account have changed to conform to the Bank’s practice of allocating funding after the signing of a grant agreement. Therefore, in some cases, the allocation in a given fiscal year no longer corresponds to the amount of approvals in the same year (Exhibit 3.1). Overall disbursements increased from US$ 37.5 million in FY 2011 to US$ 40.0 million in FY 2012, continuing the trend from the previous year, and increased again to US$ 49.3 million in FY 2013. Exhibit 3.1 Contributions, Allocations and Disbursements (US$ Million) Universalia 8 Volume I – Final Report 3.4 Regional and Sectorial Distribution of JSDF Regular  Program Grants  Exhibit 3.2 Regional Distributions of Regular Exhibit 3.3 Sectorial Distribution of Regular Program Grants by Fiscal Year of Program Grants Approved in FY13 Approval, FY09‐13 (US$ Million) (US$ million) Exhibit 3.2 shows the regional distribution of the approved grants by fiscal year of approval for the five years between 2009 and 2013. The two regions with most approved grant amounts for the entire period are Africa (AFR) and Latin America and the Caribbean (LCR). South Asia Region (SAR) is fifth in terms of total amount of approvals for the five year period, but the approved amounts for the region have been relatively steady each year, unlike AFR, EAP, LCR and MNA (Middle East and North Africa Region), which have varied significantly from year to year. While JSDF grants can support any sector of the economy, most granting activities target initiatives that directly impact the lives of poor and vulnerable groups. As shown in Exhibit 3.3, the distribution of regular program grants between sectors was very balanced, at 21% for all sectors other than mining (which was 16% of the total). This balanced distribution is not always typical of JSDF; for example, the severity of the global food crisis that began in 2011 led to a greater emphasis on agriculture in FY 2011 and FY 2012. This in turn demonstrates JSDF’s flexibility and responsiveness across sectors in seeking to reach the most poor and vulnerable during times of crisis. 3.5 Sister Poverty Reduction Funds  In order to be able to assess the managerial effectiveness and efficiency of the JSDF mechanism, the Assessment sought to undertake comparative analysis of the JSDF with comparable trust funds. However, it rapidly became apparent that at present in the World Bank, the JSDF is very much unique as a trust fund platform in that it is exclusively funded by one donor. By contrast, most other trust funds are multi‐donor platforms, which makes comparison problematic since multi‐donor trust funds have a multi‐donor governance structure – thus the trust fund is accountable to a wider range of actors and has to satisfy multiple donor priorities, while a single‐donor trust fund does not. Moreover, many other World Bank‐hosted trust funds are subject matter specific. The transverse nature of the JSDF’s funding priorities, which are not restricted to one particular subject area or 9 Universalia Volume I – Final Report theme, makes comparisons with many other trust funds problematic. Accordingly, it was determined that the most relevant comparators would be platforms that are transverse in nature and single‐ donor in financing. The three most relevant platforms are all supported by the Government of Japan and all relate to various aspects of poverty reduction and by extension, the promotion of Japanese foreign policy objectives. These three funds are: 1) Japan Poverty Reduction Fund ‐ Asian Development Bank (ADB) The JFPR was established in May 2000 as a partnership between the Government of Japan and the Asian Development Bank to provide grants for projects supporting poverty reduction and related social development activities that can add value to projects financed by ADB. As of 30 June 2013, the total JFPR funds made available totaled about US$ 615.4 million. ADB had approved 158 grant projects (US$ 422.0 million equivalent) and 124 technical assistance projects (US$ 128.1 million equivalent). 2) Japan Special Fund for Poverty Reduction – Inter‐American Development Bank (IDB) The Japan Special Fund for Poverty Reduction Program (JPO), established in 2001 with US$ 30 million and replenished with US$ 22.5 million in 2011, provides non‐reimbursable financial resources for Technical Cooperation operations and supports poverty alleviation activities impacting low‐income and vulnerable populations. Since 2001, the IDB‐JPO has funded over 145 projects for approximately US$ 50 million. Its funding levels are much smaller than those of the JSDF or the JFPR, with maximum limits of less than US$ 500,000 per initiative. 3) Human Security Trust Fund – Office of the Secretary General of the United Nations The United Nations Trust Fund for Human Security (UNTFHS) was established in March 1999 by the Government of Japan as a mechanism to promote the expansion of the use of the principles that underpin Human Security, a long‐time foreign policy objective of the Government of Japan. From 1999 through to December 2013, the Government of Japan invested some US$ 475 million in the Trust Fund. The work of the UNTFHS in large part mirrors that of the JSDF and its sister funds located at other development banks. These three very similar initiatives were launched at roughly the same time as the JSDF (in 2000), thereby demonstrating Japan’s global commitment to cutting edge approaches that address some of the most stubborn development challenges. It should be noted that a Japan‐supported fund at the African Development Bank differs in that is it analogous to the former design of the Japan Policy and Human Resources Development Fund and not the JSDF; therefore not being comparable. All of these initiatives share common characteristics with the JSDF: they are all based on a philosophy of promoting innovative and people‐centred approaches, they are all bottom‐up in their design in that they require beneficiary participation in both design and delivery, and they are all based on an assumption of the provision of leadership in catalyzing others to pick up these challenges. In essence therefore, all are related approaches. In total, since 2000, Japan has invested approximately US$ 2 billion in the JSDF and in these sister funds, which further underlines its commitment to innovative approaches to human development challenges. More details on these three sister funds can be found in Appendix VI of this Volume of the Final Report. Universalia 10 Volume I – Final Report 4 The Contribution of Individual JSDF Grants to  Poverty Reduction and Livelihood Support  The Statement of Work for this Assessment contemplated reviewing a representative sample of JSDF grants through a Portfolio Review followed by a series of in‐depth Field Missions. The most recent JSDF Guidelines gives the following basic understanding of the nature of poverty reduction: To provide grants in support of community‐driven development and poverty reduction programs that serve to enhance productivity, increase access to social and community services and infrastructure, and improve the living conditions of poor and vulnerable groups in eligible client countries of the World Bank Group. Although not formally part of the JSDF program design, the Statement of Work also asked that consideration be given to assessing grants on the basis of their contribution to Livelihood Support. Yet the JSDF granting typology does not categorize grants on the basis of Poverty Reduction or Livelihood Support. This resulted in a lack of clarity for the Assessment as a whole in that there were no pre‐determined benchmarks or explicit linkages between individual grants and poverty reduction and/or livelihood support. In terms of defining poverty reduction, there is a range of definitions that span from very simplistic ones to highly complex and multi‐dimensional approaches. The World Bank itself recognizes this in indicating that poverty is multi‐dimensional. There is however a general agreement that a very basic definition of poverty reduction can be formulated around the notion of a basic income criteria usually articulated in terms of a monetary value per day (people living on less than US$ 1.25 per day). Such basic definitions fail to recognize the inherent multi‐dimensional nature of poverty however. Thus more sensitive definitions go beyond this basic economic indicator to address the issues related to basic human needs, thereby viewing poverty as deprivation in terms of various material requirements including, but not only, food. These notions incorporate other basic needs, such as access to basic health and education services, minimally adequate and safe housing, as well as access to safe drinking water and sanitation. These factors are seen as necessary because of the observation that increased income does not necessarily translate into improved access to means of satisfying basic human needs of the type described above. Still more sensitive understandings of the nature of poverty reduction emphasize that access to commodities and services matters not as ends in themselves but as determinants of people’s capability to function. These capabilities can relate to simple functions, such as being adequately nourished and clothed, and to more complex capabilities, such as being able to participate effectively in the social life of a community. These more complex understandings incorporate the problem of social exclusion or marginalization into the idea of poverty, and are therefore much broader than even the basic needs perspective. This holistic understanding of the multi‐dimensional nature of poverty is fully in line with the Government of Japan’s articulation of what constitutes “Human Security”, a long‐standing objective of Japan’s foreign policy. What the Human Security concept is primarily about is the sense of the totality of human well‐being, dignity and respect, as well as of the notion that the totality of that individual well‐being cannot be dismantled into “economic well‐being, social well‐being, physical (health) well‐being or even absence of physical threats to life”. Human security is best conceptualized as a holistic balance between the exercise of human rights, participation in sustainable human development and living in an environment free from physical or psychological threats to personal or community security. It can be best characterized by the expression: “Freedom from fear, Freedom from want, Freedom to live in dignity”. For Human Security, the totality is more. Thus there is in a way a holistic, and to a degree existential, notion of well‐being. 11 Universalia Volume I – Final Report Based on such more holistic understandings of the nature of poverty, virtually every one of the JSDF grants would contribute to poverty reduction for the purposes of the Assessment. They would also contribute to the attainment of Human Security. Equally, there is no agreed upon definition of what constitutes Livelihood Support. However, there is a general understanding that livelihood support in the most general terms involves economic productivity and harnessing labour. In essence, a livelihood is a means of making a living. It encompasses people’s capabilities, assets, income and activities required to secure the necessities of life. To that extent, livelihood support can be seen as one element of a more comprehensive approach to overall poverty reduction. However, to address the specific request made in the Statement of Work to assess the two, a provisional approach to differentiating between Poverty Reduction and Livelihood Support was developed. It categorized the 53 grants based on whether there was an immediate and direct linkage between the activities proposed and some form of contribution to “making a living”. It included capacity building activities that were directly linked to increasing economic productivity. It did not include more general capacity building which would address the capability‐related factors described above. On this basis, 18 of the 53 grants were identified as being primarily related to Livelihood Support. Appendix VI of this first Volume identifies all 53 projects. Those highlighted in bold italics constitute those related more closely to Livelihood Support. Next, the Assessment Team established an approach to review grants both through the Portfolio Review and also those 14 grants that were selected for an in‐depth Field Mission. To begin, the grants were categorized on the basis of the JSDF design described earlier. Then the team reviewed the adequacy of the design of the grant against the existing standards and criteria laid out in the JSDF application process and then current (2011) Guidelines; the link to World Bank and/or national development strategies and/or plans. Next, an assessment of the quantity and quality of output performance was undertaken, largely on the basis of the review of World Bank periodic and end‐of‐ grant reporting. Third, World Bank reports were assessed with respect to the degree of first level performance, comparing reported performance with the anticipated outcomes set out in the application process. However, this approach only focussed on documentary review and as such, needed to be triangulated by means of Field Missions where the Assessment Team would have direct access to beneficiaries and to government officials so as to better weigh and balance the consequences of individual JSDF grants. Indeed, given the limitations of some aspects of the data set for all the grants which were reviewed, the Field Missions became the most important source of third party information about the overall performance of JSDF granting. 4.1 Individual Grant Performance Overview  Finding 1: Overall, JSDF grants are aimed at contributing to poverty reduction on an individual basis. They do so by tackling development issues that prevent beneficiaries from overcoming poverty. Documentary review, Field Missions and extensive interviewing confirmed that the nature of individual JSDF grants have made contributions to poverty reduction, by addressing a variety of development issues that bar beneficiaries from overcoming poverty. The review of 35 grants related to poverty reduction showed a high level of output attainment. Universalia 12 Volume I – Final Report Furthermore, they have done so in a diverse fashion, reflecting the complexities inherent in the nature of poverty and therefore the instruments that can be used to alleviate it. Exhibit 4.1 demonstrates that the JSDF as a granting mechanism crosses sectors and recognizes, as was described above, that poverty alleviation necessarily must address deprivation of access and empowerment. In this context, not only does the intent of the JSDF as a granting mechanism combat conditions that contribute to the continuation of poverty, it also contributes to the promotion of human security, a long‐standing objective of Japan’s foreign policy and one which recognizes a holistic approach to the sense of overall human wellbeing. Exhibit 4.1 The Diversity of JSDF Grants that on an individual basis contributed to alleviation of poverty Colombia: Strengthened social inclusion for young urban women living in hostile environments. Senegal: Addressed child begging through proactive education‐related mechanisms. Burkina Faso: Empowered rural poor to participate in planning and decision‐making. Nepal: Provided school stipends for disadvantaged groups. Indonesia: Empowered vulnerable internal migrant female workers to improve their access to social services. Moldova: Increased access to health services for those afflicted with Hepatitis C. Pakistan: Improved the quality of housing stock in mountainous regions. Swaziland: Increased access to maternal health care for rural women. Sierra Leone: Increased access to the justice system for the rural poor. Honduras: Empowered urban poor to develop community disaster risk reduction plans. Vietnam: Increased access to early childhood education for ethnic minority rural poor families. Finding 2: Similarly, the nature of JSDF grants is aimed at contributing to livelihood support on an individual case basis. They do so by tackling development issues that prevent beneficiaries from improving their livelihood. Documentary review, Field Missions and extensive interviewing confirmed that individual JSDF grants have made contributions to livelihood support. They have done so by empowering beneficiaries toward economic autonomy, which is known to be a significant factor in increasing livelihood support. The review of the 18 grants related to livelihood support also shows a high level of output attainment. As with support for poverty reduction, individual JSDF grants have adopted a wide range of approaches to livelihood support. With the exception of a very few grants designed to provide emergency employment (cash for work), the primary approaches have concentrated on building capacity among the poor, introducing new production techniques, and empowering the poor to take more active roles in the means of production and subsequent marketing and sales. Exhibit 4.2 highlights the range of approaches taken to providing means for beneficiaries to become more economically independent. This range of approaches complements the diversity of approaches used to address poverty alleviation, and in many respects underscores the fact that supporting livelihoods is but one aspect of a more comprehensive and holistic approach to poverty alleviation. 13 Universalia Volume I – Final Report Exhibit 4.2 The Diversity of individual JSDF grants that contributed to Livelihood Support Nicaragua: Supported coca farmers in isolated regions. Cambodia: Empowered landless rural poor to ensure access to farm land. Philippines: Increased farmers’ access to lending institutions and strengthening land tenure. Kenya: Supported farmers in semi‐arid regions. Grenada: Provided disaster recovery support for small farmers. India: Empowered local artisans to establish their own means of distribution. Djibouti: Generated new jobs through community‐based small scale infrastructure projects. Ethiopia: Improved the working conditions and access to social services for urban women engaged in wood carrying. Mongolia: Provided support for small scale urban infrastructure projects to employ internal migrant workers. Uzbekistan: Supported the silk industry in rural areas. Finding 3: Nearly all of the individual grants examined attained degrees of first level outcomes though it was sometimes difficult to definitively ascertain the degree of outcome achieved. The review of 53 grant data sets showed that in nearly every case, there was evidence of achievement of outcomes that was reported, with the exception of six grants (12% of the total sample of 53) which were at such early stages of implementation as to make the reporting of outcome performance unnecessary. In the case of seven other grants, outcome information was present but was minimal in content. Of these seven, six were Project type grants. These observations are based on a comparison of anticipated outcomes set down in project proposal documentation with evidence drawn from periodic and end‐ of‐grant reporting. Exhibit 4.3 View Of TTLs Regarding Whether There are Exhibit 4.3 draws upon the TTL survey Positive Outcomes in Relation to Poverty and shows the degree to which TTLs Reduction and/or Livelihood Support affirmed that there were positive outcomes in relation to poverty Strongly Agree reduction and/or livelihood support. Agree The strong preponderance of Agree or Strongly Agree responses is notable. Moderately Agree Another manner used to assess the Disagree degree to which first level outcomes were attained was through Field Strongly Disagree Missions. These were particularly Don't Know/NA important since they allowed the Assessment Team direct access to 0 10 20 30 40 50 beneficiaries. Exhibit 4.4 highlights examples of outcomes that were achieved as determined by the Field Missions. Universalia 14 Volume I – Final Report Exhibit 4.4 Examples of Outcome Performance Identified by Field Missions Nicaragua: Increased agricultural production and continuation of farmer cooperatives. Sierra Leone: As a result of training, thousands of new long‐term farming and related jobs were provided to former artisanal “blood diamond” miners. Kenya: Traditional herbalists achieved formal certification. Colombia: Community organizations designed to empower local citizens and especially young people, women and girls, continue to function as local advocates for changes and social improvement. Nepal: Community groups continue to advocate for increased education support while targets for participation in secondary schools have been exceeded, as have graduation rates among disadvantaged groups who participated in the stipend process. Cambodia: Community cooperatives continue to function after the conclusion of the grant. New rural landowners have established working farms, while new urban landowners have established small businesses. India: As a result of community empowerment and capacity building in a new business model, the employment levels and incomes of target groups of women have increased. Other examples of outcome performance were identified during the Portfolio Review. Exhibit 4.5 presents some of the most interesting examples. Exhibit 4.5 Examples of Outcome Performance Identified by Portfolio Review Ethiopia: As a result of training, 200 new jobs in bio‐farming practices were established; about 9,000 new jobs in small scale business via a new micro‐credit initiative; 100 new jobs in hand weaving created by the project. Vietnam: 6‐8% decrease in malnutrition rates among participating pre‐school children; 95% of children who participated in the programming enrolled in primary school, demonstrating school‐readiness. Thailand: Increased usage of motorcycle helmets in target provinces. Azerbaijan: After participating in the skills development program, 86% of the participants started their own enterprise, and 9.5% were employed by someone else. By the end of the project, 100% of the participants had increased income levels as compared to only 17% of those who applied to the program but were not accepted. Nicaragua: 75% of the beneficiary communities (teachers, parents and leaders) reported increased knowledge about good nutrition practices, and also reported putting them into practice. Senegal: 68% reduction in the begging migration of young people in 35 villages in Tivaouane, and 71% reduction in 15 villages in Thies. As well, new legislation was passed. Russia: An increase from 600 clients accessing legal aid to over 1,300 in three years. Worth noting is that the application process during the period under consideration (2007–2012) did not require fully elaborated baseline information at the time of grant proposal submission (though baseline studies were undertaken at the onset of project implementation), therefore making it difficult to ascertain the degree of change that occurred. As well, during this time, gender balance was unevenly addressed in the articulation of outcomes. Finding 4: There are only minor variances in programmatic effectiveness between grants directed toward poverty reduction and targeting livelihood support, and there is no variance based on geography or the use of seed fund grants. Of the 53 individual grants reviewed during this Assessment, 18 are directly linked to the concept of livelihood support that was presented at the start of this section. Of these 18 grants most closely linked to Livelihood Support, 12 (66%) were rated “High” or “Substantial” in terms of their overall effectiveness, largely focusing on delivery of outputs and 15 Universalia Volume I – Final Report generation of first level outcomes. Of the 35 related to Poverty Reduction, 31 (88%) were rated “High or “Substantial” in terms of their effectiveness, again largely focusing on the delivery of outputs and first level outcomes. The variance between the two groups can in large part be related to issues arising from the small sample size. The sample of 53 grants selected for either Portfolio Review or a Field Mission was regionally balanced. A methodology was developed so as to ensure random selection based on the regional weighting of open and closed grants from 2007 until 2012. Details of this methodology can be found in the Launch Briefing Overview which is presented as Appendix I of Volume II of this Final Report. Given the size of the sample involved and the number of geographic regions, there were insignificant statistical variances in terms of the ratings for Effectiveness for particular grants (which primarily focused on output and outcome performance) across the six geographic regions of the World Bank. This Finding also addresses the issue of whether the seed fund grant provided in a number of instances to ensure, through consultations, beneficiary participation in the design of the grant has any impact on programmatic effectiveness. The data presented in Appendix VI of this Final Report shows clearly that seed fund grants do not appear to generate different levels of performance. Finding 5: Nearly all 53 grants generated their expected outputs within timelines and budgets, although about a quarter of all the grants reviewed required extensions. Of the 53 grants examined for this Assessment, only 13 (24%) required project extensions. All the grants in question were Project grants, out of a total of 45 Project grants. Delays were determined by comparing the anticipated outputs specified in grant proposals and evidence drawn from periodic and end‐of‐grant reporting. Documentary review and interviews with TTLs indicated that the root cause for these delays lay in two areas. First, additional time was required in some instances to increase the delivery capabilities of the implementing partners. However, notwithstanding a degree of delay, there is no variance in overall grant effectiveness between NGO or government implementing partners. The second reason given for delays is slightly changing contextual circumstances over the course of the often long application process. These can include changes in beneficiary needs that have arisen since the start of the process, changing circumstances including changes in governments or their priorities, actions of other development partners, and, other external economic, social or political factors that could not be anticipated during grant design. The approved rating guide for the Assessment was explicit in stating that cost over‐runs and/or failure to deliver mandated outputs constituted some of the most negative characteristics, and that when present, would lead to at best a rating of “Modest” or more likely “Low”. The data presented in Appendix VII of this Final Report shows that few if any grants were rated “Low”, especially among the 14 that were the focus of Field Missions. More specifically, only seven Project grants reviewed (13%) experienced shortfalls and/or delays in output delivery. As well, out of the total sample of 53 grants, only one, a Project grant, demonstrated cost over‐runs. There were no discernible patterns for which grants experienced delays in output delivery or had cost over‐runs. 4.2 Contribution to Longer Term Objectives – The JSDF as a  Program  The Findings in Section 4.1 focused on individual JSDF grants. However, the JSDF also exists as a global program. In this context, it is important to explore how the JSDF, as a program hosted by the World Bank Group and supported by the Government of Japan, contributes to the newly emerging set of World Bank strategic goals. Universalia 16 Volume I – Final Report Finding 6: This Assessment showed that JSDF grants addressed development issues that prevent beneficiaries from overcoming poverty or improving their livelihood and thus were broadly in line with World Bank goals. However, during the period of time in question, the JSDF, like many other World Bank programs, did not have a formal results framework. During the period of time being considered by this Assessment (2007–2012), the JSDF, along with many other World Bank programs, did not have a program level results framework. It implicitly however had the key elements of one. Specifically, the design of the JSDF required that individual grants demonstrate the degree of their contribution to country‐level objectives (CAS or CPF). These were in turn inherently linked to the higher level World Bank objectives. As well, it should be noted that during the review period, the JSDF conducted a review of the congruency and complementarity of the work of the JSDF with that of the International Development Association. During the period of time in question, the general absence of results frameworks across the World Bank hampered its ability to report on program level results. The recent moves at the World Bank to develop a comprehensive, institution‐wide results framework offer the JSDF the opportunity to better demonstrate in a detailed manner how it as a program contributes to the World Bank’s evolving corporate framework and thus, to larger World Bank goals such as ending extreme poverty. During the preparation of this Independent Assessment, as part of the World Bank’s collective effort to improve its results focus, all elements of the World Bank, including trust fund activities like the JSDF, have been directed to develop results frameworks that will link individual programs with larger corporate objectives. This work is now underway and is expected to be completed by the end of 2014. 4.3 Grant Design  The set of Findings in this section looks at factors related to the design of JSDF grants. It is sub‐ divided into four parts that mirror the JSDF design features: Beneficiary consultation in grant design; The degree to which JSDF grants meet an unforeseen need or fill a gap; The extent to which JSDF grants promote innovation; and The degree to which JSDF grants promote sustainability and/or leverage. 4.3.1 Beneficiary Consultation in Grant Design   Consultation with eventual beneficiaries is a key requirement of the JSDF. Consultation can ensure a higher degree of effectiveness and relevance since it can help to improve the degree to which an initiative truly meets the needs of the beneficiary population. It can also ensure greater empowerment, ownership by beneficiaries and ultimately therefore, sustainability. The current seed fund grant is designed to foster greater beneficiary participation in the initial design of JSDF grants. Finding 7: There is evidence to show that beneficiaries played a role in the design of JSDF‐ supported initiatives. Documentary analysis of the 39 grants that were subject to Desktop Portfolio Review demonstrated that in virtually every case, application documents referred to the level of consultation with beneficiaries in the grant design. However, there was a degree of variance in the level of detail provided in terms of the information contained in formal applications. 17 Universalia Volume I – Final Report Interviews conducted during Field Missions Exhibit 4.6 View of TTLs regarding whether with beneficiaries and with representatives of beneficiaries were consulted as part implementing NGOs uncovered more in‐depth of the design of the grant information regarding the extent of beneficiary consultation at the design stage. Engagement Strongly Agree was sometimes direct by the beneficiaries and sometimes through NGOs or similar Agree community‐based organizations advocating on Moderately Agree behalf of beneficiaries. For example, in Senegal, the child begging initiative involved extensive Disagree community consultation during the early Strongly Disagree design stages, while in Sierra Leone, community groups took the lead in ensuring Don't Know/NA the relevance of the rural justice initiative. The 0 5 10 15 20 25 30 35 TTL Survey triangulated the importance of the role played by beneficiaries in grant design. Exhibit 4.6 shows the degree to which TTLs affirmed this level of involvement. 4.3.2 Meeting Needs and Filling Gaps  One of the design characteristics of the JSDF as a granting mechanism is that grants are intended to address human development issues that others have ignored or that are as yet not addressed in a given country. In this context, “filling a gap” means addressing an issue that a government has not addressed and that no other Official Development Assistance (ODA) partner has addressed. It can also mean that the JSDF‐supported approach differs qualitatively from the work of others, either governments or ODA partners. This gap‐filling characteristic is frequently a determining characteristic of trust‐fund like activity in that these funds tend to be targeted toward addressing new or emerging issues, and by extension, filling gaps. Therefore, individual JSDF grants in principle should not be directed toward what is generally referred to as “scaling up” or “replication”. Indeed, the catalytic nature of the JSDF would tend to imply that “scaling up” or “replication” should be undertaken on the basis of the lessons learned from a JSDF‐supported initiative. The JSDF application process requires documentation of the nature of the gap and the degree to which beneficiaries were involved in the design of the grant and the extent to which it meets their articulated needs. Finding 8: Evidence indicates that the JSDF grants filled gaps or met previously unaddressed needs. In nearly every instance, documentary review conducted as part of the Desktop Portfolio Review clearly showed the extent to which grants responded to either unaddressed needs (gaps) or newly emerging considerations, for example the introduction of new ways of addressing long‐standing human development considerations. Documentary review conducted as part of the Desktop Portfolio Review also showed the extent to which beneficiary needs were central in the design of a JSDF‐ supported initiative. This documentary evidence was triangulated and reinforced during Field Missions through interviews with government representatives, World Bank staff and beneficiaries themselves. Field Missions were able to better explore the degree to which JSDF‐supported initiatives were in response to gaps and unaddressed needs of potential beneficiaries, particularly by discussing the design process in detail. Universalia 18 Volume I – Final Report This latter factor warrants in‐depth consideration. For example, the plight of children in neighbourhoods in Colombia (TF052879), where many live amidst poverty and crime caused by drug trafficking is not a new human development challenge; , nor have there been any other efforts by others to address this situation. What this JSDF‐supported initiative did was to introduce sports and support for cultural activities as means of empowering children and their families, and thus giving individuals including children a voice in shaping their communities. Likewise, improving the economic well‐being and thus supporting the livelihood of cocoa farmers in Nicaragua (TF093115) has been addressed by other development partners. The JSDF‐supported initiative however introduced new approaches, not so much new technologies or agricultural techniques, but mechanisms new to Nicaragua aimed at empowering rural farmers so as to improve their access to markets and their mechanisms of competition. Again, this addressed a gap, or in this case, introduced new ways of thinking about the local market environment. Exhibit 4.7 highlights interesting examples of gap‐filling, and also demonstrates the transverse nature of the work supported by the JSDF. Exhibit 4.7 Examples of Gap Filling by JSDF Grants In Swaziland, a JSDF‐supported initiative addressed long‐standing gaps in social programming related to maternal health, a subject that had not previously been a priority of the government. In Russia, a JSDF grant filled a long‐standing gap in the justice system by supporting pilot projects designed to open up access to justice for disadvantaged groups and young people. In Colombia, a JSDF supported initiative responded to community requests for programming to empower young women and girls living under the threat of poverty and violence by promoting advocacy. In Cambodia, as part of a larger World Bank initiative, long disenfranchised urban and rural landless people were give a voice in land distribution decision‐making. In Honduras, as part of a larger initiative, a JSDF grant empowered previously ignored urban poor to develop their own disaster risk management plans for their communities. In Senegal, as a result of community and NGO interventions, the plight of children begging in the streets was comprehensively and proactively addressed for the first time. 4.3.3 Promotion of Innovation  The JSDF granting mechanism is based on the stipulation that the initiatives supported should be innovative. The mechanism also aims to support pilot projects that will improve the design of future World Bank projects and also inform policy. Finding 9: Many examples have been found of grants that introduced new ideas and new ways of working to address poverty reduction and/or livelihood support within the context of the particular country or situation, thus confirming the approach to defining the nature of innovation that the JSDF used during the period under review. DFPTF staff and managers reported concerns that attempting to precisely define innovation in guidelines might become excessively restrictive. At the same time, representatives of the GoJ indicated that one of their major challenges while reviewing JSDF grant applications was the extent to which a proposal was in fact innovative. The question therefore boils down to this: what constitutes innovation within the JSDF context? 19 Universalia Volume I – Final Report JSDF granting activity has been noteworthy in the degree to which it has promoted the introduction of innovative practices. During the period of the Assessment, training materials and information provided to applicants described innovation in terms of bringing new ideas to a particular environment. Numerous examples have been found of grants that clearly introduced new ideas and new ways of working to address poverty reduction and/or livelihood support within the context of the particular country or situation. Both the Desktop Review and Field Missions uncovered numerous examples of contextually innovative practices. There is also strong evidence to indicate that these projects are “new” in relation to the context in question. Exhibit 4.8 presents some examples of contextual innovations, and underlines the range of contextual innovations that have been supported by JSDF grants. Exhibit 4.8 Examples of Contextually Innovative Practices In the Philippines, farmers and financial organizations came together to address community‐based lending issues for the first time. In Moldova, for the first time, the introduction of community‐driven approaches to development enabled poor rural communities to empower themselves to address social issues. In Russia, the introduction of NGOs into the legal aid system for young offenders marked a first in terms of promoting proactive outreach. In Colombia, the inclusion of financial literacy skills into an empowerment program for young women marked a first for Colombia. In Tanzania, conditional cash transfers for poor families was introduced, a first not only for Tanzania but for Africa as a whole. In Senegal, the development of a proactive approach to combating youth begging marked a transformation from previous after‐the‐fact solutions. In rural Vietnam, a grant piloted providing early childhood education both in Vietnamese and the local dialect. In Sierra Leone, a rural justice initiative introduced non‐confrontational approaches that include using paralegals and volunteers instead of lawyers so as to increase confidence levels among the historically disadvantaged rural poor. In Cambodia, landless rural peasant families were empowered with the assistance of an NGO to take active roles in decision‐making about land distribution. In Jamaica, a Crime and Violence Observatory brought together for the first time community groups, governments and law enforcement to address long‐standing human security issues. DFPTF and GoJ stakeholders both agree that an absolute definition of innovation is impractical. For example, it would be counterproductive to define innovation within the JSDF context as solely ideas or activities that had never been introduced anywhere before since such stringent standards would exclude many very worthwhile projects with innovative elements. These stakeholders also agree that probably what was more important in relation to the understanding of what constitutes innovation would be to take into account the country context. Thus, there was a degree of emerging consensus around the notion that innovation is best understood within the context of the country in question. The following example, drawn from a JSDF grant for Nepal (TF093397), best illustrates the notion of contextual innovation. The grant involved providing funds to the Ministry of Education to allocate stipends so that disadvantaged students, primarily females, could attend secondary school. As such, on the surface, very little of this grant would seem to be innovative, especially as the Government of Nepal had provided similar stipends and scholarships in the past. However, this grant promoted several cutting edge innovations for Nepal. It introduced an objective means test to select the Universalia 20 Volume I – Final Report recipients of the stipends. It also engaged a nationally recognized NGO to design and implement the means test. Finally, the means test was developed with the direct input of the potential beneficiary community, a long‐time socially disadvantaged group in rural Nepal. Therefore, for Nepal, this seemingly traditionally‐focussed grant in fact introduced three innovative elements. It is also interesting to note that based on positive results, the Ministry of Education now intends to roll‐out this community‐based objective approach to Needs Assessments across the Ministry. Furthermore, the Ministry of Social Development has indicated that it intends to apply this community‐based objective methodology in relation to other social assistance programs. A second example, drawn from a grant provided to an NGO in Colombia (TF099171), also demonstrates the subtle characteristics inherent in contextualizing innovation at the country level. In this case, support was given to communities to encourage girls to participate in football. What was innovative here was the fact that while on the surface the initiative was reaching out to girls to encourage their participation in sport, at an equally important level, the grant involved active community participation in design and delivery and thus empowerment in some neighbourhoods that had long been beset by narcotics trafficking and the attendant social challenges. The innovation here related to the use of a recreational program to foster increased community participation and building consensus to combat a major social ill. Overall then, the Assessment Team concluded that in relation to JSDF grants, contextuality was the determining characteristic of “innovation”. The introduction of new ideas, bringing new groups of partners together or introducing new ways of working should be seen in the context of whether those actions were in fact new to the country or environment in question. Finding 10: JSDF grants have provided a mechanism for the World Bank to promote the piloting or the introduction of new techniques into a given country. They have thus contributed to the strengthening and broadening of the World Bank’s capacity to respond to country level challenges in an innovative fashion and have helped inform sector policy in the recipient country. According to a majority of senior World Bank stakeholders, one of the most appealing attributes of the JSDF is the flexibility it provides to Bank personnel to introduce new techniques and approaches, in essence to pioneer new approaches, using a grant mechanism. Importantly, and as Exhibit 4.9 shows, TTLs are of the view that there is a high rate of concepts pioneered by JSDF grants being applied elsewhere. This kind of piloting can operate in two dimensions. First, it can promote the introduction of new policies and national programming in the way governments shape their day‐to‐day activities in response to emerging human development challenges. For example, in Nepal, a JSDF‐supported initiative (TF093397) introduced an objective means of assessing social need to the Ministry of Education by means of a new policy. This “Needs Assessment” mechanism has now been integrated into other programming areas and has been transplanted into the Ministry of Social Development for use across a broad range of poverty reduction and community support programming. Therefore, the JSDF grant was instrumental in refreshing and renewing how the government as a whole operates. 21 Universalia Volume I – Final Report Second, a JSDF grant can inform the work of the Exhibit 4.9 Ideas Tested by the JSDF Grant are World Bank and refocus World Bank Country being Applied in Various Other Strategies and Priorities themselves. For Contexts example, in Moldova, the pilot project reaching out to those afflicted with Hepatitis C Strongly Agree (TF093702) has resulted in the World Bank renewing itself and reorienting its own strategies Agree to be more inclusive. By extension, these new Moderately Agree strategies have resulted in increased governmental awareness of the challenges facing Disagree those afflicted with this condition and have Strongly Disagree generated a commitment to increase programming in response to their needs. Don't Know/NA 0 5 10 15 20 25 30 35 4.3.4 Sustainability and Leverage  Incorporation of sustainability and leverage elements into the design of the JSDF grants is key since it helps to ensure continuation of the grant activities as well as the scaling up and expansion of them into ongoing activities after the grant ends. This section looks at both the incorporation of these elements into grant design, as well as at actual continuation of grant activities in the longer term. Finding 11: Some 66% of individual JSDF grants acting as pilots have resulted in expansion of the pilot project and thus demonstrate leverage. Overall, there was some variation between the leverage of Project grants versus Capacity Building grants; as well, the range of ways in which grants demonstrated leverage was notable. A fundamental design characteristic of the JSDF granting mechanism is that it should aim to generate sufficient critical mass and support so as to cross‐fertilize other projects and be transplanted into other contexts. There are several different types of such leverage that were uncovered. The first and most direct are instances where the lessons of a JSDF‐supported pilot initiative are integrated into an eventual lending activity of the World Bank. The second type of leverage occurs when the government in question, either national or sub‐national, agrees to integrate the lessons learned or techniques into their on‐going public programming. The third consists of those instances where another body, mainly the beneficiaries themselves, continues the initiative or incorporates elements of the initiative into other activities they are pursuing. This categorization is used in Exhibit 4.10 below. It is important to stress that, by design, the JSDF granting mechanism actively discourages requests for continuation of grants and in nearly all circumstances, requests for replication within any given country. Its pioneering and piloting characteristics therefore are central. As well, of the 53 grants examined for this Assessment, there were two instances where a JSDF‐ supported initiative was sustained and enlarged by another ODA partner. This is very important to underscore. The leverage of the JSDF mechanism is very different from that of many other trust fund‐ like activities in that learning and transplantation rather than merely prolongation appear to be the most significant consequences of the JSDF grant design. Universalia 22 Volume I – Final Report Exhibit 4.10 Leverage of JSDF Grants Project Grants (45 of 53 grants) No Evidence of Government World Bank Continuation or Insufficient Integration or continuation or continuation or integration by information provided continuation integration integration Other Parties to assess leverage 9 12 14 6 3 Capacity Building Grants (8 of 53 grants) 3 2 1 1 1 Exhibit 4.12 illustrates the extent of the leverage of the JSDF granting mechanism. It shows that overall, 66% of the grant sample generated some concrete form of continuation. It also clearly illustrates the degree to which JSDF grants have pioneered initiatives that have been taken up by either governments or the World Bank. More specifically, some 71% of Project grants demonstrate this concrete leverage, while only 44% of Capacity Building grants demonstrated explicit leverage activities. Exhibit 4.11 demonstrates the sustainability consideration. It highlights examples where initiatives pioneered by a JSDF grant continued to take root. Exhibit 4.11 Sustainability Highlights Drawn From Selected Field Missions Scale up/expansion of support for the silk industry in Uzbekistan via a new project for a Horticulture Strategy. A new network was created in Kenya to continue cooperation related to the application of traditional herbal medicines. Moreover, the government has decided to begin to integrate traditional medicine into the public health system. In Nicaragua, the networks and cooperatives remain in place after the end of the grant, cocoa production has increased, and farmer to farmer capacity building is continuing. In Cambodia, follow‐on World Bank support for land distribution will draw upon lessons learned, while NGOs and community groups remain active advocates for the poor. Government also remains committed to continuing the innovative practices. In Nepal, there has been continued expansion of the objectively‐based Needs Assessments for educational stipends, as well as expansion to other Ministries. In Sierra Leone, cooperatives and the microfinance body established remains active, while community leaders remain as participants in district planning councils. In Moldova, some networks to support those afflicted with Hepatitis C remain active, some local councils have absorbed the costs of community outreach, and financing of blood services has been absorbed by the national government. Somewhat surprisingly, the degree of leverage did not seem to significantly vary between the Regular and Emergency/Special windows. This is surprising since in theory, grants operating through the Emergency/Special window were designed to be short‐term in nature and thus likely not to be catalytic or indeed to emphasize leverage to the same degree. However, several examples came to light which contradicted this assumption. In Jamaica (TF096923) and Grenada (TF097967), Emergency/Special grants had catalytic effects and resulted in absorption into ongoing national programming. 23 Universalia Volume I – Final Report Also worth underlining is the wide range in the types of leverage of JSDF grants. Exhibit 4.12 highlights this range, as well as the fact that JSDF pilot activities not only continued within the context of the pilot program but were expanded upon. Such expansion was particularly notable in those cases where a government body, after coming to recognize the benefits of an initiative pioneered by the JSDF, has begun to integrate JSDF lessons learned into its overall programming. Field Missions found this occurring around the world, from Moldova in relation to Hepatitis C (TF093702) to Nicaragua with respect to new agricultural techniques (TF093115) to Cambodia in relation to government programs and policies for land distribution (TF091833 and TF091836) to Nepal where innovative social programming piloted in the Ministry of Education is now being integrated across the entire government (TF093397). Exhibit 4.12 Additional Examples of Leverage and Continuity Drawn from Selected Field Missions In India, artisans set up their own brand which is owned and managed by the community members. The initiative has also been replicated in various states receiving support from the State Level Livelihood Societies. In Gambia, the key component and lessons learned from a grant to support early childhood care will be integrated into major new national initiatives that will receive both World Bank and Government of Gambia financing. In Senegal, the lessons learned from a grant that addressed child begging have been integrated into both on‐ going World Bank and Government programming, with the Government itself having expanded the key innovative element of the grant – introduction of Islamic‐oriented curriculum and expansion of Islamic schooling in general. 4.4 Implementation Capacity  The design of the JSDF includes, among its funding criteria, the aim of strengthening the capacity of local implementing partners in terms of their ability to undertake the tasks assigned. The JSDF granting mechanism envisages two primary sets of implementing partners. The first are NGOs and other civil society organisations. This is a very important characteristic and underlines the degree to which the JSDF mechanism is designed to promote “bottom‐up” solutions. The other group of implementing partners are government bodies, either national or sub‐national. The selection of grants to be assessed was based upon multiple factors. One was related to ensuring a representative sample across these two categories of implementing partners. For greater clarity, of the total number of closed or active grants, a handful involved a joint implementing relationship between a government body and a civil society group. Of the 53 grants reviewed, 19 were implemented by an NGO or another civil society body. There are a wide range of partners. Among NGOs and CSOs, the overwhelming majority are national level organizations. They primarily include philanthropic organizations and citizen‐based community organizations. Among governmental bodies, the range is even wider. National level or sub‐national level ministries predominate, while at the sub‐national level, a number of grants were implemented by municipal governments or their agencies. Universalia 24 Volume I – Final Report 4.4.1 Effectiveness of Implementing Partners  Finding 12: The Assessment of the JSDF suggests that, overall, implementing partners are of good quality in terms of being able to deliver planned grant activities. Moreover, there are no differences in the effectiveness of individual grants between the two categories of implementing partners. Implementing partners have in general generated the anticipated outputs on time and within budget. Moreover, of the 53 grants reviewed for this Assessment, there was no evidence of implementing partners having major challenges in terms of capacity to deliver that could not be overcome. It is fair to state therefore, that JSDF implementing partners are of good quality based on this measure of quality. This Finding is in accord with several earlier Findings that demonstrated the degree of consistency of grant performance across different factors. Exhibit 4.13 is derived from the ratings for each of the 53 grants, and describes the impact of implementing partners in relation to their ability to have delivered outputs and first level outcomes. The rating guide used for the Portfolio Review and subsequently in the assessment of the grants that were subject to a Field Mission can be found in Volume 2. For each of five factors, a set of thresholds/questions was developed along with criteria to establish a 1‐4 rating, the most positive two ratings being “High” and “Substantial”. These overall ratings can be found in Appendix VII of this Volume of the Final Report. Exhibit 4.13 Impact of Implementing Partners (53 grants in total) CSO/NGO (19 grants) Government Body (34 grants) Overall Grant Rating “High” or 78% 91% “Substantial” Effectiveness Rating “High” or 67% 66% “Substantial” The degree of commitment to particular projects was evident in virtually every Field Mission and across each category of implementing partner. Both governmental and non‐governmental partners appeared to demonstrate enthusiasm for their role in implementing the grant in question. Governmental partners especially noted the capacity building that ensued in many instances, reporting that they were able to transplant these skills into other parts of the government organization or ministry in question. Finding 13: In general, the capacities of the implementing partners are analysed through an assessment process that is flexible and which contributes to effective implementation of the grants. During Field Missions and as part of interviews, it became apparent that the current approach to assessing the capacity of implementing partners was dynamic and flexible, and weighed local conditions in an appropriate manner. However, documentation provided as part of the formal application process, tended to understate these important attributes. Portfolio Review and interviews indicated a lack of familiarity with World Bank fiduciary procedures on the part of implementing agencies. The consequence of this is a delay in implementation. 25 Universalia Volume I – Final Report 4.4.2 Beneficiary Participation in Implementation  Finding 14: Field missions uncovered a pattern of ongoing participation. The community‐based nature of the JSDF implies continuing participation by beneficiaries so that changing circumstances and feedback can be accommodated. Equally, continued participation by beneficiaries during implementation strengthens their ownership of the initiatives, and contributes to ongoing sustainability. It was not always possible to conclusively identify the level of beneficiary participation during implementation solely on the basis of document review due to varying levels of information provided. However, Field Missions were able to uncover solid patterns of evidence of frequent meetings with the beneficiary communities during the life of particular initiatives. Beneficiaries themselves provided some testimony about the extent to which they were involved in the on‐going life of the initiative. Exhibit 4.14 highlights some of these instances where beneficiaries actively continued to participate in implementation, as observed during Field Missions. Exhibit 4.14 Examples of Beneficiary Participation in Implementation Nicaragua: Indigenous farmers in a remote region reported how they continued to work with the implementing NGO to ensure that the farming capacity building respected their cultural traditions and indigenous lifestyles. Cambodia: Community associations established as a result of a JSDF grant continued to engage government and NGO implementing partners, raising with them local issues about land tenure, intimidation and the need for additional training in farming techniques. Sierra Leone: Former artisanal “blood diamond” miners indicated that they worked with the implementing partner to raise on‐going concerns about the location of some of the public works activities and especially the need to redirect toward refurbishing long‐damaged river crossings. Kenya: Practitioners of traditional medicine continued to provide feedback and advice to the government agency implementing a program to better plan for and manage traditional herbal medicine practitioners. Moldova: Community groups representing those afflicted with Hepatitis C provided on‐going input and outreach services for the government agency implementing the JSDF grant. Earlier Findings addressed the question of the consultation of beneficiaries in the design of JSDF grants, demonstrating a consistent degree of participation. It is equally important however to consider whether beneficiaries continued to participate in the implementation of JSDF grants, thereby providing a degree of feedback to implementing partners and ensuring that grants continue to remain relevant and responsive to beneficiary needs. However, the quality of information regarding the question of the degree of beneficiary participation in the on‐going implementation of particular JSDF‐supported initiatives is uneven. 4.5 The Complementarity of the JSDF  Given the importance of trust funds generally to the World Bank and the growing recognition that trust fund activities need to be better linked to overall World Bank strategies, it was important to ascertain the degree to which JSDF grants complemented or otherwise supported broader World Bank strategies. Similarly, the JSDF requires that the activities it supports complement the Country Partnership Strategies (CPS). Accordingly, it is equally important to address this aspect of congruity. Universalia 26 Volume I – Final Report 4.5.1 Complementarity with Country Partnership Strategies  Finding 15: Individual JSDF grants are aligned at the country level with Country Partnership Strategies. The application process for the JSDF specifically requires evidence of congruity with CPS, and thus asks how a potential grant will contribute to these. The Assessment found that there is good congruity between JSDF grants and the various Country Partnership Strategies. Exhibit 4.15 highlights some of the most notable examples. Exhibit 4.15 Examples of Congruity with Country Partnership Strategies Senegal: The grant in support of eliminating child begging followed on the Country Assistance Strategy that speaks directly to improving the living conditions of vulnerable groups including children. Swaziland: The grant in support of maternal health was in line with a 2008 strategy note to scale up World Bank engagement by helping it address the HIV/AIDS challenge. Sierra Leone: A grant to support strengthened access to justice in rural areas complemented the World Bank’s Institutional Reform and Capacity Building Project in Sierra Leone by improving participation and accountability at the local level. Philippines: A grant to promote livelihood support among farmers complemented the Country Assistance Strategy objective of restoring sustainable economic growth with more poverty reduction and greater equity by accelerating environmentally sustainable rural development and enhancement of human development and social services for the poor. Uzbekistan: A project to revitalize Karalpakstan and the Aral Sea area complemented the Country Assistance Strategy to increase economic opportunities in the rural areas and to improve environmental conservation. Cambodia: Two small grants added new features that increased community participation in a country‐wide World Bank program in support of land redistribution. Nicaragua: A grant to strengthen agricultural production in a remote area and thus enhance livelihoods was integrated into a much larger agricultural reform program supported by the World Bank. Kenya: A grant to strengthen the capacity of farmers in semi‐arid regions complements the Kenya Forest Conservation and Management Fund. Nepal: A JSDF grant to provide secondary school stipends to disadvantaged youth was developed as a complement to the Country Assistance Strategy’s larger goals of increasing access to higher education. 4.5.2 Complementarity with Japan’s Overseas Development  Objectives  The JSDF granting mechanism is exclusively financed by the Government of Japan through its Ministry of Finance (MoF). Although the MoF is the formal counterpart with the World Bank with respect to the administration and planning of the JSDF, several other elements of the Government of Japan play key roles. Key informants said that the Ministry of Foreign Affairs (MoFA), the Embassy of Japan and the Japan International Cooperation Agency (JICA) are consulted on individual applications, to ensure that there is no duplication with the proposed grant and the Japanese bilateral programs. As well, it is implicit that JSDF grants should not only complement and support World Bank strategies but equally be representative of the international cooperation policies of the Government of Japan. Discussions held in Tokyo with representatives of these three bodies underscored the degree of collaboration between them and the collective commitment of the Government of Japan to ensure the congruity of the JSDF with Japanese overall policy objectives and also to ensure the showcasing of the contributions made by the Government of Japan. 27 Universalia Volume I – Final Report Finding 16: Individual JSDF grants are in line with the Priority Policy for International Cooperation of the Government of Japan. The Priority Policy for International Cooperation of Japan lays out three major objectives: To realize a prosperous and stable international community with freedom; To support the growth of emerging and developing economies together with the growth of the Japanese economy; and To promote human security and strengthen trust in Japan. Representatives of all three major Japanese agencies along with embassy staff affirmed that individual JSDF grants contribute to the attainment of Japan’s major priorities. More specifically, these stakeholders indicated that JSDF grants contribute to Japan’s goals in relation to global poverty reduction. They specifically identified the importance of expanding partnerships and reaching out to civil society to ensure its full engagement in all aspects of international cooperation. The JSDF focus on engaging the poorest of the poor and the disenfranchised was noted as responding to Japan’s long‐standing commitments with respect to the promotion of human security. These senior Japanese stakeholders also reaffirmed Japan’s commitment toward undertaking a holistic approach to poverty reduction and livelihood support, highlighting the US$ 2.5 billion investment made since 2000 in the four trust funds and related platforms that were described in Section 3.1 of this Final Report. The following citations from Japan’s Medium‐term Policy on Official Development Assistance may best exemplify the philosophical link between Japan’s approach to poverty reduction and JSDF granting activity: “Poverty is not only an economic dimension. It also has social and political dimensions exemplified by lack of access to basic social services… and lack of opportunities to participate in the decision‐making process”…”Therefore poverty reduction should be pursued proactively through actions that address both economic and social dimensions.” “Direct assistance to the poor occupies a significant position in efforts to reduce poverty. From the human security perspective this requires strengthening the capabilities of the poor so as to enable them to participate in the formulation of aid policies, and the project planning and implementation processes that affect their own lives.” Universalia 28 Volume I – Final Report 5 The Management of the JSDF Program  This section of the report reviews the managerial aspects of the program. We are approaching this in two ways, first, reviewing the management of the JSDF grants and, second the management of the JSDF program. Our framework for assessing the managerial aspect of the JSDF includes the planning of the grants, their implementation (with a particular focus on timeliness of implementation as well as on reporting), and finally the visibility of the Government of Japan. 5.1 The JSDF Grant Process  During the review period of 2007‐2012, the critical path for the review of applications and subsequent promulgation of implementation contracts operated within a time‐fixed “call for proposals” approach. Two or three times a year, JSDF program staff issued a general call for proposals with a fixed cut‐off date for submission of proposals. Standardized application documents were required. Each of these general calls for proposals was referred to as a “round”. It is also important to underscore that throughout its 15 year lifespan, despite the fact that NGOs, CSOs and government bodies play crucial on‐going roles in the entire JSDF process, applications for JSDF grant support have only been able to be received from World Bank personnel, and more specifically from a TTL. TTLs had the option of submitting a full scale proposal that addressed all required elements, including but not limited to, evidence of beneficiary participation in design, evidence of discussions with in‐country representatives of the GoJ and the comments of internal reviewers. They also had the choice of engaging JSDF staff in a more iterative and consultative process, which could result in a more relevant proposal. Until 2011, a Steering Committee comprised of DFPTF management (then known as CFPTO) and a number of other World Bank senior managers provided a formal internal review of applications for each round before submission of a proposal to the Government of Japan for its approval. The importance of this mechanism needs to be emphasized. The senior managerial oversight and challenge function played by the Steering Committee afforded the JSDSF with a high level of insights into relationship between a proposed initiatives and country level planning. The Steering Committee, in playing a challenge role with respect to each proposal, also constituted a Quality at Entry function. Moreover, the Steering Committee also provided a high level institutional memory. It also played an overall advisory role for the JSDF as a whole. However, this consultative process was subsequently eliminated, placing the final decision with respect to forward submission to Japan in the hands of DFPTF management. In 2012, the JSDF wound down its long‐standing approach of annual calls for proposals, largely because of the concerns about the length of the application process. This “rolling system” where the application process for an individual grant could commence at any time remains in place today. Starting in 2010, JSDF itself introduced a new mandatory preliminary document known as a concept note, which was aimed at providing an early review of the relevance and viability of the proposed development intervention, thus strengthening the overall proposal process and reducing risks inherent in the development of full proposals that might be rejected. At the same time, the preparation of full‐fledged proposals involves costs that are not recovered by the Task Teams. Of equal importance, the introduction of concept notes provided the JSDF staff a formal opportunity to play an enhanced challenge function with TTLs so as to improve the quality at entry. It should be noted that this presaged the introduction by the Bank of the Small Grant Guidelines, which required generally similar preparation requirements. 29 Universalia Volume I – Final Report For both the pre and post 2012 period, applications were then forwarded to the Office of the Executive Director for Japan of the World Bank, who subsequently transmitted these applications to the Ministry of Finance (MoF). A benchmark was established for the turnaround time of some two months. However for greater clarity, it is important to note that virtually none of the grants reviewed for this Assessment follow the 2012 and onward criteria. It should be noted that the JSDF program had and has full accountability for the application process up to and including the signing of a grant agreement with the World Bank country management in question. At that point, direct accountability for the implementation of the grant transfers to the TTL and by extension, the World Bank country level management, with JSDF management playing an oversight role. Following the signing of the grant agreement, the recipient begins implementation. Implementing partners are either NGOs, other civil society bodies, or government agencies, whether national or sub‐national. In a very few instances, joint implementation has been permitted. Finding 17: The files reviewed for this Assessment show a high degree of compliance with the application guidelines then in place. The grants and their corresponding application materials reviewed for this Assessment reflect the pre‐2012 application process described earlier. The 53 sets of grant files reviewed for this Assessment showed a high degree of compliance with specifications then in place regarding the content of formal application documents. This may be in large part due to the extensive nature of the challenge function played by DFPTF staff and managers in advance of formal submission of an application. As well, the challenge function played by the steering committee also strengthened the quality and relevance of proposals. There were no variances observed in the quality of applications between different types of grants or applications from different geographic regions. One of the most notable aspects of the standardized application documents are the comments provided by the external reviewers. In nearly all instances, these comments demonstrate rigour in the assessment of the application and provide a degree of clarity about certain key issues that may not be available elsewhere. Finding 18: The former Steering Committee played a value‐adding role, including that of ensuring Quality at Entry. The Steering Committee of the JSDF, which was briefly described above, during the period under review played value‐adding roles including that of ensuring increased Quality at Entry. This Committee, during the review period, composed of senior managers of varying elements of the World Bank played roles over and above those specifically related to the consideration of proposals. The Committee also ensured a degree of high‐level of awareness and understanding of the intrinsic value of the JSDF to the World Bank as a whole. Because of its cross‐sectorial composition, the Steering Committee gave the JSDF a sort of cross‐ WBG window, thereby enabling it to be more relevant to the World Bank as a whole. The Steering Committee played a crucial value‐adding role in relation to its challenge function concerning the review of individual proposals. It is evident through the review of Steering Committee minutes that it played an active programmatic challenge function, questioning the validity of proposals and thereby improving their quality and sensitivity. As well, it insured consistent managerial standards across the JSDF portfolio. Universalia 30 Volume I – Final Report The decision to wrap up the Steering Committee was taken after the period of review for this Assessment. Therefore, the reasons behind the decision to wrap up the Steering Committee are not germane to this present Assessment. 5.2 Timeliness  To be able to assess the timeliness of the grant application and approval process, it is first necessary to know what the timelines are. The documentation on the timeliness for the various steps in the JSDF process is uneven. As noted previously, during the 2007‐2012 period, the JSDF application process was based on a twice annual “call for proposals”, although in some instances there could be three such calls. During this period, what became important was the amount of time taken by the GoJ to consider these proposals, and subsequently, the amount of time required to move from approval by Japan to the signing of a grant. The first Finding addresses the period of time between submission of a proposal to the GoJ for its review and its approval. The second Finding discusses the amount of time taken between approval by the GoJ and the signing of the grant agreement. Finding 19: Based on the data that is available for the grant sample examined by this Assessment and during the time frame in question (2007‐2012), the average time taken to review an application by the GoJ approached twice the anticipated standard. The only time‐based performance metric for the JSDF grant application and approval process is the goal of a two month (eight week) turnaround by the Government of Japan to review a submission. Exhibit 5.1 shows that only nine of the 39 grants were processed by the GoJ in 60 days or less (a period of time that roughly corresponds with two months), or about 15% of grants. This very low success rate climbs to 17 of the 39 grants if we consider those processed in 70 days or less, with 47% (28 grants) requiring more than 70 days, bringing the average processing time to 112 days. This average is nearly double the stated goal for this process. Exhibit 5.1 Time from Transmission to GoJ to Approval by Japan (2007‐2012) Average Time for GoJ to Approve an Application 111.5 days Number of Grants % % 50‐60 days 2 3% % 61‐70 days 7 11.6% % 70+ days 28 46.6% % 50 days 8 13.3% # that could not be calculated due to data gaps 15 25% This timeline dovetails with perceptions by World Bank staff both at Headquarters and at the country level interviewed for this Assessment that the review by GoJ is overly lengthy. Yet a caveat to this is that there is a general lack of awareness among these stakeholders and others of the internal steps of the GoJ leading to approval. Importantly, discussions with the managers of similar Japan‐ supported trust funds at the Asian Development and the Inter‐American Development Bank highlighted similar issues with respect to the timeliness of the application process and specifically the timeliness of review by the Ministry of Finance and other elements of the GoJ. One of the contributing factors for such a degree of delay may have rested in the “round” nature of the then application process, with the staff at MoF being presented with numerous applications to consider at 31 Universalia Volume I – Final Report the same time. This makes it clear that the issue is not only internal to the JSDF application process but is rather located higher up the decision‐making chain. It should be noted that the shift since the end of this Assessment’s period of focus to a rolling application and approval process seems to have resulted in some saving of time. More specifically, anecdotal evidence from JSDF staff, TTLs and representatives of the GoJ point to some degree of improvement, although all recognize that more could be done. However, given the already limited resources available for the management of the JSDF, and as a result of the two stage application process (concept note and proposal), the rolling approach to grant approval seems to be problematic. More specifically, this results in unanticipated pressures and demands which could lead to slowdowns. Finding 20: During the 2007–2012 timeframe, the stages for the participation of representatives of the Government of Japan in grant approval was unclear and potentially may have resulted in duplication of efforts as well as slower turnaround times. A further contributing factor relates to the lack of a common understanding of innovation among the GoJ stakeholders. During discussions, representatives of the MoF indicated that while they were unaware of the specific practices of the 2007‐2012 era due to turnover, they currently attempt to circulate and return materials to sister Japanese agencies, MoFA and JICA, within the space of two weeks. However, discussions with representatives of these two agencies seemed to indicate that in some cases, a second round of country level consultation with representatives of the Government of Japan may ensue, especially in instances where there are questions about the links between the proposed JSDF grant and overall Japanese development cooperation strategies for the country in question. It also should be noted that during the review period, TTLs were required to consult with representatives of the GoJ in country, which included MoFA ( embassy staff) and if present JICA personnel. This implies that there is possible duplication of effort in the approval process, which could be negatively impacting upon the turnaround time for applications. A further potential factor in the slow down at this stage could be related to the lack of a common understanding of what constitutes innovation amongst GoJ stakeholders, which can provoke lengthy debates within the GoJ. This was exacerbated by the relatively rapid rate of turnover among MoF staff. Moreover, there appears to be some lack of understanding among GoJ stakeholders about the fundamental purpose of the JSDF. Some see the JSDF largely as a tool to replicate existing development interventions. Others however, recognize the fundamental nature of the JSDF, which is based on pioneering and innovation. Finding 21: There were significant delays in moving from approval of the proposal by Japan to the signing of the grant agreement. During the period of time in question (2007‐2012) and based on the grant sample, the average amount of time for a JSDF grant to move from approval by GoJ to the signing of a grant agreement is 252 days (see Exhibit 5.2). This period of time is internal to the World Bank, and it is important to emphasize that the processes leading to the signing of a grant agreement are executed through World Bank standardized processes. Various parts of the World Bank are involved in the process of developing a grant agreement, including legal services, disbursement, financial management, and procurement. DFPTF’s role in this is limited to providing clearance of the draft grant agreement. Universalia 32 Volume I – Final Report Exhibit 5.2 Time from Approval by Japan to Grant Agreement Signed (2007‐2012) N % Average number of days 252.6 ‐ % +14 days above average 34 grants 56.6% % ‐14 days above average 16 grants 26.6% # that couldn’t be calculated 9 grants 15% # that fell within the +/‐14 day band 1 grant 1.6% Several external factors appear to influence the timeliness of this phase. The requirement for recipients to prepare an operations manual can increase the amount of time required. Likewise, in certain circumstances, legislative or judicial approval by recipients is required, which can result in unpredictable delays to ratification of the signed agreement. Staff turnover at the TTL level also can influence the timeliness of the signing of a grant agreement. Finally, possible delays in approving the proposal by the GoJ can result in changed circumstances and beneficiary needs which require revisions to ensure relevance and future effectiveness of grants. This situation however, has been rectified to some degree. The recently established World Bank Small Grant Guidelines and Procedures require the development of a draft grant agreement in advance of formal proposal approval by GoJ. This has reduced the time between the approval by Japan and the signing of the grant agreement by bringing forward the development of mandatory items such as the draft grant agreement and other elements like a draft procurement plan and a draft operations manual. Reporting  Finding 22: The introduction of a new end‐of‐grant reporting instrument, the Implementation Completion Report (ICR), fills a current gap in the design of World Bank small grants. In 2012, the World Bank instituted a new reporting format for small grants, applicable to all programming, trust funds and otherwise; the Implementation Completion Report (ICR). The ICR, which is now a requirement for all JSDF grants and for all other small grants, fills a reporting gap by introducing an outcome‐based, evaluation‐like instrument. ICRs will be conducted at least six months after the formal conclusion of the grant. Unlike the former approach to end‐of‐project reporting, they will be conducted by a third party mutually acceptable to both the World Bank and the implementing partner. They will, for the most part, be focused on issues related to effectiveness and outcome performance. In essence, they could be seen as a sort of evaluation of the short to medium term outcomes that have been articulated in grant applications. While the then Guidelines for the JSDF did not require formal evaluations of individual grants, this Assessment uncovered a number of evaluation‐like instruments that had been developed as part of existing grants, and which focused greater attention on outcome performance. These ad hoc instruments, examples of which are shown in Exhibit 5.3, are obviously not standardized. However, they contain outcome and output performance information, and address issues related to beneficiary participation, implementation challenges and recommendations for future improvements. 33 Universalia Volume I – Final Report Exhibit 5.3 Samples of Evaluation‐Like Instruments Cambodia: Strengthening Good Governance in Land Distribution Cambodia: Strengthening Civil Society/Government Partnerships Nepal: Pro‐poor Secondary School Stipends Azerbaijan: Internally Displaced Persons (IDP) Youth Support Thailand: Youth Helmet Usage Afghanistan: National Solidarity Program 2 Jamaica: Conditional Cash Transfers Pakistan: Addressing the Poverty Nexus Nicaragua: Sustainable Agro‐Forestry Management Kenya: Rural Traditional Herbalists 5.3 Visibility of the Government of Japan  Finding 23: While visibility at both the level of individual grants and at the programmatic level is broadly in line with the existing guidelines, the implementation of the existing visibility guidelines is impacted by the dual lines of accountability that characterize the JSDF program model. There are many different elements related to the visibility of the Government of Japan. They exist at both the level of individual grants and in relation to the JSDF as a program. The dual lines of accountability that characterize the JSDF model impact on issues related to the visibility of Japan. With direct responsibility for the application process, staff and managers of DFPTF can ensure that as part of the application process, in‐country representatives of the GoJ are consulted about potential initiatives. As well, DFPTF staff ensure the presence of representatives of the Government of Japan during grant signing ceremonies as well as during project launch activities. However, given that TTLs and by extension, country teams, have responsibility (and accountability) for implementation, these same staff of DFPTF only have an indirect oversight and guidance role with respect to the roll out of the specific guidelines that speak to the visibility of Japan in terms of participation in events, branding of deliverables demonstrating Japan’s contributions and most importantly, measures to raise awareness among beneficiaries of the contributions of Japan. It is important to underscore that the DFPTF staff have undertaken measures to raise awareness with TTLs of the importance of Japan and have provided additional materials to encourage greater visibility and branding. At the level of the JSDF as a program, the visibility of Japan is widespread in World Bank websites and publications. JSDF program materials display logos that make clear the role of the GoJ, JSDF electronic resources are equally clear in showcasing the contributions made by Japan, and JSDF annual reports make explicit the role that the GoJ plays. Moreover, special publications such as the one spotlighting an innovative grant to empower rural women in Indonesia, “A Call To Dignity”, affirm the crucial role played by Japan. Current World Bank standardized reporting does not require specific information about issues related to the visibility of the Government of Japan, or indeed, donor visibility more generally. Therefore, the absence of information in periodic, standardized reporting is not unexpected. Universalia 34 Volume I – Final Report Field Missions uncovered some challenges in promoting greater visibility during implementation. For example, several TTLs and Country Directors/Managers explicitly said that they and many of their colleagues were reluctant to actively engage in branding of a particular donor, notwithstanding the guidelines that have been provided by JSDF program staff to do so. Moreover, it became apparent in a majority of Field Missions that the beneficiaries were only tangentially aware of the participation of what is for them an unknown foreign government. 5.4 The Program Management of the JSDF  This final set of Findings speaks to several program‐wide managerial issues. These issues are more related to strategic management considerations rather than the management of individual grants. More specifically, they focus on the understanding of the nature of the JSDF and its location within the context of the recently implemented changes to the organization of the World Bank, as well as to the level of resources presently available within DFPTF (then CFPTO) for the administration of the JSDF. Finding 24: At the moment, the JSDF is located in the Development Finance (DFi) Vice Presidency of the World Bank; however, there is no consensus regarding where it “fits” within the new structure of the World Bank Group. This Assessment of the JSDF has been undertaken simultaneously with the roll‐out of the first major reorganization of the World Bank Group in the last decade. During the course of interviews with TTLs and especially with senior managers and senior advisors of the World Bank located in Washington, there was discussion about the suitability of the current organizational location for the JSDF, within what is now the DFi Vice Presidency. Some of these stakeholders were of the view that the JSDF could better “fit” into the evolving social development area. These stakeholders were of the view that the JSDF was primarily about social policies. This however, is not entirely accurate. Exhibit 5.4 shows the range of the 53 JSDF‐supported initiatives that were reviewed as part of this Assessment. It is evident that JSDF‐supported activities go well beyond traditional understandings of what constitutes social development to encompass other areas of activity, thus demonstrating its cross‐cutting nature. Several other Headquarters‐based stakeholders expressed the view that locating the JSDF within the newly emerging Social , Urban, Rural and Resilience (SURR) Global Practice might lead to a form of institutional capture wherein the unique characteristics of the JSDF might be supplanted by overarching policies or strategies related specifically to social development. These stakeholders pointed out that the most fundamental characteristic of the JSDF was its ability to promote innovation and to catalyze willingness to experiment, largely in the country level context, and not so much to be a social development fund per se. Other stakeholders expressed different views. They indicated that the JSDF was not a sectorially‐ based trust fund in that its areas of activity cut across several of the new global practices that are being established as part of the overall reorganization of the World Bank. These observers indicated that there is an element of risk in conceptualizing the JSDF exclusively as a social development‐ related fund. Doing so would tend to de‐emphasize the importance of the promotion of innovation and ensuring community level participation. Exhibit 5.4 demonstrates the subject matter diversity of the JSDF. It shows that a significant percentage of JSDF‐supported activities fall outside of the range of social development, running from support to public administration to degrees of support for economic development. This implies that the JSDF is best considered to be a transverse kind of program whose activities cut across several of the newly developed global practices. 35 Universalia Volume I – Final Report Exhibit 5.4 Sectorial Distribution – All Grants Number of Grants in Portfolio Review Sample and Sector Field Mission Review combined Agriculture, Fishing and Forestry 9 Public Administration, Law and Justice 16 Information and Communications 1 Education 12 Finance 0 Health and Social Services 12 Industry and Trade 0 Energy and Mining 1 Transportation 0 Water, Sanitation and Flood Protection 2 53 (39 Portfolio Review sample + 14 Field Mission review) At this time, the newly emerging Development Finance Vice Presidency, the successor to CFPTO, would seem to be one choice for the JSDF to remain. Not only would the program’s cross‐cutting nature be preserved, co‐location within Development Finance would link the JSDF to the GoJ, in comparison to a potential location in a subject‐specific area. The cross‐cutting nature of the JSDF is one of its historic and most important characteristics. While retention within Development Finance as a whole is one choice, other choices exist so as to accommodate its transverse nature. For example, the Global Practices Strategies and Operations (GPSOS) unit is an organization within the restructured World Bank Group that has cross cutting oversight responsibilities. In that light, the JSDF might be equally well‐positioned to retain its fundamental core principles and not be misperceived as largely focused upon one thematic area such as social development. Indeed, location within such a cross‐cutting unit would also provide the JSDF with a broader degree of access to operational units within the World Bank. The dual lines of accountability for the JSDF (during the application process to the JSDF and during implementation, with the TTLs and by extension, the management of individual country teams) tends to favour the retention of the JSDF as a centralized entity, either in DFPTF or GPSOS because during the crucial application process, the JSDF is not subject to the priorities of any given subject‐specific global practice. These dual lines of accountability require a degree of centralization that cuts across global practices so as to ensure that activities funded under the JSDF are not only selective, but demand‐driven. Finding 25: Overall, the contention that JSDF grants require more supervision costs than provided for cannot be independently substantiated. To recall, during the review period of 2007‐2012, the JSDF allocated to a maximum of 9% of the overall value of a grant for supervisory costs if the TTL in question was based in Washington and some 5% if the TTL was based in‐country, the difference in large part being due to increased travel costs. Universalia 36 Volume I – Final Report During interviews, some TTLs expressed the Exhibit 5.5 View of TTLs Regarding the Question view that supervision costs exceed the level task teams were given adequate budget of funds allocated. Yet no substantive to carry out the grant‐making process evidence could be provided to corroborate Strongly Agree this view. Moreover, the TTL survey results tend to contradict this expressed point of Agree view, with only 26% of respondents stating Moderately Agree that there were inadequate resources (see Disagree Exhibit 5.5). Strongly Disagree One thing that emerged in conversations with TTLs is that they sometimes use Don't Know/NA creative techniques to efficiently manage 0 5 10 15 20 25 30 35 JSDF grant supervisory funds. For example, to increase the room for fiscal manoeuvre, they occasionally “piggy back” JSDF grant supervision onto larger World Bank initiatives, with a non‐resident TTL also responsible for a major loan in the same geographic area extending a supervision mission for the loan and thereby conducting supervisory activities with respect to a tangentially related JSDF grant. Finding 26: In comparison to comparable funds, the JSDF does not currently have similar levels of resources. Until August 2014, the JSDF mechanism was directly supported by less than two staff years. Only one individual is engaged on a full time basis, while two others operate on a part‐time basis, each with a resource allocation for the JSDF of less than half of a staff year. Fractional resources are available for financial administration and administrative support. This small team has a wide range of tasks: Coordinating the grant application process and playing an increased challenge function so as to improve the results focus of the JSDF. This presently includes the reviewing of concept notes and the appraisal package; Performing a clearance function of draft grant agreements, operations manual, and restructuring papers; Monitoring an active portfolio in excess of 110 grants, including Implementation Status Reports (ISR), Grant Reporting and Monitoring (GRM), the Implementation Completion Memorandum (ICM), and the Implementation Completion Report (ICR); Budget planning and monitoring, including monitoring of disbursements, developing quarterly financial reports; Developing annual and closed‐grant reports as well as other communications instruments; Maintaining front‐line liaison with representatives of the GoJ including annual resource discussions; Revising existing guidelines, not only in relation to the recommendations made by this Assessment, but on an on‐going basis to ensure continued relevance; Conducting orientation and sensitization missions to raise the profile of the JSDF and informally promote knowledge management and cross‐learning among TTLs; Organizing workshops and seminars for GoJ stakeholders, practitioners, academia and beneficiaries; 37 Universalia Volume I – Final Report Mounting oversight missions with respect to specific grants to ensure effective implementation as well as to facilitate the resolution of implementation issues; Undertaking a challenge function during implementation with TTLs to encourage a greater application of the guidelines with respect to the visibility of Japan; and Maintaining liaison with other elements of the World Bank. Key questions for this Assessment asked that a comparison be undertaken regarding resource allocations with analogous organizations. The most relevant comparisons are not with other World Bank trust funds or multi‐donor platforms. The fact that many have overarching policy‐related responsibilities for such things as public private partnerships, improved disaster resilience, and new information technologies to name only a few, results in inaccurate comparisons in that considerable levels of resources are allocated for these policy‐related functions and it is difficult to separate out those whose exclusive purpose is grant administration. As has been discussed in several instances in this Final Report, three similar trust funds exist at regional development banks, one at the Asian Development Bank and a second at the Inter‐American Development Bank, while the Human Security Unit of the Office of the Secretary General operates a very analogous trust fund, largely if not exclusively supported by the Government of Japan. The Asian Development Bank’s fund has three full‐time staff members to administer an annual budget roughly similar to that of the JSDF. In addition, at the Asian Development Bank, there are fractional staff years that support financial administration and office administration. At the Inter‐ American Development Bank, approximately two staff years are allocated to support a much smaller fund, less than half the size of the annual budget of the JSDF. At the Human Security Unit, a total of six staff years are allocated for a variety of functions, of which approximately two and a half staff years are exclusively dedicated to the management of an annual granting portfolio that is approximately similar in size and number to the JSDF. With respect to the similar funds at the Asian Development Bank and the Inter‐American Development Bank, the staff undertake a relatively similar set of duties as those described above with respect to the JSDF, recognizing that both these funds operate within the context of a regional bank as opposed to a global development bank. This is important to emphasize; unlike these other two funds, the JSDF, with less staff, operates within a global mandate. Finding 27: There is a degree of concern among some World Bank stakeholders about the relationship between the JSDF and the newly emerging priorities of the World Bank. During the course of interviews with stakeholders of the World Bank, it became evident that there is a lack of understanding about the linkages between the JSDF and the newly emerging World Bank strategic results framework that elaborates on the two major goals that President Kim articulated in early 2013, namely eradicating extreme poverty and promoting shared prosperity. Some stakeholders interviewed for this Assessment indicated that they perceived that the JSDF was only tangentially linked to these two major goals. This is in large part due to the misperception that the JSDF is in itself not well focused. It is also due to an apparent misunderstanding of the transverse nature of the JSDF wherein its catalytic and innovative properties serve to promote overall World Bank goals. Stakeholders at MoF and MoFA in Japan also expressed some degree of apprehension about the future of the JSDF. Their concerns lay in their perception that some in the World Bank did not see the degree to which the JSDF plays a role in the newly emerging priorities of the World Bank and contributes to the attainment of Japan’s foreign policy objectives, including those relating to human security. Universalia 38 Volume I – Final Report 6 Highlights of the Achievements of the JSDF  The foregoing discussion of the performance of individual JSDF grants has presented a generally positive picture. However, the full picture of the worth of the JSDF granting activity cannot be fully grasped in this sort of a disaggregated context. Accordingly, it is beneficial to present some highlights drawn from the total of 53 JSDF‐supported initiatives that were the subject of either the Desktop Portfolio Review or a Field Mission. The Statement of Work for this assignment suggested an approach based on the analysis of: the extent to which the JSDF supported poverty reduction; and how JSDF initiatives contributed to livelihood support and thus reduced poverty. This categorization forms the basis for the presentation of a series of vignettes. However, it is also appropriate to highlight two other important characteristics of the JSDF which contribute to its overall worth. They are: The extent to which JSDF‐supported initiatives are sustainable; and The degree to which innovation has been promoted. 6.1 Highlights of Achievements  Eight vignettes are presented below. While divided into the four categories described above, these eight cases show the inter‐relationships between these four primary characteristics of the JSDF. Many of the vignettes below highlight innovations; the introduction of new ways of thinking or new ways of working within the context of an individual country. Many show the extent to which such innovative approaches have been sustained. In short, these eight vignettes demonstrate the flexibility, sensitivity, responsiveness and sustainability of the JSDF granting mechanism. 6.1.1 Reducing Poverty and Supporting Livelihoods   Contributing to the reduction of poverty can involve a wide range of initiatives. Likewise, providing support for livelihoods indirectly contributes to reducing extreme poverty and increasing shared prosperity, the two strategic objectives for the World Bank that were articulated by President Kim in early 2013. The four vignettes that follow demonstrate that the challenges inherent in poverty reduction run the gamut from providing direct social assistance to individuals and communities in need; to the recognition that in a longer term context, early childhood care and development can lay the foundations for access to formal education, thereby contributing to “breaking the poverty trap”; to the recognition that securing land tenure can generate sustainable income, lifting families out of extreme poverty. The first two vignettes also showcase the country context‐based approach to innovation. In both instances, techniques that were seemingly relatively standard were introduced into completely new environments thereby transforming how solutions to human development challenges were conceptualized and subsequently implemented. Both vignettes also demonstrate the catalytic effect of individual JSDF grants. The positive short term outcomes of these two grants resulted in not only the continuation of the two initiatives themselves, but their expansion and integration into long‐term national development programming. In many instances, livelihood support relates to ensuring improved land use as well as ensuring fair and guaranteed land tenure. The third and fourth examples below focus on land. In the first instance, two very small grants played contributing roles in changing the way Cambodia undertook land distribution, one of the major poverty reduction issues facing that nation. In the second instance, a JSDF‐supported initiative to build capacity among farmers in the Philippines not only improved basic agricultural production, but also improved their capacity to manage their land, market produce and utilize their land as a means of generating capital. 39 Universalia Volume I – Final Report Tanzania – Community‐based Conditional Transfer Pilot Project  (TF090491)  This initiative, supported by a US$ 1.97 million grant, not only combated extreme poverty; it did so by introducing cutting‐edge techniques that were new to Tanzania. The need for deliberate targeting and inclusion of vulnerable groups in the social fund program was an explicit demand from the communities served by the Tanzania Social Action Fund (TASAF). TASAF is a Government of Tanzania funding organisation that provides a mechanism for local and village governments to respond to community demands for interventions that will contribute to the achievement of the MDGs. Community members expressed the view, however, that there was need for a program of direct support to vulnerable groups, as the poorest and most vulnerable groups within villages were not being served. Discussions were held with a range of stakeholders to support the design of the pilot initiative, including stakeholders in each village in two districts such as the community management committee, local leaders and the larger community, village school and health officials, and district/local government staff. The overall benefits gained by the project were the following: The community‐based CCT reached 4,998 targeted households covering 13,081 beneficiaries. Parents and guardians in targeted households now tend to seek medical care when their children are unwell. Consumption and access to services improved following the CCT program, as reflected in the reduction in illnesses among elders and infants, and increased school attendance by children. The CCT program has improved the process of managing both health and education sectors, through: the creation of more transparent systems; improvement in the quality of healthcare and education services; and improvement in family structure because of increased parental capacity to afford meals, take their children to school and access social services. The use of the Community Driven Development (CDD) approach, including administration of the CCT program through a social fund agency, together constituted the innovative aspect of the program. This was the first time in Africa that: i) a social fund agency was being used to administer a CCT program; and ii) a CCT program was being delivered through a CDD approach. This innovative approach to combating extreme poverty now will be sustained. The Government of Tanzania set aside US$ 4.5 million in additional financing under the Second Tanzania Social Action Fund Project to scale up the CCT program. Vietnam – Early Childhood Care and Development (TF052939)  Combating extreme poverty can take many shapes. This initiative from Vietnam shows the human dimension of poverty reduction and the links between reducing poverty and supporting early childhood education. In the remote mountainous regions of Vietnam, many families and communities remain unaware of the importance of the holistic development of children, particularly in their early years, and lack the information and the resources to implement best practices in child rearing. The JSDF grant provided a coordinated, comprehensive, and low‐cost Early Childhood Care and Development (ECCD) model utilizing innovative strategies to improve the environment, health and education services along with knowledge and skills of parents, caregivers, community members, and government leaders in supporting the holistic development of young children. Universalia 40 Volume I – Final Report This US$ 1.9 million initiative implemented by Save the Children contributed to increasing the percentage of children aged 3‐4 years old who are enrolled in kindergarten (the rate increased from 60% to 98%). Furthermore, its health component (promoting better access to health care services) contributed to successful outcomes. For example, there was a 6 to 8 % decrease in malnutrition rates for children. In addition to reducing poverty, this project contained a number of highly innovative measures and has proved to have been a catalytic agent for long‐term change. It piloted for Vietnam innovative approaches such as providing bilingual education to pre‐primary school children. Following project completion, the World Bank decided to implement a large‐scale ECCD project in Vietnam, investing US$ 100 million. The bilingual education model has also been picked up and implemented at the primary and secondary school levels elsewhere in Vietnam, having been integrated into the national early childhood education curriculum. As well it generated a “good practice” note. Cambodia – Strengthening Good Governance in Land Distribution  (TF091833); and Strengthening Civil Society‐Government Partnership  to Deliver Land Tenure Security (TF091836)  After decades of disruption, the land tenure system in Cambodia remains fragile at best. Large portions of both the rural and urban populations do not have legal tenure and are thus put at risk in terms of the continuation of their livelihood. Many urban and rural poor do not have any access to land, living a precarious lifestyle where employment is frequently on a day‐to‐day basis and secure access to a home, let alone social services, problematic. The Government of Cambodia has put in place a long‐term and comprehensive approach to addressing the land tenure issue which has negatively impacted on the country’s growth. Its Land Allocation for Social and Economic Development (LASED) project aims to determine tenure of land and restore land titles. The World Bank’s Country Assistance Strategy recognized the importance of government initiatives to undertake comprehensive land reform. However, it became apparent that there were conceptual gaps in the government’s approach. Accordingly, the JSDF was called upon to support three pilot initiatives that primarily focussed on engaging landless urban and rural poor people more actively in land tenure decision‐making and also increasing their economic skills, so that they could benefit from new land. Two of these grants were examined as part of this Assessment. The projects were in the range of US$ 500,000 each, and involved utilizing NGOs to reach out to the urban and rural poor, something that had not been undertaken in earlier approaches to land distribution. The first small project involved improving access of the urban disenfranchised poor to land within the city of Battambang. After extensive local consultation, the Government agreed to work with a local NGO to raise awareness among the urban poor, to give them the tools so that their voices could be heard and to provide technical support in terms of housing construction and land use. Over a period of four years, several hundred plots were distributed to previously landless families. Most importantly, the distribution pattern was fully transparent, being based on an objective Needs Assessment that was conducted by the NGO. The results of the initiative are considerable. Not only were new technical skills engendered, new community associations were established and continue to thrive. More and more people have started up or improved businesses after having official plots of land. More families now are having the connection to clean drinking water and electricity. Income generation activities are more sustainable. A range of new businesses have grown up, including: grocery, sewing, shoemaker, cookie production, hair dressing, motor repair, etc. The second small project addressed the needs of the rural poor. 41 Universalia Volume I – Final Report A national NGO worked with the government to build the capacity and voice of landless families and civil society organizations in gaining access to land and livelihood support through three main components including: Public awareness and capacity building; Piloting community support groups and NGO capacity building; Coordination and sharing lessons learned. The results of this initiative were significant. Over 3,000 families, previously landless and living in precarious circumstances, now possessed plots of their own. Virtually all of these plots are now being successfully farmed. The community associations established during the selection process for land distribution have continued and remain effective voices for the people themselves. There is early evidence that incomes among these 3,000 families have considerably increased, in large part due to the value of agricultural production. In many instances, recipients have combined more stable day wages with tending their own plots, thereby diversifying their income. Equally, new communities have emerged anchored by new schools and health posts. Entrepreneurship is beginning to take root in the form of small community‐based businesses, thus keeping wealth, to some degree, within the community itself. In short, while the JSDF‐supported initiative did not pay for the land, nor did it build the schools or health posts, it contributed to building a consensus. It supported the first‐time application in Cambodia of an objective means test as the determination for land transfer, and it built social consensus. It contributed not only to supporting existing livelihoods but in fact to shaping productive and sustainable livelihoods where none had existed before. Philippines – Community‐Managed Agrarian Reform and Poverty  Reduction Project (TF051601)  This US$2 million initiative aimed at empowering poor farmers to become owners of their land and have better infrastructure to increase their production and therefore their income. In doing so, almost 3,000 households in ten different regions benefited from infrastructure subprojects. This initiative, designed to support agricultural livelihoods, was designed as a complement to an on‐going project of the World Bank, and as such was in accord with the overall Country Strategy and themes. The farmers themselves participated in the design and implementation. The overall objective of this project was to complement and accelerate the ongoing comprehensive agrarian reform program being undertaken by the Government of Philippines. It was to do so by supporting three of the seven strategic objectives of the World Bank’s Country Strategy: (i) addressing the effects of the East Asian financial crisis and promotion of economic recovery; (ii) accelerating environmentally sustainable rural development; and (iii) enhancement of human development and social services for the poor. Three major components were successfully implemented: Social preparation, organizational development training and mobilization of agrarian reform communities and beneficiaries; Technical assistance/services in the formulation of comprehensive area development plan per community and farm business plan per beneficiary; Technical services and financing of demand‐driven small scale rural infrastructure and on‐ farm investments. In terms of the catalytic effect of this initiative, discussions are on‐going regarding the replication of this approach elsewhere in the Philippines. As well, representatives from Vietnam have conducted site visits to explore lessons learned and the possibility of replication in Vietnam. More importantly Universalia 42 Volume I – Final Report however, this initiative showed that poor farmers can be empowered to negotiate for and pay for their own land, as well as be given assistance in strengthening their long‐term economic capacity, resulting in a sustained contribution to rural poverty reduction and livelihood support. 6.1.2 Promoting Sustainability  The following two vignettes highlight two different catalytic effects of JSDF grants. The first involves a community‐based approach to land management being sustained by the communities themselves. In this instance, capacity building and community empowerment activities have been retained by the beneficiaries who continue a program of improved land use management. In the second instance, the effectiveness of cash transfers to address livelihood challenges among disadvantaged groups led the Government of Jamaica to recognize the benefits of such transfer mechanisms and to integrate these techniques into on‐going social programming. Honduras – Community Disaster Management in the Barrios of  Tegucigalpa (TF052879)  In Tegucigalpa, the capital of Honduras, poor residents living in informal settlements (barrios) suffer from frequent flooding and landslides, made worse by poor waste management systems and deficient infrastructure. In order to address this issue, the JSDF granted US$ 1,984,500 to the Municipality of Tegucigalpa to implement a pioneering, community‐driven disaster management project. The project’s goals were to “(i) build local and community capacity to manage environmental risks in the Sapo zone, an informal market area highly susceptible to flooding; (ii) institutionalize a working relationship between communities, local emergency response committees and the Municipality of Tegucigalpa, and strengthen ties to NGOs; and (iii) develop and implement new, participatory techniques for community Risk Assessment and disaster mitigation, ultimately defining a methodology for community level disaster mitigation.” Despite delays due to changes in the municipal government as well as bad weather bringing on more flooding, this initiative managed to exceed its target output by 500%. The original goal was to train and support 20 local emergency response committees; the number of communities reached was actually 109. There is strong evidence that the project has been sustained by the beneficiaries themselves, and that some communities have now taken ownership of the project (e.g. planting ornamental bushes and ensuring that the drainage ditches were well maintained). The Municipality of Tegucigalpa’s Operations Center for Disaster Emergency (CODEM) has decided to integrate this community‐driven model of disaster management into its own procedures, and it is now standard modus operandi at this level of government. Jamaica – Program of Advancement through Health and Education  (PATH) Conditional Cash Transfer Program (TF096923)  The objective of this initiative was to improve the livelihood of about 10,000 poor and vulnerable elderly and disabled persons affected by the food, fuel and financial crises, through cash transfers and improved outreach of the PATH Program. The activities were fully aligned with the World Bank’s Country Partnership Strategy (CPS) 2010–2013, particularly under Pillar II: Promoting Inclusive Growth, Results Area 2: Strengthen Human Capital. The Project was also aligned with the country’s National Development Plan (NDP) Vision 2030, contributing to the achievement of the first national goal, which seeks to empower Jamaicans to achieve their fullest potential. The grant responded to 43 Universalia Volume I – Final Report community input, having identified the need for a more targeted approach to provide social assistance to the most vulnerable and the elderly. The US$ 2.64 million grant has been successfully implemented and the development objectives have been met. It has been able to largely increase the registration of beneficiaries in the categories of poor elderly and people with disabilities meeting the target of an additional 18,519 beneficiaries. On average, 81% of the persons added over the past two years come from the elderly group. Poor elderly were the majority of new beneficiaries registered, demonstrating that there is a high uptake of social protection benefits by the elderly. The initiative also highlighted the need to link poor elderly and persons with disabilities with other services such as housing, health, legal counsel, etc. In terms of its catalytic effect, as a result of the lessons learned, the Government of Jamaica has now expanded the program by integrating the community‐based outreach practices that were piloted where community workers actively reach out to the elderly and most vulnerable, bringing social assistance programs to people who do not have the capacity to make the request themselves. 6.1.3 Promoting Innovation  Many times attempts to reduce poverty and promote social equality tend to rely on using “best practices” that others have developed. All too often, simple replication fails to recognize how unique conditions require specially developed solutions. The JSDF’s focus on support for innovative approaches enables communities to develop solutions that meet their own needs. Senegal – Fighting Child Begging and Human Trafficking in Senegal  (TF091830)  Child begging and human trafficking of young people is a persistent problem in Senegal, and the Government has had limited success in addressing it using traditional approaches. The Government’s review of its national development plans found that only 33% of the children‐related actions identified in the “vulnerable groups” component had been addressed. In addition, child beggars constitute a very marginalized cohort among those denied access to education, and could not be accommodated in the regular Education for All projects because of the specific, multi‐sectorial approach required, thereby placing them at risk to human traffickers. A group of concerned civil society members had been meeting for over a year to coordinate and strategize with support from the World Bank (and UNICEF). Their work became the basis for this US$ 1.7 million initiative. Civil society, religious leaders and government representatives conducted extensive consultations. Subsequently, they approached the World Bank Country Office for technical and political support in addressing the problem of child begging. Previous interventions focused on providing assistance to children already in the street. This was the first time that a concerted effort towards prevention was made, which was considerably cheaper than after the fact assistance, and aimed to result in much less suffering for the children and removing them from the influences of human traffickers . The project also took the only sustainable approach to address the problem, and had the advantage of putting families and communities on center stage, encouraging them to take greater responsibility for the well‐being of their children. An additional innovative component was the comprehensive approach developed to address demand (from Quranic teachers) and supply (from families) factors of child begging. Impressive results ensued: 98 local committees to protect children against begging were set up; 76,464 households were informed and educated about begging and human trafficking issues; Universalia 44 Volume I – Final Report There was a 68% reduction in the migration of young people in 35 villages in one region and 71% reduction in the 15 villages of a second; The Government of Senegal received for the first time a complete curriculum, harmonized with and validated by the Ministry of Education, the Quranic teachers and experts in Islamic education. This innovative initiative has generated catalytic effects. It has led to the Government preparing and adopting a new law to protect children from begging and human trafficking. In addition, the project contributed to the development of a curriculum for Quranic schools that is being implemented by the Government with the new International Development Association (IDA)‐financed project, the Senegal Quality Improvement and Equity of Basic Education. Sierra Leone – Developing a Model for Delivering Primary Justice  Services (TF057251)  After nearly a decade of civil war, Sierra Leone continues to face considerable economic and social challenges. Livelihoods were disrupted and poverty remains rampant. If not more importantly, the social contract between citizen and state had evaporated, resulting in a loss of confidence in the state and thereby a loss of fundamental human security. Crucial societal institutions such as the justice system lay in tatters. The JSDF was called upon to address one of these societal gaps, lack of confidence in and access to the justice system, especially as these gaps affected women and other disadvantaged groups. A US$ 900,000 capacity building initiative was approved. Informal community Needs Assessments, as well as targeted consultations with traditional leaders and representatives of various community organizations, were conducted in five chiefdoms in the Northern and Southern provinces of Sierra Leone. The assessments were used to identify the scope of justice problems and needs in these chiefdoms. This innovative initiative aimed to improve the access of disadvantaged groups in Sierra Leone to customary and formal justice and increase the accountability of justice sector institutions and their constituents. The focus of this initiative on improving access of the poor to services was consistent with the overall objective of the national devolution program, entailing increased citizen access to a wide range of basic services such as education, health, water, sanitation and transport. It also complemented the work of the World Bank’s Institutional Reform and Capacity Building Project in Sierra Leone in improving democratic participation and accountability at the local level, and ensuring that general institutional reforms are beneficial to the citizenry. The target beneficiaries were disadvantaged groups, including women. The overall benefits gained were as follows: Community dialogue meetings contributed to post‐conflict inclusion in communities, including the empowerment of beneficiaries to enable them to speak out and seek help when needed. Traditional town chiefs were included in the project implementation process through the creation of Community Oversight Boards, which assisted in bridging the gap between the formal and informal legal systems in Sierra Leone. The service of paralegals, and additional coordination with civil society to provide justice services, was included as part of the state structure. 45 Universalia Volume I – Final Report The innovations that were fostered by this initiative are subtle in nature but extremely important in the rebuilding of the social contract among individuals, civil society organizations and governments. They have included: Use of the term "justice services" instead of "legal services" to refer to both a wider range of justice problems and a wider range of problem‐solving tools as compared with conventional legal services; Use of community‐based paralegals rather than lawyers as frontline providers of justice services; Interaction between paralegal advocacy, with negotiation and organizing on one hand, and the strategic use of litigation on the other hand; Commitment to engaging both formal and customary institutions; and, Generating reform proposals from grassroots experience rather than from expert analysis. Universalia 46 Volume I – Final Report 7 Recommendations  The Recommendations are divided into two broad categories: 1) Strategic 2) Operational The recommendations provide practical advice but do not go into details of the “how to”, since determining this falls within the responsibility of World Bank and DFPTF staff and managers in consultation with representatives of the GoJ. In section 7.3, we present a mechanism that prioritizes recommendations and suggests which entities would be responsible for their implementation. 7.1 Strategic  The Strategic Recommendations are designed to reiterate the strengths of the JSDF, clarifying its role and helping it to better align with the overall strategies of the World Bank. 7.1.1 Reaffirming Strategic Relevance  Recommendation 1: The World Bank Group needs to reaffirm the value of the JSDF partnership with the Government of Japan to maintain the JSDF at, as a minimum, the present funding level over the next three years. This first Recommendation is designed to give the JSDF a window of time to be able to implement a number of operational level modifications (to be presented below) which would have the effect of strengthening its relevance, sustainability and effectiveness. It also serves to confirm the worth of the JSDF partnership as a whole for both the World Bank and the GoJ, and its current design and strategic directions. This Recommendation also calls upon representatives of the GoJ located in Washington and at Headquarters in Japan to make a contingent commitment to the JSDF so as to ensure a stable planning environment, while not limiting the annual decision‐making power of the GoJ. In so doing, more relevant anticipatory planning can be carried out by both the World Bank and the GoJ, further integrating the JSDF within the priorities of both. Recommendation 2: DFPTF managers should rapidly finalize a strategic results framework for the JSDF so as to articulate how, as an innovative global program, it contributes to and is part of the hierarchy of evolving World Bank priorities and is aligned with the strategic goals of reducing extreme poverty and increasing shared prosperity. This Recommendation has been somewhat subsumed by the requirement for all of DFPTF, including JSDF, to develop individual results frameworks for each program component. However, it has been retained herein to emphasize the importance of program level results frameworks. Such a results framework will transform the understanding of programs like the JSDF to more accurately represent it as an instrument available to the World Bank to promote innovation across a broad range of thematic areas, all contributing to the ultimate objective of reducing poverty and increasing shared prosperity. The soon to be developed results framework for the JSDF will give to DFPTF managers and decision‐ makers at the MoF a more precise instrument on which to base approval decisions and also the ability to better conceptualize how the JSDF as a whole can contribute to both World Bank objectives and also the overseas development strategies of the Government of Japan as a whole. 47 Universalia Volume I – Final Report At the time of writing, Fall 2014, managers of DFPTF have begun to draft individual results frameworks, including one for the JSDF. 7.1.2 Increasing Recognition and Building New Partnerships  Recommendation 3: The JSDF mechanism should be located within an organization in the World Bank which possesses a transverse mandate, for example Development Finance or the Global Practices Strategies and Operations (GPSOS) unit. During the course of interviews after the completion of the Field Missions for this assignment, some World Bank stakeholders suggested that the JSDF mechanism might be better located in the newly emerging Social, Urban, Rural and Resilience Global Practice. Yet, as had been described earlier in this Assessment, doing so would tend to place some of the primary characteristics of the JSDF mechanism at risk and potentially subject the mechanism to “capture” by a particular subject specific global practice within the World Bank. Earlier Findings showed that the work of the JSDF transcended what is normally understood to be “social development” or “social protection”. Therefore, placing it within that subject specific global practice could limit its overall development effectiveness. As well, while the words “social development” exist within its title, the JSDF’s real focus has been shown by the Findings of this Assessment to be related to poverty reduction through the introduction of innovation and experimentation and the promotion of “bottom‐up/community–based” approaches. These fundamental characteristics might be reduced in importance if this cross‐cutting instrument were to be located in a subject specific global practice that its own specific priorities. Retaining the JSDF mechanism within an element of the World Bank which possesses cross‐cutting roles and responsibilities, emphasizes its transverse and holistic nature and by extension, underscores the recognition that the strategic goals of the World Bank are not in themselves sectorially‐driven. In essence, the JSDF is a small, but highly relevant value‐adding element within the panoply of World Bank programming. 7.2 Operational  The second set of Recommendations speaks to more operational matters and specifically addresses some of the issues identified in various Findings. 7.2.1 Clarifying and Strengthening Guidelines   Recommendation 4: DFPTF and the GoJ should agree on a more explicit common understanding regarding the nature of innovation. During the period of the Assessment, training materials and information provided to applicants described innovation in terms of bringing new ideas to a particular environment. It should be noted that in 2013, JSDF management sought and received confirmation from Japan that the understanding of what constitutes innovation that the Steering Committee and the Secretariat had been using to assess the concept notes/proposals was still valid. This Assessment reaffirmed that innovation should be understood within a country specific context. Such an approach implies that the nature of innovation will be different in each country. For example, introducing an objectively‐based mechanism to identify and prioritize candidates for a social stipend in one country may involve introducing to that country concepts which are completely new to it, or bringing together groups of participants who have never collaborated before. In this instance, the innovation is directly contextual to the country in question and not within a global context given that Universalia 48 Volume I – Final Report such objective tools have been frequently used in other countries. This latter point is crucial to emphasize. Universalia is of the view that innovation within the framework of the JSDF should not be considered in terms of whether the actions constitute a “global first”. Rather, what is more important is whether the activities are pioneering within the country in question. Likewise, innovation should also be understood to constitute bringing to the World Bank new ways of reaching out to the neediest and also new ways of delivering programming. The crux of this contextual approach would be whether these new ways of working and collaborating are in themselves fresh within the country context. Innovation also should be understood to include the introduction of new ways, which result in more efficient and timely delivery of services to beneficiaries. Conversely, any expanded and clarified understanding of innovation also needs to conceptualize it in terms of what it is not. For example, there remains uncertainty in some quarters of the Government of Japan, but not MoF, that the JSDF mechanism should be considered as a tool to engage in “scaling up”, or transplantation of so‐called best practices into other locales. In both these instances, the JSDF is ill‐suited as a replication mechanism. The relatively small size of its budget as a whole means that a sufficient critical mass could not be developed. Equally, factors of scale tend to argue against the use of the JSDF as a replication mechanism within any given country. Doing so could lead to the weakening of the JSDF as a global initiative. It will be important to operationalize this Recommendation. JSDF may wish to consider developing a short policy paper/guidance note to foster within the GoJ a common understanding of the nature of innovation. In this way, a more durable and longer‐term “backgrounder” will be available to counteract the inevitable institutional memory loss that occurs during staff turnover. Recommendation 5: DFPTF and the GoJ should revisit the current Guidelines which address issues related to the visibility of Japan to more explicitly articulate what are minimum visibility criteria, what should be expected in different circumstances and what should be submitted to DFPTF to demonstrate visibility. Earlier Findings highlighted that the degree of visibility of the GoJ as the sponsor of the JSDF varied widely. This issue is complicated by the split lines of accountability for the JSDF described earlier. It is important to recall that the JSDF team itself has direct accountability for the application process up to the signing of the grant agreement, while accountability for implementation rests with TTLs and the World Bank management at the country level. While discussions with TTLs both as part of Field Missions and follow‐up data collection showed that representatives of the Government of Japan had been more active in supervisory and orientation missions than was generally thought; the extent of branding of JSDF grants as being a contribution by the people of Japan remains uneven. It is important to note that DFPTF recently recognized this gap and has already begun to put in place corrective measures – a new Tool Kit. This Recommendation builds on those measures by suggesting a higher degree of codification and subsequent reporting. Specifically, Universalia suggests that the current Visibility Guidelines be strengthened to add specific examples of the nature of “branding” which have been identified in the Tool Kit, and also be expanded to more clearly indicate that the GoJ strongly expects a higher degree of recognition. Examples should be given as to how to increase awareness among beneficiaries of the role played by Japan. 49 Universalia Volume I – Final Report Finally, to promote a higher degree of branding and general visibility of the contribution of the GoJ, an annual statement should be required of the TTL at the time of the preparation of the “Status of the Portfolio” report requiring each TTL to report on all visibility and branding measures that have been undertaken over the past year. Recommendation 6: In order to improve the results orientation of the JSDF grant monitoring, DFPTF should provide additional guidance related to the presentation of periodic report writing and should revisit the application format and content of the application package. The following constitute some initial considerations with respect to enhancing the quality of the application process while still utilizing standardized formats. Specific direction needs to be given to increase and standardize the level of detail in the application process with respect to: The degree to which beneficiaries are consulted in the grant design; Specifics to be provided ‐ more detail on the sequence of engagement so as to ensure that the grants are “bottom‐up”. The relationship between the proposal and the new JSDF results framework; Explicit references to the results linkages. The articulation of what constitutes “innovation” within the context of the country involved; Requirements that Concept Notes and subsequent formal applications explain in detail the nature of the innovation in question, why it is innovative in the context of the country, what groups will benefit from the innovation, what alternative there might be and why they would not be considered innovative, the likely costs of the innovative practices and the risk involved. Turning to periodic reporting which is carried out by TTLs and which uses World Bank standardized reporting formats, the JSDF needs to provide specific direction that in relation to JSDF grants, the following areas of the standardized formats need to reflect some JSDF‐specific considerations with respect to: Increasing reporting on beneficiary participation during implementation; Specific information about on‐going consultations needs to be included in each periodic report; Information about beneficiary participatory monitoring needs to be enhanced. These suggestions all should be seen in the light of an improvement in on‐going monitoring activities and strengthening the results‐focus of the JSDF. 7.2.2 Building a New Culture of Results  Recommendation 7: As a bridging mechanism, the GoJ should set aside a specific sum for up to three years to be managed by DFPTF to accommodate carrying out a small number of ICRs per year pending the transition of all JSDF grants to ISR and consequently ICR. As noted earlier, the World Bank has implemented a new end‐of‐project reporting system, the ICR, which will constitute for the JSDF a mechanism somewhat similar to outcome ( not impact) level evaluation, thus considerably improving the quality of performance reporting. Universalia 50 Volume I – Final Report At present, there are approximately 110 on‐going initiatives, most of which are preparing ISRs. While it is possible that some may independently generate evaluation‐like reports of their own accord as was uncovered through this Assessment, it would be unfair to retroactively mandate an additional expense (mounting an ICR) on an on‐going grant. Equally, it would be fiscally imprudent to seek additional support from the GoJ to provide broad coverage to retroactively finance the development of ICRs for all prior grants. This Recommendation suggests a medium‐term fix, utilizing a separate fund of approximately US$100,000 per year for three years to carry out a small number of ICRs in each year. Such an approach would provide a means of collecting improved results data for aspects of the portfolio that may not be covered by the current end‐of‐project reporting given that the more comprehensive ICR approach to JSDF grants would likely not begin to take place until 2016 given the lifespan of most JSDF grants. 7.2.3 Strengthening World Bank Capacity to Manage the JSDF   Recommendation 8: DFPTF and the GoJ need to revisit the current approach to formal approval of grants by the GoJ in an effort to reduce the current delays and to operate within existing agreements which are based on the principle of assumed consent. As well, the establishment of an Advisory Board to conduct a vetting of concept notes could be considered. This Recommendation addresses the perception that the JSDF application process is not sufficiently timely. For the sample period of 2007–2012, the average duration of 110 days between submission to the Office of the Executive Director and receipt of formal approval was effectively double the standard which was agreed to in the most recent version of JSDF guidelines (2011). It should be noted however, that with the elimination of the “rounds” system in 2012, there have been some improvements in the rapidity of the Japanese response, although not as yet meeting expectations. In the recent past (2011 and 2012), several ideas have been explored regarding mechanisms to improve timeliness, but none has been adopted. It should be noted that the JSDF’s sister poverty alleviation funds at the Asian Development Bank and the Inter‐American Development Bank also stipulate final approval by the Ministry of Finance. Discussions held with representatives of MoF, MoFA and JICA emphasized the degree to which the GoJ as a whole remains committed to retaining a strong degree of control over final approval. In addition, strong consideration should be given to establish a new Advisory Board (a sort of successor to the prior Steering Committee) charged with the responsibility to review concept notes before submission to the MoF. Comprising World Bank senior managers and including one or two representatives from the Office of the Executive Director of Japan, this new Board could play a challenge and Quality at Entry function before submission to the MoF, thereby possibly reducing the time needed by MoF at its Headquarters to conduct its review. The model suggested in this Recommendation would not alter the final authority of the GoJ. The notion of “assumed consent” would involve the World Bank and the GoJ agreeing to a specific cut‐off date, possibly the eight week benchmark set down in the current guidelines. Unless specific directions were to be received after this cut‐off date, it would be assumed that the grant in question had been approved. Care would need to be taken during negotiations to specify the kinds of additional information that might be required should a prolongation of the review process be required. 51 Universalia Volume I – Final Report Recommendation 9: DFPTF and the GoJ need to amend their agreements to provide that the current seed fund grant be transformed into a more flexible tool that would provide for a degree of advance spending so as to strengthen the capacity of implementing agencies to meet the mandated fiduciary requirements of the World Bank. This Recommendation is drawn from two complementary sets of Findings. First, it has been shown that the use of seed fund grants does not result in any appreciable difference in individual grant performance in relation to effectiveness or sustainability. Second, there are some challenges in relation to strengthening the delivery and managerial capacities of a number of local government and civil society as well as NGO implementing partners. This Recommendation would redirect the funds currently used for seed fund grants towards advance capacity‐building expenditures, thus resulting in a more efficient and effective usage of overall JSDF resources. Specifically, re‐profiling these funds would provide TTLs with a mechanism to strengthen implementing capacity, especially among CSOs and NGOs. More particularly, this Recommendation implies that the funds in question should be made available to prospective implementing agencies as soon as possible after approval by the GoJ of the grant itself, thus enabling preparatory and capacity building activities including the development of baseline studies to be undertaken in advance of the formal implementation of the grant. This naturally therefore would result in savings of time and improve the sensitivity of the implementation process. Recommendation 10: DFPTF and the GoJ need to rapidly enter into negotiations to provide additional human and financial resources so that the JSDF can become more results‐oriented. A number of Findings in this Assessment pointed to the extent to which the JSDF mechanism may be facing challenges to meet the implications of switching to a more‐result focus environment. It is necessary to highlight that the JSDF is being operated with less than two full staff years including administrative support, and only one staff year allocated on a full time basis. This is less on a proportional basis than that allocated for sister Japan‐supported funds, the most relevant comparator bodies. To recall, the Asian Development Bank’s fund has three full‐time staff members to administer an annual budget roughly similar to that of the JSDF. At the Inter‐American Development Bank, approximately two staff years are allocated to support a fund half the size of the JSDF. At the Human Security Unit, a total of six staff years are allocated for a variety of functions, of which approximately two and a half staff years are exclusively dedicated to the management of an annual granting portfolio that is approximately similar in size and number to the JSDF. More importantly however, the Findings and Recommendations of the Assessment point to increased, not decreased, workload with additional responsibilities evolving in the following areas: Enhanced coordination of the application process and playing a stronger challenge function so as to strengthen the quality of applications to ensure that results will be in line with the Fund results framework; More active monitoring of an active portfolio in excess of 110 grants, recognizing the complexity and sensitivity of the split lines of accountability for the implementation of JSDF grants; Analyzing and subsequently reporting on program performance in relation to a new JSDF results framework; Developing more sensitive reporting on the visibility of Japan, monitoring and playing a challenge function with respect to the new Implementation Completion Reports, which if Universalia 52 Volume I – Final Report made sensitive to the JSDF’s unique needs, will considerably improve the results focus of the JSDF itself; Surprisingly, after nearly 15 years of existence, there remain some 20 low and lower middle income countries that have not been reached by the JSDF. Conducting additional outreach and sensitization missions with country level staff to increase awareness of the benefits of the JSDF funding mechanism and reach out to countries that have not yet tapped into the program; Mounting oversight missions so as to strengthen quality control and foster increased compliance by TTLs with implementation‐related guidelines; Establishing more formalized approaches to knowledge management and cross‐learning among TTLs so as to broaden understanding and usage of some of the pioneering techniques (lessons learned). These suggested improvements will strengthen the results focus of the JSDF. They will also more closely link it with higher level World Bank goals. 7.3 Setting Priorities and Responsibilities  This Assessment presented two broad categories of Recommendations: Strategic and Operational. Yet even this level of categorization is not sufficient. These Recommendations need to be categorized on the basis of the degree of their urgency. Two categories related to urgency are presented below, along with the identification of responsibility and overall timeliness. These two categories are: Urgent to implement – Recommendations which are central to the strategic direction of the JSDF mechanism as a whole and which underpin all subsequent recommendations. Important to implement – Recommendations that are largely operational in nature and constitute a range of activities that can be implemented by Development Finance and whose implementation would be highly important to improving the day‐to‐day operations of the JSDF mechanism and strengthening its contribution to the attainment of World Bank strategic objectives. All these Recommendations have a relatively short timeframe, a maximum of two years. As well, in line with the Statement of Work, responsibilities for implementation are suggested. Exhibit 7.1 Recommendation Matrix Recommendation Responsibility Timeliness Urgent to implement Immediate (within 6‐8 months) Recommendation 1: The World Bank Group needs to DFPTF/ the Senior Immediate reaffirm the value of the JSDF partnership with the Management of the Government of Japan to maintain the JSDF at, as a minimum, World Bank Group and the present funding level over the next three years. GoJ Recommendation 2: DFPTF managers should rapidly finalize DFPTF Immediate a strategic results framework for the JSDF so as to articulate how, as an innovative global program, it contributes to and is part of the hierarchy of evolving World Bank priorities and is aligned with the strategic goals of reducing extreme poverty and increasing shared prosperity. 53 Universalia Volume I – Final Report Recommendation Responsibility Timeliness Recommendation 3: The JSDF mechanism should be located DFPTF and The Senior Immediate within an organization in the World Bank which possesses a Management of the transverse mandate, for example Development Finance or World Bank Group the Global Practices Strategies and Operations (GPSOS) unit. Recommendation 4: DFPTF and the GoJ should agree on a DFPTF and GoJ Immediate more explicit common understanding regarding the nature of innovation. Recommendation 5: DFPTF and the GoJ should revisit the DFPTF and GoJ Immediate current guidelines which address issues related to the visibility of Japan to more explicitly articulate what are minimum visibility criteria, what should be expected in different circumstances and what should be submitted to DFPTF to demonstrate visibility. Recommendation 6: In order to improve the results DFPTF Immediate orientation of the JSDF grant monitoring, DFPTF should provide additional guidance related to the presentation of periodic report writing and should revisit the application format and content of the application package. Recommendation 8: DFPTF and the GoJ need to revisit the DFPTF and GoJ Immediate current approach to formal approval of grants by the GoJ in an effort to reduce the current delays and to operate within existing agreements which are based on the principle of assumed consent. . As well, the establishment of an Advisory Board to conduct a vetting of concept notes could be considered. Recommendation 10: DFPTF and the GoJ need to rapidly DFPTF and GoJ Immediate enter into negotiations to provide additional human and financial resources so that the JSDF can become more results‐oriented. Important to implement Recommendation 7: As a bridging mechanism, the GoJ DFPTF and GoJ Within 1 program should set aside a specific sum for up to three years to be year managed by DFPTF to accommodate carrying out a small number of ICRs per year pending the transition of all JSDF grants to ISR and consequently ICR. Recommendation 9: DFPTF and the GoJ need to amend their DFPTF and GoJ Within 1 program agreements to provide that the current seed fund grant be year transformed into a more flexible tool that would provide for a degree of advance spending so as to strengthen the capacity of implementing agencies to meet the mandated fiduciary requirements of the World Bank. Universalia 54 Volume I – Final Report 8 Summation  What can be said in summation about the JSDF and its overall worth? The breadth and scope of the JSDF reaches well beyond traditional social development considerations, emphasizing the degree to which the challenges of poverty reduction and increasing levels of shared prosperity cut across traditional ways of thinking. This body of programming however faces some challenges not only in relation to ensuring its day‐to‐day efficiency. Like any other initiative, the JSDF needs to evolve to meet ever changing demands. The JSDF mechanism also contributes to the foreign policy objectives of the Government of Japan thereby creating a synergy between these objectives and the goals of the World Bank Group. Assessments such as this can tend to de‐personalize the consequences and benefits of catalytic activities such as those supported by the JSDF. Even the sort of case study vignettes laid out earlier in this Assessment do not do justice to the degree of change, the degree of hope and confidence and the degree of empowerment that JSDF grants have engendered. Early in March 2011, large segments of the island of Honshu in Japan were devastated by the Tohoku earthquake and tsunami. The short‐ term loss of life and devastation is only now being fully understood, with the longer term implications as yet not fully comprehended. Japan had been dealt a body blow. Support from around the globe began to pour into Japan, some from most unlikely sources. In 2010, a JSDF grant was approved to provide emergency assistance to unemployed residents of the city of Ulan Bator, Mongolia’s capital. This grant was designed to support urban infrastructure restoration and was targeted at reaching out to formerly rural peasants who had left their lands in search of economic opportunities in the burgeoning capital of Mongolia. This grant was one that was reviewed for this assignment. During the TTL interview, something not included in the World Bank standardized reports came to light. The TTL recounted that when news of the Tohoku earthquake and tsunami began to circulate in Ulan Bator, the community leaders responsible for the design and implementation of the JSDF grant reached out to the beneficiaries themselves, asking these already desperately poor people to come together to make a collective contribution to the victims of the earthquake and tsunami. These people, living at or below the poverty line, had come to learn that the civic works project that was employing their family members and neighbours and thus not only providing economic benefits but also building skills, confidence and self‐respect, was the result of a contribution from the people of Japan. Several thousand dollars was rapidly raised and presented to the Japanese Embassy. Moreover, at the same time, all across the world, other JSDF beneficiaries came together through their community groups or as individuals to express their solidarity with the people of Japan. They wished to be one. They wished to express their gratitude. The World Bank assembled all these messages into a Book of Hope and Solidarity which was presented to the Government of Japan. These expressions of global solidarity may be the best way to conclude this Assessment of the JSDF. It highlights what is truly important, the solidarity among diverse peoples, the extent to which hope can be engendered and the centrality of mutual support in the face of natural and human challenges. 55 Universalia Volume I – Final Report Appendix I List of Findings Finding 1:  Overall, JSDF grants are aimed at contributing to poverty reduction on an individual basis. They do so by tackling development issues that prevent beneficiaries from overcoming poverty.  Finding 2:  Similarly, the nature of JSDF grants is aimed at contributing to livelihood support on an individual case basis. They do so by tackling development issues that prevent beneficiaries from improving their livelihood.  Finding 3:  Nearly all of the individual grants examined attained degrees of first level outcomes though it was sometimes difficult to definitively ascertain the degree of outcome achieved.  Finding 4:  There are only minor variances in programmatic effectiveness between grants directed toward poverty reduction and targeting livelihood support, and there is no variance based on geography or the use of seed fund grants.  Finding 5:  Nearly all 53 grants generated their expected outputs within timelines and budgets, although about a quarter of all the grants reviewed required extensions.  Finding 6:  This Assessment showed that JSDF grants addressed development issues that prevent beneficiaries from overcoming poverty or improving their livelihood and thus were broadly in line with World Bank goals. However, during the period of time in question, the JSDF, like many other World Bank programs, did not have a formal results framework.  Finding 7:  There is evidence to show that beneficiaries played a role in the design of JSDF‐ supported initiatives.  Finding 8:  Evidence indicates that the JSDF grants filled gaps or met previously unaddressed needs.  Finding 9:  Many examples have been found of grants that introduced new ideas and new ways of working to address poverty reduction and/or livelihood support within the context of the particular country or situation, thus confirming the approach to defining the nature of innovation that the JSDF used during the period under review.  Finding 10:  JSDF grants have provided a mechanism for the World Bank to promote the piloting or the introduction of new techniques into a given country. They have thus contributed to the strengthening and broadening of the World Bank’s capacity to respond to country level challenges in an innovative fashion and have helped inform sector policy in the recipient country.  Finding 11:  Some 66% of individual JSDF grants acting as pilots have resulted in expansion of the pilot project and thus demonstrate leverage. Overall, there was some variation between the leverage of Project grants versus Capacity Building grants; as well, the range of ways in which grants demonstrated leverage was notable.  57 Universalia Volume I – Final Report Finding 12:  The Assessment of the JSDF suggests that, overall, implementing partners are of good quality in terms of being able to deliver planned grant activities. Moreover, there are no differences in the effectiveness of individual grants between the two categories of implementing partners.  Finding 13:  In general, the capacities of the implementing partners are analysed through an assessment process that is flexible and which contributes to effective implementation of the grants.  Finding 14:  Field missions uncovered a pattern of ongoing participation.  Finding 15:  Individual JSDF grants are aligned at the country level with Country Partnership Strategies.  Finding 16:  Individual JSDF grants are in line with the Priority Policy for International Cooperation of the Government of Japan.  Finding 17:  The files reviewed for this Assessment show a high degree of compliance with the application guidelines then in place.  Finding 18:  The former Steering Committee played a value‐adding role, including that of ensuring Quality at Entry.  Finding 19:  Based on the data that is available for the grant sample examined by this Assessment and during the time frame in question (2007‐2012), the average time taken to review an application by the GoJ approached twice the anticipated standard.  Finding 20:  During the 2007–2012 timeframe, the stages for the participation of representatives of the Government of Japan in grant approval was unclear and potentially may have resulted in duplication of efforts as well as slower turnaround times. A further contributing factor relates to the lack of a common understanding of innovation among the GoJ stakeholders.  Finding 21:  There were significant delays in moving from approval of the proposal by Japan to the signing of the grant agreement.  Finding 22:  The introduction of a new end‐of‐grant reporting instrument, the Implementation Completion Report (ICR), fills a current gap in the design of World Bank small grants.  Finding 23:  While visibility at both the level of individual grants and at the programmatic level is broadly in line with the existing guidelines, the implementation of the existing visibility guidelines is impacted by the dual lines of accountability that characterize the JSDF program model.  Finding 24:  At the moment, the JSDF is located in the Development Finance (DFi) Vice Presidency of the World Bank; however, there is no consensus regarding where it “fits” within the new structure of the World Bank Group.  Finding 25:  Overall, the contention that JSDF grants require more supervision costs than provided for cannot be independently substantiated.  Universalia 58 Volume I – Final Report Finding 26:  In comparison to comparable funds, the JSDF does not currently have similar levels of resources.  Finding 27:  There is a degree of concern among some World Bank stakeholders about the relationship between the JSDF and the newly emerging priorities of the World Bank.  59 Universalia Volume I – Final Report Appendix II List of Recommendations Recommendation 1:  The World Bank Group needs to reaffirm the value of the JSDF partnership with the Government of Japan to maintain the JSDF at, as a minimum, the present funding level over the next three years.  Recommendation 2:  DFPTF managers should rapidly finalize a strategic results framework for the JSDF so as to articulate how, as an innovative global program, it contributes to and is part of the hierarchy of evolving World Bank priorities and is aligned with the strategic goals of reducing extreme poverty and increasing shared prosperity.  Recommendation 3:  The JSDF mechanism should be located within an organization in the World Bank which possesses a transverse mandate, for example Development Finance or the Global Practices Strategies and Operations (GPSOS) unit.  Recommendation 4:  DFPTF and the GoJ should agree on a more explicit common understanding regarding the nature of innovation.  Recommendation 5:  DFPTF and the GoJ should revisit the current Guidelines which address issues related to the visibility of Japan to more explicitly articulate what are minimum visibility criteria, what should be expected in different circumstances and what should be submitted to DFPTF to demonstrate visibility.  Recommendation 6:  In order to improve the results orientation of the JSDF grant monitoring, DFPTF should provide additional guidance related to the presentation of periodic report writing and should revisit the application format and content of the application package.  Recommendation 7:  As a bridging mechanism, the GoJ should set aside a specific sum for up to three years to be managed by DFPTF to accommodate carrying out a small number of ICRs per year pending the transition of all JSDF grants to ISR and consequently ICR.  Recommendation 8:  DFPTF and the GoJ need to revisit the current approach to formal approval of grants by the GoJ in an effort to reduce the current delays and to operate within existing agreements which are based on the principle of assumed consent. As well, the establishment of an Advisory Board to conduct a vetting of concept notes could be considered.  Recommendation 9:  DFPTF and the GoJ need to amend their agreements to provide that the current seed fund grant be transformed into a more flexible tool that would provide for a degree of advance spending so as to strengthen the capacity of implementing agencies to meet the mandated fiduciary requirements of the World Bank.  Recommendation 10:  DFPTF and the GoJ need to rapidly enter into negotiations to provide additional human and financial resources so that the JSDF can become more results‐oriented.  Universalia 60 Volume I – Final Report Appendix III Key Stakeholders Engaged Cambodia  Name Title Section/Organization CHAMROEUM, Mudita TTL World Bank Office , Cambodia SOW, Alassane Country Manager World Bank Office , Cambodia KHIM, Sarin Implementation Officer World Bank Office , Cambodia THON, Heng Chan Director Wathnakeap, (Implementing Partner) Ministry of Land and Social Economic BORAMY, Sarath Project Director Development Ministry of Land and Social Economic PE A, Cheam Project Coordinator Development VONG, Sarinda Executive Director Cooperative Association of Cambodia National Committee of Sub‐National BUNLONG, Chhun Systems Analyst Democratic Development TAMAMITSU, Shinichi 1st Secretary Embassy of Japan, Cambodia BORING, David Country Director Habitat for Humanity , Cambodia Ministry of Land and Social Economic THAVIRAK, Sim Deputy Director Development, Kampong Cham District Ministry of Land and Social Economic VONG, Konkea Advisor Development, Kampong Cham District Ministry of Land and Social Economic RITHY, Hang Song Operations Advisor Development, Kampong Cham District 61 Universalia Volume I – Final Report Colombia  Method of Name Title Section/Organization consultation AGUDELO OSORIO, Daisy Project Officer Fundación Fútbol con Corazón In person Agencia Presidencial para la ARISTIZABAL, Juan David Project Officer In person Cooperación – APC ARRIETA, Cristina Project Manager Fundación Restrepo Barco In person Founder and AZOUT, Samuel Fundación Restrepo Barco In person Chairman BERNAL BLANCO, Paola Project Officer World Bank In person ESCANDON, Leonardo Operations Analyst World Bank In person GALINDO PARDO, Ana Maria Project Officer Fundación Restrepo Barco In person Agencia Presidencial para la GIRALDO, Ángela Project Officer In person Cooperación – APC Prime Secretary and INAGAKI, Osamu Chief of International Embassy of Japan In person Cooperation Agencia Presidencial para la JIMENEZ, Catalina Project Officer In person Cooperación – APC Senior Education LAVERDE, Martha World Bank In person Specialist MARIN, Danny Cooperation officer Embassy of Japan In person Coordinator of Agencia Presidencial para la ORTIZ, Gloria In person multilateral relations Cooperación – APC Acting Country SELLEN, Daniel World Bank In person Manager Parents of young beneficiaries living in La Paz and El Pueblo Communities In person Young beneficiaries living in La Paz and El Pueblo Communities In person Young beneficiaries living in Flor de Monte In person Young beneficiaries living in San Onofre In person Universalia 62 Volume I – Final Report India  Method of Name Title Section/Organization consultation International Finance BARMAN, Subrata Senior Operations Officer Corporation (World Bank In person Group) Project Manager Self Employed Women’s BHATNAGAR, Smita In person Coordinator Association (SEWA) Senior Rural Development GHATATE, Vinayak World Bank Group In person Specialist LAXMAN, Karthik Consultant World Bank Group In person Consultant, Monitoring & MEHTA, Varsha World Bank Group In person Evaluation Self Employed Women’s NANAVATY, Reema Executive Director In person Association (SEWA) RUHL, Onno India Country Director World Bank Group In person SHOBHA, Shetty Practice Manager, Agriculture World Bank Group In person Members of Gyan Vigyan Kendra teams from various regions across Gujarat, at SEWA Gram In person Mahila Haat in Ahmedabad Executive committee members, Visavdi Gyan Vigyan Kendra, Surendranagar District In person SEWA Master Trainers, Visavdi Gyan Vigyan Kendra, Surendranagar District In person Beneficiaries (Salt farmers, Surendranagar District) In person 63 Universalia Volume I – Final Report Japan and the Philippines  Name Title Section/Organization HIRAI, Tomoko Sr. Communications Officer World Bank, Japan Office TSUKAGOSHI, Yasuske Special Representative World Bank, Japan Office SUGIURA, Tatsuya Dep. Director Development Institutions Bureau, MoF TSUSHIMA, Horonari Section Chief Development Institutions Bureau, MoF INAMAI, Osamu Analyst Development Institutions Bureau, MoF Office of Global Issues and Development NASU, Takenori Dep. Director Partnerships JICA Office of Global Issues and Development TAKAGI, Akihiro Analyst Partnerships JICA Office of the President SAKANE, Koji Assistant JICA Global Issues and Cooperation Division IEDA, M. Director MoFA Global Issues and Cooperation Division TOKUDA, K. Analyst MoFA SENGA, Kunio CEO Save the Children, Japan MIYASHITAQ, Hiroshi Director International Program, Save the Children, Japan YOSHIDA, Katsuya Manager International Program, Save the Children, Japan Office of Co‐Financing SATO, Kazushi Senior Specialist Asian Development Bank Office of Co‐Financing KASAHARA, Hiroki Senior Specialist Asian Development Bank Office of Co‐Financing OBLEPIAS, Ronaldo Consultant Asian Development Bank Azaracon Senior Operations Officer World Bank, Philippines Office Universalia 64 Volume I – Final Report Kenya (SCBFFE)  Method of Name Title Section/Organization consultation Forest Extension Services/ Kenya KARIUKI, Patrick Deputy Director In person Forest Service KIMONYE, Benard Farmer/Beneficiary SCBFFE In person MUTHIU KARIUKI, Peter Forestry Officer SCBFFE /Kenya Forest Service In person NDETI, Janei Project Manager SCBFFE /Kenya Forest Service In person NYAGA, Micael Farmer Facilitatior SCBFFE /Kenya Forest Service In person NDUNGE NDUI, Dometila Farmer/Beneficiary SCBFFE In person NJIRU KAMUENDE, Jacob Farmer/Beneficiary SCBFFE In person Nema Farm Forestry Group Gachoka Division Mbweere Beneficiaries/FFG SCBFFE FGD South Evurore Division Mbweere North Sub County Farm Beneficiaries/FFG SCBFFE FGD Forestry Group Milevo Farm Forestry Group in the Mbweere South Sub Beneficiaries/FFG SCBFFE FGD County Muungano Farm Forestry Beneficiaries/FFG SCBFFE FGD Group (FFG) Regional Project Japan International Cooperation SHIMIZU, Eiichi In person Formulation Advisor Agency/Kenya Office SYOMBUA MUIA, Beth Farmer/Beneficiary SCBFFE In person 65 Universalia Volume I – Final Report Kenya (Rural Traditional Herbalists Project)  Method of Name Title Section/Organization consultation Health Sector Japan International Cooperation AGATA, Naphtali In person Consultant Agency External Evaluation MAHUNNAH, Rogasian Muhimbili University Workshop Consultant Professor of University of Nairobi/ Kenya MUKIAMA, Titus Botany/ Project Working Group for Medicines In person Director and Aromatic Plants Board Member/ Independent Pharmacy and Poisons Board, ORWA, Jennifer In person Evaluation Ministry of Medical Services Consultant RAMANA, Gandham Task Team Leader The World Bank In person Regional Project Japan International Cooperation SHIMIZU, Eiichi Formulation In person Agency Adviser Regional representatives, National Traditional Health Practitioners Association Stakeholders’ meeting Universalia 66 Volume I – Final Report Moldova  Method of Name Title Section/Organization consultation Director of Policy and BIVOL, Stela PAS Center In person Research Director of Medic National Center for Blood CEBOTARI, Svetlana In person transfuziolog Transfusion Ludmila Chitic, faculty, COBYLYANSKY, Svetlana Deputy director In person Medical College COCUL, Svetlana Coordinator Japanese Embassy In person CORMAN, E. Procurement Specialist World Bank In person COSTIUC, Nina Mayor Budesti village In person Financial Management DRUTA, Oxana World Bank In person Specialist GUBAN, Irina Health Consultant World Bank In person MARCU, Larisa VCT Counselor Local hospital In person SECK, Abdoulaye Country Manager World Bank In person SEVCIUC, Viorica VCT counselor in Balti Local hospital In person SIRBU, Sergiu Vice Mayor Stauceni village In person National Center of TODERASCU, Lilia M&E Specialist In person Management in Health 67 Universalia Volume I – Final Report Nepal  Name Title Section/Organization Senior Operations Officer and PRADHAN, Bigyan World Bank Office, Nepal Acting Manager ARJUL, Mohan Senior Officer, Education World Bank Office, Nepal DIR, Amit Officer, Education World Bank Office, Nepal BHANDARI, Uddhav Consultant , Education World Bank Office Nepal NISHITANI, Chisako Second Secretary Embassy of Japan, Nepal Economic Development Section RAJBHANDARI, Madhusudan Manager Embassy of Japan, Nepal KAWAMATA, Yamato Representative JICA, Nepal SHERPA, Ang, Pasang Program Officer JICA, Nepal AWASTHI, Lava Deo Director General Ministry of Education Early Childhood Development, DAWADI, Divya Deputy Director Ministry of Education Gender Equity Development KANDAL, Durya Deputy Director Ministry of Education Finance Section, CHITRA, K C Finance Oficer Ministry of Education Research Inputs and Development Action LEKHAK, Hem Consultant Ltd. LEKHAK, Raj Chairperson CPROC BADHAL, Navim Member CPROC PANDAY, Narendra Raj Member CPROC KAPILVESTU, Chandraruta Principle Rastriya District High School Universalia 68 Volume I – Final Report Nicaragua  Method of Name Title Section/Organization consultation ANDIRA WATSON, Carmen Consultant World Bank in Nicaragua In person BARRIOS, Raul Country director World Bank in Nicaragua In person Senior Social GONZALES, Mary Lisbeth Development World Bank In person Specialist KAGAMI, Mitsuhiro Ambassador Japan Embassy in Nicaragua In person SILVERIO La Campesina In person Institutional Technical URENA, Arturo ACICAFOC In person Manager 69 Universalia Volume I – Final Report Sierra Leone (project 1)  Method of Name Title Section/Organization consultation BROWN, Ato Country Manager World Bank Country Office In person Baminfoma Cassava Farmers CONTEH, Daniel Secretary In person Cooperatives Project Facilitator FODAY, Sahr Life after Diamond Project In person (Roving) GANDA, Peter Operations Officer World Bank Country Office In person KAMANDA, Aiah Paramount Chief Farandu Community In person KANU, John Project Manager Life after Diamond Project In person Project Facilitator NGANJAH, Aioh Poul Life after Diamond Project In person (Koidu District) Advisor to the Village Development SAMUKA, Chief Section Chief In person Committee (VDC) Social Capital Project, SESAY, Sullay B. Project Coordinator Decentralization Secretariat, In person Government of Sierra Leone Project Facilitator TENGBEH, Sia Christiana Life after Diamond Project In person (Mimikoro District) Vice‐Principal, Njala Secondary School, Staff and Student In person Kamara Primary School In person Locals from eight communities at the Farandu Bridge Construction site In person Universalia 70 Volume I – Final Report Sierra Leone (project 2)  Method of Name Title Section/Organization consultation BROWN, Ato Country Manager World Bank Country Office In person DURA III, Alimamy Paramount Chief Safroko Limba Chiefdom In person GANDA, Peter Operations Officer World Bank Country Office In person Project Formulation Japan International Cooperation HASEGAWA, Toshihisa Adviser/ In person Agency Representative National Commission for Social KAMARA, Isatu Deputy Commissioner In person Action KOROMA III of KALASONGOIA, Alimany Paramount Chief Bumbuna Chiefdom In person Backarie Yellah Project Formulation Japan International Cooperation SANO, Akihira In person Adviser Agency Regional Coordinator, National Commission for Social TURAY, Osman In person North Action Agriculture Business Units, Safrako Limba Chiefdom In person Beneficiairies, Mabamba Communication Centre, Safrako Limba Chiefdom In person Council of Elders, Kalansongoia Chiefdom In person Farmer Field Schools, Safrako Limba Chiefdom In person Micro‐enterprise Groups, Safrako Limba Chiefdom In person Representatives, Bombali District Council (Deputy Chairman; Administrative Head; Head of In person District‐Ministry of Agriculture) Technical staff, National Commission for Social Action In person Youth beneficiairies, Kalansongoia Communication Centre, Bumbuna Chiefdom In person 71 Universalia Volume I – Final Report Uzbekistan  Method of Name Title Section/Organization consultation KAMATA, Takuya Country Manager World Bank In person KHIDIROV, Dilshod Sr. Rural Development Specialist World Bank In person Deputy Minister of Agriculture KHIDIROV, Ya. Government In person and Water Resources NAKAMURA, Shinichior First Secretary Japanese Embassy In person RUSTAMOV, Hurshid Coordination Officer UNDP In person Project Implementation Association of Silk YULDASHEC, Satimbay In person Manager Producers Pilot center leader and project beneficiaries at Turtkul District in Karakalpakstan In person Pilot center leader and project beneficiaries at Ellikkala District in Karakalpakstan In person Pilot center leader and project beneficiaries at Beruni District in Karakalpakstan In person Universalia 72 Volume I – Final Report World Bank HQ and IDB stakeholders engaged  Name Title Section/Organization Concessional Finance and Global SO, Jaehyang Director, Trust Funds Partnerships, World Bank Concessional Finance and Global BLEOMENKAMP, Sandra Manager, Trust Funds Partnerships, World Bank Concessional Finance and Global JOSEPH, Yolaine Senior Operations Officer Partnerships, World Bank Concessional Finance and Global PUTRA, Eka Operations Officer Partnerships, World Bank Concessional Finance and Global DIAW, Mohamed Operations Analyst Partnerships, World Bank Concessional Finance and Global McClain, Loreine Operations Analyst Partnerships, World Bank Concessional Finance and Global KUIJPER, David Advisor Partnerships, World Bank Concessional Finance and Global NAKAGURA, Makoto Advisor Partnerships, World Bank Office of the Executive Director, Japan, KURASAWA, Shinichiro Advisor World Bank PSWARAYI‐RIDDIHOUGH, Operational and Quality Services, South Director Idah Asia, World Bank Fragile States, Conflict and Social BANNON, Ian Sector Manager Development, World Bank Strategy and Operations, Africa, World WALLISER, Jan Director Bank Urban and Social Development, MENA, BOUSQUET, Franck Sector Manager World Bank Regional Trust Fund KELLER, Barbry LAC Region, World Bank Coordinator Capacity Development and SALLAH, Tijan Manager Partnerships, Africa, World Bank Independent Evaluation Group, World TOMONAGA, Jiro Evaluator Bank MOTO, Kasukake VP (retired) Resource Mobilization, World Bank Grants and Co‐Financing, Inter‐ MUTSUURA, Gora Senior Operations Specialist American Development Bank Grants and Co‐Financing, Inter‐ ABE, Masaru Operations Specialist American Development Bank 73 Universalia Volume I – Final Report Appendix IV TTL Survey – An Overview The electronic survey of TTLs comprised a major tool to secure more qualitative information while also lending itself to quantification. As described above, approximately 190 TTLs were identified by DFPTF management. An invitation was sent to each by DFPTF management and was followed up with reminders on two occasions. The survey was closed on in late April 2014, allowing TTLs as much time as possible to respond. The survey, which can be found in Volume II (Launch Briefing Overview) of this Final Report, is comprised of a series of Likert scale‐based questions wherein respondents are asked to identify their degree of agreement or disagreement with approximately 40 specific questions. In addition, write‐in capacity was provided. Some 98 TTLs responded. As described earlier, a decision was taken to re‐open the survey for the month of March and into early April so as to increase the response rate and thus facilitate greater levels of disaggregation within the Final Report. Respondents were asked to identify themselves on the basis of several basic characteristics; whether their primary current region of work; and, whether they had had any previous experience with administering or applying for a JSDF grant. Table 1 illustrates the geographic distribution. In several instances, respondents chose to identify several regions, thus resulting in a higher count than 98 responses. Table 1. TTL Distribution by Region Region Number Africa 34 East Asia Pacific 23 Europe & Central Asia 11 Latin America & Caribbean 22 Middle East & North Africa 5 South Asia 12 Of the 98 responses, virtually all indicated that they had had experience with two or more JSDF‐ supported grants. While this may seem to be of great significance, care should be taken in attempting to extrapolate this number across the JSDF as a whole. It is equally possible that these 98 are more motivated as individuals due to their prior multiple exposures to the JSDF process and as such were more willing than others to participate in the survey. Universalia 74 Volume I – Final Report Administering Two or More JSDF Grants 40 36 34 30 28 20 10 0 2 3 4+ Moreover, anecdotal evidence provided by DFPTF management showed a number of instances where TTLs had contacted DFPTF management, indicating their unease at participating in a survey about a subject which was now somewhat in the distant past for them. Turning to the responses themselves, with a base number of only 98 responses and given the use of Lichter scales, disaggregation can become very problematic. Small scale sampling error combined with relatively low levels of statistical reliability given the respondent pool, result in methodological limitations and the inability to responsibly disaggregate down to the individual responses on the basis of geographic region. However, an initial analysis of the raw data for these 98 responses showed very little if any statistically reliable variances. For example, in relation to a number of questions, comparisons between the responses based on geographic region shows high degrees of similarity within the range of statistical probability given the size of the initial pool of respondents. In short, it appears that geographic considerations were not relevant with respect to the survey responses. The same can be said for the other differentiating characteristic, whether a respondent had been involved with two or more grants. Again, the degree of commonality of response was striking. Finally, in terms of general remarks, the specific findings that follow show very high levels of support for the JSDF as a whole for its Relevance, for its Sustainability and for the quality of Programme Management. In general therefore, the survey portrays a highly positive view of the JSDF’s worth from the standpoint of TTLs. 75 Universalia Volume I – Final Report Appendix V Grants Subject to Assessment NOTE: Grants Identified in Bold Italics are those categorized as related to Livelihood Support Grant Portfolio Sample – 39 grants  Notes: Sector Codes List: (1) Agriculture, Fishing & Forestry, (2) Public Administration, Law & Justice (3) Information & Communications, (4) Education, (5) Finance, (6) Health and Social Services, (7) Industry & Trade, (8) Energy & Mining, (9) Transportation, (10) Water, Sanitation and Flood Protection Africa  Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Community‐Based Conditional Cash TF090491 Project Tanzania Regular N 2 N GOV Transfer Pilot TF091830 Project Senegal Regular Y 6 N Fighting Child Begging in Senegal NGO Community‐Based Early Childhood Care TF093722 Project Gambia Regular Y 4 N GOV and Development Delivering Maternal Child Health and TF094080 Project Swaziland Regular Y 6 N Other Social Services Care to Vulnerable GOV Populations Emergency Food Security and TF097191 Project Comoros Emergency N 2 N Unemployment Support Through Cash GOV for Work Sierra Developing a Model for Delivering TF057251 Capacity Regular N 2 N GOV Leone Primary Justice Services TF055618 Project Somalia Special N 6 N Puntland Primary Health Services NGO Energy, Gender And Poverty TF054213 Project Ethiopia Regular N 8 N Alleviation: The Ethiopian Women GOV Fuelwood Carriers Universalia 76 Volume I – Final Report E. Asia & Pacific  Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Community Led Infrastructure TF099673 Project Mongolia Emergency N 2 N Development for the Urban Poor of GOV Ulaanbaatar Empowering Women Overseas Migrant TF091169 Project Indonesia Regular N 4 N NGO Workers TF095058 Project Indonesia Regular N 4 N Sustaining Women's Leadership NGO Community‐Managed Agrarian TF051601 Project Philippines Regular N 2 N GOV Reform And Poverty Reduction Project Vietnam: Early Childhood Care And TF052939 Project Vietnam Regular N 4 N NGO Development East Asia and Pacific TF051602 Capacity Regular N 3 N East Asia: Asean School Net NGO Region as a whole Indonesia: Support For Female Headed TF053442 Capacity Indonesia Special N 6 N Households In Conflict Areas (Ngo‐ NGO Executed) TF090486 Project Thailand Regular N 1 N Community Youth Helmet Use Program NGO Europe and Central Asia   Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Russian Strengthening Access to Justice for the TF090655 Capacity Regular Y 2 N NGO Federation Poor Rayon Community and Plumbing TF090657 Project Azerbaijan Regular N 10 Y GOV Rehabilitation Program TF090489 Project Azerbaijan Regular N 6 N IDP Youth Support Project (IDP‐YSP) GOV 77 Universalia Volume I – Final Report Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Pilot Community Driven Development TF051709 Project Moldova Regular N 6 N GOV Project Latin America and Caribbean   Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Introducing innovative arts‐based TF057249 Capacity Colombia Regular N 4 N Education to displaced and violence‐ NGO affected communities Microenterprise Development for the TF091178 Capacity Honduras Regular N 1 N GOV Poor and Indigenous People Empowering Young Women Affected by Violence in Colombia: An Innovative TF093829 Capacity Colombia Regular N 4 N NGO Approach towards Sustainable Socio‐ Economic Inclusion Jamaica Community Crime and Violence TF094380 Project Jamaica Regular N 2 N GOV Prevention Program El Addressing Youth Violence through TF094811 Project Regular N 2 N NGO Salvador Cultural and Music Learning Small Farmer Vulnerability Reduction TF097967 Project Grenada Emergency N 2 N GOV Initiative Honduras: Community Disaster TF052879 Project Honduras Regular N 2 N Management In The Barrios Of GOV Tegucigalpa TF096923 Project Jamaica Emergency N 6 N Conditional Cash Transfer Program GOV Food Emergency Support Program for TF097212 Project Nicaragua Emergency N 4 N GOV School Children Universalia 78 Volume I – Final Report Middle East & North Africa  Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Crisis Response: Employment and TF097217 Project Djibouti Emergency N 2 Y GOV Human Capital Social Safety Net Community Health and Other Social TF099374 Capacity Tunisia Regular N 6 N GOV Services Collaborative Project Yemen, Yemen: Community Water TF054230 Project Regular N 10 N NGO Republic of Management Project South Asia  Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Pro‐poor Targeted Secondary School TF093397 Project Nepal Regular Y 4 N GOV Stipend Local Level Nutrition Interventions TF097171 Project Sri Lanka Emergency N 6 N GOV for the Northern Province Afghanistan ‐ Support to Basic Package of Health and Other Social TF095919 Project Afghanistan Special N 6 N Services (Strengthening Health and GOV Other Social Services Activity for Rural Poor Addressing the Poverty‐Built TF090416 Project Pakistan Regular N 2 N NGO Environment Nexus TF052849 Capacity India Regular N 4 N Sewa Capacity Building NGO Making Globalization Work For TF058021 Project India Regular N 1 N NGO The Rural Poor In India Empowering Poorest of Poor Women and Young Girls‐Focusing TF051707 Project Sri Lanka Special N 2 N NGO on the Most Affected Areas of Hambantota (South) & North & East 79 Universalia Volume I – Final Report Grant Window Seed TF No. Country Sector Restructured Grant Name Implementer Type Type Grant Support for the roll‐out of the TF092433 Project Afghanistan Special N 2 N National Solidarity Program 2 GOV (NSP2) Grants for Field Mission Review – 14 (year approved added)  Notes: 1) Sector Codes List: (1) Agriculture, Fishing & Forestry, (2) Public Administration, Law & Justice (3) Information & Communications, (4) Education, (5) Finance, (6) Health and Social Services, (7) Industry & Trade, (8) Energy & Mining, (9) Transportation, (10) Water, Sanitation and Flood Protection Africa  Restru TF No. Year Grant Type Country Window Seed Sector Grant Name Implementer ctured Capacity Building of Rural TF090490 2007 Capacity Kenya Reg. No 6 N NGO Traditional Herbalists Support to Community Based Farm TF093520 2009 Project Kenya Reg. Yes 1 Yes Forestry Enterprises in Semi‐Arid Gov. Areas in Kenya Sierra Leone Rapid Response Growth Poles: TF097914 2010 Project (fragile Emerg. No 2 N Community‐based Livelihood and Gov. state) Food Support Program Sierra Leone Artisanal Mining Community TF099111 2010 Project (fragile Reg. No 1 N Development and Sustainable Joint state) Livelihoods Universalia 80 Volume I – Final Report East Asia‐Pacific  Grant Perf. TF No. Year Country Window Seed Sector Restructured Grant Name Implementer Type Rating Strengthening Good TF091833 2008 Capacity Cambodia Reg. Yes 1 Yes S Governance in Land NGO Distribution Strengthening Civil Society‐Government TF091836 2008 Capacity Cambodia Reg. Yes 1 Yes MS NGO Partnership to Deliver Land Tenure Security Europe and Central Asia   Grant Perf. TF No. Year Country Window Seed Sector Restructured Grant Name Implementer Type Rating Karakalpakstan Silk TF057070 2006 Project Uzbekistan Reg. No 1 Yes MU Development Pilot Gov. Project Preventing Hepatitis B&C TF093702 2009 Project Moldova Reg. No 6 No HS Gov. in Moldova South Asia  Grant Perf. TF No. Year Country Window Seed Sector Restructured Grant Name Implementer Type Rating Economic Empowerment TF098626 2011 Capacity India Reg. Yes 4 No S NGO For Women Sewa Capacity Building TF052849 2005 Capacity India Reg. No 4 No S NGO Project Pro‐poor Targeted TF093397 2009 Project Nepal Reg. Y 4 No S Gov Secondary School Stipend 81 Universalia Volume I – Final Report Latin America and the Caribbean   Grant Perf. TF No. Year Country Window Seed Sector Restructured Grant Name Implementer Type Rating Alternative Indigenous and Afro‐descendants Sustainable Agro‐Forestry TF093115 2009 Capacity Nicaragua Reg. No 1 Yes MU NGO Management for the Autonomous North Atlantic Region of Nicaragua Access to Opportunities for TF093141 2009 Capacity Colombia Reg. Yes 2 No S NGO Young People in Colombia Soccer Together: Rethinking How to Improve Gender TF099171 2011 Project Colombia Reg. No 4 No S NGO Equity and Inclusion in the Education System Universalia 82 Volume I – Final Report Appendix VI Other Funds Supported By Japan Japan Fund for Poverty Reduction (JFPR), Asian Development Bank  The JFPR was established in May 2000 as a partnership between the Government of Japan and the Asian Development Bank to provide grants for projects supporting poverty reduction and related social development activities that can add value to projects financed by ADB. In 2010, the JFPR expanded its scope of grant assistance to provide technical assistance grants in addition to project grants. Currently JFPR grants are intended to focus on activities which: Respond directly to the needs of the poorest and most vulnerable groups through new and innovative methods; Support initiatives that lead to rapid, demonstrable benefits with positive prospects of developing into sustainable activities; or Build ownership, capacity, empowerment and participation of local communities, non‐ governmental organizations (NGOs) and other civil society groups to facilitate their involvement in projects financed by ADB. As of 30 June 2013, the total JFPR funds made available totaled about US$615.4 million. ADB had approved 158 grant projects (US$422.0 million equivalent) and 124 technical assistance projects (US$128.1 million equivalent). Japan Special Fund for Poverty Reduction – Inter‐American  Development Bank (IDB)   The Japan Special Fund for Poverty Reduction Program (JPO), established in 2001 with US$30 million and replenished with US$22.5 million in 2011, provides non‐reimbursable financial resources for Technical Cooperation operations and supports poverty alleviation activities impacting low‐income and vulnerable populations. The IDB‐JPO has a proven track record in implementing community‐based development projects in a number of sectors, in numerous countries, and with many local community development organizations. Since 2001, the IDB‐JPO has funded over 145 projects for approximately US$50 million. Its funding levels are much smaller than those of the JSDF or the JPRF, with maximum limits of less than US$500,000 per initiative. JPO finances Community‐Based Projects (CBPs) that have a direct effect on poor and vulnerable groups. The CBPs supports: Well‐targeted poverty reduction and social development activities that respond directly to the needs of socially and/or economically disadvantaged people; and Capacity building of organizations working with low‐income communities to provide sustainable solutions for poverty. 83 Universalia Volume I – Final Report Human Security Trust Fund of the United Nations (UNTFHS)   The United Nations Trust Fund for Human Security (UNTFHS) was established in March 1999 by the Government of Japan as a mechanism to promote the expansion of the use of the principles that underpin Human Security, a long‐time foreign policy objective of the Government of Japan. From 1999 until December 2013, the Government of Japan invested some US$475 million in the Trust Fund. The work of the UNTFHS in large part mirrors that of the JSDF and its sister funds located at other development banks. The Human Security approach is based on the following five principles: The work must be people‐centered reflecting their needs and not what we want to do for them, the more traditional and interventionist approach to development; It must be multi‐sectoral. The human condition cannot and should not be artificially divided into slices. You cannot simply “fix” an economic problem and expect real change in the overall condition of a community if there remain social challenges to their well‐being, etc; It must be comprehensive. Partial approaches will leave gaps; It must be context specific – related to real needs and tailored to meet them – as opposed to the notion of replicating actions here there and everywhere with little reference to the subtleties of varying situations; It must be prevention‐oriented – not just fixing a problem for today, but building a lasting foundation to promote this overall sense of well‐being. UNTFHS‐funded projects have: Underscored the practical approach of Human Security for addressing multifaceted insecurities in a contextually relevant and targeted manner; Provided new perspectives for the development of actions, particularly where existing responses have proved insufficient; Resulted in a deeper analysis on the causes and manifestations of complex and interconnected threats and their impact on the survival, livelihood and dignity of local communities; Highlighted the lack of protection and empowerment conditions that can perpetuate insecurities; Supported the development of mitigation and resilience‐building measures that have resulted in notable increases in the Human Security of the most vulnerable; Provided examples of successful multi‐stakeholder collaborations that have fortified existing partnerships and promoted avenues for new collaborations among UN organizations, Governments and local communities; Since the inception of the Fund in March 1999, over 300 projects have been approved with a total cumulative budget of nearly US$400 million. Universalia 84 Volume I – Final Report Appendix VII Rating of Individual Grants Note regarding Ratings. A 1‐4 scale is used for each of the five assessed factors with 1 representing “High”(all rating criteria met with one or more exceeded) , 2 representing “ Substantial (all rating criteria met save for one or two), 3 representing “Modest ( more than two rating criteria not met) and 4 representing “Low” (most of the rating criteria not met). More detail can be found in Volume II, Launch Briefing Overview where the detailed rating guide can be found. Desk Review Grants  TF No. Country Seed Projects Relevance Effectiveness Sustainability Supervision Efficiency Overall Rating TF057251 Sierra N Developing a Model for 1 2 2 2 2 2 Leone Delivering Primary Justice Services TF055618 Somalia N Puntland Primary Health 2 2 3 2 2 2 Services TF054213 Ethiopia N Energy, Gender And Poverty 2 2 3 2 2 2 Alleviation: The Ethiopian Women Fuelwood Carriers TF051601 Philippines N Community‐Managed 2 2 1 2 3 2 Agrarian Reform And Poverty Reduction Project TF052939 Vietnam N Vietnam: Early Childhood 1 2 2 2 2 1 Care And Development TF051602 East Asia N East Asia: Asean School Net. 2 2 2 2 2 2 and Pacific TF053442 Indonesia N Indonesia: Support For 2 3 3 3 0 3 Female Headed Households In Conflict Areas (Ngo‐ Executed) TF090486 Thailand N Community Youth Helmet Use 2 2 2 2 2 2 Program TF090489 Azerbaijan N IDP Youth Support Project 2 2 4 2 0 2 (IDP‐YSP) TF051709 Moldova N Pilot Community Driven 2 1 1 2 1 1 Development Project 85 Universalia Volume I – Final Report TF No. Country Seed Projects Relevance Effectiveness Sustainability Supervision Efficiency Overall Rating TF052879 Honduras N Honduras: Community 2 2 2 2 3 2 Disaster Management In The Barrios Of Tegucigalpa TF096923 Jamaica N Conditional Cash Transfer 2 2 2 2 2 1 Program TF097212 Nicaragua N Food Emergency Support 1 2 2 2 2 2 Program for School Children TF054230 Yemen, N Yemen: Community Water 2 2 1 3 3 2 Republic of Management Project TF090416 Pakistan N Addressing the Poverty‐Built 2 2 3 3 0 2 Environment Nexus TF052849 India Sewa Capacity Building 2 2 3 3 3 2 TF058021 India N Making Globalization Work 1 3 2 3 3 2 For The Rural Poor In India TF051707 Sri Lanka N Empowering poorest of poor 1 3 2 2 3 2 women & young girls‐ focussing on most affected areas of Hambantota (south) & north & east TF092433 Afghanistan N Support for the roll‐out of the 1 1 1 1 2 1 National Solidarity Program 2 (NSP2) TF090491 Tanzania N Community‐Based 2 2 2 2 2 1 Conditional Cash Transfer Pilot TF091830 Senegal Y Fighting Child Begging in 1 2 2 2 2 1 Senegal TF093722 Gambia Y Community‐Based Early 2 3 2 2 3 2 Childhood Care and Development TF094080 Swaziland Y Delivering Maternal Child 2 3 3 2 3 3 Health and Other Social Services Care to Vulnerable Populations Universalia 86 Volume I – Final Report TF No. Country Seed Projects Relevance Effectiveness Sustainability Supervision Efficiency Overall Rating Tf097191 Comoros N Emergency Food Security and 1 1 0 2 1 1 Unemployment Support Through Cash for Work TF099673 Mongolia N Community Led 2 3 0 2 0 2 Infrastructure Development for the Urban Poor of Ulaanbaatar TF091169 Indonesia N Empowering Women 2 2 2 3 3 2 Overseas Migrant Workers TF095058 Indonesia N Sustaining Women's 1 3 3 1 3 3 Leadership TF090655 Russian Y Strengthening Access to 2 1 2 2 1 1 Federation Justice for the Poor TF090657 Azerbaijan N Rayon Community and 2 3 3 3 3 3 Plumbing Rehabilitation Program TF057249 Colombia N Introducing innovative arts‐ 2 2 2 2 3 2 based Education to displaced and violence‐affected communities TF091178 Honduras N Microenterprise Development 2 3 4 3 3 2 for the Poor and Indigenous People TF093829 Colombia N Empowering Young Women 2 3 2 2 3 2 Affected by Violence in Colombia: An Innovative Approach towards Sustainable Socio‐Economic Inclusion TF094380 Jamaica N Jamaica Community Crime 1 1 1 1 2 1 and Violence Prevention Program TF094811 El Salvador N Addressing Youth Violence 2 2 3 2 2 2 through Cultural and Music Learning TF097967 Grenada N Small Farmer Vulnerability 2 3 2 2 3 3 Reduction Initiative 87 Universalia Volume I – Final Report TF No. Country Seed Projects Relevance Effectiveness Sustainability Supervision Efficiency Overall Rating TF097217 Djibouti N Crisis Response: Employment 2 3 4 2 3 2 and Human Capital Social Safety Net TF099374 Tunisia N Community Health and Other 2 0 0 2 3 2 Social Services Collaborative Project TF097171 Sri Lanka N Local Level Nutrition 2 3 3 3 3 3 Interventions for the Northern Province TF095919 Afghanistan N Afghanistan ‐ Support to Basic 2 4 3 4 4 3 Package of Health and Other Social Services (Strengthening Health and Other Social Services Activity for Rural Poor Grants Reviewed by a Field Mission  TF No. Country Seed Projects Relevance Effectiveness Sustainability Supervision Efficiency Overall Rating TF090490 Kenya N Capacity Building of Rural 2 1 1 2 2 1 Traditional Herbalists TF093520 Kenya Y Support to Community Based 1 3 1 3 3 2 Farm Forestry Enterprises in Semi‐Arid Areas in Kenya TF097914 Sierra N Rapid Response Growth Poles: 2 2 2 2 2 2 Leone Community‐based Livelihood and Food Support Program TF099111 Sierra N Artisanal Mining Community 1 2 2 2 2 2 Leone Development and Sustainable Livelihoods TF091833 Cambodia Y Strengthening Good 1 1 1 1 2 1 Governance in Land Distribution TF091836 Cambodia Y Strengthening Civil Society‐ 1 1 1 1 2 1 Government Partnership to Deliver Land Tenure Security Universalia 88 Volume I – Final Report TF No. Country Seed Projects Relevance Effectiveness Sustainability Supervision Efficiency Overall Rating TF057070 Uzbekistan N Karakalpakstan Silk 1 3 2 2 3 2 Development Pilot Project TF093702 Moldova N Preventing Hepatitis B&C in 1 2 1 1 1 1 Moldova TF098626 India Y Economic Empowerment For 1 2 2 3 2 2 Women TF052849 India N Sewa Capacity Building Project 1 2 2 3 2 2 TF093397 Nepal Y Pro‐poor Targeted Secondary 1 2 1 2 2 2 School Stipend TF093115 Nicaragua N Alternative Indigenous and 1 2 2 1 3 1 Afro‐descendants Sustainable Agro‐Forestry Management for the Autonomous North Atlantic Region of Nicaragua TF093141 Colombia Y Access to Opportunities for 1 1 2 3 3 2 Young People in Colombia TF099171 Colombia N Soccer Together: Rethinking 1 1 2 3 3 2 How to Improve Gender Equity and Inclusion in the Education System 89 Universalia Volume I – Final Report Appendix VIII Statement of Work REQUEST FOR PROPOSALS (RFP) FOR INDEPENDENT ASSESSMENT OF JAPAN SOCIAL DEVELOPMENT FUND (JSDF) RFP NO. 14‐0029 STATEMENT OF WORK (SOW) A. BACKGROUND  1. Introduction. In 2000 responding to the rising poverty and plummeting livelihoods among Asia’s vulnerable communities caused by the financial crisis, the Government of Japan (GoJ) established the Japan Social Development Fund (JSDF). The Fund developed into a unique instruments for improving human security of the poorest affected by failures of social security programs and institutions in the face of crises, disasters and conflicts. The objective of the Fund of providing grants in support of innovative and pioneering approaches such as, community‐driven development (CDD) and poverty reduction programs that serve to enhance productivity, increase access to social and community services and infrastructure, and improve the living conditions of poor and vulnerable groups in eligible client countries of the World Bank Group4 (WBG), continues to make it a unique and flexible instrument for poverty alleviation and livelihood support and complement Bank initiatives (IDA/IBRD loans, credits or grants). 2. Since its inception, the GoJ has allocated US$ 770 million to the Fund financing 395 JSDF projects in 47 eligible countries. The GoJ continues to use the Fund to respond to emerging needs. To that extent it added seed fund grants in 2002 to facilitate consultations with isolated or marginalized groups and, in the same year, created the Afghanistan (special) window to support the country’s reconstruction and transition to political, economic and social stability. In 2004 following the disastrous tsunami in the Indian Ocean, the GoJ allocated funds for the post‐tsunami reconstruction and in 2005 for the Pakistan earthquake. In 2010, in response to the global food, fuel and financial crisis, the GoJ created the JSDF Emergency Window. 3. JSDF grant financed projects are administered by the WBG’s Global Partnership & Trust Fund Operations (DFPTF) of the Concessional Finance and Global Partnership Vice Presidency (CFPVP) as the Trustee5 of the Fund. The GoJ and DFPTF agreed, during the December 2012 annual consultations, to undertake an independent assessment of the achievements and operation of the Fund since the last strategic evaluation6 in 2007 4 JAPAN SOCIAL DEVELOPMENT FUND (JSDF) FY11‐13 POLICY GUIDELINES AND PROGRAM ALLOCATION, November 2010. 5 OP 14.40 – Trust Funds – A trust fund is a financing arrangement set up with contributions from one or more donors, in some cases, from the World Bank Group which establishes and administers trust funds as a complement to IDA and IBRD financing to promote development and aid effectiveness by leveraging its capacity and development knowledge. 6 World Bank – Japan Social Development Fund, Strategic Evaluation of the Japan Social development Fund (JSDF), Final Report, 1st June 2007, ITAD LTD, UK. Universalia 90 Volume I – Final Report 4. The GoJ and the WBG want the assessment to contribute to the on‐going efforts to enhance the results of the Fund and improve its operation. Contributions to poverty alleviation and sustainable livelihood achieved with the large number of diverse grants financed from the regular, special and emergency windows of the Fund have been important, but have not been systematically assessed since 2007 nor have they been explained to the general public and people interested in development. Following a short discussion of the improvements and changes that have been made to the Fund and to the Fund’s operations since the 2007 strategic evaluation, The assessment is expected first to review how and how much JSDF grants are contributing to poverty alleviation among beneficiaries. Second, if and how grants are helping to sustain livelihood of beneficiaries. Third what operational and administrative issues and problems stand in the way of the efforts to improve the Fund’s operation. The assessment of these three aspects of the Fund should be analyzed and described in a comprehensive report that brings together quantitative and qualitative evidence of poverty alleviation and livelihood support, as well as, case‐studies and interviews with beneficiaries, and recommendations for improving the Fund’s operation. The consultant is also expected to produce a short illustrated report that explains the poverty alleviation and livelihood support activities of the Fund to the general public. B. SCOPE OF WORK  1. Assess the contributions the activities funded from the regular window have made between July 2007 and December 2012 to alleviate the poverty of the beneficiaries and their communities. The focus would be on the outcomes of a sample of completed grants that funded: cash and food for work programs; home, shelter and social infrastructure rehabilitation; education and health service improvements, and related poverty reducing objectives. If feasible include grants that are close to completion in order to expand the sample size. Separately, assess the outcomes in terms of poverty alleviation of completed (and almost completed) grants under the special windows and the emergency window. If possible these outcomes should be aggregated in a way to show the typical manner in which the Fund’s activities contribute to poverty alleviation. 2. As part of this assessment, the consultant should explain how the approach of the Fund to consult beneficiaries and involve them in the selection, design and implementation of the poverty alleviation activities together with NGOs/CSOs and local governments contributes to the sustainability of the activities and enhances access to services and institutions by vulnerable groups. This should be explained in the context of the fact that JSDF grants are designed to reduce the vulnerability of the poorest, often directly only over the duration of the grant project. If possible, the consultants should give an indication as to how many groups of beneficiaries gained access to permanent poverty alleviation programs as a result of the grant activities. 3. In addition, to give a sense of the complexity of poverty alleviation through CDD activities, the consultants should describe the beneficiaries in the aggregate in terms of income or poverty level, gender, age, obstacles that prevent the beneficiaries to benefit from poverty alleviation programs and services, and other issues and circumstances that cause these beneficiaries to be vulnerable. 4. Building on the analysis of the poverty alleviation outcomes of JSDF grants, the consultants should assess the completed (and almost completed) grants that contribute to livelihood improvements. These include grants that assist entrepreneurs, informal businesses, small farmers or fisherman and similar beneficiaries who with the help of small grants either engage in new livelihood activities or expand the scope of the livelihood activities they are already 91 Universalia Volume I – Final Report involved in. As in the assessment of poverty alleviation grants, the consultants should describe the beneficiaries in the aggregate in terms of income levels, age and gender, and, if possible, provide evidence of the sustainability of the livelihood activities supported or initiated with the grants. 5. Although livelihood improvements among vulnerable groups are difficult to achieve and sustain with small grants, the consultant should single out and explain those grants that have been particularly effective in not only sustained livelihood activities, but also led to improvement in the overall well‐being of the beneficiaries (nutrition, health, education, access to social services, etc.), and had multiplier effects on the communities the beneficiaries belong to (cheaper or better food, market access, community self‐help activities, etc.). 6. Any lessons that can be derived from the experience with poverty alleviation with the grants should be extracted, especially lessons that would inform the design and approaches of future grants. Similarly, any lessons that can be extracted that would help improve the design of future livelihood supporting grants should be discussed. 7. In addition to deriving lessons that would inform the design of future grants, the consultants are expected also to propose ways in which the Fund would best be able to develop innovative ways to achieve the goal of reducing extreme poverty and increasing shared prosperity President Jim Yong Kim called for during the 2013 Spring Meetings. The first goal of reducing extreme poverty aims at reducing the percentage of people living on less than US$ 1.25 a day (extreme poverty) to 3 percent by 2030. Ending extreme poverty will mean addressing problems of fragile and conflict‐affected countries where human development lags the rest of the world. Single Donor grants such as JSDF ones have shown to be uniquely suitable for piloting new, innovative and flexible ways to reduce poverty and improve livelihood among vulnerable groups affected by crises and conflicts. The consultants should interview key stakeholders within the WBG including possibly members of the change management team to determine how best the Fund could contribute to the goal of reducing extreme poverty, particularly in fragile and conflict‐affected countries. 8. In assessing the grants in the poverty and livelihood samples, the consultant should examine the issues the task team encountered in identifying, preparing, implementing and closing the grant projects. Especially issues around unit, country, regional, JSDF and Donor comments and approvals, time and effort to consult with beneficiaries and Donor agencies and to produce concept notes, proposals and related documents, include risk mitigation ‐, quality control ‐, and fiduciary measures, and design an effective results framework. The consultant should examine the time it takes to prepare a grant request and interview a randomly selected number of task teams to get a sense of the problems and obstacles they face in putting together a successful grant project. This should be assessed in the context that grants are meant to provide immediate relief to vulnerable groups especially in emergency or disaster situations. Essential in this assessment is the examination of the procedures for review and approval within the WBG and by the Donor. The consultant is expected to come up with explicit recommendations to further simplify, streamline and improve the grant application and approval process, including the existing requirements regarding the Donor visibility, scrutiny, commentary and approval procedures and the uncertainties and risks attached to that. It may be difficult to assess the impact of the recent changes in procedures with the introduction of the small grants guidelines and the role of the Donor agreed to in the December 2012 consultations, but the consultant is expected to project the potential impact of these changes on the time and effort required to obtain a JSDF grant for a vulnerable group. Universalia 92 Volume I – Final Report C. DELIVERABLES  The assessment should produce two products: 1. One would be a comprehensive report on the contributions to poverty alleviation and sustainable livelihood JSDF grants bring to the vulnerable beneficiary groups. If feasible the report would have a separate chapter on poverty alleviation outcomes and experiences, a separate chapter on livelihood improvement outcomes, experiences and a section on lessons learned. These chapters may be enriched with boxes with case studies, interview results and opinions from stakeholders and beneficiaries. A third separate chapter should be devoted to the discussion of the difficulties and problems with the application process and the suggestions by task teams, managers and others to improve the process. This chapter should also reflect the discussions with the Donor and the follow‐up actions to be undertaken to simplify the process agreed upon with the Donor, OPCS and DFPTF. The report will comment on the visibility of Japan as a donor and recommend actions to improve donor visibility. 2. The second product of the assessment should be a short (no more than 30 pages) illustrated report7 that explains the contributions and operations of the Fund to the general public and persons interested in development. The potential audience for this report would be Japanese officials in IDA eligible countries, academics and citizens. Other audiences would include community, civic and social policy officials, academics and staff of NGOs/CSOs in IDA eligible countries. The broader audience would be social and economic development practitioners and students world‐wide who would like to learn how an important Donor such as Japan partners with the WBG to assist vulnerable and poor communities in poverty alleviation. 7 An example of the kind of report the Donor and WBG would like to see is: CATALYST FOR PROGRESS, The Japan Policy and Human Resources Development Fund. CFP, the World Bank, the Government of Japan. October 2012. 93 Universalia Volume I – Final Report Appendix IX Core Bibliography JAPAN SOCIAL DEVELOPMENT FUND (JSDF) FY11‐13 POLICY GUIDELINES AND PROGRAM ALLOCATION (Amended October 2011) JSDF Annual Report 2013 (unpublished) ISDF Annual Report 2012 JSDF Annual Report 2011 GOVERNMENT OF JAPAN – WORLD BANK GROUP ANNUAL CONSULTATIONS TOKYO – DECEMBER 10‐11, 2012 JAPAN SOCIAL DEVELOPMENT FUND (JSDF) PROGRAM FUNDING APPROVAL AND STRATEGIC USE OF GRANTS JAPAN SOCIAL DEVELOPMENT FUND (JSDF) FY14‐16 POLICY GUIDELINES AND PROGRAM ALLOCATION JAPAN SOCIAL DEVELOPMENT FUND GUIDANCE NOTE ON VISIBILITY OF JAPAN Strategic Evaluation of the Japan Social Development Fund (JSDF) FINAL REPORT, World Bank , 2007 2012 Trust Fund Annual Report, World Bank 2013 2013 Trust Fund Annual Report, World Bank, 2014 A Call to Dignity, World Bank , 2012 Strengthening the Results Focus in Trust Fund Management, World Bank 2010 Fighting Poverty With Knowledge, World Bank A Stronger, Connected Solutions, World Bank Group, World Bank, 2013 Japan Social Development Fund, Special Window for Afghanistan, World Bank 2007 JSDF Processing Procedures, World Bank, 2013 JSDF CASE LAW DECLINED PROPOSALS, World Bank, 2010 JAPAN SOCIAL DEVELOPMENT FUND (JSDF) PROCEDURES FOR PROCESSING JSDF SEED FUND GRANTS DOCUMENTS, World Bank, 2007 ADB’s Japan Funds: Japan Fund for Poverty Reduction, Asian Development Bank Operational Evaluations Department, 2007 Call for Proposals from Civil Society Organizations for Community‐Based Development Projects Japan Special Fund Poverty Reduction Program (JPO) Managed by the Inter‐ American Development Bank (IDB), Inter‐American Development Bank, 2013 Japan Fund for Poverty Reduction Annual Report 2012 Asian Development Bank, 2013 Japan International Cooperation Agency, Annual Report 2013 Universalia 94