Document of The World Bank FOR OFFICIAL USE ONLY Report No: PP4737 INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT AND/OR INTERNATIONAL DEVELOPMENT ASSOCIATION PROJECT PAPER ON A PROPOSED GRANT IN THE AMOUNT OF US$ 0.765 MILLION TO THE ECOLE NATIONALE D'ADMINISTRATION PUBLIQUE (ENAP) FOR A ENAP GLOBAL ACTION FOR CAPACITY-BUILDING IN MONITORING AND EVALUATION November 29, 2021 Global CURRENCY EQUIVALENTS (Exchange Rate Effective November 29, 2021) Currency Unit = US$ US$ 1 = CAD$ 1.28 FISCAL YEAR July 1 – June 30 Global Director: Sophie Sirtaine Practice Manager: Dugan Fraser Task Team Leader(s): Fabio Pittaluga, R Jason Thomas Mayfield ABBREVIATIONS AND ACRONYMS AFREA African Evaluation Association AWPB Annual Work Program and Budget CLEAR Centers for Learning on Evaluation and Results CLEAR FA Center for Learning on Evaluation and Results Francophone Africa CMU Country Management Unit DA Designated Account ECD Evaluation Capacity Development ENAP École nationale d’administration publique ESF Environmental and Social Framework EvalMENA Middle East and North Africa Evaluators Network FCV Conflict, Fragility, and Violence FIFE Forum Francophone de l’évaluation FM Financial Management GEI Global Evaluation Initiative GP Global Practice GPEDC Global Partnership for Effective Development Cooperation GRS Grievance Redress Service IAD International Affairs Directorate ICR Implementation Completion Report IDA International Development Association IEG Independent Evaluation Group IFR Interim Financial Reports ISR Implementation Status and Results Report IPDET International Program for Development Evaluation Training IPF Investment Project Financing MDTF Multi-Donor Trust Fund M&E Monitoring and Evaluation MENA Middle East and North Africa MIS Management Information System NEC National Evaluation Capacities Conference NGO Non-governmental organization PIFED Programme international de formation en évaluation du développement PPSD Project Procurement Strategy for Development RETF Recipient - Executed Trust Fund SAFIRH Système Approvisionnement, Finances, Immobilisations, Recherche, Rémunération, Ressources humaines SDG Sustainable Development Goals TAQYEEM International Training Program for Development Evaluation in Arabic TOR Terms of Reference UN United Nations WB World Bank WBG World Bank Group The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) BASIC INFORMATION Is this a regionally tagged project? Country (ies) No Financing Instrument Classification Investment Project Financing Small Grants Approval Date Closing Date Environmental and Social Risk Classification 01-Dec-2021 31-Oct-2024 Low Approval Authority Bank/IFC Collaboration GDD No Please Explain Proposed Development Objective(s) The project’s specific development objective is to improve monitoring and evaluation, capacity, and use in supported developing countries in the Francophone Africa and MENA regions. In doing so, the project will seek to build partnerships with other ECD providers in these regions to generate synergies. This project aims to contribute to the higher-level development objective of the GEI MDTF, namely, to improve monitoring and evaluation frameworks, capacity, and use in supported developing countries for improved evidence- informed policymaking. Components Component Name Cost (USD Million) Strengthen M&E Capacities of Individuals and Organizations in Francophone Africa and 0.40 MENA, with a stronger focus in the Maghreb sub-region initially Generate and disseminate M&E Knowledge 0.18 Management, accountability and learning 0.18 Page 1 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Organizations Borrower : Ecole Nationale d'Administration Publique (ENAP) Implementing Agency : Ecole Nationale d'Administration Publique PROJECT FINANCING DATA (US$, Millions) SUMMARY-NewFin1 Total Project Cost 0.77 Total Financing 0.25 Financing Gap 0.52 DETAILS -NewFinEnh1 Non-World Bank Group Financing Trust Funds 0.25 Global Evaluation Initiative Multi-Donor Trust Fund 0.25 Expected Disbursements (in USD Million) Fiscal 2022 2023 2024 2025 Year Annu 0.13 0.13 0.00 0.00 al Cumu 0.13 0.25 0.25 0.25 lative INSTITUTIONAL DATA Financing & Implementation Modalities Situations of Urgent Need of Assistance or Capacity Constraints [✔] Fragile State(s) [ ] Fragile within a [✔] Small State(s) [✔] Conflict [ ] Responding to non-fragile Country Natural or Man-made Disaster Page 2 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Other Situations [ ] Financial Intermediaries (FI) [ ] Series of Projects (SOP) [ ] Performance-Based Conditions (PBCs) [ ] Contingent Emergency Response Component (CERC) [ ] Alternative Procurement Arrangements (APA) [ ] Hands-on Expanded Implementation Support (HEIS) Contributing Practice Areas OVERALL RISK RATING Risk Category Rating Overall ⚫ Low COMPLIANCE Policy Does the project depart from the CPF in content or in other significant respects? [ ] Yes [✔] No Does the project require any waivers of Bank policies? [ ] Yes [✔] No Page 3 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Environmental and Social Standards Relevance Given its Context at the Time of Appraisal E & S Standards Relevance Assessment and Management of Environmental and Social Risks and Impacts Relevant Stakeholder Engagement and Information Disclosure Relevant Labor and Working Conditions Relevant Resource Efficiency and Pollution Prevention and Management Not Currently Relevant Community Health and Safety Relevant Land Acquisition, Restrictions on Land Use and Involuntary Resettlement Not Currently Relevant Biodiversity Conservation and Sustainable Management of Living Natural Not Currently Relevant Resources Indigenous Peoples/Sub-Saharan African Historically Underserved Traditional Not Currently Relevant Local Communities Cultural Heritage Not Currently Relevant Financial Intermediaries Not Currently Relevant Legal Covenants Conditions PROJECT TEAM Bank Staff Name Role Specialization Unit Team Leader(ADM Fabio Pittaluga SLCSO Responsible) R Jason Thomas Mayfield Team Leader IEGCD Procurement Specialist(ADM Blandine Marie Wu Chebili EPSPA Responsible) Page 4 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Financial Management Atul B. Deshpande EPSPF Specialist(ADM Responsible) Social Specialist(ADM Chaohua Zhang OPSIS Responsible) Environmental Specialist(ADM Xiaoxin Shi OPSEF Responsible) Angela Nyawira Khaminwa Safeguards Advisor/ESSA LCROS Dugan Ian Fraser Program Manager IEGCD Escarlata Baza Nunez Counsel LEGDF Geraldine Mayela Alonso Counsel LEGLE Ghersi Jessica Marie Meckler Team Member IEGCD Rodolfo Tello Abanto Social Specialist SLCSO Tobias Johan Thomas Fast Team Member IEGDG Vivian Nwachukwu-Irondi Counsel LEGDF Maria Gonzalez de Asis E&S Primary PM SLCSO Extended Team Name Title Organization Location Page 5 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) WORLD ENAP GLOBAL ACTION FOR CAPACITY-BUILDING IN MONITORING AND EVALUATION TABLE OF CONTENTS I. STRATEGIC CONTEXT ...................................................................................................... 7 A. Country Context ................................................................................................................. 7 B. Sectoral and Institutional Context ..................................................................................... 9 C. Higher Level Objectives to which the Project Contributes ............................................. 12 II. PROJECT DEVELOPMENT OBJECTIVES ............................................................................ 14 A. PDO ................................................................................................................................... 14 B. Project Beneficiaries ......................................................................................................... 14 C. PDO-Level Results Indicators ........................................................................................... 15 III. PROJECT DESCRIPTION.................................................................................................. 16 A. Project Components ......................................................................................................... 16 B. Project Cost and Financing ............................................................................................... 20 IV. IMPLEMENTATION........................................................................................................ 21 A. Institutional and Implementation Arrangements ........................................................... 21 B. Results Monitoring and Evaluation ................................................................................. 23 C. Sustainability .................................................................................................................... 24 V. KEY RISKS ..................................................................................................................... 24 A. Overall Risk Rating and Explanation of Key Risks ........................................................... 24 VI. APPRAISAL SUMMARY .................................................................................................. 24 A. Legal Operational Policies ................................................................................................ 26 B. Environmental and Social................................................................................................. 26 VII. WORLD BANK GRIEVANCE REDRESS .............................................................................. 27 VII. RESULTS FRAMEWORK AND MONITORING .................................................................... 28 Page 6 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) I. STRATEGIC CONTEXT A. Country Context 1. Effective monitoring and evaluation (M&E) systems are essential to effective public policy making. They enable institutions to assess the effectiveness of policy decisions and programs, monitor progress towards national goals and the Sustainable Development Goals (SDGs), and course correct as needed to accelerate progress. They also enhance public accountability for results and provide opportunities for dialogue between citizen and public institutions. The 2015 Evaluation Year, along with the United Nations (UN) General Assembly’s adoption of the Agenda 2030 and the SDGs, strengthened the movement towards increased governmental awareness and interest in evaluation while raising expectations about the pace of M&E capacity development at national levels. A strong demand for individual evaluation training and capacity development emerged from this movement and important steps have been made towards the institutionalization of evaluation, although progress varies between countries. 2. As the implications of the Covid-19 pandemic are felt around the world – with the poverty rate in West and Central Africa increasing almost three percent in 20201 – strong M&E systems supporting evidence- based policymaking are needed now more than ever before. To define effective policies to fight the disease, limit its socio-economic repercussions, and help societies build back better post-pandemic, governments will need relevant data to plan, allocate resources, and track progress to improve their ability to measure the impact of policies and adjust those policies based on such evaluations. 3. However, in many developing and emerging economies, M&E systems are underdeveloped and are not sufficiently institutionalized, and capabilities to provide quality M&E services are insufficient. A recent study by the Global Partnership for Effective Development Cooperation (GPEDC) showed that, while ninety-one percent of national development strategies approved after 2015 explicitly refer to the 2030 Agenda and the SDGs, only thirty-five percent of them have the required data and systems to track implementation. In many countries, this stems from the lack of a conducive legal and regulatory environment to create demand for M&E services on the part of public institutions; insufficient capabilities to procure, provide, and use evaluations; weak accountability mechanisms on the use of evidence and results; and weak frameworks to institutionalize the use of M&E in decision-making. 4. As a response to these pressures, many countries across the world, including in Francophone Africa and in the Middle East and North Africa (MENA) region, have increased their interest and demand for M&E in recent years, which the École nationale d'administration publique (ENAP) is well positioned to serve. Indeed, there is a continued need for evaluation capacity development (ECD) across the world through capacity building programs such as the ones developed by ENAP. Such programs are critical to bringing global expertise and local best practices together, so that key stakeholders in evaluation have the opportunity to develop skillsets in evaluation that can be leveraged in their in-country work. With its long- standing experience in providing ECD services globally, in French and now also in Arabic, and its reputation for excellence, ENAP is perfectly positioned to support evaluation stakeholders better understand and use evaluation for policy and decision making and to contribute to the strengthening of the capacities of M&E 1 Supporting a Resilient Recovery: The World Bank’s Western & Central Africa Region Priorities 2021 – 2025. Page 7 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) systems and practitioners in the Francophone Africa and MENA regions. 5. Since 2010, many countries in the MENA region have been affected by an unprecedented wave of political transitions and social uprisings. A key driver of the Arab Spring was an increasing and widespread dissatisfaction with the quality of life in the region, including limited formal employment opportunities, poor quality of public services, and a lack of public accountability.2 In this context, the development and strengthening of M&E systems and capacities of both public actors and local M&E stakeholders is particularly important. M&E functions play an essential role in improving service delivery, enhancing accountability of public institutions, fostering good governance, and rebuilding citizens’ trust in government. 6. Countries in Francophone Africa also face an urgent need for stronger M&E systems and frameworks, as the region faces multifaceted, wide-reaching challenges. Such challenges include but are not limited to conflict and violence, gender disparities, food insecurity, and climate change. These challenges have been accentuated by the Covid-19 pandemic, which will likely result in heavy economic and social cost for countries in the region. The pandemic threatens to push up to forty million people into extreme poverty in Sub-Saharan Africa alone.3 As a result, the need for good governance and evidence-informed public policies is critical to ensure better development outcomes for all. Stronger M&E systems will enable governments throughout Francophone Africa to make institutions more efficient, accountable, and ready to help communities build back better post-pandemic. 7. Despite persisting barriers to the institutionalization of M&E, the MENA region has seen a growing interest in M&E overall. For instance, the 2012 launch of the Middle East and North Africa Evaluators Network (EvalMENA) galvanized M&E professionals at the regional level. Further, national evaluation associations (including the Moroccan Evaluation Association, the Palestinian Evaluation Association, the Jordan Evaluation Association, the Tunisian Evaluation Association, the Lebanese Evaluation Association, and the Egyptian Development Evaluation Network) have been established to promote evaluation throughout the region and create communities of practice for MENA-based evaluation professionals. 8. The Francophone Africa region also witnessed a growing demand for the use of evidence in decision and policy making. Many administrations in the region have demonstrated an increasing awareness and interest in evaluation and its use in policymaking as well as in evidence-based decision making. Some countries made progress in institutionalizing evaluation through laws and regulatory frameworks, while civil society played an increasingly important role in supporting the professionalization of evaluation and building evaluation capacity. Lastly, recent Francophone or bilingual evaluation events have enabled Francophone evaluators and leaders from different countries to connect and share knowledge to address common challenges. For example, the Forum Francophone de l’évaluation (FIFE) (Dakar 2014, Marrakech 2016, Ouagadougou 2019) and the African Evaluation Association (AFREA) conferences (Yaounde 2014, Kampala 2017, Abidjan 2020) have greatly contributed to the sustained interest in the institutionalization of evaluation in Francophone Africa. 9. With the post-pandemic push towards a better and more resilient recovery, much work remains to be done to support North African and MENA governments’ recognition and formalization of the use of 2 https://www.worldbank.org/en/news/feature/2015/10/21/middle-class-frustration-that-fueled-the-arab-spring 3 https://www.worldbank.org/en/region/afr/overview#1 Page 8 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) evaluation in decision-making. In general, national capacities to support the institutionalization of evaluation remain weak, and while there are many similarities across the countries in these two regions, they also have heterogeneous economic, cultural, and political contexts in which the demand for greater accountability varies widely. In a part of the world undergoing deep transformation and facing multiple, interconnected crises – including sanitary, geopolitical, economic, humanitarian, migratory, etc. – the development and strengthening of M&E systems and capacities of both public actors and local M&E stakeholders are crucial factors in the achievement of national development goals and the SDGs. As such, awareness building, advocacy efforts, and the training of M&E stakeholders at all levels are needed now more than ever in order to promote national evaluation policies and strategies and to build more effective national M&E systems. 10. In order to bridge the significant gap in M&E capacity in Francophone Africa and MENA, a strong focus on strengthening M&E systems and capacities at all levels of government (national, sub-national, and local) is critical to drive progress on the SDGs and other development initiatives and to ensure that policies and decisions are based on evidence of what works. Enhancing M&E in Francophone Africa and MENA is particularly important given the many challenges facing the regions. As a multisectoral, cross-cutting field, M&E offers a means to improve public service delivery across all areas of public administration. This demand for M&E support was strongly emphasized during various regional launch events of the Global Evaluation Initiative. In MENA this brought together representatives from governments, civil society, international organizations, and experts from across the region in April 2021 who voiced a strong demand for ECD services to help rebuild better from COVID-19 and reach the 2030 Agenda. A similar event was held for the Africa region in November 2020. B. Sectoral and Institutional Context 11. In FY21 the Independent Evaluation Group (IEG) launched the Global Evaluation Initiative (GEI), a partnership that aims to catalyze and bring together key actors and experts in the evaluation field to help governments in developing countries place evidence at the heart of decision making . GEI is an inclusive global partnership committed to developing country-owned, sustainable M&E frameworks and capacities to promote the use of evidence in public decision-making, enhance accountability, and achieve better results. ENAP is one of the key implementation partners of GEI. ENAP is ideally positioned to helps GEI achieve its mandate in Francophone African and in the MENA regions. 12. Since 2011, ENAP has been a leader in providing ECD services in the Francophone Africa and MENA regions to government institutions, and other public actors and M&E stakeholders . ENAP is a Francophone higher education institution based in Quebec, Canada and is one of the largest public administration universities in the world. Since its creation in 1969, it has positioned itself as a leader in the renewal of a modern and effective public administration in Quebec and abroad. ENAP’s services include in-person continuing education programs in Canada and abroad, as well as remote learning options. Its International Affairs Directorate (IAD) coordinates ENAP’s international work, including ECD services to clients from the Francophone Africa and MENA regions, in French and in Arabic. Over the last decade, ENAP has trained over two thousand people from over thirty countries — seventy percent of which from the public administration sector — in evaluation. 13. In Africa, ENAP has trained five hundred French-speaking individuals since 2016, and in 2021, with the Page 9 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) support of the World Bank, ENAP developed the International Training Program for Development Evaluation in Arabic (TAQYEEM), the first ever global M&E training program in Arabic. During the inaugural edition of the TAQYEEM course in June 2021, the program reached thirty-six people from five Arabic-speaking countries4 and was highly regarded by the course participants5. ENAP’s work has targeted some of the poorest and most fragile countries in the world in order to enhance in-country knowledge and skills to plan for and use evaluation for adaptive management, manage for results, and ultimately, develop more relevant and effective policy interventions. By focusing on addressing the M&E gap in French- and Arabic-speaking developing countries, ENAP aims to bring M&E systems, practices, and capacities to developing countries most in need of support. 14. The first edition of the TAQYEEM program allowed ENAP to learn key lessons that will help in the growing of the program with GEI’s support. TAQYEEM was designed in collaboration with key ECD actors in the region (e.g. Islamic Development Bank and Saoudian Impact Center Alathar) and this allowed ENAP to deepen its understanding of intra-regional differences and the diverse types of M&E services demanded by public administrations in the region. These lessons will help ENAP consolidate its approach and strategy to reach those heterogenous audiences. TAQYEEM also highlighted the dearth of M&E scientific literature in Arabic, confirming the importance of ENAP’s contribution in this field. On a more practical level, TAQYEEM helped refine the Arabic terminology needed to adapt content to sub-regional contexts. 15. ENAP is uniquely placed to support GEI’s work in capacity building in the Francophone Africa and MENA regions based on its past work in French- and Arabic-language ECD activities. Since 2011, ENAP has collaborated with IEG to develop and implement a French version of the International Program for Development Evaluation Training (IPDET), an English-language executive M&E training program. The French language program, known as the Programme International de Formation en Évaluation du Développement (PIFED), has grown to be a well-known course for Francophone evaluators and utilizes cutting-edge pedagogical techniques (more information on ENAP’s pedagogy in Box 3). Further, ENAP has adapted and customized the PIFED content for its newly developed TAQYEEM program. As such, ENAP’s core training programs are characterized by culturally relevant content and strong pedagogy, and through its past collaborations with the IEG, ENAP has demonstrated its commitment to cultivating global public goods, especially for the Francophone and Arabic communities. 16. As an implementing partner of GEI, ENAP is uniquely positioned to deliver the results that GEI wants to achieve in the Francophone Africa and MENA regions. In particular, with this grant, ENAP will solidify its Arabic-language offerings to create new opportunities for ECD in Arabic for key stakeholders in the MENA region. Additionally, ENAP will facilitate communication between evaluators and M&E specialists in the regions and globally to create stronger networks and peer learning in French and Arabic. By strengthening connections between the international, regional, and national evaluation communities, ENAP will enable evaluators in the Francophone Africa and MENA regions to build on lessons learned around the world and apply them to their specific context. 17. ENAP will draw on strong, collaborative partnerships to amplify its work. ENAP will coordinate with other GEI partners in Francophone Africa and MENA, such as the Center for Learning on Evaluation and 4 Algeria, Libya, Mauritania, Morocco, and Tunisia 5 100% of participants responded that they were satisfied or highly satisfied with the training. Page 10 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Results Francophone Africa (CLEAR FA) and the Islamic Development Bank, to build synergies and avoid duplication of efforts, thus increasing the overall reach of ENAP’s work. ENAP will also collaborate with local actors who are influential in the national and regional M&E spaces. In doing so, ENAP will optimize the strengths, knowledge, and capacities of its partners to provide effective and contextually relevant M&E solutions to developing countries in the Francophone Africa and MENA regions. With its collaborative approach and regional focus, ENAP’s work is aligned with the GEI and it is well placed to contribute to the intended outcomes of GEI. Further information about GEI and ENAP’s alignment with GEI is in Box 1. Page 11 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Box 1. The Global Evaluation Initiative The Global Evaluation Initiative (GEI) is a partnership centered on the vision that better evidence contributes to better policies, and ultimately to better lives. GEI expects to be a catalyzer, bringing together key actors and experts in the evaluation field to help governments in developing countries place evidence at the heart of decision making. GEI intends to act as a broker, helping developing countries find solutions and financing for their country M&E systems and capacity development, building on their strengths and endogenous knowledge, and leveraging the best available local, regional, and global knowledge. GEI anticipates playing a critical role in fostering M&E knowledge generation and sharing it globally and locally, so that M&E knowledge generated in one country in the world is made available to others as relevant for greater learning and effectiveness. GEI’s partners will support Evaluation Capacity Development (ECD) in developing countries, fostering evidence-informed decision-making through enhanced M&E frameworks, capacity, and use. GEI’s central aim is to strengthen the M&E frameworks and capacities of governments and other stakeholders in developing countries and the use of M&E evidence by these stakeholders, by establishing a global partnership of ECD providers and experts supported by a hybrid Trust Fund. GEI will support individuals, governments, and other organizations where there is a market failure in terms of the supply of, or demand for, ECD services (e.g., ECD service providers are unavailable or inadequately meeting market demands). Through GEI, more countries will be able to establish and use M&E systems as well as conduct and use evaluations to inform organizational learning, accountability, and decision-making. This will contribute to more relevant and effective policy interventions, better responses to shocks, and ultimately progress towards national development goals and the Sustainable Development Goals (SDGs). GEI will partner with key ECD players around the world and coordinate with them to achieve GEI’s strategic priorities and outcomes. Partnerships will focus on leveraging entities’ areas of strength and comparative advantage. For instance, partners such as the regional Centers for Learning on Evaluation and Results (CLEAR Initiative) will leverage their strengths in providing technical assistance to strengthen M&E systems in developing countries. ENAP, with its expertise in providing M&E training and its reputation for excellence, will be an essential partner in supporting GEI’s work in building M&E stakeholders’ capacity. Under the GEI umbrella, ENAP will play a critical role as a provider of trainings that merge global expertise with local knowledge and practice. As an associate partner of the GEI and a renowned institution focused on developing and improving the M&E capabilities of key stakeholders in French- and Arabic-speaking developing countries, ENAP will contribute to the results expected by the GEI. Through GEI, ENAP will also aim to strengthen its collaboration in-country with other GEI partners and World Bank country offices in the Francophone Africa and MENA regions. C. Higher Level Objectives to which the Project Contributes 18. The World Bank Group’s (WBG) strategic priorities and commitments set out in the Forward Look, Capital Increase Policy Package, International Development Association (IDA) replenishment documents, and IFC 3.0 strategy demonstrate the level of ambition needed to reach the WBG’s twin Page 12 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) goals and contribute to the achievement of the SDGs. M&E has an important role to play in monitoring the progress towards these goals, enhancing good governance, transparency, and accountability in client countries, thus ultimately contributing to better development outcomes. As an implementing partner of GEI focused on developing and improving the M&E capabilities of key stakeholders in developing countries, ENAP will contribute to helping the WBG and its clients in the Francophone African and MENA contexts to enhance their effectiveness in meeting such strategic priorities. 19. ENAP’s support to the development of M&E systems and capacities in the Francophone Africa region aligns well with the WBG’s ongoing work in the West and Central Africa region. In its regional strategy for 2019 – 2023, the WBG identified the urgent need to make institutions more efficient and accountable in order to enable governments to respond to complex service delivery needs in rapidly changing contexts. M&E plays a crucial role in enabling public institutions to track implementation and course correct as needed, thus enhancing efficiency of public services and enabling greater transparency. ENAP will support local institutions to improve effective service delivery and contribute to overall good governance in the region. Further, the WBG’s regional strategy offers support to countries experiencing conflict, fragility, and violence (FCV). Through its support to professionals in FCV contexts, ENAP will support governments and organizations better monitor and evaluate programs in challenging settings so that programs address the constantly changing needs of vulnerable populations. 20. The grant is also well aligned with the WBG’s MENA Regional Strategy6, in which structural reforms, governance, and service delivery are key focus areas. The Covid-19 pandemic has exacerbated key development challenges in the MENA region, including access to jobs and economic opportunities for youth and women, limited trust in government, poor returns on investments in human capital, and uncompetitive economies. By providing evidence on what works and what doesn’t, M&E is a fundamental tool that can enhance the efficient use of resources in public service programs and support the restoration of trust in government entities. As such, this grant is well aligned with the WBG’s MENA Regional Strategy7, in which structural reforms, governance, and service delivery are key focus areas. Additionally, the WBG’s focus on advocating for data transparency, including filling urgent data gaps and supporting evidence- based policymaking, aligns with the aim of this grant. 21. Finally through its contribution to the GEI partnership, ENAP will also support the WBG’s outcome orientation agenda. In 2005, the WBG rolled out results-based management at the country level and, in 2021, continues to build on this experience to strengthen outcome orientation. ENAP will, among other things, contribute to GEI’s support to the WBG outcome orientation agenda by building a cadre of professionals in client countries who are trained on monitoring, evaluation, learning, and adaptive management practices, and better equipped to place outcomes and results at the heart of their decisions. 6 Middle East and North Africa (MENA) Regional Update: Preventing a lost decade, Unlocking the region’s potential, 2021. 7 Middle East and North Africa (MENA) Regional Update: Preventing a lost decade, Unlocking the region’s potential, 2021. Page 13 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) II. PROJECT DEVELOPMENT OBJECTIVES A. PDO 22. The project’s specific development objective is to improve monitoring and evaluation, capacity, and use in supported developing countries in the Francophone Africa and MENA regions. In doing so, the project will seek to build partnerships with other ECD providers in these regions to generate synergies. 23. This project aims to contribute to the higher-level development objective of the GEI multi-donor trust fund (MDTF), namely, to improve monitoring and evaluation frameworks, capacity, and use in supported developing countries for improved evidence-informed policymaking. B. Project Beneficiaries 24. The grant will target and benefit a range of stakeholders in Francophone Africa and MENA regions: (i) government agencies (including monitoring and evaluation agencies, line and sectorial ministries and sub- national governments) looking for sustained engagements to resolve or satisfy their M&E capacity needs; (ii) civil society organizations that help strengthen M&E systems; and (iii) evaluation professionals who need to build their capacity to conduct high quality evaluations; and (iv) parliamentarians and decision- makers who play an important role in strengthening national M&E systems, without necessarily practicing it on a daily basis. 25. ENAP will support the targeted beneficiaries at two levels: (a) Organizational, where support is provided to build M&E capacities of institutions such as government agencies (particularly but not limited to evaluation and training units), national schools of public administration, evaluation networks, and civil society organizations; (b) Individual, which focuses on building the capacity of individuals whose knowledge, skills, and competencies are essential to achieving any system-level impact or culture change. 26. At all levels, ENAP targets various audiences (including evaluators, users and commissioners of evaluation, researchers, policymakers, and managers of evaluation) and employment levels (including executive, managerial, technical, and professional). ENAP will prioritize beneficiaries who represent underserved demographics and geographies to ensure that evaluation training is available to those who are best placed to use this knowledge. ENAP has always encouraged women’s participation in evaluation training, and they remain another key target group to benefit from ENAP’s services. 27. During the next few years ENAP will have a preferred focus on countries, institutions and individuals in the Francophone Africa and MENA regions most likely to have a systemic impact on the use of evidence in their respective country/institution. The stakeholders and activities will be demand-driven but also chosen taking into account a strategic client prioritization criteria, including: i) countries’ and clients' institutional conditions and readiness for M&E; ii) likelihood that the activities will lead to expected outcomes and impacts; iii) likelihood that the activities will provide good demonstration effects Page 14 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) for other potential beneficiary; iv) likelihood that the activities will lead to other demands for relevant services and long term engagements with the strategic client; v) strategic clients' intere st and ability to replicate capacity building efforts;  vi) possibility to leverage the engagements to develop public goods and knowledge; vii) possibility to establish new partnerships and leverage alternative sources of funding; viii) existing relationships and partnerships; and ix) low-income states, fragile , and conflict affected economies, and small states in the Francophone Africa and MENA regions. The project team at ENAP will engage regularly with GEI’s Global Team and partners to identify shifts in the countries’ context and future opportunities for engagement in other countries. 28. Ultimately, the final beneficiaries of this grant are the citizens of the Francophone Africa and MENA regions, who will benefit from the use of evidence for better policies, enhanced accountability, and good governance. C. PDO-Level Results Indicators 29. Underlying Theory of Change informing this project. The proposed project is predicated on a Theory of Change that links ENAP’s training and hands-on support activities to results at the institutional levels where training recipients work and operate. As ENAP works predominantly with public sector officials, it is expected that trainees will be able to use the knowledge acquired for improved use of evidence in policy- and decision-making processes. In this context, capacity building activities are expected to increase knowledge of evaluation theory and practices at individual and organizational levels. On the job coaching and technical assistance will provide additional means to move from theory to practice. As participants acquire new knowledge and learn how to apply it, they are likely to adopt new and/or improved behaviors (ways of working) and practices (e.g., develop quality M&E frameworks, adopt new/improved M&E plans, policies, etc.) that in turn would likely result in improved evidence-based decision-making processes. To that effect, showing concrete results in the early stages of project implementation would help in building trust with clients and potentially generate the demand for more upstream M&E services. Box 2: Underlying Theory of Change Page 15 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Participants Increased ENAP training and champion and Increased use of awareness and Participants adopt hands on support to support evidence for policies knowledge of more M&E practices move from theory institutionalization and institutuional evaluation theory and behaviors to practice and use of M&E practices and practice practices 30. The GEI has identified a core set of key indicators that are standard to all GEI grants. As an implementing partner to GEI, ENAP is committed to contributing to reaching the indicators’ targets and to collect and provide the necessary data and information for GEI to aggregate them with those of other partners for accountability purposes. 31. The key PDO results indicators for this grant are: (a) Percentage of participants that provide positive feedback on training relevance, quality and knowledge acquired (b) Number of ECD service requests coming to ENAP as a result of training activities with client countries (c) Expanded institutional, thematic, or geographic coverage through new or existing partnerships III. PROJECT DESCRIPTION A. Project Components 32. The project’s development objective will be achieved through the provision of high quality and relevant technical assistance, trainings, and knowledge sharing activities that target government agencies, civil society organizations, evaluation professionals, parliamentarians, and decision-makers in developing countries in the Francophone Africa and MENA regions. It is expected that the institutions and individuals who benefit from ECD activities financed under this grant will serve as change agents in their respective institutions, countries, and regions and promote the use of M&E for evidence-based decision making. The main expected activities to achieve results are the following. 33. COMPONENT 1. Strengthen M&E Capacities of Individuals and Organizations in Francophone Africa and MENA, with a stronger focus in the Maghreb sub-region initially Page 16 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) 34. Under this component, ENAP will focus on building and sustaining the M&E capacity of individuals and institutions through the provision of training and support to improve national and/or institutional M&E systems. Activities will include French- and Arabic-language M&E training for local stakeholders (M&E specialists, evaluators, government officials, and parliamentarians/decision-makers), training-of-trainers sessions, mentoring, hands-on support for evaluations, and support for analytical/research activities related to M&E and decision making. In some cases, ENAP may deliver customized solutions to institutions that request assistance on specific contexts or sectors. Given the limited resource available for this project, ENAP will coordinate with World Bank local and regional offices to establish priority countries and/or areas (sectors) for intervention. Specifically, ENAP will work with the Poverty and Governance Global Practices (GPs) to ensure contributions to the broader agendas on data quality and availability for broader policies and programs. Criteria for selectivity are included in the description of “project beneficiaries� section in this document. 35. Specifically, this program will finance the following activities under Component 1: (a) Delivery of French-language training courses to key audiences in Francophone Africa. Depending on the evolution of the Covid-19 pandemic, in year one, ENAP will deliver either distance-learning or in-person training activities to target beneficiaries in Francophone Africa. (b) Expand and deliver the Arabic-language training program TAQYEEM in the MENA region. During the first year of the grant, ENAP’s activity in the MENA region will be structured around the continued development of the TAQYEEM program, utilizing the lessons learned during the June 2021 program to enhance the course. Advanced training content that complements the basic TAQYEEM training will be developed to enrich the program. The training tools implemented for the first edition of the program will be refined, and ENAP will continue to identify international experts to join their academic team and will conduct training-of-trainers sessions. At least two editions (distance-learning or in-person) of the program are expected to be delivered during the course of the grant. These activities will allow ENAP to consolidate the TAQYEEM program and turn it into a customizable product for the MENA region. For any work in the MENA region under this grant, ENAP will rely on the findings of a rapid situational analysis of the M&E landscape in the MENA region (currently being coordinated by the GEI Global Team) to provide the necessary background analysis. (c) Provide technical assistance to French- and Arabic-speaking institutions, based on client demand. Relationships with client countries and institutions often begin with a specific training (e.g. PIFED, TAQYEEM), after which countries may request additional services in the form of advisory services and/or technical assistance. It is expected that ENAP’s work in Francophone Africa and the MENA region may result in such demand for follow-up services. Depending on the opportunities leveraged by training activities in French and Arabic, and contingent upon the availability of additional funding, technical assistance may be offered to client countries in year two and three of the project. Additionally, ENAP intends to collaborate with CLEAR FA to assess the state of national M&E systems in the Francophone Africa region and identify countries that would most benefit from ECD activities. This will be important to define more precisely the roles of ENAP and CLEAR FA in the Francophone Africa region, avoiding duplication of efforts and Page 17 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) maximizing the impact of GEI-supported work in the region. (d) Build capacity of selected institutions in Francophone Africa and MENA to improve their M&E systems and functions. In addition to training, institutions often require additional support to incorporate M&E practices and frameworks into their work as well as to formalize the role of M&E in their processes. As such, ENAP will strategically supplement its trainings with additional hands- on support to enhance evaluation functions in client institutions through targeted activities. Such activities may include mentoring, hands-on guidance during evaluation exercises, guidance on M&E strategy development, and developing follow-up content for specific stakeholder groups such as parliamentarians. Additionally, to support the institutionalization of evaluation capacity development in client countries, ENAP will build the capacity of local partners (e.g. local universities and research centers) using a “training-of-trainers� model following a tested approach already utilized by ENAP to build “satellite� centers. Box 3. Highlights of ENAP’s training philosophy and pedagogy ENAP’s training utilize a skill-based approach focused on practical knowledge and know-how and varied pedagogical methods to maximize learning. The ENAP training approach is characterized by i. A dynamic teaching method that emphasizes the involvement of participants; ii. Concrete practical exercises that aim to apply the content learned; and iii. Trainers who have vast experience with training practicing public managers, both in Canada and internationally. ENAP will also promote synergies among partners to enhance training offerings and opportunities that are informed by and relevant to local contexts, and share lessons learned from global programs. Further, there is potential to connect these activities with ongoing and future work of other GEI partners and World Bank country offices to amplify the overall impact of ECD in developing countries. The content of ENAP’s trainings will be tailored to the specific needs of the intended participants. ENAP uses diagnostic/analysis activities to identify the topics and themes that will be the most useful and impactful for participants. These activities may be at (i) the initiation/awareness level (i.e., the use of evaluation in program or policy-making, the role of decision-makers in the public policy cycle, the role of evaluation in the public policy cycle, evaluation as a tool for transformational development, the role of the public service/parliaments in the national M&E systems, etc.); (ii) at the level of methods and core concepts in evaluation (i.e., different evaluation approaches, Theory of Change, evaluation questions, the evaluation matrix, techniques and strategies for the communication of results, management of the evaluation process, management of evaluation teams, ethical principles and standards for evaluators, etc.); or (iii) on more advanced methods and concepts (i.e., quantitative and qualitative methods in evaluation, statistical methods and impact evaluation tools, economic evaluation methods, cost-benefit evaluation, equity-focused and gender-sensitive evaluations, evaluation in the context of the SDGs, the establishment of a monitoring and evaluation systems, etc.). As part of GEI, there is scope to explore linkages between ENAP’s trainings and follow up job competency development through partnerships with the GEI Global Team and other GEI partners. Page 18 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) 36. COMPONENT 2: Generate and disseminate M&E Knowledge 37. Activities under this component will focus on supporting research, analytical and thought leadership work. Specifically, this program will finance the following activities under Component 2: (a) Develop knowledge products on best practices for the successful institutionalization and use of evaluation. Using its technical assistance support to institutions and governments in-country, ENAP will document good practices for the development of cultures of evaluation in governmental entities. Examples of such work include, but are not limited to, the development of guidance notes on French and Arabic M&E terminologies, best practice notes, and guidance material, among others. Additional research will be identified for future years of the grant, depending on the financial and human resources available. (b) Organize webinars, panels, and round table events for knowledge sharing in French and Arabic. ENAP will lead and participate in collaborative initiatives to pool knowledge on best practices and lessons learned in evaluation capacity development to stimulate the development of innovative and efficient approaches and practices in Francophone Africa and MENA. ENAP will focus on sharing new knowledge on best practices, lessons learned, new approaches, success stories, and more, in collaboration with other GEI partners. To do so, ENAP will organize and participate in webinars, panels and virtual round tables on relevant topics at regional and national levels in Francophone Africa and MENA. ENAP will also continue to promote peer-to-peer learning and knowledge sharing through its alumni community of practice that it will extend to alumni from Francophone Africa and MENA reached under this grant. Ultimately, ENAP aims to stimulate more experience and knowledge sharing activities in French and Arabic. (c) Participate in global M&E knowledge sharing events to promote lessons learned from the Francophone Africa and MENA regions. To do so, ENAP will participate in global and regional events such as the gLOCAL Evaluation Week, FIFE, EvalMENA conference, the National Evaluation Capacities (NEC) Conference, etc., to advocate for stronger M&E systems, share knowledge, and learn from others. ENAP will leverage the global footprint of the GEI to bring more attention to the Francophone and Arabic-speaking world of evaluation. 38. This component will also include activities aimed at sharing M&E knowledge, be it through organizing events aimed at disseminating ENAP's learnings and experiences, convening and fostering collaboration in Francophone Africa and MENA among ECD providers. 39. COMPONENT 3: Management, accountability and learning 40. This component will finance the overall program management, including administration, oversight, and assessments for each event and training, as well as for the entire program. This component will also support ENAP’s efforts to incorporate feedback and lessons learned into its ECD approach and future activities. (a) In order to continuously learn and incorporate feedback to support a culture of learning, ENAP will put in place a series of measures illustrated below. Page 19 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) (i) Systematically distribute surveys to participants at the end of each training program. These surveys will focus on participants’ satisfaction with the training (e.g., the content of the course, the quality of the speakers, the balance between theory and practice, the logistics and general organization of the training) and assess participants’ level of knowledge before and after the training. These surveys will also collect data on the participants’ experience to develop a better understanding of what works well, what can be improved, and participants’ needs and expectations, which will allow the ENAP team to adjust their training programs to produce the best possible results. (ii) Conduct after-action reviews to document best practices and areas of improvement for future ECD activities. (iii) Send tracer surveys to participants six months after training programs to document long-term effects of the trainings, as well as to gather insights about the factors favoring or hindering the transferability of learning in participants’ workplace. (iv) Produce activity reports documenting best practices, lessons learned, and recommendations. (v) Regularly send surveys to course alumni and newsletters to analyze its training offers and adapt its services to meet the needs of its audience. (vi) Wherever possible, organize a meeting with women to better understand women’s experiences in the field of evaluation and their specific capacity building needs. This will enable ENAP to adapt their trainings to better meet the needs of women and enhance their opportunities to access ECD opportunities. 41. The results of these activities will be incorporated into the preparation of subsequent trainings to integrate, when possible, the necessary adjustments to adapt their ECD approach to maximize the overall satisfaction of participants and achieve the program’s expected outcomes. 42. Accountability mechanisms: ENAP has a strong tradition of utilizing accountability mechanisms to ensure the quality of its ECD work. As per the requirements of programs, ENAP relies on a number of accountability systems, such as oversight committees, the support of logistical and content experts, and more. Additionally, ENAP ensures the quality of its ECD work through the support of professors within the ENAP system who provide advice on the best approaches and latest trends in evaluation. ENAP also has an internal team of scientific advisors who are responsible for updating ECD material periodically to ensure its alignment with best practices in the field of evaluation and with international quality standards. Lastly, the team is responsible for coaching the trainers and holding debrief sessions with them to learn from their experiences and collect their advice for continuous improvement. These accountability mechanisms will continue to support ENAP’s ECD activities under this grant. 43. Linkages to the GEI M&E Framework: ENAP will ensure that its internal M&E activities align with the overall GEI M&E Framework, which lays out an ambitious plan for evaluative exercises across the GEI network. As part of this, ENAP will provide data through the GEI Management Information System (MIS) and will link into other planned evaluative exercises as relevant. B. Project Cost and Financing Page 20 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Project Components Project cost Trust Funds Counterpart Funding Component 1 130,000.00 GEI 0.0 Component 2 60,000.00 GEI 0.0 Component 3 60,000.00 GEI 0.0 Total Costs Total Project Costs Total Financing Required 250,000.00 IV. IMPLEMENTATION A. Institutional and Implementation Arrangements 44. Founded in 1969, ENAP is a Francophone higher education institution of public administration in Quebec, Canada. The University has several regular professors, adjunct professors, and guests from a variety of disciplines, as well as active research groups in fields such as management, politics, public programs, and governance. ENAP is a branch of the Université du Québec network. 45. ENAP has been active on the international scene for over fifty years and has contributed to the reflection and evolution of public administration best practices. Over the years, ENAP has developed an expertise in knowledge and capacity building in public administration. ENAP has developed its international expertise by conducting several major international development projects, implementing interventions in over thirty countries, and training thousands of professional, mid- and high-level public managers and executives from around the world on the best practices of public management. 46. ENAP will be the Recipient and sole implementing entity for the project. There will not be any sub- implementing agencies for this project. ENAP is an entity with separate legal personality with powers to enter into contracts with international organizations. 47. ENAP will be responsible for the overall management and implementation of the project. A project director, acting under the authority of ENAP’s Director of International Affairs, will be responsible for the management and monitoring of the project. The project director will oversee the planning of project activities and resources and be the point of contact with the GEI/World Bank for any administrative or operational question with the implementation of the grant. The project director will monitor the implementation of the project and act as a bridge, when necessary, between the Bank and the various services of ENAP (financial, legal, procurement, etc.) to ensure compliance with the contractual agreement. 48. The project director will have the support of an accounting technician who will liaise with the finance Page 21 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) department for the payment of expenses, budget monitoring, data entry and report production. 49. The project manager plans the budget, but all expenses are subject to authorization by the person holding proper powers according to the level of the expenditure, as prescribed in ENAP’s internal Regulation. Each expense is processed by the finance department using ENAP’s Système Approvisionnement, Finances, Immobilisations, Recherche, Rémunération, Ressources humaines (SAFIRH) system upon proper authorization. Monthly financial monitoring of the project will be carried out. 50. Project activities aimed at target beneficiaries will be led by two ECD Experts who will be mobilized on the project team. They will have access to administrative and logistical support from administrative technicians. 51. Statutory meetings of the project team will be held every two weeks to follow the schedule and deadlines, address operational issues, discuss strategies, and make any adjustments to the work plan if necessary. The team will set up a project monitoring tool to ensure that the activities are carried out and that deadlines are met. A data collection mechanism will be put in place to ensure the monitoring of indicators and project performance. In terms of financial monitoring, ENAP’s SAFIRH system will be used within the framework of this project. Financial monitoring and oversight will be carried out in collaboration with the finance department who will ensure the compliance of the project finances with the Canadian and Quebec laws and the requirements of the World Bank. 52. Project team and responsibilities (a) Project manager: Louise Picard (i) Point of contact of ENAP for GEI (ii) Responsible for project management and implementation (iii) Oversight of the project's compliance with the rules of the World Bank, including the aspects regarding financial management, the Environmental and Social Framework (ESF) and procurement (iv) Responsible for monitoring and reporting, including on project progress and social performance, grievance mechanism and inclusion measures of underserved groups (b) ECD Experts: Aref Ben Abdallah, Elodie Roy (i) Conduct studies and needs analysis (ii) Responsible for identifying the capacity development strategies to be deployed (iii) Design, develop and implement ECD activities (iv) Coordinate development and knowledge sharing activities (v) Develop and deploy strategic partnerships (vi) Contribute to data collection and reporting activities (c) Administrative technician: Nathalie Tremblay, Lina Pageau (i) Responsible for logistical and administrative support for project activities (ii) Responsible for managing registrations for training activities and liaising with participants Page 22 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) (d) Accounting technician: Sophie Fortier (i) Under the supervision of the project director, ensure budget monitoring (ii) Enters information into the ENAP accounting system, generates reports (iii) Processes project expenses according to rules and regulations (iv) Liaise with the ENAP finance department B. Results Monitoring and Evaluation 53. ENAP, GEI, and the World Bank Group will monitor the implementation and results of activities supported by each Project component as well as the achievement of PDO level indicators (see Annex 1 M&E framework). 54. There is a systematic approach to the monitoring and evaluation of the project’s activities and results , as described in Component 3 under the Project Components section above. ENAP already has well- established internal mechanisms for the monitoring and evaluation of its activities. These mechanisms will be complemented with additional M&E tools to track the outcome level results and meet the WBG requirements, as well as align project M&E with the M&E framework of GEI. 55. The monitoring and evaluation of ENAP activities and results will be conducted as follows: (a) Reporting on activities and immediate results: In addition to the grant standard requirements and conditions, ENAP will collect, document, and share systematic information on its activities. This will include: (i) Ongoing monitoring at the ‘activity’ level through the GEI MIS – including main objective, type of activity, target audience, dates, duration, implementing partners, and other descriptive parameters as agreed through the MIS. (ii) Feedback on trainings and workshops collected through post-training participant surveys, including parameters on quality, usefulness and increase in knowledge and/or skills collected through standardized feedback forms, and instructor feedback sessions. (b) Measuring outcomes and using evidence to make program decisions: To track outcome level results and impact of the trainings, as well as make informed changes in the program’s content and delivery, ENAP will: (i) Use tracer studies and systematically collate and aggregate feedback survey results to distill and incorporate lessons on pertinent issues such as workshop pedagogy, content, and more into program decision making. (ii) Aggregate and report the results of ENAP’s activities through the GEI MIS. (iii) In addition to self-reported data, the project task team leader will conduct an annual supervision and monitoring mission and prepare an Implementation Status and Results Report (ISR). An Implementation Completion Report (ICR) will be prepared at the closing of the project to assess the achievements of the project against its development objectives. 56. Throughout the project implementation, ENAP will ensure that its internal M&E activities align with the overall GEI M&E Framework and provide data through the GEI MIS. Page 23 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) C. Sustainability 57. The project will be operated in a perspective of reaching long-term effects with regards to the interest and commitment in ECD of targeted beneficiaries and to leverage the project to produce positive externalities that go beyond the scope of the project. 58. To do so, strategic communication and advocacy actions in favor of a stronger evaluation culture will be deployed. These actions will aim at the target clientele of the project so as to ensure high visibility and generate a lasting enthusiasm for the development of evaluation capacities. Actions will be deployed to communicate its relevance from a perspective of influence and awareness. The development and broadening of strategic partnerships will also anchor the strategic alignment and the expected results of the project with those of other key players, so as to create synergies as well as lever and multiplier effects. Sharing activities with other GEI actors will also help create a climate favorable to the sustainability of this collective project. V. KEY RISKS A. Overall Risk Rating and Explanation of Key Risks 59. Overall risks for this project are low. Activities financed are limited to training, knowledge generation and sharing. VI. APPRAISAL SUMMARY 60. Given the limited scope of this project, it is not possible to estimate economic benefits and costs of this project. 61. Procurement Assessment. Procurement activities will be carried out in accordance with the World Bank’s Procurement Regulations for Borrowers under Investment Policy Financing (Fourth Edition, November 2020) (“Procurement Regulations�), the Guidelines on Preventing and Combating Fraud and Corruption in Projects Financed by IBRD Loans and IDA Credits and Grants’ revised as of July 1, 2016; an d the provisions stipulated in the Procurement Plan for the entire project life, approved on 25 October 2021. The Procurement Plan will be updated at least annually or as required to reflect the actual Project implementation needs. 62. The ENAP will be responsible for all procurement and contracting related queries and processing, including management and compliance with fiduciary requirements. A procurement capacity assessment was carried out. ENAP’s financial department, staffed only with financial management analysts, will offer support and assistance to ensure procurement documents and processes follow adequate standards. ENAP has implemented satisfactorily several major international cooperation projects over the past fifty years. 63. In accordance with the Regulations, a short form Project Procurement Strategy for Development (PPSD) Page 24 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) has been prepared by ENAP with the support from the Bank’s team, for the purpose of identifying the most appropriate procurement approach for the project. This project will be composed by mainly consulting services without major risks identified, and only small value contracts. The overall procurement risk for this project is assessed as moderate since the Project Implementation Unit is not staffed with any procurement specialist and has limited experience with Bank’s procurement rules and procedures. 64. Financial Management Assessment. A Financial Management (FM) capacity assessment of the Recipient of the Bank grant, ENAP, was carried out as required under the Financial Management (FM) Manual for World Bank Investment Project Financing (IPF) Operations, February 2017, and the Guidance Note on Small Recipient - Executed Trust Fund (RETF) Grants, October 2017. ENAP is a branch of the Université du Québec network and is established as a higher education institution to offer university-level courses and conduct research in public administration and training and continuing education of public administrators. The Bank legal team has confirmed that ENAP is an entity with a separate legal personality and with powers to enter into contracts with international organizations, the same as a university, and therefore eligible to receive financing. 65. The objective of the FM assessment was to review ENAP’s financial management, reporting and oversight arrangements and assess if they meet the minimum requirements of financial management and oversight arrangements as per the IPF FM Manual and the Small RETF Grants Guidance Note. The FM capacity assessment of the ENAP was conducted based on the completed FM assessment questionnaire submitted by them. The assessment covered the areas of budgeting, staffing, accounting, financial recording and reporting, funds flow, accountability, internal control and oversight and external audit arrangements. The overall FM risk at project preparation is assessed as “Moderate� – this will be evaluated during project implementation and revised if necessary. 66. As per the project institutional and implementation arrangements described in section IV (A) of this document, ENAP will be the grant recipient and sole implementing entity for the project and will be responsible for the overall management and implementation of the project. There will not be any sub – implementing entities for the project. If, during the project implementation, it is decided that the Bank grant funds will flow from ENAP to any other partner institutions, the Bank may conduct an assessment/due diligence of the FM arrangements of such institutions. The scope of ENAP external audits may also be expanded to include the audit of such funding to other institutions. The Grant Agreement/legal documentation may need to be revised to reflect this and any other related issues. 67. Fund Flow and financial reporting – ENAP will open a segregated Designated Account (DA) in a commercial bank under terms and conditions acceptable to the World Bank, to receive the grant funds for implementing the agreed activities. ENAP will get their Annual Work Program and Budget (AWPB) approved by the Bank before the commencement of their fiscal year. Funds will be disbursed either as reimbursement of expenditure incurred or on an advance basis (if requested), the advance being based on a six months’ projection of expenditures for the agreed activities. ENAP will send to the Bank six – monthly Interim Financial Reports (IFR), within forty-five days after the end of each semester, in the agreed IFR format. The IFRs will reflect the “actual expenditures� incurred on the various components and activities during the reporting period and the funds required for the next six months (after netting off the unspent balances from the previous advance) for implementing the project. Page 25 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) 68. Disbursement and eligible expenditures: There will be a single expenditure category “Consulting services, Non-consulting services, Training and Operating Costs under the project�. The items of expenditure under Training and Operating Costs will be defined in the Grant Agreement. 69. External Audit and Compliance – The Bank Grant to ENAP will be audited annually by an external auditor acceptable to the Bank and under Terms of Reference agreed in advance with the Bank. The auditor will verify that the grant funds have been spent in line with the provisions of the Grant Agreement and the financial management requirements of the Bank, and provide a Management Letter highlighting the significant findings of the audit. The Audit Report and the Management letter will be submitted to the Bank no later than six months after the end of ENAP fiscal year. ENAP will also share their annual entity audit report with the Bank. The option of the current external auditors of ENAP also covering the audit of the Bank grant as part of their annual audit, under agreed Terms of Reference (TORs), can also be considered. ENAP will fully comply with the WB public disclosure policy of audit reports once they are accepted by the Bank. 70. Supervision strategy - The Bank will review the implementation of FM arrangements and FM performance, identify corrective actions if necessary, and monitor fiduciary risks. FM supervision will be conducted mostly via desk reviews with one formal annual supervision mission (virtual or in person depending on pandemic conditions). 71. Environmental and Social assessment. The environmental and social risks from the grant activities are expected to be low. No environmental risk or potential impacts are expected on the biophysical environment, human health and safety, and/or valued environmental components from the project activities. The project activities are not expected to incur significant adverse social impacts and risks given the range and nature of activities anticipated under the grant. A.. Legal Operational Policies . Triggered? Projects on International Waterways OP 7.50 No Projects in Disputed Areas OP 7.60 No . B. Environmental and Social 72. The project is focusing on capacity building in monitoring and evaluation. Its activities are limited to the diagnostic assessment of M&E systems and institutional capacity, design of M&E training materials, generation of knowledge products, delivery of online and onsite training, organization of learning workshops and events as well as technical advisory support to beneficiary institutions in M&E. There are no civil works or equipment procurement involved. 73. The capacity-building efforts will have indirect positive environmental effects by supporting the strengthening, monitoring, and evaluation capacity in the beneficiary countries over natural resources, protected areas, forests, and a range of ecosystems, as well as management of (global) environmental issues such as climate change, pollution, waste, and degradation of natural systems. The project will give priority consideration in the training Page 26 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) programs to potential beneficiaries who represent underserved demographics, geographies, and sectors, including vulnerable disadvantaged and indigenous groups. The project will ensure broad information dissemination to raise awareness on the engagement of the project activities to facilitate broad participation in the training program. 74. The environmental and social risks from the grant activities are expected to be low. No environmental risk or potential impacts are expected on the biophysical environment, human health and safety, and/or valued environmental components from the project activities. The project activities are not expected to incur significant adverse social impacts and risks given the range and nature of activities anticipated under the grant. However, there could be risks associated with the employment conditions of staff by ENAP for carrying out project activities, potential sexual harassment, and gender discrimination among the project delivery team and beneficiary participants in the potential face-to-face project activities as a result of people’s interactions in an office environment. However, these risks are considered low considering that the project interactions are anticipated to remain largely virtual in the current pandemic situation. Key stakeholders involved in the project include ENAP project staff, the trainers of virtual and on-site workshops, as well as the potential participants of the workshops from government agencies, non-governmental organizations (NGOs), think tanks, the private sector, and other agencies from targeted countries. The project will ensure broad information dissemination of its training activities and engagement to facilitate broad participation of potential beneficiaries, particularly the marginalized groups, NGOs, and other institutions who may represent vulnerable segments of society for maximum project development benefits. The project is not expected to incur major stakeholder risks. VII. World Bank Grievance Redress 75. Communities and individuals who believe that they are adversely affected by a World Bank (WB) supported project may submit complaints to existing project-level grievance redress mechanisms or the WB’s Grievance Redress Service (GRS). The GRS ensures that complaints received are promptly reviewed in order to address project-related concerns. Project affected communities and individuals may submit their complaint to the WB’s independent Inspection Panel which determines whether harm occurred, or could occur, as a result of WB non- compliance with its policies and procedures. Complaints may be submitted at any time after concerns have been brought directly to the World Bank's attention, and Bank Management has been given an opportunity to respond. For information on how to submit complaints to the World Bank’s corporate Grievance Redress Service (GRS), please visit http://www.worldbank.org/en/projects-operations/products-and-services/grievance- redress-service. For information on how to submit complaints to the World Bank Inspection Panel, please visit . www.inspectionpanel.org. Page 27 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) VII. RESULTS FRAMEWORK AND MONITORING Results Framework COUNTRY : World ENAP Global Action for Capacity-building in Monitoring and Evaluation Project Development Objectives The project’s specific development objective is to improve monitoring and evaluation, capacity, and use in supported developing countries in the Francophone Africa and MENA regions. In doing so, the project will seek to build partnerships with other ECD providers in these regions to generate synergies. This project aims to contribute to the higher-level development objective of the GEI MDTF, namely, to improve monitoring and evaluation frameworks, capacity, and use in supported developing countries for improved evidence-informed policymaking. Project Development Objective Indicators Unit of Data Source / Responsibility for Indicator Name Corporate Baseline End Target Frequency Measure Methodology Data Collection Name: Percentage of Percentag 50.00 85.00 Data collection after Online survey ENAP - Activity participants that provide e each training leader positive feedback on training relevance, quality and knowledge acquired Description: This indicator will monitor the participants level of satisfaction regarding the relevance, quality and level of knowledge acquired in ENAP’s tra inings to ensure it maintains a high level throughout the project. This will be determined based on the participant’s perception, gathered through an online survey that participants will be Page 28 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Unit of Data Source / Responsibility for Indicator Name Corporate Baseline End Target Frequency Measure Methodology Data Collection asked to fill after completing the training. 'Percentage of participants' is the number of individuals who provide positive feedback (i.e., choose a rating of 3 or higher on a 5 point (Likert) rating scale), divided by the total number of individuals who answered the survey. Name: Number of ECD Number 0.00 3.00 Project tracking tool ENAP - Project service requests coming manager to ENAP as a result of training activities with client countries Description: GEI is looking at significantly scaling-up support to governments and other institutions around the world to equip them with adequate M&E systems and capacities. It is expected that the project will stimulate ECD interest in the regions of intervention during project implementation. This indicator will provide information to this effect. ‘ECD’ means Evaluation Capacity Development and can encompass a range of different services (e.g. needs asses sment, trainings, advisory services). Name: Expanded Number 0.00 5.00 Annual Project tracking tool ENAP institutional, thematic, or geographic coverage through new or existing partnerships Description: GEI’s vision is to work in partnerships with others to provide high quality M&E systems development support to governments and other target groups around the world. This indicator will measure the number of actors engaged in GEI’s multi -stakeholder effort as a result of ENAP’s activities and the capacity of the project to mobilize strategic partners who will play a role in the sustainability of the project in the long term. This indicator will count new or expanded partnerships involved in ENAP’s ECD activities during project implementation. Page 29 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Intermediate Results Indicators Unit of Data Source / Responsibility for Indicator Name Corporate Measur Baseline End Target Frequency Methodology Data Collection e Name: Number of Number 0.00 9.00 Annual. Project tracking tool ENAP project training courses and/or manager workshops delivered by ENAP Description: This indicator tracks the number of training courses that will be delivered during project implementation. Training courses are sessions in which participants are trained according to learning objectives. It can occur through long-term or short-term technical courses, seminars, workshops, presence and/or online courses. Name: Total number of Number 0.00 225.00 Collected after every Participants lists. ENAP project people who received training delivered. manager. training (disaggregated by gender and country) Description: This indicator will be used to monitor the scope of ENAP’s training activities. ‘People who received training’ include those who have participate d in a training course delivered throughout the project, such as long-term or short-term technical courses, seminars, workshops, presence and/or online courses. An individual who receives training multiple times (e.g., participates in different training courses) can be counted multiple times for this indicator. Name: Number of Number 0.00 3.00 Annual. Project monitoring tool. ENAP project requests for further manager. engagement [e.g. trainings, advisory, workshops etc.] received by ENAP as a result of activities carried out in MENA and/or West Africa Page 30 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Unit of Data Source / Responsibility for Indicator Name Corporate Measur Baseline End Target Frequency Methodology Data Collection e Description: This indicator provides information on the interest of the supported organizations to continue building capacity after benefiting from training or support service offered by ENAP. The indicator will document the number of requests addressed to ENAP for the entire project for additional support to those initially planned. ‘ECD’ means Evaluation Capacity Development and can encompass a range of different services (e.g. needs assessment, trainings , advisory services). Name: Number of Number 0.00 400.00 After every event Participants lists. ENAP activity individuals attending delivered. leaders. knowledge sharing events (disaggregated by gender/country/region) Description: This indicator will be used to monitor the scale and reach of ENAP’s knowledge sharing events. ‘Knowledge sharing events’ are activities through which knowledge (namely, information, skills, or expertise) is exchanged or shared. Events such as panels, round tables, conferences, informative webinars, etc. will be considered. ‘Attending’ means that an individual has participated to some extent in the event. The individual may or may not have attended in full. An individual who participates in more than one event can be counted multiple times for this indicator. Name: Number of Number 0.00 20.00 Annual. Project monitoring tool. ENAP project analytical, technical, and manager. communication knowledge products created by ENAP Description: This indicator captures the amount of knowledge products created by ENAP and made possible with funding from this grant. ‘Knowledge products’ could include, but not be limited to: articles, studies, assessments, videos, infographics, reference documents, practical guides. Page 31 of 35 The World Bank ENAP Global Action for Capacity-building in Monitoring and Evaluation (P177237) Target Values Project Development Objective Indicators FY RESULT_FRAME_TBL_ PD O Indicator Name Baseline End Target Percentage of participants that provide positive feedback on training relevance, quality 50.00 85.00 and knowledge acquired Number of ECD service requests coming to ENAP as a result of training activities with 0.00 3.00 client countries Expanded institutional, thematic, or geographic coverage through new or existing 0.00 5.00 partnerships Intermediate Results Indicators FY RESULT_FRAME_TBL_ IO Indicator Name End Target Number of training courses and/or workshops delivered by ENAP 9.00 Total number of people who received training (disaggregated by gender and country) 225.00 Number of requests for further engagement [e.g. trainings, advisory, workshops etc.] received by ENAP as a result of activities carried out in MENA and/or West Africa 3.00 Number of individuals attending knowledge sharing events (disaggregated by 400.00 gender/country/region) Number of analytical, technical, and communication knowledge products created by ENAP 20.00 Page 32 of 35