Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR127371 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF 10846, TF 12036 and TF A-2289 ON GRANTS IN THE AMOUNT OF US$ 10.449 MILLION TO THE EAST AFRICAN COMMUNITY AND NEW PARTNERSHIP FOR AFRICA’S DEVELOPMENT AGENCY FOR THE AFCC2/RI-AFRICAN MEDICINES REGULATORY HARMONIZATION PROJECT ( P128332 ) June 29, 2018 Health, Nutrition & Population Global Practice Africa Region Click here to enter text. CURRENCY EQUIVALENTS Exchange Rate Effective November 9, 2017 FISCAL YEAR July 1 - June 30 Regional Vice President: Makhtar Diop Country Director: Paul Noumba Um Senior Global Practice Director: Timothy Grant Evans Practice Manager: Magnus Lindelow Task Team Leader(s): Apollo Muhairwe ICR Main Contributor: Yvonne Nkrumah, Lea Philippe Kagan ABBREVIATIONS AND ACRONYMS AF Additional Financing AM Aide Memoire AMRH African Medicine Regulatory Harmonization AU African Union BE Bioequivalence BMGF Bill & Melinda Gates Foundation CDC Center for Disease Control CHAI Clinton Health Access Initiative CTD Common Technical Document EAC East African Community ECOWAS Economic Community of West African States FM Financial Management GCG Global Cooperation Group GMP Good Manufacturing Practices ICH International Conference of Harmonization ICR Implementation Completion Report ICT Information and Communication Technology IDA International Development Association IMS Information Management System ISR Implementation Status and Results Report JA Joint Assessment MER Medicine Evaluation & Registration MDTF Multi Donor Trust Fund MTR Mid-Term Review NEPAD New Partnership for Africa’s Development NMRA National Medicine Regulatory Agency NMRO National Medicine Regulatory Officer NPO National Professional Officer PAD Project Appraisal Document PCN Project Concept Note PMPA Pharmaceutical Manufacturing Plan for Africa PPB Pharmacy & Poisons Board PS Partner States PV Pharmacovigilance QMS Quality Management System REC Regional Economic Community RF Result Framework RIAS Regional Integration Assistance Strategy RTT Regional Technical Team RTWG Regional Technical Working Group SADC Southern African Development Community SC Steering Committee SOP Standard Operating Procedure TFDA Tanzania Food & Drug Administration TMEA Trade Mark East Africa TWG Technical Working Group UHC Universal Health Coverage USAID United States Agency for International Development WHO World Health Organization TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 10 II. OUTCOME .................................................................................................................... 12 A. RELEVANCE OF PDOs ............................................................................................................ 12 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 14 C. EFFICIENCY ........................................................................................................................... 20 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 22 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 22 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23 A. KEY FACTORS DURING PREPARATION ................................................................................... 23 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 24 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 26 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 26 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 29 C. BANK PERFORMANCE ........................................................................................................... 29 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 31 V. LESSONS AND RECOMMENDATIONS ............................................................................. 32 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 34 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40 ANNEX 3. PROJECT COSTBY COMPONENT ............................................................................ 42 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 43 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 45 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 46 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) DATASHEET BASIC INFORMATION Product Information Project ID Project Name AFCC2/RI-African Medicines Regulatory Harmonization P128332 Project Country Financing Instrument Africa Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Related Projects Relationship Project Approval Product Line Additional Financing P155163-Africa 21-Apr-2016 Recipient Executed Activities Medicine Regulatory Harmonization Project Organizations Borrower Implementing Agency East African Community NEPAD, EAC Secretariat Project Development Objective (PDO) Original PDO To harmonize medicines registration systems and to improve efficiency and enhance transparency in medicines registration among the East African Community Partner States. Page 1 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 5,549,200 5,549,200 5,549,200 TF-10846 1,500,000 1,500,000 1,500,000 TF-12036 3,400,000 3,400,000 3,151,184 TF-A2289 Total 10,449,200 10,449,200 10,200,384 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 10,449,200 10,449,200 10,200,384 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 21-Mar-2011 13-Jul-2012 16-May-2014 30-Jun-2017 29-Dec-2017 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 31-Dec-2014 4.63 Other Change(s) 11-May-2017 8.85 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Unsatisfactory Modest Page 2 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 14-Oct-2012 Satisfactory Satisfactory 0 02 17-May-2013 Satisfactory Moderately Satisfactory .50 03 30-Dec-2013 Satisfactory Moderately Satisfactory 1.49 04 18-May-2014 Satisfactory Satisfactory 2.53 05 25-Nov-2014 Satisfactory Satisfactory 3.43 06 11-May-2015 Satisfactory Satisfactory 4.28 07 01-Dec-2015 Moderately Satisfactory Moderately Satisfactory 5.44 08 13-Jun-2016 Moderately Satisfactory Moderately Satisfactory 5.55 09 29-Dec-2016 Moderately Satisfactory Moderately Satisfactory 7.16 10 30-Jun-2017 Moderately Satisfactory Moderately Unsatisfactory 9.11 11 28-Dec-2017 Moderately Satisfactory Moderately Satisfactory 10.20 SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 100 Health 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100 Health Systems and Policies 100 Health System Strengthening 100 Page 3 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) ADM STAFF Role At Approval At ICR Regional Vice President: Obiageli Katryn Ezekwesili Makhtar Diop Country Director: Yusupha B. Crookes Paul Noumba Um Senior Global Practice Director: Ritva S. Reinikka Timothy Grant Evans Practice Manager: Olusoji O. Adeyi Magnus Lindelow Task Team Leader(s): Gandham N.V. Ramana Apollo Muhairwe ICR Contributing Author: Yvonne Nkrumah Page 4 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Access to medicines is a tremendous challenge to global public health. According to the WHO World Medicines Situation Report (2011), one third of the world’s population lacks access to essential medicines. A significant portion of this population lives in Low and Middle-Income Countries (LMICs), especially in Africa. For these countries to attain Universal Health Coverage (UHC) and meet the SDG (3.8)1, increased investments are critical not only to ensure that medicines are available and affordable, but also that they are safe, effective, and of good quality. 2. Health sector challenges: Strong governance of the pharmaceutical sector and effective, independent, and transparent regulatory systems provide the necessary foundation for greater access to medicines. Each country is obligated to regulate the pharmaceutical products sold on its market in order to assure the safety, quality, and efficacy of these products. However, a National Medicine Regulatory Agency (NMRA) situational assessment conducted by WHO and EAC in 20092 showed that countries in the East Africa Region3 lacked sufficient regulatory capacity to approve medicines for sale on their respective markets in a timely manner. Weak regulatory capacity in the region combined with lack of standardization and non-transparent processes for medicines registration affects availability of essential medicines and vaccines for high burden diseases in the East African Community (EAC). In 2012, East Africa had a high burden of disease, mainly caused by communicable diseases, such as HIV, TB and lower respiratory infections. With nearly 133.5 million inhabitants in the region at the time, treatment of communicable diseases—of which access to medicines is essential— was a major priority. 3. Global regulatory context: In 1990, harmonization of medicines registration requirements became a global trend through the International Conference of Harmonization of Technical Requirements for the Registration of Pharmaceuticals for Human Use (ICH). This has led to the convergence of technical standards and risk assessment concepts for medicine registration between ICH countries, including the USA, Europe and Japan. The ICH Global Cooperation Group (GCG)4 facilitates effective engagement and coordination among different regional medicine regulatory harmonization initiatives. The EAC Treaty of 1999, as amended in 2007, established the Partner states medicines regulatory harmonization as a goal. To advance this goal and increase access to medicines, the EAC Council of Ministers endorsed a comprehensive EAC regional proposal for medicine regulatory harmonization. The aim was to strengthen institutional capacity and adhere to standardized norms for medicines regulatory harmonization in the region. In 2010, the ICH formally accepted the EAC as a member of the GCG in recognition of its efforts to promote medicine regulatory harmonization. 1SDG 3.8: Achieve universal health coverage including financial risk protection, access to quality to essential healthcare services and access to safe, effective, quality and affordable medicines and vaccines for all. 2A situational analysis of NMRAs of Member States was performed by experts from EAC with support of WHIO between 27th July and 28th August 2009 to review the capacity in terms of medicine policy, legal and regulatory framework, procurement, distribution and management within the EAC 3 Republics of Burundi, Kenya, Rwanda, Uganda and the United Republic of Tanzania. 4ICH Global Cooperation Group was formed in 1999 as a subcommittee of the ICH Steering Committee. Its objective is to make information available on ICH activities and guidelines to the public. The GCG created a set of principles intended to guide its activities as it answers request for information and response to non-ICH regulators and industry. For more on the GCG, see Link Page 5 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 4. Sectoral and institutional context: The EAC has 6 National Medicines Regulatory Agencies (NMRAs)— the Tanzania Food and Drugs Authority (Tanzania, Mainland); Zanzibar Food and Drugs Board (Tanzania, Zanzibar); Pharmacy and Poisons Board (Kenya); Ministry of Public Health (Burundi); Ministry of Health (Rwanda); and National Drug Authority (Uganda)—whose institutional capacity, infrastructure, and medicine registration requirements vary widely. For instance, manufacturers must fill multiple and different forms in each country as they seek to obtain licenses to market new products. In addition, Rwanda and Burundi perform very limited regulatory functions due to a lack of human resources, capacity, and legislative instruments. In the EAC region, these regulatory shortcomings have resulted in: (i) significant delays in the registration and distribution of essential medicines; (ii) poor growth of the local pharmaceutical industry, which is important for improving access to essential medicines; and (iii) high risk of poor quality medicines moving across the EAC Partner States. To address these challenges, several development partners are collaborating with NMRAs in the region to strengthen the various areas of regulatory functions. At the continental level, the African Medicines Regulatory Harmonization (AMRH)5 initiative was launched in 2009 to establish and improve standards and requirements related to the regulation of and access to safe, high-quality medicines for the African population. Under the AMRH initiative, the World Bank created a Multi Donor Trust Fund (MDTF071682) to support regional economic communities (RECs) to improve regulatory harmonization. Recognizing the alignment of the AMRH initiative with the EAC Treaty of 1999, the EAC Partner States—through the EAC Secretariat and with technical support from the New Partnership for Africa’s Development (NEPAD) and WHO—prepared the first regional MRH proposal to receive a grant under the AMRH. 5. Link to Higher level objectives. The grant—the EAC Medicine Regulatory Harmonization Project (the Project)—was aligned with the World Bank’s “Africa Regional Strategy�6 to enhance governance and public- sector capacity to improve systems for delivering basic health services. Medicines regulation is a core public health function. Improved governance and harmonization of medicine registration requirements with international standards will improve access to medicines and contribute to improved health outcomes. In addition, the project was fully consistent with the following regional strategies: (i) the Regional Integration Assistance Strategy (RIAS)7, specifically Pillar III on Coordinated Interventions to provide Regional Public Goods; (ii) the EAC Fourth Strategic Development Plan (2011-2015), which prioritized improving access to safe, effective, affordable and quality medicines (Section 7.4); and (iii) the NEPAD’s AMRH Strategic Plan (2011- 2015). At the global level, the project contributed to the achievement of SDG 3, which aims to provide access to safe, effective, quality and affordable essential medicines and vaccines – which is important for attaining UHC. 6. The rationale for the Bank’s engagement in the AMRH initiative was in alignment with two areas of the Bank’s global strategies: 1) the 2007 Health Nutrition and Population Strategy which focuses on governance, accountability, and transparency; and 2) the orientation of IDA-16 to support initiatives that foster regional integration. Furthermore, the MDTF (TF071682) supported the effort to facilitate regional medicines 5The AMRH Consortium consists of the New Partnership for African Development, (NEPAD) Coordinating Agency, World Health Organization (WHO), Pan African Parliament (PAP), Bill and Melinda Gates Foundation (BGMF), UK Department for International Development (DFID), Clinton Health Access Initiative (CHAI), and Swiss Development Cooperation, US Government. 6 The Africa Regional Strategy (2011-2021) was developed by the World Bank to support Africa’s transformation and provides the framework in which to embed country strategies. 7The RIAS provides a strategic framework to guide the World Bank’s assistance in support of regional integration and regional programs in Sub-Saharan Africa (SSA). Page 6 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) harmonization in 2 to 3 Regional Economic Communities (RECs) in Africa, with the EAC being the first. The World Bank was uniquely placed not only to facilitate the AMRH initiative, but also to manage the MDTF for the following reasons: (i) strong convening power; (ii) comparative advantage in fiduciary oversight; (iii) complementarity with existing engagement at country and regional levels; and (iv) ability to develop and scale- up standardized approaches for public sector reforms to improve services for the poor. Project Development Objectives (PDOs) 7. To harmonize medicines registration systems and to improve efficiency and enhance transparency in medicines registration among East African Community Partner States. Key Expected Outcomes and Outcome Indicators 8. The key expected outcomes were: harmonized medicines registration systems; improved efficiency in medicines registration; and enhanced transparency in medicines registration among EAC Partner States. The following four PDO indicators were used to measure project performance: (i) NMRAs participating in the harmonized medicine registration based on internationally recognized policies and standards; (ii) NMRAs piloting electronic submission of applications for registering medicines; (iii) NMRAs sharing regulatory policies, legislation, guidelines and information on registered medicines on their websites; (iv) NMRAs receiving International Standards Organization (ISO-9001) certification on quality management system. In addition, there were four intermediate outcome indicators: (i) Harmonized guidelines, including SOPs and manuals for registration of medicines and GMP inspection developed based on internationally recognized policies and standards; (ii) NMRAs operationalizing a common information management system for medicine registration developed by the regional technical working group; (iii) Staff trained in medicine regulation; and (iv) NMRAs implementing quality management systems. An additional intermediate indicator on developing a comprehensive strategy for strengthening pharmacovigilance in EAC Partner States was added during Additional Financing. 9. The Theory of Change presented in Figure 1 illustrates how the project’s activities and related outputs were to contribute to the expected outcomes. Page 7 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) Theory of Change INPUTS/ ACTIVITIES OUTPUTS OUTCOME IMPACT Establish Technical Working Groups CTD/Common documents for Establish Steering Committee medicines registration and GMP developed and approved by SC Develop guidelines for medicines registration and CTD, and Good Joint assessments of MER &GMP Manufacturing Practices inspections conducted Regional Pharmacovigilance Conduct joint assessments of business plan developed regional medicines applications and Joint GMP/ audit Feedback from stakeholders inspections reflected in common documents Harmonized Medicines Conduct a PV gap analysis Registration systems Develop a PV strategy Consult QMS compendium developed Improved availability of key stakeholders on new quality essential medicines in guidelines NMRAs ISO 9001 certified EAC Improved Efficiency in Medicines Registration Develop compendium for Increased staff capacity on Systems Quality Management Systems medicine regulation (QMS, GMP, Conduct external audit for ISO IMS, etc) certification NMRAs receiving medicines Enhanced Transparency Build staff capacity on medicine applications electronically in Medicines regulation (QMS, IMS, GMP, Registration Systems Bioequivalence etc) and project NMRAs sharing relevant public management information (e.g. policies, regulations) Develop NMRAs websites Procure and install ICT NMRAs sharing regulatory equipment for IMS information and decisions (e.g. confidential information) Figure 1: Theory of Change Components 10. The original project was supported by a grant of US$5.549 million to EAC and US$1 million to NEPAD. The project consisted of two components. 11. Component 1 - regional coordination and capacity building for medicines regulatory harmonization (US$2.40 million) to support activities of harmonization at regional level and establishment of a regional steering committee for direction and oversight and a project coordination team. The core activities supported by Component 1 included: (i) creating Technical Working Groups(TWGs) for each regulatory function; (ii) developing standardized guidelines and a Common Technical Document (CTD) for medicines registration and GMP by the regional technical working groups; (iv) establishing a Steering Committee; (iv) conducting a Pharmacovigilance (PV) gap analysis and developing a regional PV strategy; (v) stakeholder consultations on standardized guidelines; and (vi) capacity building of NMRA staff. Page 8 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 12. Component 2 - institutional development of national medicines regulatory authorities (US$3.13 million) to support NMRAs capacity building and activities at national level. Although the activities to be implemented by NEPAD (mainly regional coordination and advocacy) were mentioned as part of Component I, funding for these activities was not allocated from the estimated cost of US$ 2.40 million for this component. Rather, NEPAD was financed (US $1 million – TF12036) by the MDTF under a separate grant agreement signed in September 28, 2012.Component 2 supported activities at the NMRA/country level covering costs of personnel, in-country workshops and consultations, consultant support, supply of ICT equipment and software, and development of QMS and IMS, including development/upgrading of NMRAs websites. 13. To achieve the PDO, the project was to implement the above activities to improve the following medicine regulatory functions at both national and regional levels by standardizing regulatory requirements and enhancing institutional capacity for implementation: (i) Medicines Evaluation and Registration (MER): Standardization of medicine registration requirements - guidelines, including standard operating procedures (SOPs) and manual for medicines registration based on internationally recognized policies and standards. (ii) Good Manufacturing Practice (GMP): Standardization of requirements for GMP, which is the minimum standard that medicine manufacturers must meet in their production processes. Typically, as part of medicine registration, a NMRA inspects the manufacturing site of the product submitted for marketing authorization to verify compliance with GMP standards. (iii) Quality Management Systems (QMS): Development and utilization of harmonized Quality Manuals and SOPs, and ISO certification. (iv) Information Management Systems (IMS): Establishing and operationalizing common information systems for medicines registration. (v) Pharmacovigilance(PV): Development of a strategy to strengthen PV in the region (added during Additional Financing) 14. In terms of the responsibilities of the implementing partners, at the regional level, the EAC Secretariat was responsible for implementing the project through a dedicated Project Coordination Team. NEPAD was responsible for advocacy and engaging with EAC senior policy makers and ministers, as well as facilitating coordination across the regional economic blocks. At the national level, the NMRAs were responsible for project implementation in their respective countries. Each lead NMRA constituted a TWG for which it was responsible. For example, the Tanzania NMRA was the lead for the TWG on Medicines Evaluation and Registration. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 15. The PDO remained the same. Page 9 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) Revised PDO Indicators 16. A new activity on Pharmacovigilance (PV) and a corresponding intermediate result indicator were added as part of the Additional Financing project. The new indicator reads as follows: A comprehensive strategy for strengthening PV in EAC Partners States is prepared. Revised Components 17. The original project was intended to be implemented over 5 years. However, a 3-year project was approved due to a limitation of resources in the MDTF. For this reason, and others discussed later, in 2016, Additional Financing of US$3.9million was approved ($3.4 million to EAC and $0.5 million added to NEPAD grant under TF12036) for the project to complete activities and scale-up successful interventions. Although neither of the project components was revised, a new activity on PV was added to Component 1. The aim of this new activity was to develop a strategy to strengthen PV in the EAC region to support efforts to enhance patient safety – a key regulatory function which was not included in the project initially. At the time of AF, the project had supported the following activities: (i) establishment and operation of the Steering Committee; (ii) development of standardized protocols for medicines registration, GMP and QMS; (iii) delivery of a series of capacity building events for the staff of the EAC and NMRAs in medicines registration; and(iv) joint assessment of 5 market authorization applications. Other Changes 18. Two level II restructurings were approved, in December 2014 and May 2017 respectively, as well as an Additional Financing (in 2016) for the project. Further details are provided below. 19. First restructuring: A level II restructuring was approved in December 2014 to extend the Closing Date of the project from December 31, 2014 to June 30, 2016. The restructuring was necessary to prevent the interruption of implementation while the requests from EAC and NEPAD for Additional Financing of the second phase of the project were being considered by the Bank. 20. Additional Financing (AF): An AF in the amount of US$3.4 million and US$0.5 million was granted to the EAC and NEPAD respectively, bringing the total project funds to US$ 10.45 million: $ 1.924,936 was added to Component 1, bringing the total estimated cost to approximately $4.48 million while$1.475, 064 was added to Component 2, increasing its total estimated cost to $4.96 million. The NEPAD Grant (TF12036) increased to $1.5 million. The new disbursement estimates were modified accordingly. The AF supported the EAC to: (i) complete activities being implemented; (ii) scale-up successful interventions in the EAC Partner States to maximize development impact and results; and (iii) sustain NEPAD’s technical assistance with a focus on political advocacy and enhancing coordination among partners and stakeholders. 21. Second restructuring: A second level II restructuring was approved in May 2017 to extend the Closing Date from June 30, 2017 to December 29, 2017. This was due to multiple factors including implementation delays between the end of activities under the original project and the start of the Additional Financing and political issues such as: (i) the inclusion of South Sudan as a member of the EAC regional economic community in 2016, requiring some adjustments in activities; and (ii) political tensions in Burundi, which slowed down the implementation of the activities requiring participation from all the countries. The inclusion of South Sudan in Page 10 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) the EAC, however, did not significantly impact the cost of project. The project made a conscious effort upon the accession of South Sudan to the EAC to incorporate the country into the project where feasible to allow them to learn from the other countries and build capacity. The inclusion of South Sudan was only for regional and already planned activities. For example, a representative of South Sudan attended the last supervision mission in November 2017 and the Steering Committee held in December 2017. For these reasons, the ICR will not include reference to South Sudan. 22. In addition to updating the RF to include an indicator for PV, the target dates for indicators were changed to align with the new revised Closing Date of December 29, 2017. Rationale for Changes and Their Implication on the Original Theory of Change 23. As noted earlier, the first level II restructuring was necessary to ensure that ongoing activities would not be interrupted while waiting to get the AF approved. The second level II restructuring, on the other hand, was to complete remaining activities that had been delayed due to political tensions in Burundi. As explained previously, the AF was consistent with the initial plan to extend the project upon successful completion of the first phase of the project and availability of funds. Adding the PV activity was very relevant as it is a fundamental part of a well-functioning medicines regulatory system to ensure the monitoring of medicines and patient safety. Furthermore, the inclusion of PV to the project was important given that a 2009 NMRA situational study showed that the EAC countries significantly lacked capacity in this key area. The above changes were not expected to have any implication on the theory of change as the PDO and the project components essentially remained the same. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 24. Relevance of PDO is rated as High. 25. The project is consistent with Pillar 2 of the World Bank’s ten -year Africa Strategy 2011: “vulnerability and resilience�, addressing the high risk of idiosyncratic shocks, including health system reforms. Governance and public-sector capacity are the cornerstones of the Africa Strategy (World Bank, 2011). 26. At the (Africa) regional level, the project remains aligned with the agenda set at the continental level as it contributes to promoting access to medicines and developing a scientific and regulatory framework in Africa by building stronger national medicines regulatory authorities. For instance, the African Union Executive Council Decision {EX.CL/Dec.857 (XXVI} in 2015 endorsed the milestones for the establishment of a single medicines regulatory agency in Africa within the context of the AMRH initiative and the Pharmaceutical Manufacturing Plan for Africa framework (PMPA). The AMRH initiative, by aligning with the African medicines regulations, helped to facilitate the implementation of the continental free trade agreement launched in March 2017 that enables free movement of goods, including pharmaceuticals. Page 11 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 27. Similarly, at the EAC level, the project is aligned with the EAC Vision 2050 focusing on access to and distribution of medicines through harmonized health and health-related legislation and regulations. A Regional Pharmaceutical Manufacturing Plan of Action (2017-2027) was approved by the Council of Ministers in 2017 to advance access to medicines goals and to enhance inclusive growth in the health sector. The project is consistent with this plan as developing the local industry by strengthening the regulatory capacity was a key aspect envisaged to improve access to medicines in the region. Furthermore, the project objectives contributed to joint actions towards prevention and control of diseases implemented under the EAC common market by ensuring a strong collaboration across national medicines regulatory agencies to accelerate the availability of good quality medicines in the region. 28. At the national level, all country health sector strategies also support improving access to essential medicines. As an example, the Big Four agenda8 announced by the Kenyan government in February 2018 focuses on universal health coverage and the development of the local pharmaceutical industry. In Rwanda, the Medical and Pharmaceutical laboratory, which ensures the quality of imported and locally manufactured medicines, is a key component of the 2015 Health Sector Policy. The project supports both priorities by ensuring better regulation of medicines marketed in EAC and enhancing the capacity of local industry through robust regulation. 29. Finally, the project contributes to the development of the main regulatory functions recommended by the WHO as a priority for strengthening national medicines regulatory authorities, namely: (i) product registration; (ii) licensing of manufacturing, importation, and distribution and (iii) control of medicine promotion and information. Therefore, the project’s focus on harmonizing regulatory requirements for medicines registration remains highly relevant in the context of improving access to quality, efficacious, and safe medicines in East Africa. 8The Big Four Agenda was announced by the Kenyan government on February 2018 and includes 4 priorities to implement by 2022: (i) boosting food security, (ii) rolling out affordable healthcare, (iii) creating 500,000 jobs and (iv) building low-cost housing. Page 12 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 30. Achievement of PDO/ Efficacy is rated– Modest. The assessment of the project PDO for this ICR is unpacked into the following 3 objectives: • To harmonize medicines registration systems among the EAC Partner states • To improve efficiency in medicines registration among the EAC Partner states • To enhance transparency in medicines registration among the EAC Partner states The discussion below summarizes achievement of the three objectives. Harmonization of medicines registration systems among EAC member states: Rating- Modest 31. The PDO indicator for this objective is: NMRAs participating in the harmonized medicine registration based on internationally recognized policies and standards. The target, as per the RF, is 3 out of 6 NMRAs, which was exceeded. Although this target was exceeded, harmonization of medicine registration systems (i.e. full interoperability of national systems with regional system) was not achieved as explained below. 32. To achieve this objective, the project sought to standardize requirements for medicines registration, including GMP—which at appraisal stage varied across countries—at national and regional levels. At project appraisal there was no regional procedure for medicine registration. In other words, medicines could only be registered at the national level. Recognizing the critical importance of ownership to the success of the regional harmonization process, the project established many Regional Technical Working Groups (TWG) including —MER, GMP and PV—made up of technical experts from all of the NMRAs to standardize medicines registration requirements for both national and regional applications. For PV, the focus of the TWG was to develop a regional PV strategy. WHO provided technical assistance for the development of the harmonized guidelines. In addition to the establishment of the TWGs, a Project Steering Committee (SC) was established to review and adopt the technical guidelines for the various regulatory areas. To strengthen political buy-in at the highest level within government, NEPAD provided political advocacy by developing an AU Model Law on Medical Products Regulation to facilitate the regional harmonization effort. Tanzania, Zanzibar, Kenya, Rwanda, and Burundi either adopted the Model Law or are currently in the process. 33. By Mid-Term Review (2014), a compendium (Guidelines, SOPs, Manual) for MER and GMP each had been developed by the respective TWG and endorsed by the Steering Committee. In 2015, a Common Technical Document (CTD) was approved and adopted for the region. Since then, 5 out of the 6 NMRAs (except Burundi) have, to a large extent, adapted the compendium and all 6 NMRAs are using the CTD. As the CTD is based on the WHO and ICH CTDs, it is easier for manufacturers to leverage the dossiers they use in most international submissions in the EAC countries. Thus, manufacturers no longer must fill multiple forms in each country in the region to obtain licenses to market new products. The project also met its target of developing a PV strategy for strengthening and harmonizing PV systems in the region. The strategy was developed in September 2017, and presented at the 12th Steering Committee meeting held in Uganda from March 6 to March 9, 2018. Page 13 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 34. Despite the progress noted above, the objective of harmonization of medicines registration systems in the EAC region has not been achieved. This is because standardizing medicines registration requirements per se is not an end but a means to an end. In fact, utilization of standardized requirement at both national and regional levels is necessary for harmonization. To elaborate, so far, the project has succeeded in standardizing medicines registration requirements at both national and regional levels. In terms of utilization, at the EAC level, there seems to be full compliance with the regional standards. However, at country level, compliance has only been partially achieved. In interviews with NMRAs, the ICR team noted that countries were at different levels in terms of utilization of the standards for several reasons: (i) lack of human resources (technical expertise); (ii) lack of physical infrastructure for implementation; and (iii) inconsistency in the application of new standards. For example, in Kenya, due to lack of bioequivalence (BE) study centers in the region, the board of the NMRA (PPB) decided to exempt local manufacturers for BE studies. As a result, Kenya is the only country in the region that allows a waiver for bioequivalence (BE) studies for local manufacturers. See Table 1 below for the status of utilization of the standardized guidelines by country. Given the varying regulatory capacity in the region at appraisal stage, the project team should have been realistic in terms of what was feasible regarding harmonization of medicines registration systems in the EAC region. With 3 less resourced NMRAs in the region –Rwanda, Burundi and Zanzibar (based on the 2009 NMRA situational assessment that informed the project) –attaining harmonization within 5 years was ambitious. Despite this caveat, the indicator for this objective (3 out of 6 NMRAs participated in the harmonized medicines registration system), was met. However, the essence of medicine registration harmonization is to have all countries involved adopting and utilizing the standard requirements. Standardizing the requirements for medicines registration would have been a more appropriate indicator. For these reasons, achievement of this objective is rated Modest. Table 1: Utilization of Standardized Guidelines by Country at project closing Regulatory areas Kenya Tanzania Uganda Burundi Zanzibar Rwanda Comments MER Introduction of Yes Yes Yes Yes Yes Yes Common Technical document (CTD) Compliance Yes Yes Partially Yes Yes Yes Uganda requires additional information with regional for some products in order to complete standards for the joint assessments process. joint assessments Bioequivalence No Yes Yes Yes Yes Yes Due to lack of BE study centers in the studies region, the PPB has exempted local required for manufacturers of BE studies. generics at national level GMP Recognition of Partially Partially Partially partially partially Yes Lack of clarity on the relationship joint GMP between a national and regional GMP inspection certification. certificate National GMP inspections are still required. Use of Yes Yes Yes Partially Partially partially Lack of capacity to fully utilize the standardized standardized guidelines. GMP guidelines Page 14 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) QMS Implementatio Partially Yes Partially No Yes No Kenya and Uganda were in the process n of QMS and of getting certified. ISO certification Burundi and Rwanda were not able to implement QMS due to a lack of capacity. Source: BCG End-term Evaluation Report and key informants’ interviews during ICR mission Improve efficiency in medicines registration among the EAC member States: Rating – Modest 35. To improve efficiency in medicines registration at both national and regional levels, the project supported the following activities:(i) introduction of the CTD format for medicines registration; (ii) introduction of a joint assessment (JA) process; (iii) introduction of joint GMP inspections; (iv) electronic submission of applications; (v) establishment of a QMS system; and (vi) staff training in all regulatory areas. Before the project, it took more than two years on average to register medicines at the national level, including the process for inspection of GMP for registered medicines or new applications. As noted earlier, at the regional level, there was no system in place for medicine registration. Hence, this project established a new system for registering medicines at the regional level. The following explains the extent to which these activities have contributed to improving efficiency in medicines registration. 36. As mentioned earlier, the introduction of a CTD format made medicines registration more efficient by standardizing the format across countries in the region. As a result, the timeline for registration reduced significantly from 24 months on average to 10 to 12 months on average9. However, this reduction may not necessarily have been solely due to the CTD format. Other factors to consider are: increased human resource capacity, improved quality of applications, and increased response rate of applicants to queries.10 37. As for the JA process, although the project RF did not provide a target, an interview with TFDA, the lead NMRA for MER, confirmed that an informal target of 18 products registered by year 5 was set. Of the 49 applications submitted for JA, only 9 have been recommended for registration between 2014 and December 2017 (Figure 2). In comparison to the ZaZiBoNa Initiative11 which seeks to harmonize medicines registration systems for 4 countries – Zambia, Zimbabwe, Botswana and Namibia — where 156 products12 were jointly assessed and 90 of them recommended for registration between July 2015 and November 2017 (within a 2 and half year period), the EAC project target of 18 products was less ambitious. Although both initiatives had different approaches, they had similar objectives, were carried out over a similar period, and presumably had similar capacity to review and recommend applications at the regional level. 9 Information sourced from Boston Consulting Group end-term evaluation Report and confirmed by clients during the ICR mission. 10 Information sourced from Boston Consulting Group end-term evaluation. 11 The ZaZiBoNa Initiative promotes a collaboration model to facilitate access to good-quality medicines through work-sharing in assessment of medicines and inspection of medicine manufacturing and testing facilities among 4 countries: Zambia, Zimbabwe, Botswana and Namibia. For more on this initiative, see Link 12 Information sourced from the Boston Consulting Group end-term evaluation Report. Page 15 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 38. Likewise, the RF did not provide a target for national registrations using the harmonized guidelines. However, various aide memoires (AMs) and clients’ progress reports to EAC indicate a target of at least 75 products registered for Kenya, Tanzania and Uganda; and at least 50 products registered for Zanzibar, Rwanda, and Burundi. At project closing, Kenya (1676), Uganda (277), Tanzania (232), and Rwanda (437) surpassed the target. However, Burundi (18) and Zanzibar (42) did not meet the target. Figure 213: Number of Applications in the Joint Assessment Process between 2014 and December 2017 39. Good Manufacturing Practice (GMP) is a crucial element of a robust registration system. Under the project, 14 joint inspections were conducted and 11 certificates issued as part of the harmonization process. These joint inspections were supposed to reduce the number of inspections conducted by each NMRA, resulting in a more efficient system. As there was no target in the RF nor the AMs for joint inspections, it is difficult to determine whether 14 joint inspections are significant in terms of how they contribute to assessing the achievement of this objective (efficiency). The ICR team notes, though, that this is a commendable achievement since there was no system for joint inspections prior to the project. However, during the ICR mission, NMRAs and other stakeholders raised several challenges associated with the joint inspection system which affects efficiency. Key among these is a lack of clarity on the relationship between a national and regional GMP certification. It is not always clear if a regional GMP certification supersedes a national GMP and would therefore waive the requirement for a national GMP certificate to be renewed if a regional GMP certificate is issued prior to the expiration of a national GMP. In practice, however, this is not the case as national and regional GMPs inspections are conducted in parallel, defeating the very purpose of a joint GMP system. Other challenges included inadequate coordination between the MER and GMP audits for product registration; difficulty with scheduling inspections (inspectors not given enough notice, issuance of travel clearance for inspectors, etc.); and the lack of a single point of contact for the payment of GMP certificate fees. 13Information sourced from the Boston Consulting Group End-term Evaluation Report and confirmed by clients during ICR mission. Page 16 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 40. Regarding the electronic submission of applications, all NMRAs started piloting the electronic submission of applications for registering medicines, exceeding the target of 3 NMRAs. Although this indicator is not described in the RF, within the context of medicines registration, electronic submission of applications entails applying through an online platform; whereas application in electronic format involves applying through CD or USB. In the EAC region, NMRAs are currently receiving applications in an electronic format either through their website portals (Uganda, Kenya and Rwanda) or through CDs or USBs (Burundi, Tanzania and Zanzibar). During the ICR mission, most stakeholders applauded this significant milestone in medicines registration, which is contributing to a more efficient system of submission of applications. Linked to electronic submission of applications is an intermediate result indicator for at least 3 countries to have operationalized a common IMS for medicines registration. The aim of the IMS was to facilitate information sharing on medicines registered across NMRAs in order to avoid duplication of efforts in MER and GMP specifically, and the other regulatory areas in general, resulting in improved efficiency. The IMS consists of two components: (i) national systems (NMRAs); and (ii) a regional portal linked to the national systems. 41. At project closing, 4 NMRAs out of 6 (except Burundi and Zanzibar) had either operationalized or upgraded their national IMS to be linked to the regional platform, exceeding the target. The regional portal was developed, but it was not functional at the end of the project. While the target for this indicator was achieved, there is very little (at best) information on medicines registration applications being shared among the NMRAs. Similarly, interoperability remains an issue as IMS in the NMRAs are not the same: the NMRAs in Kenya and Tanzania have different systems from the other NMRAs. Furthermore, there is still a lack of clarity on what kind of information should be shared on medicines registration among the NMRAs and who should have access. While the 10th Steering Committee report gives an indication of information that will be shared among NMRAs through the IMS—including registering of products, registered premises, registered GMP sites, recalls and alerts, Partner States guidelines, laws and policies, client service charters, publications, EAC harmonized technical guidelines and statistical data— this appears to be very comprehensive and would not likely have been achieved within the project period. These issues require policy dialogue by the Council of Ministers to move this agenda forward and ensure cooperation. 42. The implementation of a Quality Management System (QMS) that was supposed to result in ISO certification of NMRAs was also intended to improve efficiency by standardizing and strengthening regulatory and non-regulatory processes in all NMRAs. Only Tanzania and Zanzibar received ISO certification before the end of the project, therefore partially meeting the target of 3 NMRAs. Kenya and Uganda applied for certification and are on track to be certified by June 2018. Uganda became ISO certified in June 2018. Burundi, on the other hand, currently does not have adequate technical and financial capacity to implement a QMS while Rwanda lacks an autonomous regulatory agency. Feedback from the NMRAs during the ICR mission suggests that the establishment of a QMS greatly improved efficiency and quality of work. Specifically, quality of services improved due to a customer survey, better clarity of timelines for every stage of the process for medicines regulation, as well as inputs and outputs for each regulatory function. In sum, a QMS has contributed significantly to improved internal business processes. 43. Finally, the project supported staff training in medicines regulation, which enhanced capacity and contributed to improved efficiency. A total of 233 staff members from NMRAs and the EAC were trained Page 17 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) during the project on areas such as MER, GMP and QMS—surpassing the target of 75 staff set initially. Furthermore, capacity was enhanced through the following: twinning of strong NMRAs with less resourced NMRAs (e.g. Zanzibar mentioned that the twinning activities were very beneficial for the less resourced agencies such as itself); peer-to-peer learning through joint assessments and joint inspections; and cross- country learning through development of guidelines and compendia for different regulatory functions. During the ICR mission, strong improvement in terms of capacity-building for MER and GMP was reported by most of the NMRAs. A WHO Benchmarking study conducted in February 2018 confirmed these improvements, especially for Tanzania, which was rated level 3 on a 4-point scale (with 4 being the highest) of NMRA maturity. Table 2 below summarizes the achievement for the PDO indicators, Intermediate Result Indicators and other indicators used to measure the PDO- 2 objective (efficiency). Table 2: Achievement and rating of PDO-2 (Efficiency) PDO Indicator Intermediate Result Indictor (IRI) Other Baseline Target Achievement Rating of Indicator Efficiency 1) NMRAs participating in the 1) Harmonized guidelines Introduction 0 3 NMRAs 6 NMRAs, Modest harmonized medicines including SOPs and manuals for of the CTD Target registration based on registration of medicines and format exceeded internationally recognized GMP inspection developed based Joint 0 18 products Target not policies and standards on internationally recognized Assessment recommended achieved policies and standards Assessments 0 75 products Target not at national registered by achieved level TZ, KQ and UG by year 5 50 products registered by BR, RW and ZR by year 5 Joint GMP 0 No target set No target set inspections but 14 inspections conducted 2) NMRAs started piloting the No indicator No indicator 0 3 NMRAs 6 NMRAs, electronic submission of Target applications for registering exceeded medicines No PDO indicator 2) NMRAs operationalizing No indicator 0 3 NMRAs 4 NMRAs, common information Target management system for medicine exceeded registration developed by the regional technical working group No PDO indicator 3) Staff trained in medicine No indicator 0 75 staff 233 staff, regulation Target exceeded 4) NMRAs received ISO 9001 4) NMRAs implementing quality No indicator 0 3 NMRAs 2 NMRAs certification on quality management system Target management systems partially achieved Informal targets (not in the RF) are italicized and bolded. TZ - Tanzania; KQ -Kenya; UG – Uganda; BR -Burundi; RW – Rwanda; and ZR -Zanzibar. Page 18 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) Enhance transparency in medicines registration among the EAC member states: Rating- Substantial 44. The last of the 3 project objectives was to enhance transparency in medicines registration. The PDO indicator for this objective is: NMRAs sharing regulatory policies, legislation, guidelines and information on registered medicines on their website. The target, as per the RF, is 3 out of 6 NMRAs, which was exceeded. 45. To enhance transparency towards the public and among NMRAs, the project supported the development of a regional website and national websites for each NMRA for sharing their respective policies, legislation, guidelines, etc. In assessing this objective, the ICR authors applied the relevant aspects of the Transparency and Confidentiality section of the WHO Benchmarking Tool for the Review of Drug Regulatory Systems, listed and described in Annex 6. This tool was developed to assess national medicines regulatory systems and it defines transparency as “the degree to which regulatory procedures and decision criteria are made public and communication between the regulatory authority, its clients, and the consumers�. 46. At project closing, a regional website (EAC MRH) had been developed (www.mrh.eac.int). Four NMRAs (Tanzania, Uganda, Kenya and Zanzibar) had either developed or updated their national websites and have links to the EAC MRH website. In the case of Rwanda and Burundi, websites can be accessed through their respective ministries of health. All NMRAs are either sharing most public documents such as regulations and guidelines on their websites or have links to the regional website where such public documents can be found, thus, exceeding the target of 3 NMRAs. 47. Furthermore, efforts were made during the development of standardized guidelines and compendia to involve all stakeholders, including the pharmaceutical sector (local manufacturers, importers, and International Federation of Pharmaceutical Manufacturers and Associations (IFPMA), to provide feedback. For instance, in Kenya, a stakeholder meeting was convened to discuss the implementation of the new guidelines for medicines registration. Tanzania and Zanzibar launched customer satisfaction surveys to improve the quality of services. NMRAs in Kenya, Tanzania and Zanzibar indicated that enhanced transparency has contributed to more accountability and has contributed to increased internally-generated revenue. Justification of Overall Efficacy Rating 48. Of the 4 PDO indicators, 3 were either achieved or surpassed and one was partially achieved. In terms of intermediate result indicators, all 5 were either achieved or surpassed. Despite these achievements, utilization of standardized documents—which was central to effectively harmonizing medicines registration systems in the sub-region—was not consistent across countries. For this reason, as well as limited national capacity, the project was unable to fully achieve the overall project development objective, which focused on harmonization—rather than standardization. Similarly, there were significant shortcomings in efficiency that led to suboptimal results. Thus, the overall efficacy rating is Modest. Page 19 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) Overall PDO To harmonize medicines To improve efficiency To enhance registration systems in medicines transparency in among the EAC member registration among the medicines states EAC member states registration among the EAC member states Modest Modest Modest Substantial C. EFFICIENCY Assessment of Efficiency and Rating - Modest 49. Implementation efficiency is considered Modest. Project implementation was guided by the Operations Manual which included a detailed execution plan. At project closing, 98 percent of the grant had been disbursed. All the activities including IMS, QMS, and the implementation of the joint activities were started as planned. However, overall efficiency was affected by several factors. Implementation of the project had delays of about 14 months which impacted the planned activities and the expected outcomes. They included (i) delays in recruitment of project staff; (ii) delays in disbursement and (iii) delays in approval of AF. The delays in recruitment due to internal processes and negotiations affected efficiency and slowed down the implementation. The first AM in November 2012 indicated that only regional activities had started, with national activities only beginning in April 2013. Delays in disbursement also occurred during the project. For instance, NMRAs annual reports indicated that these delays resulted in most joint activities not being undertaken on schedule. In addition, the EAC did not provide financial support to NMRAs to implement activities in a timely manner. Lastly, the implementation was interrupted for a period of 6 months due to lack of resources during the period when the AF was being processed. The EAC Secretariat was not able to use retroactive financing made available under the AF due to lack of its own resources or funding from other donors. Although the Operations Manual provided terms of reference for the EAC, NMRAs and partners, implementation efficiency was affected by poor coordination among different stakeholders leading to duplication of effort and suboptimal results. For example, the organization of joint assessments encountered significant delays and led to fewer products having been recommended for registration than planned. 50. Regardless of the above bottlenecks, gains made by each NMRA by implementing the different activities that contributed to institutional strengthening improved the overall efficiency. For instance, through IMS implementation, Kenya PPB increased its revenue significantly and could leverage internal and external resources from the government, Trade Mark East Africa (TMEA), Clinton Health Access Initiative (CHAI) and Center for Disease Control (CDC). 51. Cost-benefit: The costs for this project were minimal at US$ 0.06 per person based on an estimated population of 168.8M in the region in 2016 and the total project costs of US$ 10.2M.From an economic point of view, the use of substandard medicines can have serious negative public health externalities, such as the development of medicines resistance, one of the major public health threats. In this case, consuming substandard medicines by single individuals could lead to costs for the entire society in the form of reduced Page 20 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) medicine efficacy and increased risks of death. Thus, from a health systems perspective, effective regulation ensures that already strained resources (public or private) for health are not wasted, and costs for additional medical consultations and hospitalizations are avoided. 52. At the macroeconomic level, the reduction in regulatory barriers and efficiencies through standardized, simplified, and internationally recognized registration processes has been shown to be associated with greater domestic and foreign investments, which, in turn, are expected to create jobs and boost the economic productivity of the country. A recent study by the University of Cambridge (2012) demonstrated that the regional harmonization of the regulatory framework in Eastern Africa brings significant economic benefits to both regulators and pharmaceutical companies. Under a scenario of harmonized medicines regulation, the 5- year costs per product for pharmaceutical companies were estimated to drop from US$ 5.7M to US$ 1.7M, while the 5-year revenue for the regulator was to increase from US$72,800 to US$ 117,200. Furthermore, assuming a 50 percent increase in the number of medicines registered in the region because of the improved registration process, the regulator’s 5-year revenue is expected to increase from US$ 52.2 million to US$ 140.6 million. D. JUSTIFICATION OF OVERALL OUTCOME RATING Overall Rating Relevance Efficacy Efficiency Moderately High Modest Modest Unsatisfactory E. OTHEROUTCOMESANDIMPACTS (IFANY) Gender 53. The design of the project did not specifically consider the project’s direct impact on gender. Both the PAD and AF documents name the following as the key beneficiaries of the project: NMRAs, Ministries of Health, and citizens of East African Community Partner States. For this reason, there is no data available measuring positive or negative impact on gender. The ICR authors note, however, that many female technical staff from the NMRAs and EAC benefitted from the numerous capacity building activities supported by the project to strengthen institutional capacity. 32 percent of staff trained in various aspects of medicines regulation were female. Furthermore, medicines for reproductive health and for children were considered in the call for Expression of Interest for applicants. Institutional Strengthening 54. Institutional strengthening was at the core of the project as the two components specifically focused on this aspect. The 6 NMRAs benefitted from capacity building in various aspects of medicine regulation. At the institutional level, capacity of all 6 NMRAs was enhanced in medicines evaluation and registration, GMP, QMS and IMS—all of which contributed to improved efficiency in the discharge of their regulatory functions. For Page 21 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) example, Kenya reported that the project contributed to systems improvement as PPB systems were upgraded as part of IMS; QMS and systems audits resulted in improved transparency and accountability; MER enhanced quality of medicine assessments and reduced the time taken to assess products at national level from 24 to 10 - 12 months. Furthermore, the project successfully helped Zanzibar transition from the Food, Drug, and Cosmetic Board under ministry supervision, to a semi-autonomous Zanzibar Food and Drug Agency. The project, which included formal trainings and a twinning program with Kenyan PPB, also helped the newly established ZFDA in building solid capabilities in a short time, adopting more advanced regulatory guidelines as well as becoming ISO certified. 55. In addition to the above, staff capacity in all NMRAs was increased as those who trained as Assessors in medicines registration, Inspectors for GMP, and Lead Auditors for QMS compliance and implementation of quality management systems subsequently served as trainers in their areas of expertise in all the respective NMRAs. All NMRAs indicated that the development of the compendia for GMP, MER, and GMP helped expand their capacity as they learned from more skilled and qualified colleagues from other NMRAs and technical experts from SwissMedic and WHO. Similarly, participation in joint assessments and joint GMP contributed immensely to increasing capacity in these regulatory areas. The joint assessments and GMP promoted exchange of technical knowledge and facilitated cross-country learning, which in turn, increased knowledge and capacity. 56. At the EAC Secretariat, the project helped improve capacity in project management skills, procurement and financial management. The project team participated in training in Bank disbursement and procurement policies and processes, as well as reporting obligations to better equip them to comply with Bank rules. 57. The project also enhanced infrastructure capacity at the national level through the installation of complete video conferencing facilities and 2 data severs in each of the 6 NMRAs and at the EAC Secretariat. These developments facilitated virtual meetings and reduced the number of face-to-face meetings. Poverty Reduction and Shared Prosperity 58. Safe, quality, and efficacious medicines play an important role in healthcare. Weak medicine regulatory systems lead to circulation of poor quality medicines, resulting in low medicine efficacy, which in turn, could contribute to increased mortality and morbidity. By enhancing regulatory capacity in the EAC region, this project has contributed to increasing the availability of much-needed quality and safe medicines and vaccines for communicable diseases, especially for the poor. Furthermore, improved efficiency and transparency in medicines registration is likely to lead to more competitive markets, improved access to new medicines, better quality of medicines in circulation, and ultimately better health outcomes. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 59. The PDO, while relevant and timely, was rather ambitious given the wide variation in regulatory capacity across NMRAs in the EAC region. This was raised in the internal PCN review held in May 2011 and the meeting Page 22 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) recommended the team to consider changing the objective from achieving a harmonized system to facilitating the process of countries adopting similar registration systems or standardizing the registration protocols. An NMRA situational assessment conducted in 2009 among others— which informed the design of the project—revealed that 3 of the 6 NMRAs in the project (Burundi, Rwanda and Zanzibar) had limited personnel (Rwanda and Zanzibar) and inadequate technical capacity and physical infrastructure (Burundi). They also lacked enabling laws that would allow them to carry out regulatory functions needed to approve market authorization for medicines. This was compounded by the fact that in order to achieve harmonization, these weak NMRAs would have had to build institutional and legislative capabilities comparable to the strong NMRAs (Tanzania, Uganda and Kenya). Moreover, additional capacity would have had to be built in even the strong NMRAs. A similar harmonization effort in the Association of South East Asian Nations (ASEAN) took over 10 years. In developing the PDO, the project considered lessons learned from other regional projects, one of which was being realistic in what can be achieved in a regional operation which is inherently complex, but this lesson was not applied. Furthermore, no action was taken to correct these identified gaps during the AF and restructuring. 60. Project preparation was carried out using a participatory approach, and included formal engagement with key stakeholders, such as local industry groups (pharmaceutical companies), implementing agencies (EAC and NMRAs), technical partners (WHO and NEPAD), and development partners (Bill &Melinda Gates Foundation) to identify gaps to be supported by the project. Likewise, well-defined institutional and implementation arrangements clarifying the roles and responsibilities of implementing partners were designed to avoid duplication of efforts and enhance coordination. A results framework informed by the WHO/NEPAD institutional assessments established a baseline for the project. 61. The overall implementation risk at appraisal was deemed to be high. Key risks were summarized in the PAD and appropriate mitigation measures were put in place at preparation. These included measures to enhance operational and technical capacity of both EAC and NMRAs and ensuring effective coordination between Partner States and EAC, and between EAC and other key partners supporting the project (i.e., NEPAD and WHO). However, while the risks posed by the weaker NMRAs were correctly assessed at preparation, they were not reassessed during implementation. B. KEY FFACTORS DURING IMPLEMENTATION 62. Factors that positively affected project implementation • Recruitment of full time project staff at the EAC Secretariat and the National Medicines Regulatory Officers (NMROs) for each of the NMRAs ensured adequate attention to project implementation. • Training of EAC staff on World Bank procurement and financial management processes and policies contributed to improved compliance with PFM. • Conducting external audits helped to ensure credibility and accountability; it also allowed to identify issues early on to put in place corrective measures, (e.g. 2013 external audit). • The constitution of the technical working groups for each regulatory function ensured greater collaboration among NMRAs’ staff and promoted harmonization and standardization of practices in the region. • Establishing the Steering Committee ensured a continuing dialogue on policy and technical aspects of the project among stakeholders and was a platform to address implementation bottlenecks. Page 23 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) • NEPAD’s political advocacy role was critical in addressing and resolving high level policy issues in Member States (Burundi and Rwanda) in order to accelerate implementation. • The involvement of the pharmaceutical industry provided an opportunity for constructive feedback on the development of regulatory guidelines. • The grant became effective in July 2012 and first disbursement was made in November 2012. Despite the delay in accessing the project funds, the EAC facilitated implementation of the regional activities using its own resources and World Bank video conferencing facilities. This prevented further delays in implementation. 63. Factors subject to implementing entities control Coordination: As noted earlier, this project involved regional and national activities, making coordination key for this project. During the ICR mission, it was often emphasized that coordination of regional activities was challenging, leading sometimes to delays. For planning purposes and to ensure good attendance, joint assessments of regional medicine applications and joint inspections should have had a clear calendar of upcoming activities and meeting documents circulated sufficiently in advance to allow for preliminary review. This was not the case: meetings were sometimes held without sufficient notice and therefore assessors, inspectors, and other technical experts could not make the necessary arrangements to attend or obtain clearance from their respective ministries/agencies. For instance, assessors were expected to receive the documents well in advance of the meetings to enable the first assessor to review and the second assessor to review the work of the first assessor prior to discussion of an application at the joint assessment. The lack of coordination may be partly due to the lack of clarity of roles and responsibilities both within the EAC Secretariat (project staff) and between the EAC Secretariat and the NMRAs. This was further compounded by the lack of clarity on the division of labor between the EAC Secretariat and the NMRAs for each of the Technical Working Groups. Furthermore, NMRAs faced challenges in coordinating joint activities and their own national activities. This is because joint activities were not always s een as part of NMRAs’ staff daily priorities due to conflicting tasks at national level. As a result, efficiency of the joint meetings was hindered, with less qualified staff from the NMRAs attending the meetings and the meetings being rescheduled, leading to implementation delays. Human resources and organizational capacity: Project implementation at the country level was delayed. This was because of the length of time required to fully comply with EAC procedures for recruitment of project staff and the 6 National Medicines Regulatory Officers (NMROs). As a result, the appointments were completed in March 2013 and these staff started working from April 2013, almost one year after project effectiveness. Thus, the limited EAC staff supporting the project were overstretched with several responsibilities. This challenge, coupled with lack of support for NMROs at country level, resulted in implementation delays in both national and regional level activities. However, once project staff and NMROs were appointed, implementation began in earnest and gained steam. In addition, the main thrust of the project was to improve capacity to enable NMRAs to effectively participate in the harmonized registration system. Despite strong capacity building in less resourced NMRAs, Burundi, Rwanda and Zanzibar still face understaffing issues and lack of key expertise. For example, with only 6 pharmacists in Burundi, it was unable to meet the target of registering 50 medicines using the new CTD format by the end of the project. Similarly, with only 12 pharmacists in Zanzibar, the discharge of new regulatory functions (MER and GMP) Page 24 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) is creating mounting pressure on the limited technical staff. Overall the slow start of the project due to prolonged recruitment of project staff, coupled with other implementation challenges (late submission of no objections), weak project management capacity, as well as a break in implementation (between January and April 2016, pending approval of the AF) contributed to delays and resulted in a February 2017 request by the Secretariat for a no-cost extension of the project to December 2017 in order to complete the remaining activities. Legislation and regulations: An enabling legislative framework is a prerequisite for harmonizing requirements for medicines registration. Standardized requirements for medicine registration need to be aligned with national laws on medicines registration. Although the Model Law for Medicinal Products Regulation was introduced by NEPAD to facilitate harmonization efforts, all countries are still not at the same stage in domesticating this law. 64. Factors subject to World Bank control Retroactive financing to avoid interruption in implementation: In 2016, it became apparent that there would be a financing gap of US$119,000 between January and March. During the January 2016 implementation supervision mission, the EAC Secretariat informed the task team that they would not be able to provide any funds to continue implementing project activities, including staff salaries. Thus, the Secretariat requested support from the Bank in the form of retroactive financing as the Additional Financing was being considered. Consequently, an amount of US$ 120,000 was made available as retroactive financing under the proposed project to finance activities implemented between January 1, 2016 and the date of the Amended and Restated Agreement for the Additional Financing. However, because the EAC Secretariat had no funds to pre-finance project activities, the retroactive financing was eventually never utilized. 65. Factors outside of government control Conflict and instability: Political issues at national level also impacted the implementation of the project leading to delays. As noted previously, political tension in Burundi slowed down the implementation of the activities that required participation from all the countries, leading to one of the two reasons for the second level II restructuring. Furthermore, in 2015, implementation was scaled down due to a travel ban in Burundi caused by political instability. Specifically, Trade Mark East Africa (TMEA), the firm supporting countries with IMS, could not travel to Burundi due to the political instability. Page 25 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) V. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Implementation 66. M&E implementation is rated Modest 67. The task team produced 11 ISRs during project implementation (which was adequate for the duration of the project) and included sections on issues for management actions. The ISRs tracked the indicators in the RF. In addition, the AMs also tracked the indicators in the RF as well as indicators such as the number of medicines assessed and registered at national and regional levels, which were not included in the RF, but necessary for monitoring efficiency. 68. According to the PAD, because this was a system strengthening project, M&E data was primarily qualitative. M&E data was to be generated from administrative records, information made available on NMRAs’ and EAC’s websites, and by documented reports. Data collection tools included Excel spreadsheet and Word tables. In practice, however, the frequency of NMRAs reporting to EAC was inconsistent. For example, while NMRAs were required to provide annual progress reports, they sometimes produced quarterly reports. Similarly, details on the number of medicines assessed and registered using the CTD also varied, with some NMRAs providing more details than others. Furthermore, there was no reporting on NMRAs sharing public information (policies, guidelines, etc.) on their websites, although this was supposed to be reported on quarterly. Annual reports were shared with partners but data/information was not validated as stated in the PAD (WHO National Professional Officers were expected to validate country data). The ICR authors note that NMRAs’ reports were detailed and included sections on Challenges and Recommendations, which provided an opportunity to take corrective actions and provide the needed support to NMRAs lagging in implementation. 69. Finally, gaps in M&E implementation included (i) the lack of an M&E focal point on the project staff at the EAC Secretariat; and (ii) the absence of reporting on M&E by NMRAs. Although M&E implementation had gaps, no changes were made during AF and restructuring. M&E Utilization 70. M&E utilization is rated Modest. 71. Data collected during implementation was used to inform corrective measures to improve overall project achievements. For instance, implementation issues raised in the annual progress reports and data collected on number of medicines assessed and registered at national and regional levels as well number of joint GMP inspections were also presented to the different stakeholders including partners (WHO, NEPAD,) and donors (BMGF) at the Steering Committee meetings. The Committee deliberated upon these issues and proposed practical solutions for addressing them. The EAC Secretariat and the task team also reviewed data collected for the project as well as implementation challenges raised in the annual and quarterly reports and proposed recommendations for timely action. Page 26 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) Justification of Overall Rating of Quality of M&E 72. Overall M&E is rated as Modest. Overall rating M&E Design M&E Implementation M&E Utilization Modest Modest Modest Modest Page 27 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Procurement 73. The main procurement for the project included ICT infrastructure, hardware and software, and office equipment for the EAC and NMRAs. Initially, the EAC had limited procurement capacity and knowledge of Bank procurement procedures, resulting in the late start in procurement activities (late 2014). To address this challenge, EAC enhanced its procurement capacity by recruiting additional procurement staff who were then trained in procurement under World Bank guidelines and procedures, which enhanced knowledge and skill in procurement activities under the project. Also, EAC project procurement staff made constant and regular recourse to the Bank Procurement Specialist to clarify procurement issues. This enabled the EAC project team to successfully complete most of the packages that were planned to be procured under the project. At project closing, the contract for Translation of Compendium Documents in French for Burundi was the only contract that was not completed; however, this contract was initiated late and there were indications that the deliverable would not be ready by the project closing date. As a result, the contract had to end with some of the documents having not been translated. It was made clear to the EAC Secretariat at the time of the mission in November 2017 that goods and services not delivered by the project closing date were not eligible for financing under the project. Financial Management 74. At project design, to minimize delays in funds flow and reduce the number of audits, the task team adopted a virtual budget for disbursement of project funds to NMRAs. This method allowed NMRAs to procure small value services for training, workshops, and individual consultants following agreed procedures within the approved work plan. Payments were then made directly by the EAC Secretariat to vendors and consultants. For operational costs, each NMRA incurred the expenditures and shared documented claims seeking reimbursement from the EAC Secretariat. During implementation, however, there were delays in processing reimbursements for operational cost to NMRAs. These delays were addressed by simplifying the reimbursement process. 75. Regarding the virtual budget, NMRAs raised concerns about this arrangement during implementation. For example, some of the training expenditure budgeted at the country level was also shown as expenditure incurred at the regional level. By MTR, it was clear that this arrangement was not working well and was identified as one of the operational challenges for the project. To address this challenge, the EAC Secretariat was requested to clarify the processes/arrangement for the virtual budget in the Operations Manual, but this was never done. During interviews with NMRAs, the virtual budget was identified as one of the project weaknesses due to a lack of transparency on how project funds earmarked for virtual budgeting were spent. 76. Furthermore, there were delays in submitting the first initial advance request ($500K), the first 2 IFRs (July 2012 – September 2012 and October 2012 – December 2012), and monthly withdrawal applications. After project staff were trained in Bank FM procedures and policies and additional staff were hired (recruitment of project accountant), FM improved significantly. By MTR, the project team was submitting timely, quality external audits, IFRs, and withdrawal applications to the Bank. Additionally, FM issues raised Page 28 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) during supervision missions and by the external auditors were addressed in a timely manner. 77. Finally, in the first year of implementation, the disbursement rate was low due to a few factors, including delays in: (i) recruiting project staff, (ii) submitting monthly withdrawals, and (iii) procuring ICT software and hardware. As noted above, once the EAC Secretariat hired additional FM staff and trained existing staff, by MTR there was a slight increase in disbursement from 38 percent (March 2014) to 52 percent (June 2014). Although disbursement improved, the overall disbursement rate by the end of the project was 98 percent with a small unutilized balance, but this did not affect achievement of project objectives. C. BANK PERFORMANCE Quality at Entry 78. The task team worked closely with the EAC, NMRAs, technical partners (NEPAD & WHO) and donors (BMGF), and consulted with relevant stakeholders, including industry, during project preparation to ensure greater alignment between project objectives and key strategic documents at regional and continental (AU) levels. Consultation also ensured strong endorsement and commitment by NMRAs. 79. Project design was simple with only two components. The project was informed by institutional assessments of NMRAs conducted by WHO, and it sought to address some of the key recommendations from the assessment. Establishing TWGs, with the NMRAs taking the lead in the development of harmonized documents for medicines registration ensured ownership. Recognizing that several partners were involved in the implementation of the project, the task team proposed a clear institutional and implementation arrangement, with well-defined roles and responsibilities of key partners. Mitigation measures were put in place to address key risks identified at appraisal, which for this project were cross cutting—from operational to governance risks. Furthermore, to the extent possible, necessary measures were put in place to enhance the capacity of both the EAC Secretariat and NMRAs, and to provide project management support. 80. Despite the above, there were gaps in design which were not properly addressed such as the broadness of the PDO and issues with the results framework, including selection of indicators, gaps in indicators, and the lack of clarity of some of the indicators. Also, the two key areas lacking in the task team composition were a seasoned Operations Officer, and an M&E Specialist to guide the team in the formulation and review of the PDO and the results framework. In view of these gaps, Bank quality at entry is rated Moderately Unsatisfactory. Quality of Supervision 81. Besides the first supervision mission, joint supervision missions—led by the task team together with representatives from WHO and NEPAD—were conducted with the EAC Secretariat throughout the project. In addition to the joint supervision missions, the task team conducted separate missions (immediately after the EAC missions) to NEPAD in May and November of 2017. The mission team was diverse and included procurement and FM staff. Except for the November 2016 and 2017 missions, the NMRAs were not invited to most of the supervision missions; however, there were several technical missions to the project countries. These technical missions were conducted in each of the project countries and provided an opportunity to Page 29 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) interact with the NMRAs. Furthermore, industry was fully engaged in the harmonization process and their feedback obtained and considered to improve the process. For example, during the November 2017 supervision mission, industry was invited to share their feedback. Also, the task team along with WHO and NEPAD, participated in most stakeholder consultation/validation meetings for pharmaceutical policy, draft CTD and guidelines for GMP inspections. In addition to above, the task team also participated in all Steering Committee meetings where implementation progress and challenges were discussed and addressed. 82. As noted earlier, the task team conducted 11 supervision missions (two per year) and submitted 11 ISRs. Missions addressed implementation bottlenecks as well as procurement, financial management, and project management challenges. However, there were some shortcomings: (i) In 2014, 3 missions were conducted (March, June & December) whereas only 1 mission was conducted in 2015; (ii) issues around the virtual budget reappeared several times in the AMs but the EAC did not clarify the processes/arrangement for the virtual budget in the Operations Manual. As noted earlier, because of this, NMRAs felt that the virtual budget was not transparent and did not work well. Furthermore, while the project had an Operations Officer on the team, the staff was a technical expert in medicines regulation but was unfamiliar with Bank operations. Having a seasoned operational staff person on the project could have helped avoid the delays in the preparation of the AF, and address gaps in the project design and the RF during restructuring and AF. 83. Bank quality of Supervision is rated Moderately Unsatisfactory. Justification of Overall Rating of Bank Performance 84. Overall Bank performance is rated as Moderately Unsatisfactory. Overall rating Quality at Entry Quality of Supervision Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory D. RISK TO DEVELOPMENT OUTCOME 85. The EAC remains deeply committed to its Vision 2050 to strengthen and harmonize health and health- related policies, strategies and plans, including medicines regulations. Although some of the risks noted at appraisal are still relevant, such as capacity variance among NMRAs, the risk to development outcome is considered moderate given continued commitments from the EAC and NMRAs to sustain the medicines regulatory harmonization initiative. For instance, a roadmap for the implementation of the next phase of the program has been developed. In addition, a Technical Cooperation Framework, allowing bilateral recognition of approvals of medicines registration (an aspiration of the project which was achieved by project closure) has been created and will be approved by the Council of Ministers in April 2018. 86. At national level, most of the Partner States have passed or updated their laws to comply with the Model Law for Medicinal Products Regulation. This law will allow Partner States (Rwanda and Burundi) who at project closing did not have the appropriate legal framework in place to do so, thereby creating an enabling environment to sustain the harmonized standards and practices developed through the project. Also, there is political will on the part of Partner States to sustain and build on investments made by the project. For instance, in Zanzibar, the Ministry of Health has mainstreamed full regulatory functions into the government budgeting process after project closing. Page 30 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 87. Furthermore, sustainability of project investments can also be guaranteed through the fees charged by NMRAs for regulatory services. For example, Kenya and Tanzania NMRAs are currently able to maintain and sustain their IT related activities—hitherto supported by the project—from their own internally generated resources. For regional activities—joint assessments and joint GMP inspections—which were introduced and funded by the project, the Steering Committee held on March 2018 discussed at length a short to medium term sustainability plan, including fees for regional activities to fund and sustain these activities. To operationalize the sustainability plan, Partner States would need to adopt a regional policy on joint reviews and develop an operational plan detailing how these activities will be coordinated, how regional fees will be set, and how regional fees will be distributed among NMRAs in the region. For the regional PV developed with project funds, resources have been secured from USAID to fund implementation. 88. Finally, a few activities have been undertaken by the EAC and the NMRAs since project closing to maintain momentum. For instance, a Regional Technical Team (RTT) involving staff members from different NMRAs has been created to ensure continued commitment to regional activities. The RTT is expected to, among other activities, provide technical support and facilitate joint regulatory activities (joint assessments, joint inspections and PV). V. LESSONS AND RECOMMENDATIONS The project provides useful lessons that could inform similar regional projects in the other regional economic communities such as ECOWAS and SADC. 89. Establish a realistic, coherent, and clear PDO in order to improve project performance and effectiveness. Future regional projects seeking to harmonize medicines registration should establish a realistic PDO that is aligned to the RF, and is clear in terms of what the project seeks to achieve and how progress will be monitored, tracked, and revised. Medicine registration harmonization is a long-term process and involves NMRAs with varying capacities (technical and financial). For this reason, it is recommended that the design of such projects consider a phased approach starting with standardizing registration requirements, aligning these with national polices, building institutional capacity at national level, then moving towards regional harmonization of systems (full interoperability of national systems with regional system). The RF could be informed by the WHO Data Collection Tool for the Review of Drug Regulatory Systems, which provides an array of indicators for each regulatory function. 90. Assess risks and develop and implement mitigation measures before risks impact project performance. Risk assessments and mitigation measures are critical to the achievement of such a complex regional project. Therefore, project design should take into consideration these inherent risks, develop appropriate mitigation measures and review these regularly during implementation. Ongoing review is essential to ensure that mitigation measures remain relevant and can be applied before project performance is affected. Ongoing review can also help to determine if appropriate alternative measures should be considered. Issues raised at Quality at Entry in this ICR could have been resolved during Page 31 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) restructuring and AF. 91. Jointly develop clear institutional and implementation arrangements to address the complex coordination requirements inherent to regional harmonization of medicines registration. These arrangements (i.e., division of labor) should clearly be outlined during project design, explicitly formalized, and then reviewed and revised during implementation in order to ease coordination. This is particularly important in cases where key partners must combine or leverage their capacities. During project preparation, both the PAD and Operations Manual articulated the roles of the EAC Secretariat, the EAC project team, Steering Committee, NEPAD, NMRAs and other technical partners involved in the project. During implementation, however, the delineation of roles became blurred, particularly between the EAC Secretariat and the MRH team and between the EAC Secretariat and the NMRAs. There was also a lack of distinction between partner/policy coordination (NEPAD and the EAC Secretariat) from technical coordination (NMRAs and EAC Secretariat). Reviewing and adjusting institutional and implementation arrangements during implementation could help avoid duplication of efforts and better clarify roles. 92. Consider operational details of virtual budgeting and clearly communicate them to clients in order to ensure transparency and accountability. While the introduction of virtual budgeting was an innovative solution to address the complexity of managing funds for small value services across 6 NMRAs, the lack of transparency in its use, undermined its overall utility. To address this in the future, operational details could be provided in an Operational Manual and discussed with clients to create ownership and build confidence in the process. Also, implementation arrangements for virtual budgeting should be reviewed from time to time and corrective measures applied to improve transparency. 93. Take into account the capacity and limitations of implementing agencies in order to avoid unnecessary setbacks. Specifically, the internal processes of an implementing agency should be taken into consideration during project design. Delays in the EAC project were mainly due to internal bureaucratic processes of the implementing entity—the EAC Secretariat. The delay in recruitment due to internal processes and negotiation impacted efficiency and slowed down implementation. Similarly, the EAC was unable to provide financial support, in a timely manner, to NMRAs to implement activities due to lengthy . approval processes required for disbursement of funds/implementation of activities. 94. Allocate additional resources for institutional strengthening in less-resourced NMRAs to facilitate their eventual full participation in the harmonized system over the long term. For example, while twinning activities and other capacity building efforts were very beneficial for the less resourced NMRAs (Rwanda, Burundi, and Zanzibar), significant time—beyond five years—is necessary to bridge the gap between less- resourced NMRAs and the stronger NMRAs as this requires a number of factors, including: (i) greater political will and support; (ii) additional investments in human resources and infrastructure, and; (iii) an enabling legal and regulatory environment. Similarly, the varied capacity across NMRAs affects the pace of harmonization and could potentially serve as a disincentive for less resourced NMRAs to utilize harmonized standards in the long term. In the future, similar projects should invest additional resources in less resourced NMRAs and consider, during design phase, the potential impact of political economy challenges on achieving development objectives. Page 32 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project (P128332) 95. Leverage Bank operations at the country level to address emerging issues. Some of the issues on weak institutional capacity and inadequate infrastructure, which became evident during implementation, could have been incorporated into Bank operations at the country level. This lesson is especially relevant since there was a strong alignment between the regional project objectives and national health sector strategies in all the project countries. In addition, as this regional project was endorsed by the EAC Council of Ministers as a priority for the region, and the Council is responsible for setting overall policy for health in . the region, it could have advocated for the next phase of the project to be funded through regional IDA. Page 33 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome:Harmonized medicines registration systems; improved efficiency in medicines registration; and enhanced transparency in medicines registration. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion NMRAs participating in the Number 0.00 0.00 3.00 6.00 harmonized medicine registration based on 12-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 internationally recognized policies and standards Comments (achievements against targets): Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion NMRAS started piloting the Number 0.00 0.00 3.00 4.00 electronic submission of applications for registering 12-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 medicines Page 34 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion NMRAs sharing regulatory Number 0.00 0.00 3.00 3.00 policies, legislation, guidelines and information 12-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 on registered medicines Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion NMRAs received ISO Number 0.00 0.00 3.00 2.00 9001:2015 certification 11-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 Comments (achievements against targets): A.2 Intermediate Results Indicators Component:Harmonize medicine registration systems Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Harmonized guidelines Number 0.00 0.00 3.00 5.00 Page 35 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) including SOPs and manuals 12-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 for registration of medicines and GMP inspection developed based on internationally recognized policies and standards Comments (achievements against targets): Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion NMRAs operationalizing Number 0.00 0.00 3.00 3.00 common information management system for 12-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 medicine registration developed by the regional technical working group Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Staff trained in medicine Number 0.00 0.00 75.00 233.00 regulation 11-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 Comments (achievements against targets): Page 36 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion NMRAs implementing quality Number 0.00 0.00 3.00 4.00 management systems 12-Jul-2012 31-Dec-2014 29-Dec-2017 29-Nov-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Gap analysis report and a Yes/No N Y Y Y strategy for pharmacovigilance 17-May-2016 31-Dec-2014 29-Dec-2017 29-Nov-2017 formulated and ready for implementation Comments (achievements against targets): B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: to harmonize medicine registration systems 1. NMRAs participating in the harmonized medicine registration based on internationally recognized policies and standards Outcome Indicators 2. No outcome Indicator 3. NMRAs received ISO 9001 certification Page 37 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) 1. Harmonized guidelines including SOPs and manuals for registration of medicines and GMP inspection developed based on internationally recognized policies and standards Intermediate Results Indicators 2. Gap analysis report and a strategy for Pharmacovigilance formulated and ready for implementation 3. NMRAs implementing quality management systems 1. Harmonized guidelines and standards developed Key Outputs by Component 2. Pharmacovigilance strategy developed (linked to the achievement of the Objective/Outcome 1) 3. ISO certification received by 2 countries and QMS implemented in 4 countries Objective/Outcome 2: to improve efficiency in medicine registration 1. NMRAs participating in the harmonized medicine registration based on internationally recognized policies and standards Outcome Indicators 2. NMRAS started piloting the electronic submission of applications for registering medicines 3. NMRAs received ISO 9001 certification 1. Harmonized guidelines including SOPs and manuals for registration of medicines and GMP inspection developed based on internationally recognized policies and standards 2. NMRAs operationalizing common information management system Intermediate Results Indicators for medicine registration developed by the regional technical working group 3. NMRAs implementing quality management systems 4. Staff trained in medicine regulation Key Outputs by Component 1. Harmonized guidelines and standards developed (linked to the achievement of the Objective/Outcome 2) 2. IMS implemented in 5 countries Page 38 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) 3. ISO certification received by 2 countries and QMS implemented in 4 countries 4. 233 staff trained Objective/Outcome 3: to enhance transparency in medicine registration 1. NMRAs sharing regulatory policies, legislation, guidelines and information on registered medicines on their websites Outcome Indicators 2. No outcome indicator 3. NMRAs received ISO 9001 certification 1. 2. NMRAs operationalizing common information management system Intermediate Results Indicators for medicine registration developed by the regional technical working group 3.NMRAs implementing quality management systems 1. Each NMRA has a website and a regional website has been developed Key Outputs by Component 2. IMS implemented in 5 countries (linked to the achievement of the Objective/Outcome 3) 3. ISO certification received by 2 countries and QMS implemented in 4 countries Page 39 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Apollo Muhairwe Task Team Leader(s) Gisbert Joseph Kinyero, Raymond Joseph Mbishi Procurement Specialist(s) Mercy MataroSabai Financial Management Specialist Claudia Ocana Counsel Alexandra C. Bezeredi Social Safeguards Specialist Gandham N.V. Ramana Team Member Harriet E. N. Kiwanuka Team Member Cassandra De Souza Team Member Juliet Allen Gombya-Ssembajjwe Team Member Chitambala John Sikazwe Team Member TandileGuguZizileMsiwa Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 40.375 134,946.32 FY13 0 3,062.84 Total 40.38 138,009.16 Supervision/ICR FY13 54.312 169,334.25 Page 40 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) FY14 50.306 193,537.88 FY15 44.400 159,662.69 FY16 56.475 313,923.87 FY17 68.900 627,154.64 FY18 72.637 478,892.94 Total 347.03 1,942,506.27 Page 41 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) ANNEX 3. PROJECT COSTBY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Regional Coordination and Capacity Building for 2.40 4.48 100 Medicines Regulatory Harmonization Institutional Development and Strengthening of National Medicines 3.13 4.96 100 Regulatory Authorities (NMRAs) Total 5.53 9.44 100 Page 42 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) ANNEX 4. EFFICIENCY ANALYSIS Economic Benefits and Costs from supporting the AMRH Initiative The economic and societal benefits of investments to strengthen the medicines regulatory processes are understood to be significant, despite the availability of quantitative evidence due to lack of routine data collection on medicines use and from underdeveloped pharmacovigilance systems in LMICs, and the inherent challenges in estimating the costs from delayed access to medicines or the cost of administering substandard medicines. The expected economic benefits resulting from the EAC project are highlighted below. The costs for this project were minimal at US$ 0.06 per person based on an estimated population of 168.8M in the region in 2016 and the total project costs of US$ 10.2M. The circulation of safe, effective and quality medicines: From an economic point of view, the existence of market failures justifies the need for regulating the medicines industry and ensuring the circulation of safe, effective, quality medicines. Firstly, the market is characterized by an ‘information asymmetry’ between those who manufacture/sell medicines and patients/consumers, who are unable to assess the quality, safety or efficacy of medicines purchased. Secondly, the use of substandard medicines can have serious negative public health externalities such as the development of medicines resistance, one of the major public health threats. In this case, consuming substandard medicines by single individuals could lead to costs for the entire society in the form of reduced medicine efficacy and increased risks of death. For example, a study conducted by WHO in 2003 on the quality of antimalarial medicines showed that in sub-Saharan African countries only 58 percent of the medicines tested had an acceptable level of chloroquine content and only 25 percent had acceptable dissolution properties. The poor-quality chloroquine may be among the causes of the high rate of resistance in these countries. Effective medicines regulation can therefore minimize the economic costs resulting from unsafe, inefficacious, suboptimal quality medicines otherwise borne by patients, health systems, and the overall economy. Finally, patients accessing safe and effective medicines respond to treatment promptly and effectively, thus avoiding prolonged therapies, long-term complications, and even premature death. For example, the International Policy Network estimated the death toll caused by fake drugs for malaria and tuberculosis alone at 700,000 cases per year. From a health system perspective, effective regulation ensures that already strained resources (public or private) for health are not wasted, and costs for additional medical consultations and hospitalizations are avoided. Improved investments, higher revenue for regulators, and growth of the local pharmaceutical industry: At the macroeconomic level, the reduction in regulatory barriers through standardized, simplified, and internationally recognized registration processes has been shown to be associated with greater domestic and foreign investments, which, in turn, are expected to create jobs and boost the economic productivity of the country (Cambridge, 2012). Specifically, the market for medicines in Africa is expected to represent a US$ 45 billion opportunity by 2020, thus representing an immense prospect for economic development for the region (IMS, 2014). In turn, higher economic growth is expected to translate in higher investments in health (Bedir, 2016), and thus improve population’s health and economic productivity. A recent study by the University of Cambridge (2012) demonstrated that the regional harmonization of the regulatory Page 43 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) framework in Eastern Africa brings significant economic benefits to both regulators and pharmaceutical companies. Regulators benefit from notable increases in fee revenue, which can provide the previously lacking resources for increasing capacity, improving technologies, and enforcing regulations and thus preventing counterfeit/substandard medicines entering and circulating on the market. At the same time, pharmaceutical companies save significant resources from labor division and fast-tracked processes, which reduce the human capital costs. The study estimated the average human capital costs sustained by pharmaceutical companies in each country under a scenario of no harmonization: 100 full-time equivalent (FTE) days for inspections, 5-7 FTE for submitting the registration documents and 300-420 days to complete the submission. Under a scenario of harmonized medicines regulation, the 5-year costs per product for pharmaceutical companies were estimated to drop from US$ 5.7M to US$ 1.7M, while the 5- year revenue for the regulator to increase from US$72,800 to US$ 117,200. Furthermore, assuming a 50 percent increase in the number of medicines registered in the region because of the improved registration process, the 5-year regulator’s revenue is expected to increase from US$ 52.2 million to US$ 140.6 million. Page 44 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS SUMMARY PROGRESS REPORT ON IMPLEMENTATION OF THE EAST AFRICAN COMMUNITY MEDICINES REGULATORY HARMONIZATION (EAC-MRH) PROJECT 1.0 FIVE (5) YEAR PROGRESS REPORT ON IMPLEMENTATION OF EAC-MRH PROJECT Project Objective: To harmonize medicines registration systems and to improve efficiency and enhance transparency in medicines registration among the East African Community Partner States. Goal: To establish a harmonized and functioning medicines regulatory system within in accordance with national and internationally recognized policies and standards. Launched: 30th March 2012 Objectives 1. To implement an agreed common technical document for registration of medicines in EAC Partner States 2. To implement a common information management system for medicines registration in each of the EAC 3. To implement a quality management system in each of the EAC Partner States’ NMRAs 4. To build regional and national capacity to implement medicines registration harmonization in the EAC 5. To create a platform for information sharing on the harmonized medicines registration system to key 6. To develop and implement a framework for mutual recognition based on Chapter 21, Article 118 of the East African Community Treaty East African Community: -Republics of Burundi, Kenya, Rwanda, Uganda, South Sudan and United Republic of Tanzania. The Republic of South Sudan became a full member of the EAC on 5th September 2016. EAC Secretariat: Coordination and Harmonization EAC Partner States National Medicines Regulatory Authorities (NMRAs): Implementing Agencies. AMRH Partners: - WHO /Swissmedic(Technical), AU-NEPAD Agency (Policy Advocacy), BMGF, DFID& SDC (Funding), WB (Funding & Administrator of AMRH Trust Fund), MSH|SIAPS, USAID, USP/PQM (Technical &Financial) Regulatory Key Year Indicators of Progress Function Milestones Milestone/Critical Success Indicators to be Achieved Medicines EAC CTD for March 2015 An agreed common (i) EAC Compendium of Registration registration technical document Harmonized Technical of medicines for registration of Guidelines for Registration of implemented medicines in EAC Medicinal Products adopted by Page 45 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) in at least Partner States the 29th Meeting of the EAC three EAC established by end of Council of Ministers Partner year 2 (EAC/CM29/Decision 36) on States by the 20th September 2014 end of year three and in (ii)All EAC Partner States all Partner NMRAs with exception of the states by end Republic of Burundi and of year 5 Republic of South Sudan are implementing EAC CTD March 2017 At least 75 medicines Data as of May 2017 registered per NMRA under the (i) PPB, Kenya: -1676 MRH scheme by Medicinal Products registered TFDA, NDA and 2475 products submitted PPB by end of year 5 At least 100 (ii) NDA, Uganda: -705 applications Medicinal Products registered submitted to out of 837 products submitted NMRAs under the CTD format by the (iii) TFDA, Tanzania: - end of year 5 76Medicinal Products registered out of 799 products submitted March 2017 At least 50 medicines Data as of May 2017 registered per NMRA under the (i) PTF, Rwanda: -297 MRH scheme by Medicinal Products out of 792 ZFDB, Burundi and products submitted Rwanda by end of (ii)DPML, Burundi: - 10 year 5 Medicinal Products registered At least 100 out of 90 products submitted applications submitted to (iii)ZFDB,Tanzania: -19 NMRAs under the Medicinal Products registered CTD format by the out of 64 products end of year 5 March 2017 At least 18 medicines Thirty-eight(38) applications approved under the received for EAC joint joint assessments assessment and registration scheme by end of year 5 All medicinal products dossier have been evaluated Eight (8) products recommended for joint registration Page 46 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) Good EAC March 2015 EAC Joint GMP (i) EAC Compendium of GMP Manufacturing Common Inspections Guidelines for Inspections of Practices GMP Conducted and Joint Medicines Manufacturing sites Inspection Decisions/Outcomes adopted by the 29th Meeting of Guidelines Made the EAC Council of Ministers adopted least (EAC/CM29/Decision 36) on three EAC 20th September 2014 Partner States by the (ii) A total of fourteen (14) EAC end of year Joint GMP Inspections three and in conducted. Eleven (11) all Partner Pharmaceutical Manufacturing states by end Facilities have been issued with of year 5 EAC GMP Compliance Certificate. Three (3) Manufacturing sites awaits CAPA implementation and review Pharmaceutical Manufacturing Facilities Sites Location: - India (7), Uganda (2), Tanzania, Kenya (2) Egypt (2) and Morocco (1) Quality Quality March 2015 At least 3 NMRA ISO (i) EAC Compendium of QMS Management Management certified by the end Guidelines for NMRAs Systems (QMS) System of year 3 adopted by the 29th Meeting of implemented the EAC Council of Ministers in each of the (EAC/CM29/Decision 36) on EAC Partner 20th September 2014 States' NMRAs by (ii) Review of EAC QMS end of year 3 Requirements and Manual online with ISO 9001:2015 is ongoing (iii) TFDA have been ISO 9001:2008 and currently is in the process of recertification for ISO 9001:2015 External Audit for NDA to be conducted in November 2017 For PPB, pre-inspection by external audit has been done. Formal inspection to be carried out 31st December 2017 ZFDB, Pre-assessment and assessment by external auditors. Plans to apply for ISO 9001:2015 Certification Information A common March 2016 (i) Status of implementation of Page 47 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) Management information IMS (see separate sheet) System management system for (iii) Videoconference Facilities drug procured and installed in all six registration (6) EAC Partner States NMRAs operational in all EAC (ii) Procurement of 40 Tablets Partner States’ NMRAs and linked regionally by end of year 4 Policy, Legal and EAC March 2017 Six (6) NMRAs (i) EAC Cooperation Regulatory framework recognizing Framework Agreement Reforms for mutual regulatory decisions developed and validated. recognition made by other Awaiting adoption by the EAC based on NMRAs based on Policy Organs Chapter 21, mutual recognition Article 118 of framework by end of (ii) EAC Medicines and Health the East year 5 Technologies Policy developed African and validated. Awaiting Community adoption by EAC Policy Organs Treaty developed (iii) EAC Medicines and Health by end of Technologies Strategic Plan year 5 developed and will be reviewed and validated in September 2017 (iv) Enactment of National Laws to establish semi- autonomous NMRAs – Rwanda Food and Drugs Authority (RFDA) Act of 2014, Republic of Rwanda and Zanzibar Food and Drugs Authority Act of 2017, United Republic of Tanzania (v) Reviewed national laws against the AU Model Law on regulation of Medicinal Products, for the Republic of Rwanda, Burundi and Kenya. A desk review will be conducted forZFDB URT and Republic of South Sudan, while for TFDA- Tanzania, review and stakeholder’s consultation will be conducted later (vi) High Level Policy Advocacy on establishment of ABREMA with Page 48 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) Parliamentarians of the Republic of Burundi Capacity Building A pool of March 2015 (i)At least 2 regional (i) Trained 35 NMRAs staff on in Regulatory experts centers of excellence QMS lead auditing Sciences and created at in training assessor Quality national and and GMP inspector (ii)Trained 30 Assessors from Management regional established in the EAC NMRAs level with EAC region by end competency of year 5 (iii) Trained 40 EAC NMRAs in regulatory staff on basic and advanced sciences and (ii) At least 25 EAC GMP project and NMRA MRH management project staff trained (iv) Trained EAC NMRAs Staff on management of on the use and application of RCORE’s MRH by end of year IMS established 5 for (v) Trained 12 staff from EAC Assessment, (iii) At least 24 Partner States NMRAs on GCP Inspections, assessors trained in and Clinical (Contract) assessment of Research Organization (CRO) quality, safety and Inspections efficacy of medicines by end of year 5 (vi) Three (3) Twinning Sessions conducted for each set (iv) At least 24 of EAC Partner States NMRAs inspectors trained on in Dossier Evaluation and GMP GMP inspection by Inspections: - PPB/ZFDB; end of year 5 TFDA/DPML/SDFC; NDA& PTF (vii) Two (2) EAC RCORE for Medicines Registration and Pharmacovigilanceare operational i.e. TFDA/MUHAS and PPB/Nairobi University Pharmacovigilance (i) EAC PV Proposal endorsed by the EAC Policy Organs (ii) Development of EAC PV Systems Assessment Tool and Manual (iii) Assessment of PV Systems in EAC Partner States NMRAs and Piloting of the PV Tool conducted from 16th February to 10th March 2017. (iv) EAC Pharmacovigilance Business Plan developed, awaiting approval by Ministers in May 2018. (v) EAC Expert (vi) Seven (7) EWG established to undertake technical work of EAC-MRH project: - Working Group MER, GMP, QMS, IMS, Regulatory Reforms, PV & PMS (vii) Performance: -Good, activities conducted through VC and face to face meetings Forum of Heads of (i) Established by the 12thOrdinary meeting of the EAC Sectoral Council of NMRAs Ministers of Health held on 22nd June 2016 (EAC/Health/12SCM/Decision 038)to oversee implementation of EAC MRH programme (ii) Benchmarking visit to European Medicines Agency, 17-19 May 2017 (iii) 7th Forum of Heads of NMRAs held on 27th to 28th July 2017 recommends development of a concept note for a business case to establish East African Community Medicines Agency (iv) A task force has been established to oversee development of a concept note Page 49 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) EAC-MRH (i) Established by the 5thOrdinary meeting of the EAC Sectoral Council of Ministers Steering of Health held on 1st April 2011(EAC/Health/SCM 05/Decision 19)to provide Committee administrative and technical oversight of implementation of the EAC MRH programme (ii) Considered and approved a set of EAC harmonized guidelines for MER, GMP, QMS and IMS End-Term (i) End term evaluation of EAC-MRH Programme conducted and concluded in Evaluation of December 2017 by Boston Consulting Group. EAC-MRH Programme Sustainability of (i) Roadmap/Strategy for post EAC-MRH implementation developed, awaits EAC-MRH approval by EAC Ministers of Health Programme 2.0 CHALLENGES (i) The current application procedure for EAC central registration and EAC joint GMP inspections use the national systems of Tanzania Food and Drugs Authority (TFDA) of the United Republic of Tanzania and National Drug Authority (NDA) of the Republic of Uganda respectively for application filling and coordination of the process; (ii) The EAC region has not yet harmonized its regulatory fee structures and guidelines. Applicants are required to pay application fees according to each of the EAC Partner States requirements; (iii) The timelines for review and registration of medicinal products at regional level have been prolonged primarily due to lack of a regional system and structure dedicated to handle and process applications; non-existence of streamlined application and payment procedures; low quality dossiers by applicants; and sluggish response to queries by applicants; (iv) Slow progress in the establishment of two autonomous medicines regulatory agencies in the Republic of Rwanda and Burundi which hinders integration in medicines regulatory aspects; 3.0 PROPOSED SOLUTIONS Continuity of the EAC centralized product registration and joint GMP inspections is a priority for the Forum of Heads of NMRAs as the current funding phase for the EAC MRH is indicated to be 31st December 2017. The following are proposed solutions; (i) Streamlining administrative and EAC procedures for assessment and GMP inspections both at National and Regional level: EWG for MER & GMP; (ii) EAC to define timeframes for applicants of Product registration to respond to queries; a lapse period should be included that makes the application unviable: EWG on MER (iii) Enhancing capability of EAC shared portal to facilitate sharing of information including assessment reports and assessment outcomes: validation of the IMS system conducted from 30 th August to 1st September 2017; (iv) EAC Secretariat and NMRAs to carry out advocacy including sharing of materials about joint procedures widely to stakeholders; (v) EAC to follow risk based approaches in assessment and GMP inspections/Finalize and desk review procedure for GMP inspections for SRA approved and WHO prequalified products (vi) Appointment of technical officers at NMRAs level to continue implementation of EAC-MRH program at Partner States level; (vii) Development of the concept note for Business Case on EAC Medicines Agency: Interim, Lean coordination unit to be established to sustain joint activities; (viii) Roadmap/Strategy for Post EAC-MRH Implementation to be developed to define EAC priorities and stepwise implementation of regulatory systems strengthening and harmonization initiatives; and Page 50 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) (ix) Mobilization of financial resources and fully integration of Republic of South Sudan into EAC-MRH initiatives. 4.0 ACKNOWLEDGEMENT The EAC Secretariat acknowledges the strong partnership and collaboration by the African Medicines Regulatory Harmonization Programme Partners namely the World Health Organization (WHO), African Union New Partnership for Africa’s Development (AU-NEPAD Agency), Bill and Melinda Gates Foundation (BMGF) and other Development Partners who played a critical role in the success of the project. Our sincere gratitude also goes to Swiss Development Corporation (SDC), Swissmedic, Trademark East Africa (TMEA), Management Science for Health (MSH)/SIAPS, US/PQM and United States Agency for International Development (USAID) The role-played by Implementing agencies, the EAC Partner States and the National Medicines Regulatory Authorities (NMRAs) cannot be overemphasised. The EAC Secretariat Health Department team led by both Dr. Stanley S. Sonoiya and Ms. Jane H. Mashingia was a success factor. The EAC Secretariat management and supporting directorates of Information Technology (IT), Procurement, Financeand Human Resources, EAC Policy Organs and East African Legislative Assembly (EALA) contributed to the success of the project. The East Africa Community Medicines Regulatory Harmonization Programme is a blueprint for the continent, lessons learnt and documented best practices should guide design of similar project for other Regional Economic Communities (REC’s) to address challenges of access to medicines and health technologies. ONE PEOPLE ONE DESTINY Page 51 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) Borrowers’ comments: Borrower/Implementer Comments East African Community • Paragraph 32: The procedure for medicines registration exists and is being reviewed. • In Table 1, the meaning of partial implementation is unclear, and consider NDA is ISO 9001:2015 Certified since April 2018; • Paragraph 37: The initial approach by ZaZiBoNa and EAC MRH were a bit different, we can check and verify. • Table 2, consider information in Annex 5 (Borrower’s Completion Report) and the Report of the 10th EAC MRH Programme Steering Committee on the numbers of registered products per NMRAs. • Paragraph 64: EAC had no resources to fund EAC –MRH activities in January 2016 while the additional financing was being processed. Kenya Pharmacy and • The Pharmacy and Poisons Board adopted the EAC guidelines effective Poisons Board January 2015. • Stakeholder sensitization on EAC Medicines Registration guidelines was done in March 2018. • The decision to exempt local manufacturers of bioequivalence (BE) studies was agreed between the Board and the industry due to lack of BE Study centers in the region, but not due to a lack of alignment between the standardized guidelines and national laws as indicated by World Bank in their report. • The Board is planning to develop a list of products to be exempted from BE as the institution develop an implementation plan for BE compliance. Rwanda Food and • By the time the project was closing, the law establishing Rwanda FDA had Drugs Administration been gazette. Also, Rwanda was the first to implement the standardized guidelines for registration and GMP in January 2015. • Rwanda FDA currently performs all regulatory functions. Page 52 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) WHO Data Collection Tool for the Review of Drug Regulatory Systems The WHO Data Collection Tool for the Review of Drug Regulatory Systems was developed to assess national medicines regulatory systems. It aims at strengthening national regulatory system and control capacity through an assessment of the situation, the identification of specific needs, and the provision of appropriate technical support and training. Specifically, the Tool reviews the existing legal framework, regulations and controls activities with regard to medicinal products and medical devices in order to assess the national regulatory capacity against a set of predefined parameters. It identifies gaps and develops strategies to address them. The structure of the Tool is based on the main regulatory functions of medicine regulatory system: the registration of the pharmaceutical products; the licensing of the different actors; the regulatory inspection; the medicine control laboratory; the monitoring of clinical trials; the control of medicines promotion and advertisement; national regulatory authority, including transparency and confidentiality. There are 18 questions on transparency and confidentiality, 6 of which are used to assess efficacy of transparency in the ICR. These are as follows: • There is a documented policy on public disclosure of information with exemptions/exceptions • Information on legislation, regulations, procedures and guidelines are publicly available and are kept up to date on websites or other mechanisms used to ensure the proper availability of information • The information on decisions are publicly available, including the negative decisions in specific cases (i.e. when legislation permits) in a timely manner • The Information on outputs of regulatory functions performed are publicly available and kept up to date (web sites or other mechanisms) • A competent contact person or a public relations unit is established and known by the interested parties • The NRA regularly organizes meetings with the key stakeholders and open days for the public Page 53 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) References: (1) Key Bank Documents: Project Appraisal Document Aide Memoires Operations Manual Implementation Status and Results Report Sequence 11. December 2017. Available at: Link Implementation Status and Results Report Sequence 10. June 2017. Available at: Link Implementation Status and Results Report Sequence 09. December 2016. Available at: Link Implementation Status and Results Report Sequence 08. June 2016. Available at: Link Implementation Status and Results Report Sequence 07. December 2015. Available at: Link Implementation Status and Results Report Sequence 06. May 2015. Available at: Link Implementation Status and Results Report Sequence 05. November 2014. Available at: Link Implementation Status and Results Report Sequence 04. May 2014. Available at: Link Implementation Status and Results Report Sequence 03. December 2013. Available at: Link Implementation Status and Results Report Sequence 02. May 2013. Available at: Link Implementation Status and Results Report Sequence 01. October 2012.Available at: Link (2) Borrower ICR (3) Data sets used to assess outcomes EAC MRH Annual reports NMRAs Annual Reports for EAC MRH Steering Committee Reports Technical Working Group Reports Heads of NMRAs Forum Reports (4) Analytical studies related to the project EAC. 2009. East Africa Community National Medicines Regulatory Agencies Situational Analysis GIZ. 2011. Baseline Survey of the Local Pharmaceutical Manufacturing Capacity for Human and Veterinary Medicines and Medical Supplies within the East African Community Partner States. The Boston Consulting Group. 2017. EAC Medicines Regulatory Harmonization Program: End –term Evaluation. (5) Other documents African Union. 2015. Executive Decision Council {EX.CL/Dec.857 (XXVI}. Available at: Link Bedir S. (2016). Healthcare Expenditure and Economic Growth in Developing Countries. Advances in Economics and Business 4(2): 76-86. Cambridge Executive MBA (2012). Models for Sustainable Financing of NMRA's in Developing Countries. Page 54 of 55 The World Bank AFCC2/RI-African Medicines Regulatory Harmonization Project ( P128332 ) East African Community.1999. EAC Treaty. Available at: Link East African Community. 2011. EAC Development Strategy (2011/12 – 2015/16). Available at: Link East African Community.2016. EAC Vision 2050. Available at: Link IMS (2014). Africa: A ripe opportunity understanding the pharmaceutical market opportunity and developing sustainable business models in Africa. NEPAD. 2011. African Medicines Regulatory Harmonization Program Strategic Plan (2011-2015). Available at: Link NEPAD.2016. African Medicines Regulatory Harmonization Program Strategic Plan (2016-2020). Available at: Link World Bank.2008. Regional Integration Assistance Strategy for Sub-Saharan Africa. Available at: Link World Bank. 2011. Africa’s Future and the World Bank’s Support to It. Available at: Link World Health Organization. 2007. WHO Data Collection Tool for the Review of Drug Regulatory Systems. Available at: Link World Health Organization.2011. The World Medicines Situation Available at: Link International Policy Network Press Release (2013). Fake drugs kill over 700,000 people every year - new report. Retrieved from: http://archive.is/ipW8i. World Health Organization (2016). Improving the quality of medical products for universal access. Retrieved from: http://www.who.int/medicines/regulation/fact-figures-qual-med/en/. Accessed on 27/05/2018. Page 55 of 55