GPR 57633 THE MEDICINES FOR MALARIA VENTURE GLOBAL PROGRAM REVIEW Volume 1 Issue 2 THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions--the International Bank for Reconstruction and Development (IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their envi- ronment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP Improving Development Results Through Excellence in Evaluation The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group. IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses on assessment of IFC's work toward private sector development, and IEG-MIGA evaluates the contributions of MIGA guarantee projects and services. IEG reports directly to the Bank's Board of Directors through the Director-General, Evaluation. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank Group's work, and to provide accountability in the achievement of its objectives. It also improves Bank Group work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. The Medicines for Malaria Venture June 26, 2007 Sector, Thematic and Global Evaluations http://www.globalevaluations.org ©2008 Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://www.globalevaluations.org E-mail: grpp@worldbank.org All rights reserved This volume is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. It is part of an ongoing series that reviews global and regional partnership programs in which the World Bank is engaged as one of the partners. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. IEG encourages the dissemination of its work and permits these reviews to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring body. Cover photo: A blood-engorged female Aedes albopictus mosquito feeding on a human host. The mosquito's geographic origin is believed to be somewhere in Africa. Photo by James Gathany, courtesy of the US Centers for Disease Control and Prevention. ISBN-13: 978-1-60244-103-3 ISBN-10: 1-60244-103-0 Printed on Recycled Paper Independent Evaluation Group Knowledge Programs and Evaluation Capacity Development (IEGKE) E-mail: grpp@worldbank.org Telephone: 202-458-4497 IEG Mission: Improving Development Results Through Excellence in Evaluation The Independent Evaluation Group (IEG) of the World Bank reviews global and regional partnership programs (GRPPs) in which the Bank is engaged as one partner among many for two main purposes: (a) to provide accountability in the achievement of the program's objectives by providing an independent opinion of the program's effectiveness, and (b) to identify and disseminate lessons learned from the experience of individual GRPPs. The preparation of a global or regional program review (GPR) is contingent on a recently completed evaluation of the program, typically commissioned by the governing body of the program. The first purpose includes validating the findings of the GRPP evaluation with respect to the effectiveness of the program, and assessing the Bank's performance as a partner in the program. The second purpose includes assessing the independence and quality of the GRPP evaluation itself and drawing implications for the Bank's continued involvement in the program. Assessing the quality of GRPP evaluations is an important aspect of GPRs, since encouraging more consistent evaluation methodology and practice across Bank-supported GRPPs is one of the reasons why IEG embarked on this new product in 2005. IEG annually reviews a number of GRPPs in which the Bank is a partner. In selecting programs for review, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming sector studies; those for which the Executive Directors or Bank management have requested reviews; and those that are likely to generate important lessons. IEG also aims for a representative distribution of GPRs across sectors in each fiscal year. A GPR is a "review" and not a full-fledged "evaluation." It assesses the independence and quality of the relevant evaluation; provides a second opinion on the effectiveness of the program; assesses the performance of the Bank as a partner in the program; and draws lessons for the Bank's engagement in global and regional programs. The GPR does not formally rate the various attributes of the program. A GPR involves a desk review of key documents, consultations with key stakeholders, and a mission to the program management unit (secretariat) of the program if this is located outside of the World Bank or Washington, DC. Key stakeholders include the Bank's representative on the governing body of the program, the Bank's task team leader (if separate from the Bank's representative), the program chair, the head of the secretariat, other program partners (at the governance and implementing levels), and other Bank operational staff involved with the program. The writer of a GPR may also consult with the person(s) who conducted the evaluation of the GRPP. Each GPR is subject to internal IEG peer review, panel review, and management approval. Once cleared internally, the GPR is reviewed by the responsible Bank department and the secretariat of the program. Comments received are taken into account in finalizing the document, and the formal management response from the program is attached as an annex to the final report. After the document has been distributed to the Bank's Board of Executive Directors, it is disclosed to the public on IEG's external Web site. ii Abbreviations and Acronyms ACT Artemisinin combination therapy (a relatively recently developed approach to malaria drugs) ADAC Access and Delivery Advisory Committee (MMV) AIDS Acquired Immune Deficiency Syndrome APAC Authorization for Phase III [Clinical Trials] Advancement Committee (MMV) AU African Union BCG Boston Consulting Group CDC Centers for Disease Prevention and Control (USA) CEO Chief executive officer CFO Chief financial officer CODE Committee on Development Effectiveness (World Bank) CRO Contract research organization DALY Disability-adjusted life year (a measure of the burden of disease and injury) DCG Donor Coordination Group (for PD-PPPs, now known as the PD-PPPP Funders Group) DDT Dichloro-diphenyl-trichloroethane (a noxious insecticide used to kill malarial mosquitoes) DDW Diseases of the developing world DFID Department for International Development (United Kingdom) DGF Development Grant Facility (World Bank) DNDi Drugs for Neglected Diseases Initiative (MSF) EDL Essential drugs list EMEA European Medicines Evaluation Agency (EU drug regulatory body) ESAC Expert Scientific Advisory Committee (MMV) EU European Union FDA Food and Drug Administration (USA) G­8 Group of Eight (major industrial countries) GATB Global Alliance for Tuberculosis Drug Development GAVI Global Alliance for Vaccines and Immunization GFATM Global Fund to Fight AIDS, TB, and Malaria GFHR Global Forum for Health Research GMP Good manufacturing practices of medicines (WHO) GPG Global public good GPP Global Programs and Partnerships (World Bank) GPR Global Program Review (IEG­World Bank) GRPP Global and regional partnership programs GSK Glaxo-Smith-Kline (a multinational pharmaceutical firm) HIV/AIDS Human immunodeficiency virus/AIDS HNP Health, nutrition and population sector (World Bank) IAVI International AIDS Vaccine Initiative ICH International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use IEG Independent Evaluation Group (World Bank Group) IFFim International Financial Facility for Immunization IFFnd International Financial Facility for neglected diseases (proposed) IOM Institute of Medicine (USA) IOWH Institute for One World Health (a health research program) I-PRSP Interim Poverty Reduction Strategy Paper IPR Intellectual property rights Logframe Logical framework MDGs Millennium Development Goals MMV Medicines for Malaria Venture MP Member of Parliament (United Kingdom) MSF Médecins sans Frontières (Doctors without Borders) MVI Malaria Vaccine Initiative n/a Not applicable (or not available, according to context) NCE New chemical entity (drug R&D) nd No date NDA National drug authority (for drug regulation and registration) NGO Non-governmental organization NIH National Institutes of Health (USA) OECD Organization for Economic Cooperation and Development OED Operations Evaluation Department, now IEG (World Bank Group) PA Personal assistant PDO Project development objective (World Bank) PD-PPP (Health-related) Product development public-private partnership PPP Public-private partnership PQ Prequalification (of drugs for international procurement and financing) PRSP Poverty Reduction Strategy Paper R&D Research and development RBM Roll Back Malaria Partnership (located in WHO) SDC Swiss Agency for Development and Cooperation SWAP Sector-Wide Approach (a methodology for donor coordination around a country's sector program) TB Tuberculosis TDR Special Program for Research and Training in Tropical Diseases TOR Terms of reference UAE United Arab Emirates UN United Nations UNAIDS Joint United Nations Program on HIV/AIDS UNDP United Nations Development Program UNICEF United Nations Children's Fund USAID United States Agency for International Development VP Vice president WHO World Health Organization Y/N Yes /No Fiscal Year of Program January 1 - December 31 Director-General, Independent Evaluation Mr. Vinod Thomas Director, Independent Evaluation Group (World Bank) Mr. Ajay Chhibber Manager, Sector, Thematic and Global Evaluations Unit Mr. Alain Barbu Global Programs Coordinator, Sector, Thematic and Global Evaluations Unit Mr. Chris Gerrard Consultant Mr. A. Edward Elmendorf v Contents ABBREVIATIONS AND ACRONYMS................................................................................................... II PROGRAM AT A GLANCE: MEDICINES FOR MALARIA VENTURE ...............................................IX KEY BANK STAFF RESPONSIBLE DURING PERIOD UNDER REVIEW ..........................................X GLOSSARY...........................................................................................................................................XI PREFACE............................................................................................................................................ XV SUMMARY ........................................................................................................................................ XVII 1. PROGRAM OBJECTIVES, ACTIVITIES, FINANCIAL RESOURCES, AND GOVERNANCE 1 Objectives and Activities ........................................................................................................ 1 Financial Resources................................................................................................................ 6 Governance .............................................................................................................................. 7 2. THE EXTERNAL EVALUATION OF MMV ............................................................................... 9 Scope, Process, and Approach.............................................................................................. 9 Independence and Quality.................................................................................................... 10 Findings and Recommendations, and MMV Response..................................................... 11 Impact of the Evaluation .......................................................................................................12 3. THE EFFECTIVENESS OF MMV ........................................................................................... 13 Relevance ............................................................................................................................... 13 Efficacy ................................................................................................................................... 17 Efficiency................................................................................................................................ 20 Governance ............................................................................................................................ 22 Sustainability of the Program and Prospects for the Future ............................................ 23 4. WORLD BANK PERFORMANCE IN THE MMV PARTNERSHIP ......................................... 25 The World Bank and Health Research ................................................................................ 30 5. LESSONS................................................................................................................................ 33 vi REFERENCES..................................................................................................................................... 35 ANNEX A. EVALUATION FRAMEWORK FOR GLOBAL PROGRAM REVIEWS............................ 37 ANNEX B. MMV OBJECTIVES, COLLABORATION PRINCIPLES, AND PROGRAM ACTIVITIES 45 ANNEX C. PROGRAM TIMELINE: KEY INTERNAL AND EXTERNAL EVENTS ............................ 55 ANNEX D. MEMBERS OF KEY MMV GOVERNING BODIES........................................................... 59 ANNEX E. MMV FINANCES ............................................................................................................... 64 ANNEX F. PERSONS CONSULTED .................................................................................................. 66 ANNEX G. RECOMMENDATIONS OF THE INDEPENDENT EVALUATION AND PROGRAM RESPONSE ......................................................................................................................................... 68 ANNEX H. RESPONSE OF THE PROGRAM TO IEG'S GLOBAL PROGRAM REVIEW ................ 70 Boxes Box 1. Product Development Public-Private Partnerships (PD-PPPs)........................................... 2 Box 2. MMV's Mission.......................................................................................................................... 3 Box 3. MMV's Public-Private Partnership in Action: The GSK Mini-Portfolio ............................... 5 Box 4. MMV's Objectives and Strategies Are Consistent with the Bank's Global and Sector Strategies............................................................................................................................... 14 Box 5. What Is Involved in Access and Delivery of New Drugs? .................................................. 17 Box 6. Options for Future Bank Engagement with MMV ............................................................... 29 Box 7. Health Research and Agricultural Research: Comparison Between MMV and the CGIAR .............................................................................................................................. 31 Tables Table 1. MMV: Income Received and Pledged, by Source, 2000­2006 .......................................... 6 Table 2. Share of MMV Funds by Source, Initial Plan and Actual, 2000­2005............................... 7 Table 3. MMV Expenditures and Personnel -- Plans and Actuals, 2000­2006............................ 18 Table 4. MMV's Drug Project Pipeline -- Plans and Actuals, 2000­2006 .................................... 19 Table 5. MMV Expenditures, 2000­2006 (US$ millions) ................................................................. 20 Table 6. Disease Burden and Funding Comparison, 2001­2002 .................................................. 21 Table 7. The Bank's Performance as a Partner in MMV ................................................................. 27 Table 8. Global Partnership Programs on Malaria and Other Communicable Disease.............. 32 Annex Table 1. Assessing the Independence and Quality of the Evaluation .............................. 37 Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program......... 38 vii Annex Table 3. Assessing the Bank's Performance as a Partner in the Program ...................... 43 Annex Table 4. Common GRPP Activities ....................................................................................... 44 Annex Table 5. MMV: Downstream Access Prerequisites ............................................................. 50 Annex Table 6. Access Prerequisites: Key Partners and Partner Activities................................ 51 Annex Table 7. MMV Statement of Income and Expenditures for Years Ending December 31 . 64 Annex Table 8. MMV Balance Sheet as of December 31 ................................................................ 65 Annex Table 9. The World Bank's Financial Participation in MMV, 2000­2006 .......................... 65 Annex Table 10. Status of Recommendations and Program Response, as of March 2007 ....... 68 Figures Figure 1. The Long Road to a New Medicine ..................................................................................... 4 Figure 2. MMV's Public and Private Partners .................................................................................... 5 Figure 3. MMV Organization Chart...................................................................................................... 8 Figure 4. The Drug R&D and Supply Chain ..................................................................................... 16 Figure 5. MMV: Available Income and Annual Expenditures on R&D Projects, Administration and Access and Delivery Activity -- Actual, 2000­2007, and Projected, 2008­2010 ..... 24 Figure 6. World Bank: A Small and Declining Share in MMV Financial Resources .................... 26 Annex Figure 1. MMV Portfolio, July 2006: Products at All Stages of Development .................. 47 Annex Figure 2. MMV: Successful Project Transition Rates Used for Portfolio Planning and Management ................................................................................................................... 48 Annex Figure 3. The Evolving MMV R&D Portfolio: An Increasing and Dynamic Number of Projects for Drug Discovery and Clinical Development, 2000­2006, with Terminations.................................................................................................................. 48 Annex Figure 4. MMV: Towards Access and Delivery of New Malaria Drugs .............................. 52 ix Program at a Glance: Medicines for Malaria Venture Start date November 1999 Mission MMV is a not-for-profit foundation dedicated to reducing the burden of malaria in disease endemic countries by discovering, developing and delivering new affordable antimalarial drugs through effective public-private partnerships. Objectives To establish and manage a portfolio of R&D activities for new malaria drugs, leading to at least one new drug on the market by 2010 and one new drug every five years thereafter, that are affordable to low income consumers and patients in developing countries. Activities MMV invests in new drugs by inviting, screening, selecting, financing, and supervising competitive R&D proposals from contract research organizations and other partners in industry, government, and academia. Each potential drug project is subjected to annual renewal or termination, based on advice of the MMV Expert Scientific Advisory Committee. MMV brokers and manages industry-academia research collaborations on potential products. Potential drug products transition through phases from early discovery based on academic research, drug development in increasingly demanding clinical trials, and ultimately, registration with competent national drug authorities, which permits marketing in the public and private sectors. MMV expects its first new product(s) to be registered, prior to the original target, in 2008. With a successful R&D portfolio, MMV has begun to concern itself with the access of patients to new MMV drugs and their delivery at the country level. It has established an Access and Delivery Advisory Committee and initiated a work program of policy and institutional analysis of the operational implications for public and private sector marketing and distribution of its new drugs at the country level, beginning in Uganda. Beyond investing in drug R&D, MMV engages in knowledge sharing and dissemination of information on new malaria drugs, and advocacy of attention to malaria. Its emerging work on access and delivery may lead it into facilitating communication among practitioners and to supporting national level policy, institutional and technical reforms. WBG contributions Active participation in discussions leading to establishment of MMV in 1999. Annual DGF financial support of $500,000 or $750,000 from 2000 through 2006, for a total of $4,750,000 through 2006. Other donor $151 million through 2006. contributions Additional funds pledged (as of December 31, 2006) bring the total to $273 million through 2010. The Bill and Melinda Gates Foundation has contributed 60 percent of MMV's financial resources. MMV also has `mini-portfolios' of R&D with three major private pharmaceutical enterprises, which contribute in kind. Location Offices in Geneva and New Delhi. Research contracts and collaboration at about 80 sites throughout the world. Governance and Organized as an independent non-profit foundation under Swiss law, management comparable to a US 5.01(c)(3) organization. Latest program-level "Independent Review of Medicines for Malaria Venture," report of a four- evaluation person team led by Professor Adetokunbo Lucas, DFID Health Resource Center, May 2005. x Key Bank Staff Responsible during Period under Review Position Person Period Global Program Team Leader Ok Pannenborg 1999­2004 Olusoji Adeyi 2004­present Director, Health, Nutrition and Richard Feachem 1995­1999 Population Sector Christopher Lovelace 1999­2002 Jacques Baudouy 2003­2007 Cristian Baeza (Acting) 2007 ­ present Vice President, Human David de Ferranti 1996­1999 Development Network Eduardo Doryan 1999­2001 Jozef Ritzen 2001­2003 Jean-Louis Sarbib 2003­2006 Joy Phumaphi 2007 ­ present Trust Fund Operations Arif Zulfiqar, Director June 1999 ­ present Global Programs & Margaret Thalwitz, Director May 2004 ­ present Partnerships Program Manager Position Person Period President and Chief Executive Christopher Hentschel 1999 ­ present Officer, MMV xi Glossary Artemisinin A new approach to malaria treatment which combines several drugs, combination therapy including drugs based on an ancient Chinese medicinal plant known as (ACT) artemisinin. ACT treatment is gradually becoming the treatment of choice under many African countries' drug and treatment protocols. ACTs are much more expensive than current standard treatments that have lost their potency. Clinical trials Clinical trials are used to determine whether new drugs or treatments are safe and effective. Trials are in four phases: · Phase I tests a new drug or treatment in a small group of normal human volunteers (normally 20­80 persons) to gain early evidence of effectiveness and safety · Phase II expands the study of effectiveness and safety to a larger group of patients (100­300) with the disease or condition under study to evaluate effectiveness for a particular indication · Phase III expands the study to an even larger group of patients (3,000­ 10,000) to evaluate the overall benefit/risk relationship and provide an adequate basis for physician labeling · Phase IV trials take place after the drug or treatment has been licensed or marketed and provide additional information on the drug's risks, benefits, and optimal use. Devolution or exit A proactive strategy to change the design of a program, to devolve some of strategy its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. Disability-adjusted A measure of life lost to disease or injury which permits comparison across life year health conditions, countries and years. Donor For MMV, any organization or entity that makes a financial contribution to the program that is reflected in MMV's audited financial statements. Contributors of in-kind support, which has been quite substantial in the case of private pharmaceutical enterprises, are not considered donors by MMV. Drug A substance intended for use in the diagnosis, cure, mitigation, treatment or prevention of a disease. Drug development The phases of drug development (normally 5­8 years) from preclinical development through Phase III clinical trials. Clinical development takes place after a compound receives investigational new drug status from the U.S. Food and Drug Administration or equivalent authority in order to obtain the right to test the drug on humans. Drug discovery An early stage of drug R&D, after basic research; drug discovery (normally 5­ 7 years) includes identification of the disease target, such as malaria, that accounts for symptoms; screening of libraries of chemical compounds; identification of compounds whose chemical structures appear to have greatest impact on the target, including those thought to be most promising, and selection of candidates for preclinical and clinical development. Drug registration Formal approval for marketing of a drug by the competent public authority. xii Efficacy The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. The term is also used as a broader, aggregate measure -- encompassing relevance and efficiency as well -- of the overall outcome of a development intervention such as a GRPP. Efficiency The extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results in order to achieve the maximum possible outputs, outcomes, and impacts with the minimum possible inputs. Evaluation The systematic and objective assessment of an ongoing to completed policy, program, or project, its design, implementation, and results. The aim is to determine the relevance and achievement of its objectives, and its developmental effectiveness, efficiency, impact, and sustainability. Generic drugs Non-proprietary pharmaceutical products. Genome The total complement of genes found in a higher life form such as a mosquito. Governance The structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. It is the framework of accountability and responsibility to users, stakeholders and the wider community, within which organizations take decisions, and lead and control their functions, to achieve their objectives. Identification In malaria drug R&D, identification of a biological system or target, the inhibition of which will result in parasite death. Impacts Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. Independent An evaluation that is carried out by entities and persons free from the control evaluation of those involved in policy making, management, or implementation of program activities. This entails organizational and behavioral independence, protection from interference, and avoidance of conflicts of interest. Indication A symptom or circumstance indicating the advisability or necessity of a specific medical treatment or procedure. Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. Legitimacy As a criterion for assessing governance and management, the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large. Logical framework or A management technique that is used to develop the overall design of a logframe program or project, to improve implementation monitoring, and to strengthen evaluation, by presenting the essential elements of the program or project clearly and succinctly throughout its cycle. It is a "cause and effect" model which aims to establish clear objectives and strategies based on a results chain, to build commitment and ownership among the stakeholders during the preparation of the program or project, and to relate the program's or project's interventions to their intended outcomes and impacts for beneficiaries. Malaria endemic A country in which malaria prevails constantly. country xiii Management The day-to-day operation of a program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Monitoring The continuous assessment of progress achieved during program implementation in order to track compliance with a plan, to identify reasons for noncompliance, and to take necessary actions to improve performance. Monitoring is usually the responsibility of program management and operational staff. Neglected diseases Diseases that have received relatively little attention from researchers and policy makers in the industrial world but have significant effects in the tropics. Malaria, TB, and a number of less well known tropical diseases are in this category. The commercial profit motive does not provide sufficient incentive for levels of R&D that could significantly reduce the burden of these diseases. By way of comparison, R&D on so-called "orphan drugs" for rare diseases in the industrial world receives incentives under legislation in the United States, Japan, Australia, the European Union, Singapore and Korea. New chemical entity A new chemical compound to be used in a new drug, as distinct from a reformulation of existing compounds to create a new drug. Outcomes The achieved or likely short-term and medium-term effects of the outputs of a development intervention. Oversight One of the core functions of the governing body of a program: Monitoring the performance of the program management unit, appointing key personnel, approving annual budgets and business plans, and overseeing major capital expenditures. Partners In most global program reviews by IEG, partners are understood as stakeholders who are involved in the governance or financing of the program (including the members of the governing, executive, and advisory bodies). In the case of MMV and in the present report, the term "partners" is used more broadly, to include all entities with which MMV engages in malaria drug R&D and access and delivery activities. Thus, both MMV's industrial and academic collaborating institutions are considered partners, as are collaborating entities in malaria endemic countries where MMV is supporting access and delivery activities. Pharmacovigilance Pharmacovigilance is detection, assessment, understanding and prevention of adverse reactions of patients to drugs -- a response to a drug which is noxious and unintended, and which occurs at doses normally used. Preclinical (drug) This refers to the testing of experimental drugs in a test tube or in animals, development before trials in human beings are carried out. Prequalification Prior approval by a competent authority, such as WHO in the case of drugs, previous to the initiation of another action, such as World Bank-financed procurement. Prequalification is based entirely upon the capability and resources of prospective bidders to perform the particular contract satisfactorily. Prequalification requirements frequently include certification by WHO following a "good manufacturing processes" (GMP) inspection. Public goods Goods which produce benefits that are non-rival (many people can consume, use, or enjoy the good at the same time) and non-excludable (it is difficult to prevent people who do not pay for the good from consuming it). If the benefits of a particular public good accrue across all or many countries, then the good is deemed a global or international public good. xiv Relevance The extent to which the objectives and design of the program are consistent with (a) the current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. Resistance Ability of an organism to develop strains that are impervious to specific threats to their existence. The malaria parasite has developed strains that are resistant to drugs such as chloroquine. The Anopheles mosquito, which transmits the malaria parasite to human beings, has developed strains that are resistant to DDT and other insecticides. Ability to avoid or delay development of resistance is important in R&D for new malaria drugs. Shareholders The subset of donors that are involved in the governance of the program. Therefore, this does not include individual (particularly anonymous) donors who choose not to be so involved, or who are not entitled to be involved if their contribution does not meet the minimum requirement, say, for membership on the governing body. In the case of MMV, the sole donor that presently meets this definition is the Bill and Melinda Gates Foundation. Stakeholders The parties who are interested in or affected, either positively or negatively, by the program. Stakeholders are often referred to as "principal" and "other", or "direct" and "indirect". While other or indirect stakeholders -- such as taxpayers in both donor and beneficiary countries, visitors to a beneficiary country, and other indirect beneficiaries -- may have interests as well, these are not ordinarily considered in evaluations unless a principal stakeholder acts as their proxy. MMV has tended, somewhat in contrast, to see "stakeholders" as financiers, and has only broadened the notion to include others in developing countries in recent years, particularly as it has begun to consider access and delivery issues. Sustainability When the term is applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When the term is applied to organizations or programs themselves, the extent to which the organization or program is likely to continue its operational activities over time. Toxicity A measure of the degree to which something is poisonous. Transparency As a criterion for assessing governance and management, the extent to which a program's decision-making, reporting, and evaluation processes are open and freely available to the general public. This is a metaphorical extension of the meaning used in physical sciences -- a "transparent" objective being one that can be seen through. Value for money The extent to which a program has obtained the maximum benefit from the outputs and outcomes it has produced with the resources available to it. Vector An invertebrate animal, such as a mosquito, capable of transmitting an infectious agent without itself becoming infected. Sources: Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards. Independent Evaluation Group ­ World Bank, 2007, for evaluation terms; Webster's Third International Dictionary, G&C Merriam and Company, 1971, Stedman's Medical Dictionary, 24th Edition, Williams and Wilkins, 1982, Wikipedia; and MMV Web site, for drug-related terms. xv Preface The Medicines for Malaria Venture (MMV) was established in 1999 as an independent Swiss foundation to finance the research and development of new malaria drugs that are affordable in malaria endemic countries. It is a product development public-private partnership (PD-PPP). It was established as a "virtual" non-profit pharmaceutical enterprise, because its R&D is carried out entirely by partners who win MMV contracts after submitting proposals in response to MMV's calls for competitive submissions. Its initial goal was to register one new drug with competent national drug authorities by 2010, and to register at least one additional new drug every five years thereafter. Following initial successes in fund-raising and the establishment of a sound R&D pipeline, MMV changed its founding mission of "discover, develop, register" to "discover, develop, deliver." The activities in MMV's work program to improve the access and delivery of its new malaria drugs, which began in 2006, represent a small share of MMV expenditures. In 2004 the Donor Coordination Group (DCG) on PD-PPPs -- which includes the World Bank and which supports MMV and other ventures for R&D on new products of critical interest to developing country health -- decided to commission an external evaluation of MMV. MMV's Board of Directors was not involved in the design of the evaluation, but cooperated with it. MMV's management was highly responsive and facilitated the work of the team. The evaluation was prepared by a four-person team led by Dr. Adetokunbo Lucas, a distinguished Nigerian Professor of International Health and former Director of the Special Program of Research and Training on Tropical Diseases (TDR). The DFID Health Resource Center managed the evaluation on behalf of the DCG. Following discussions with MMV management on the draft report, the team completed the final report in May 2005. MMV has largely accepted the findings and recommendations of the evaluation, and has used it in its dialogue with financiers. This Global Program Review (GPR) assesses the quality and independence of the 2005 evaluation of MMV; provides a second opinion on the effectiveness of MMV's work; assesses the performance of the Bank as a partner of MMV; and draws lessons for the future of MMV. It covers the period from the beginning of MMV to the present, including key developments during the last two years since the external evaluation was completed in May 2005. MMV was chosen for a GPR because it provides lessons for the design and operation of other global programs -- in particular, for public-private partnerships for health research, and for international support of health research more generally. The Review follows IEG's Guidelines for Global Program Reviews (Annex A). It is based on a desk review of relevant documents including, in addition to the 2005 evaluation, MMV annual reports, consultant studies, journal articles, and Web sites, and discussions in Geneva and London with MMV Board members, MMV managers, MMV staff, and knowledgeable observers. A mission to MMV took place in March 2007. Telephone and office interviews with other stakeholders and people knowledgeable about MMV and health research on developing country problems, including with World Bank staff, complemented the interviews with MMV personnel. xvi IEG gratefully acknowledges all those who made time for interviews, in particular MMV governance members, management, and staff. A list of people consulted can be found in Annex F. Copies of the draft GPR were sent to MMV management, to the Bank unit which is responsible for the Bank's involvement with MMV (the Health, Nutrition and Population Department), and to other Bank units that have responsibility for the Bank's engagement with global programs more generally (the GPP Group, Trust Fund Operations, Operations Policy and Country Services, and the Quality Assurance Group). Their comments have been taken into account in finalizing this GPR. The formal response of MMV management can be found in Annex H. xvii Summary Objectives, Activities, Financial Resources, and Governance 1. MMV is a product development public-private partnership (PD-PPP) that was established as a response to the withdrawal of major pharmaceutical firms from malaria drug research. MMV was an experiment when it was established in 1999, but it is no longer regarded as such, since its business model -- and PD-PPPs more generally -- have become an accepted part of the landscape of drug R&D on diseases of developing countries. While the specific objective that was established on MMV's founding -- the registration of one new malaria drug by 2010 -- has yet to be reached, it is extremely likely that it will be achieved or even exceeded in 2008 or 2009. In this respect, MMV must be considered a success. 2. MMV functions as a "virtual" pharmaceutical research and development company by screening, selecting, financing, and overseeing a portfolio of competitive R&D projects for antimalarial drugs that will be affordable in poor countries. MMV is a "virtual" enterprise because it has no laboratories or research facilities of its own, and carries out its research entirely through other research organizations in the public and private sector throughout the world. A key activity is brokering partnerships linking academic and industry researchers. When a project is not advancing appropriately through the MMV research pipeline, then MMV terminates its financial support. The specific value added of MMV lies in its proactive management of the R&D pipeline. It functions as an efficient allocator of public and private resources to finance potential new malaria drugs. MMV's relatively large portfolio permits it to enjoy internal efficiencies in resource allocation across candidate drugs that could not be realized with a small portfolio. 3. MMV's success led its key donor partners, its Board of Directors and its management to reformulate its founding mission in 2004 from "discover, develop, register" drugs to "discover, develop, deliver" drugs. This shift in mission has led MMV to initiate a work program on improving the access and delivery of malaria drugs. 4. From its inception in 1999 through the end of 2006, MMV has successfully mobilized $273 million in payments and pledges receivable through 2010 from its supporters in the public, private and philanthropic sectors. The Bill and Melinda Gates Foundation has provided 60 percent of MMV resources and pledges, and the World Bank 3 percent. 5. MMV has adopted a corporate approach to governance. Its Board of Directors, which consists of eleven distinguished individuals from industry, academia, and WHO, meets twice a year. Although MMV's donors (aside from the Gates Foundation) are not represented on MMV's Board, MMV is increasingly engaging donor and developing country stakeholders in its work through its annual stakeholders' meetings. MMV's Expert Scientific Advisory Committee (ESAC), its Authorization for Phase III [Clinical Trials] Advancement Committee (APAC) and its Access and Delivery Advisory Committee (ADAC) are also crucial elements in its governance structure. xviii The External Evaluation of MMV 6. MMV's donor partners, including the Bank, commissioned an external evaluation in 2004. MMV's Board cooperated with the evaluation even though it was not involved in commissioning it. Completed in May 2005, the evaluation was carried out by a four-person team led by a distinguished Nigerian public health professor, Dr. Adetokunbo Lucas. DFID oversaw the evaluation on behalf of the donors, and the evaluation report contributed directly to additional financial support for MMV from DFID and the Wellcome Trust. 7. The evaluation team and report were fully independent of MMV management. The evaluation addressed all key aspects of MMV's work, including its governance, its portfolio of new drug R&D projects, the desirability of starting MMV work on access and delivery of new antimalarial drugs, and MMV's financial position, resource requirements, and sustainability. 8. The evaluation found that MMV has made "tremendous progress" and that it is likely to achieve its specific objective of registering one new malarial drug before 2010. The recommendations included supporting the expansion of MMV's mandate to access and delivery, strengthening MMV's engagement with the Roll Back Malaria Partnership (RBM) and the Special Program of Research and Training on Tropical Diseases (TDR), strengthening MMV's portfolio management with new expertise, new tools and additional staff, and undertaking special efforts to establish effective collaborative mechanisms between MMV and WHO. The evaluation proposed an independent review of MMV's interaction with TDR and RBM, but no such review has been carried out. 9. MMV's management has successfully drawn upon the report, particularly in its consultations with donors. The evaluation team recommended that donors should increase their financial commitments to MMV and explore using a replenishment model along the lines of the Global Fund to Fight AIDS, TB and Malaria (GFATM). No such replenishment model has been considered by MMV's donors. MMV is fully aware that, despite its considerable fund- raising success, it needs to expand the number of its donors, and that it faces substantial financial gaps even for currently planned R&D activities. The Effectiveness of MMV 10. This global program review finds that MMV is an effective PD-PPP. RELEVANCE 11. The objectives of MMV are fully consistent with current global challenges in the health sector, and with the needs of beneficiary countries and groups. There is a strong international consensus on the desirability of reducing the prevalence of malaria in disease endemic countries (which are mostly low-income countries) and of the need for global collective action in order to do so. This consensus is reflected in the establishment of RBM in 1998, in the inclusion of combating malaria in the Millennium Development Goals in 2000, and in the creation of GFATM in 2002. The discovery and development of new malaria drugs is a global public good that is undersupplied by the commercial private sector, and there is a growing consensus that PD-PPPs are a sound vehicle for this. It is less evident that MMV's new access and delivery work represents a global public good. Such work will have global public good xix characteristics to the extent that lessons learned in one country are readily transferable to other countries, and to the extent that some policy and institutional issues in relation to access and delivery are international in scope. 12. The Bank's involvement in the founding of MMV was a concrete expression of its 1997 HNP Sector Strategy, which foresaw that the Bank would cooperate with the product pipeline for health-related goods needed by poor people, including malaria drugs. The new 2007 HNP Sector Strategy sets its central goal as strengthening health systems for results on the ground for poor people, and sees the Bank's comparative advantages as including advice to governments on the regulatory framework for public-private collaboration in the health sector -- a crucial element of MMV's new access and delivery agenda. EFFICACY 13. MMV has clearly achieved one of its initial objectives -- of establishing and managing a portfolio of candidates for new malaria drugs passing in appropriate fashion through the various phases of drug discovery and clinical development that precede registration by national drug regulatory authorities. The public sector target price for a full course of treatment is one dollar or less, which is within the performance metrics of MMV. 14. MMV's total expenditures and staff have generally expanded more rapidly than planned, which reflects both prudent planning and success in mobilizing financial resources to support its R&D work. Both the external evaluation and MMV's principal financial supporter (the Gates Foundation) have found that the MMV team should be expanded and strengthened to maintain the momentum generated in its first five years. Along with its expenditures and staff, the number of drug projects at various stages of discovery and development in its R&D portfolio has also exceeded plans. By 2004 MMV's portfolio had become the largest malaria drug research effort ever mounted. However, it will continue to be important to have on-going replenishment of the pipeline of new candidate drugs, in the face of (a) the risks, indeed the expectation, that a number of candidates for new products will be dropped before or at various transitions in the R&D process, (b) the probability that the malaria parasite will develop resistance to new drugs placed on the market, and (c) the need for multiple drugs for different malaria indications (such as children, adults, pregnant women) and for back-up drugs for use if one drug fails to cure the disease in specific patients 15. As MMV moves its access and delivery agenda to the country level, its ability to establish and manage operational linkages with new and different actors will become increasingly important. Because malaria is largely managed at the household level, MMV rightly sees private sector distribution as key to success in reaching the vast majority of the poor population in endemic countries with new drugs. However, MMV will also need to come to grips with a large number of difficult policy and institutional issues, at both the global and national level, before its access and delivery work can be fully effective. Its present lack of institutional comparative advantage in access and delivery work contrasts sharply with its effectively established comparative advantage on malaria drug R&D. xx EFFICIENCY 16. Management and administration, including Board and stakeholder meeting expenses, have represented about 12 percent of MMV spending since 1999, and have been declining as a share of total expenditures. MMV's expected level of spending to bring an average malaria drug to the market is about $150 million, compared with an estimated $800 million in spending by a "big pharma" firm to bring a new chemical compound to the drug market. World-wide R&D funding for malaria in 2001­2002 was about $6.20 for each Disability- Adjusted Life Year (DALY) lost to the disease. This compared with $24.26 per DALY on HIV/AIDS research and with $10.88 per DALY on tuberculosis research. GOVERNANCE 17. MMV's legitimacy has grown in recent years, due to the positive external evaluation, its success in fund-raising, its increasing engagement of researchers in endemic countries, and its holding key meetings in countries where malaria is widespread and MMV-sponsored research is underway. Nonetheless, to further enhance its legitimacy as its Board gives increasing attention to wide strategic issues, MMV may wish to consider increasing developing country membership on its Board and broader engagement with NGOs and others in the design and execution of its access and delivery work program. 18. In light of the commercial and IPR risks involved in MMV's work, confidence and confidentiality are an important feature of its governance. All Board members, all MMV personnel, and all advisory committee members are required to sign a confidentiality and conflict of interest agreement before being allowed access to relevant scientific data. While there have been several conflict of interest issues affecting members of the ESAC, on the whole MMV seems to have handled conflict of interest issues reasonably well. SUSTAINABILITY AND FUTURE PROSPECTS 19. MMV's future spending and financial requirements can be expected to increase. Needs substantially exceed income currently pledged through 2010, and pose a formidable fund- raising challenge to MMV and its financial partners. As of March 2007, MMV estimated its financial gap between $300 and $400 million, according to the budget scenario chosen, but MMV also has a truly remarkable fund-raising record. 20. While the relevance, efficacy, and efficiency of MMV's R&D activities are evident, it is too early to reach conclusions on the relevance, efficacy and efficiency of MMV's new and highly demanding downstream access and delivery activities. This work demands individual and organizational skills, and involves interfaces that are not traditional for MMV. It remains to be seen to what extent and how MMV will be able to reconcile its private sector entrepreneurial style with the public sector requirements for resolution of policy and institutional issues in access and delivery. World Bank Performance in the MMV Partnership 21. The Bank's roles in the MMV partnership have been limited, largely to its work as a co-founder and a small financier. The Bank's HNP leadership played a critical role in the establishment of MMV, and the Bank has made annual DGF grants to MMV of $500,000 or xxi $750,000 from fiscal years 2000 to 2006 inclusive, but these funds represent only 3 percent of MMV resources. The Bank's engagement was important in building confidence between the public and private sectors in support of a new and -- at the time -- untried venture. The Bank's willingness to contribute DGF resources has been important to other official donors, even though the Bank's contribution has represented a declining share of MMV resources. Otherwise, the Bank has been a relatively silent partner: it has not served on any of the governing bodies or committees of MMV, and has only irregularly attended its annual stakeholders' meetings. 22. The Bank's convening power at the global level could become important again in the future, if stakeholders would like the Bank to play a role in reaching common understandings on partner roles in MMV's access and delivery activities. At the country level, the Bank has had almost no contact with MMV activities. Now, with the advent of MMV's access and delivery agenda and the Bank's Malaria Booster program, the possibilities and needs for building linkages between MMV and the Bank's country operations are growing. This new activity merits allocation of Bank budgetary and human resources. The Bank has been largely disengaged from performing oversight of MMV, there being no terms of reference for the staff concerned and minimal administrative budget support. While the details would have to be worked out, future oversight activity could include strategic dialogue and assurance of appropriate public-private balance in MMV's work. 23. Reputational risks to the Bank were high at the time of MMV's establishment, since PD-PPPs were untried. While current risks to the Bank would appear to be low, reputational and country operations risks could increase in the future if the Bank withdraws precipitously or does not engage with MMV at the country level. A disengagement strategy was not considered at the outset and receives little attention now, since the DGF provides its grants to MMV under Window 1 (its long-term grant window). In the future, the issue may be less about disengaging than about redefining the Bank's role in the partnership. Lessons 24. The following lessons have emerged from the experience of the Bank's partnership with MMV: · PD-PPPs represent an innovative and sound model for the discovery and development of new health and medical products of vital public concern to developing countries, including new malaria drugs. As exemplified by the case of MMV, however, such PD-PPPs raise particular issues including achieving legitimacy, establishing an appropriate planning and monitoring framework, and ensuring financial sustainability. · MMV's entry into access and delivery issues raises new policy and institutional challenges for MMV as well as new opportunities for exploiting synergies with the Bank's country operations. Effective coordination and consultation with other key players at the global and country levels will also be essential. · As the interviews undertaken for this study make clear, highly focused global programs such as MMV may look upon the Bank too narrowly, as primarily a source xxii of funds, and could pay greater attention to other possible roles and opportunities to engage the Bank and benefit from multiple Bank roles. Possible new roles for the Bank with respect to advocacy, strategy, and access and delivery are evident at both the global and country levels. · The Bank needs a more proactive and conscious approach to its participation in public-private partnerships such as MMV, but the Bank's budget and incentive systems discourage this. The relevant Bank Departments and task managers need to redefine and communicate the roles which the Bank is willing to play in PD-PPPs as their needs for Bank engagement evolve, as their work programs change, and as the Bank's priorities shift. The Bank's strategic engagement in such partnerships also calls for consultation with its donor partners, as in the Donor Coordination Group for PD-PPPs, which may have expectations of the Bank in relation to oversight and other matters. 1 1. Program Objectives, Activities, Financial Resources, and Governance Objectives and Activities 1.1 Due to the high cost of pharmaceutical research and development (R&D) and a product market limited largely to relatively poor developing country patients, commercial private sector R&D on new malaria drugs slowed in the last decades of the 20th Century and came to a virtual standstill in the 1990s. Public sector research was largely concentrated on basic science and gave little attention to development and commercialization of new products. Faced with this situation and the significant role of malaria in the burden of disease in poor countries,1 a number of key actors -- including WHO, TDR (the Special Program for Research and Training in Tropical Diseases, located in WHO), the Rockefeller Foundation, SDC (the Swiss Agency for Development and Cooperation), British public and private institutions, the pharmaceutical industry, and the World Bank -- began discussions in 1997 concerning the possibility of establishing a public-private partnership for malaria drug R&D. These discussions culminated in the official launch of the Medicines for Malaria Venture (MMV) in November 1999 as a private, non-profit foundation established under Swiss law, with headquarters in Geneva. Initially located with its TDR "incubator," but then moved to separate office space, MMV is one of four recently established "product development public- private partnerships" (PD-PPPs, Box 1). 1.2 Under an agreed mission statement approved by its Board of Directors (Box 2), MMV's specific objective at the time of its founding was to replenish the research pipeline and to register with stringent national drug authorities at least one new product by 2010 and at least one additional product every five years thereafter. MMV functions as a "virtual" pharmaceutical R&D company by screening, selecting, financing, and overseeing a portfolio of R&D projects for antimalarial drugs that will be affordable in poor countries. MMV is a "virtual" enterprise because it has no laboratories or research facilities of its own, and carries out its research entirely through other research organizations in the public and private sector throughout the world.2 A key activity is brokering partnerships linking academic and industry researchers. Intellectual property rights for products developed with MMV support typically remain with MMV. 1. In 1990 malaria was the seventh most important source of loss of disability-adjusted life years (DALYs) in developing countries, representing 2.6 percent of the total burden of DALYs in these countries. (Christopher L. J. Murray and Alan D. Lopez, eds., 1996, The Global Burden of Disease). In 2001 malaria was still reported to be 2.6 percent of lost DALYs in developing countries (Dean Jamison et al., eds., 2006, Disease Control Priorities in Developing Countries, 2nd Edition). Malaria remains among the most significant of infectious diseases. 2. The notion of a "virtual" enterprise is gaining favor in the pharmaceutical industry. Under it a small but skilled and experienced set of agents effectively contracts for the various component parts that would make up an integrated pharmaceutical company, including R&D. Critical skills are writing and managing contracts and managing interfaces with partners. The "virtual" enterprise reduces transactions costs internal to the normal enterprise by making maximum use of information technology and integrative skills (C. James Attridge and Alexander S. Preker, 2005, "Improving Access to Medicines in Developing Countries -- Application of New Institutional Economics to the Analysis of Manufacturing and Distribution Issues"). 2 Box 1. Product Development Public-Private Partnerships (PD-PPPs) Product development public-private partnerships (PD-PPPs) for medical equipment and pharmaceuticals are a new and promising development on the institutional landscape for health improvement in developing countries. PD-PPPs emerged as multinational enterprises disengaged from health R&D on developing country problems. While private enterprise saw drug markets for neglected diseases as non-commercial and of little interest, public actors and foundations sought ways to bring multinational enterprises back into the field while at the same time recognizing that drug development was best left to industry. The Special Program for Research and Training in Tropical Diseases (TDR) -- which is an international public sector entity that is sponsored by WHO, the World Bank and others -- has encouraged the formation of PD-PPPs, even though it was unable to organize PD-PPPs on its own. By 2004 there were four active PD-PPPs in the field of drug development: · Medicines for Malaria Venture (MMV) -- the first PD-PPP and the subject of this review, with 23 projects then under way · TB Alliance, with 9 projects · Drugs for Neglected Diseases Initiative (DNDi) of Médecins sans Frontières (MSF), with 6 projects · Institute for One World Health, with 3 projects. PD-PPPs do not start from a specific candidate product to be developed, but from a survey of the field. Then they promote parallel development of a range of candidate products, in a portfolio of projects. Portfolio management -- a notion borrowed from pharmaceutical enterprise and venture capital -- aims to manage the risk of failure by pursuing the development of a range of products simultaneously. PD-PPPs have been financed by an influx of new public and especially foundation funds. Donors, including the World Bank, established a Donor Coordination Group on PD-PPPs in 2004. One of the Group's first tasks was to oversee the independent evaluation of MMV in 2004­05. The Group has since been renamed PD-PPP Funders Group, but the original name is used in this report. Sources: Moran et al. "The New Landscape of Neglected Disease Drug Development," London School of Economic and Political Science and Wellcome Trust, 2005; and Widdus, Roy and Katherine White, "Combating Diseases associated with Poverty -- Financing Strategies for Product Development and the Potential Role of Public-Private Partnerships," Initiative on Public-Private Partnerships for Health, Global Forum for Health Research, November 2004. 1.3 MMV follows a managed innovation business model that does not lend itself to ex ante investment planning like a typical Bank-financed public sector project, captured in a logframe established prior to final approval. Indeed, no logframe has been prepared by or for MMV.3 However, an initial business plan was completed in the year 2000, with support from the Boston Consulting Group. An updated plan for 2003-2007 was prepared in 2003, also with consulting assistance, and a new plan for 2007-2012 is now under preparation. Within the overall vision of supporting the development of new malaria drugs, MMV's specific objectives have evolved flexibly as its work programs and portfolio of drug R&D projects have changed and as its financial position has improved. In the last several years, for example, it has added activities aimed at facilitating the marketing and distribution of the products of its R&D. 3. In the Bank's internal Operations Portal, the registration of new products by 2010 is given as the sole PDO indicator for MMV; no other measurements or intermediate outcome indicators are given. 3 Box 2. MMV's Mission "Medicines for Malaria Venture is a not-for-profit foundation dedicated to reducing the burden of malaria in disease endemic countries by discovering, developing, and delivering new affordable antimalarials through effective public-private partnerships." Source: Approved by MMV's Board of Directors and included in MMV's 2003, 2004 and 2005 Annual Reports. 1.4 In fact, MMV is a long-term program requiring continuous adaptation and change, rather than a single project. MMV's portfolio management aims to increase the probability that any given candidate drug will transition through the various stages of drug R&D successfully, and to eliminate unsuccessful candidates as early as possible. Therefore, Bank oversight of the program is also quite different from supervision of a tradition Bank investment project. Oversight entails ensuring that the organization has the funds and latitude to manage its portfolio with maximum chances of success, monitoring its governance (including continuous and appropriate balance between public and private interests), following its work program and stakeholder meetings, and ensuring appropriate ex post evaluation. 1.5 As a product development public-private partnership (PD-PPP), MMV publicly invites competitive R&D proposals for malaria drugs. An Expert Scientific Advisory Committee (ESAC) screens letters of interest submitted in response to its calls for proposals, explores research proposals, interrogates researchers, and advises on the proposals before selection and during execution of MMV-financed research contracts. Final decisions are taken by the MMV management. MMV's principles for grant allocation have not been gathered into a single policy statement, and vary from one call for proposals to another, according to the state of its portfolio. However, any research proposal is expected to have the potential to lead to an affordable drug.4 The MMV Chief Scientific Officer and his staff broker partnerships between academic and industrial researchers and provide continuing oversight of projects in the research portfolio. The ESAC assists MMV in balancing its portfolio with projects at different stages on the long road to a new medicine (Figure 1), and in managing project terminations. An Authorization for Phase III Advancement Committee (APAC) provides input, advice and recommendations on the advance of late-stage projects to the very expensive and critical Phase III clinical trials which, if successful, lead to product registration. 1.6 If an R&D project is not advancing appropriately through the various phases of the MMV research pipeline, MMV terminates its financial support. Reasons for termination of projects in 2005 included slow progress, toxicity in preclinical studies, no clear use for the potential product in Africa, lack of a pharmaceutical partner, lack of advantage over other products, and failure to meet MMV's standard target treatment profile, including affordability in developing countries.5 4. In this respect, MMV is unlike a public sector grant agency, such as NIH, which is oriented towards basic scientific research rather than the identification and promotion of potential medical treatments and drug products. 5. MMV, Annual Report 2005. 4 Figure 1. The Long Road to a New Medicine Clinical Data Registration Analysis Full Development Studies in 100-300 Patients (Phase II) Candidate Medicine Tested in 3-10,000 Patients (Phase III) Large Amounts of Candidate Medicine Synthesized Extensive Safety Studies Studies in Healthy Candidate Volunteers Phase I Formulations Developed Exploratory Development Early Safety Project Team Studies Synthesis Screening and Plans of Compounds Discovery Source: MMV. 1.7 MMV staff come from the commercial private sector, and a private sector entrepreneurial spirit characterizes the MMV organizational culture. As of the end of 2006, MMV had 35 projects at various stages in the research pipeline. Beyond these individual projects, MMV has developed mini-portfolios of projects with three of its industry partners. These collaborative relationships cover GSK in Spain (Box 3), the Broad Institute of MIT/Harvard and Genzyme Pharmaceuticals, and Novartis Institute for Tropical Diseases. MMV has approximately 80 partners, around the world (Figure 2). The specific value added of MMV lies in its proactive management of the R&D pipeline. It functions as an efficient allocator of public and private resources to finance potential new malaria drugs. MMV's relatively large portfolio permits it to enjoy internal efficiencies in resource allocation across candidate drugs that could not be realized with a small portfolio. 1.8 MMV expects to register three or possibly four new malaria drugs with stringent national drug authorities over the next year or two. It is therefore virtually certain to meet its initial goal of registering one new drug by 2010. 1.9 MMV's success led its key donor partners, its Board of Directors and its management to reformulate its founding mission in 2004 from "discover, develop, and register" drugs to "discover, develop, and deliver" drugs. This change was particularly driven by concern among MMV's principal financiers that it might succeed in registering new drugs and that these drugs would subsequently face institutional and policy hurdles in poor countries that would effectively inhibit access to them. In light of this change, MMV established an Access 5 Box 3. MMV's Public-Private Partnership in Action: The GSK Mini-Portfolio · MMV provides funding and management. · University of California at San Francisco (UCSF) contributes parasitology research experience and investigates drug candidates that might inhibit the malaria parasite from attacking human blood cells. · GlaxoSmithKline (GSK) -- a major pharmaceutical enterprise -- contributes experience in medicinal chemistry to identify compounds that selectively inhibit the key malaria parasite enzyme. · GSK contributes certain rights under its patents relevant to malaria treatment. · GSK contributes laboratory facilities and services needed to optimize and select drug candidates for development. · MMV gets free and unfettered access to key research outputs, including intellectual property rights within its field of interest. Source: "MMV Concept Paper for the World Bank," downloaded from the Bank's operations portal. and Delivery Advisory Committee (ADAC) in 2006, which is chaired by the Executive Director of the Roll Back Malaria (RBM) Partnership. (RBM is located in WHO not far from MMV in Geneva.) In lieu of directly addressing the global policy and institutional problems that might impede access and delivery of its new malaria drugs, MMV decided to begin with intensive, country-specific studies -- Uganda being the first country to partner with MMV in such activity. The Boston Consulting Group is also working with MMV on these matters.6 On the whole, MMV has adopted a fairly general approach to access and delivery of its new Figure 2. MMV's Public and Private Partners Liverpool School of Tropical Medicine University of California, San Francisco University of Liverpool Oxford University University of Washington GlaxoSmithKline LSHTM Immtech International STI Korea Shin Poong Pharm. Inc. GlaxoSmithKline F. Hoffmann-La Roche University of Iowa Novartis Pharma WHO/(TDR) Yale University Sigma-Tau Industrie Farmaceutiche Riunite Albert Einstein Holley Pharma College of Medicine GlaxoSmith Kline Ranbaxy Laboratories Limited WRAIR Tres Cantos Texas A&M National Institutes of Health (NIH) University of North Carolina Chapel Hill BIOTEC University of Mississippi University of Nebraska Medical Center Mahidol University University of Buea, Cameroon = Pharmaceutical partner = University/institute = National research institute Monash University = Clinical trial site ( = under review) = International agency Source: MMV. 6. BCG carried out a detailed "Planning for Success" study on post-registration issues for MMV in 2005. 6 drugs. Its limited specificity may turn out to be helpful to MMV as its work program evolves in response to occasionally conflicting needs and pressures. 1.10 Annex B provides a more detailed description of MMV's objectives, collaboration principles, and program activities, including its work on (a) drug discovery and development and (b) access and delivery of its new drugs. Annex Figure 1 on MMV's portfolio of R&D projects in July 2006 shows that MMV has products at all stages of discovery and development. It captures the various phases through which a candidate drug must pass before it is registered with competent national drug authorities for marketing. Financial Resources 1.11 From its inception late in 1999 through the end of 2006, MMV has successfully mobilized $273 million in payments and pledges receivable through 2010 from its supporters in the public, philanthropic, and private sectors. (Table 1) 1.12 MMV's sources of funds have been very different from expectations in MMV's initial business plan (Table 2). The difference is principally attributable to the Bill and Table 1. MMV: Income Received and Pledged, by Source, 2000­2006 Donor Years US$ Millions Percent of Total Governments United Kingdom (DFID) 2000­2010 29.2 11% Netherlands 1999­2009 17.2 6% Ireland 2006­2008 10.8 4% Unites States (USAID) 2004­2007 8.0 3% Switzerland (DEZA/SDC) 2000­2007 4.5 2% Subtotal 69.8 26% UN Agencies World Bank (DGF) 2000­2006 4.8 2% WHO (Roll Back Malaria) 2001 only 3.5 1% Subtotal 8.3 3% Philanthropy and Corporate Bill and Melinda Gates Foundation 2000­2010 165.0 60% Wellcome Trust 2002­2010 20.8 8% Rockefeller Foundation 2000­2006 5.8 2% Exxon Mobil Foundation 2000­2008 2.9 1% BHP Billiton 2004­2006 0.8 0.3% Individual Donors 2003 & 2005 0.006 0.002% Subtotal 195.2 71% Grand Total 273.2 100% Source: Peter Potter-Lesage, "Presentation to Spanish Agency for Development Cooperation," March 5, 2007, MMV. 7 Table 2. Share of MMV Funds by Source, Initial Plan and Actual, 2000­2005 UN Agencies Government Foundations Corporations and other Int'l Agencies and Corporate Organizations Foundations Initial Business Plan 15% 40% 30% 15% Actual, 2000­2005 3% 20% 77% 1% Source: Peter Potter-Lesage, "Presentation to Spanish Agency for Development Cooperation," March 5, 2007, MMV. Melinda Gates Foundation, which has provided $165 million or 60 percent of MMV's resources. Private industry has provided less financial support through firms and corporate foundations than was anticipated at start-up, but has provided important support through in- kind contributions in the execution of MMV-supported research. For business reasons these enterprises have declined to quantify the value of such in-kind contributions, but discussions with MMV staff and with GSK suggest that these must be quite substantial. Governance 1.13 MMV has adopted a corporate approach to governance. Its Board of Directors, consisting of eleven distinguished individuals from industry, academia and WHO, meets semi-annually and plays an active role in the organization. MMV's statutes prohibit Board members from serving more than two three-year terms. The MMV Board has separate committees responsible for audit, personnel remuneration, and nominations (Figure 3). Succession is organized through the Board's Nominating Committee. The Board has discussed establishment of a Development Committee with responsibility for fund-raising, but has so far decided not to create one. Aside from the Bill and Melinda Gates Foundation, MMV's donors are not represented on its Board of Directors. 1.14 MMV has an annual public "stakeholders' meeting," somewhat comparable to the annual general meeting of a private corporation, except that the Board of Directors is not formally accountable to the stakeholders' meeting.7 The stakeholders' meeting normally takes place just after a Board meeting. MMV's initial business plan consciously chose to provide for representation of financial partners through the stakeholders' meetings rather than through the Board of Directors. Starting in 2004, MMV has adopted the practice of having the stakeholders' meeting and one of the semi-annual Board meetings in a malaria endemic country in order to enhance its responsibility to its developing country partners and beneficiaries. Meetings have been held in Mozambique, Thailand, Uganda, and Zambia.8 The 2007 meeting took place in Uganda, where MMV launched major access and delivery studies. 7. Nearly all residents of malaria endemic countries can be considered indirect stakeholders. In MMV, however, the term is used somewhat more narrowly to include only stakeholders who are directly concerned, including financial partners and Ministry of Health and other public officials. (See glossary.) 8. A separate special meeting of the Board was held in the United Arab Emirates. 8 Figure 3. MMV Organization Chart Remuneration Board of Board of Audit & Nominations Directors Directors Committee Committees President & Chief President & Chief PA to CEO & Executive Officer International Web Executive Officer Operations Administrator Director Chief Business CFO & Director VP VP Scientific Dev. Donor Global Public Operations Officer Mgr Relations Access Affairs Expert Scientific Authorization for Access and Delivery Advisory Committee Phase III Advancement Advisory Committee (ESAC) Committee (APAC) (ADAC) Source: MMV. 1.15 Three expert advisory committees -- for drug R&D, for review prior to launching Phase III clinical trials, and for access and delivery of MMV drugs -- form an important part of the MMV governance structure. The Expert Scientific and Advisory Committee (ESAC), which is responsible for reviewing R&D proposals and progress, consists currently of nineteen drug development experts and research scientists from private industry and academia. Members of the ESAC devote considerable time and thought to their MMV activities, largely on a pro bono basis. Membership has been adjusted and expanded as MMV has required different types of highly specialized expertise for its review of individual research projects and advice on balancing its portfolio among candidate drugs at different stages of discovery and development. 1.16 The Access and Delivery Advisory Committee (ADAC) consists of a more diverse group of fourteen members, particularly (over 80 percent) from malaria endemic countries. Observers are invited to ADAC meetings from the World Bank, the Global Fund to Fight AIDS, TB, and Malaria (GFATM), the Roll Back Malaria (RBM) Partnership located in WHO, UNICEF's Procurement and Supply Division, and the WHO GMP Department. 1.17 The Authorization for Phase III Advancement Committee (APAC), provides input, advice and recommendations for the advancement of late-stage projects onto the very expensive and critical Phase III clinical trials which, if successful, lead to product registration. Unlike other MMV advisory committees, members of the APAC participate in an institutional capacity (see Annex D). The critical practical effect of the membership structure of the APAC is to give WHO and the Bill and Melinda Gates Foundation a formal voice, beyond their ongoing informal contacts, in decisions to move MMV products into Phase III clinical trials. 9 1.18 MMV's governance structure is completed with an internal Executive Management Committee consisting of the President and CEO and his senior staff. At the end of 2006 the total staff was 21. The Executive Management Committee takes decisions on the basis of the advice of MMV's expert advisory committees. The enterprise is widely perceived to have a proactive management that is responsive to others' concerns. The number of staff is increasing as the work of MMV expands in quantity and scope, and personnel issues are becoming increasingly important in MMV's management: The Chief Scientist is soon to retire, a Medical Director is being hired, and efforts to recruit a senior Vice President for Access and Delivery are currently on hold. 2. The External Evaluation of MMV Scope, Process, and Approach 2.1 The 2005 external evaluation of MMV was commissioned by its donor partners, including the Bank, and not by the MMV Board. The members of the Board viewed the evaluation as primarily meeting the needs of donors. However, this seems to have had no negative consequences for the terms of reference or conduct of the evaluation. MMV's management and staff fully accommodated the evaluation, and the MMV Board appears to have been involved where necessary. The evaluation team presented their report to the Board, which has responded positively to the evaluation results. As the principal sponsor, the DFID Health Resource Centre managed the evaluation, while involving other donors (including the World Bank) in drafting the terms of reference, selecting the evaluation team, and reviewing the draft study. 2.2 The joint evaluation was a good example of donor coordination in an environment where donors were grappling with the problem of measuring effectiveness. Comments from Bank staff on the draft terms of reference stated that the scope of work and approach were very much in line with Bank thinking, but called for greater specificity on how the evaluation would integrate country-level stakeholders and perspectives, and expressed some concern that the evaluation was not be commissioned by the MMV Board. The final terms of reference were detailed, and particularly responsive to the needs of DFID. The selection of the evaluation team was not competitive. 2.3 The evaluation was carried out by a team of four specialists in different technical areas relevant to MMV's work. It was led by a distinguished Nigerian health researcher, Prof. Adetokunbo Lucas, who is a retired Professor at the Harvard School of Public Health and former Director of the Tropical Disease Research (TDR) program based at WHO. The other members of the team were Keith Bragman, a British expert in clinical pharmaceutical development with extensive public and private sector experience in chemotherapy for infectious diseases; Alan Fairlamb, a British drug research expert devoted to the development of better treatments for tropical parasitic diseases; and Hassan Mshinda, a Tanzanian expert in malaria, parasitology, entomology, research, epidemiology and clinical trials. Industry and financial perspectives were less represented on the team than might have been appropriate. The evaluation was carried out in late 2004 and completed in early 2005; the final report was issued in May. Members of the evaluation team visited three MMV-supported research sites, 10 and observed an ESAC meeting. A draft report was presented by the evaluation team to the MMV staff, and their feedback was taken into account. The cost of the evaluation was about $150,000. Independence and Quality 2.4 The evaluation team and report were fully independent of MMV, despite the fact that the world of malaria drug experts is a rather narrow one heavily concentrated in the United Kingdom. The evaluators were responsible to the informal donor coordination group (DCG) which commissioned the evaluation, and accountable to and financed by it.9 The evaluation process was managed by DFID on behalf of the DCG, and the effective accountability was to DFID. The TOR were drafted largely by DFID, and the evaluation principally served to meet DFID's needs.10 Procedures for review of the draft report were not specified in the terms of reference. From interviews carried out for this study and examination of the evaluation report, it is apparent that the evaluation team did not hesitate to disagree with MMV and to make recommendations that would not be routinely accepted by MMV. The full text of the evaluation is available on the MMV Web site. 2.5 The evaluation addressed all key aspects of MMV's work, including its governance and management, its portfolio of new drug R&D projects, the desirability of starting MMV work on access and delivery of new antimalarial drugs, and MMV's financial position and resource requirements. The evaluation team could perhaps have been more selective in its coverage of issues, but the TOR asked for many issues to be addressed. The evaluation did not follow a results-based framework based on objectives and targets established at MMV's inception, but this would probably have been inappropriate in light of the dynamic management of the project portfolio and the rapidly changing nature of the requirements for work by MMV. For instance, an evaluation framework based entirely on a logframe established at MMV's inception in 1999 might have overlooked the critical issue of access and delivery. From the standpoint of the principal sponsor, DFID, the final report was fully responsive to the objective of the evaluation to provide an independent assessment to inform DFID financing decisions affecting MMV. 2.6 In retrospect it appears that the terms of reference and team composition could have given greater attention to the business and financial management aspects of MMV's work, for which the team drew extensively on MMV materials but offered little independent assessment.11 The approach which the team took to the evaluation was fairly consistent with IEG's expectations for an established program over five years old. The Sourcebook for Evaluating Global and Regional Programs calls for the evaluation of an established program like MMV to focus on the functioning of operations as designed, the sources and uses of funds, whether targets are being met, whether goals are being met, and whether the strategic 9. At the time the evaluation was commissioned in late 2004, the members of the Donor Coordination Group were The Bill and Melinda Gates Foundation, the Rockefeller Foundation, the Wellcome Trust, DFID, the Swiss Agency for Development and Cooperation (SDC), the Netherlands, USAID, and the World Bank. 10. As discussed below, the evaluation was quite successful and important in this respect. 11. This observation is not intended to suggest weak financial management, but only to underscore the critical importance of financial resource management and mobilization to the work of MMV. 11 direction is appropriate.12 The MMV evaluation also addressed outcomes in terms of likely drug product registrations, sustainability, and the need for long-term, continuing support -- evaluation issues which are more characteristic of evaluations of more mature programs. Findings and Recommendations, and MMV Response 2.7 The principal findings of the evaluation were: MMV has made "tremendous progress" and is likely to achieve its specific objective, before 2010, of one new antimalarial drug registered by 2010 and one new drug every five years thereafter. MMV has successfully mobilized academia and the private sector for new drug development in highly productive partnerships. MMV has established an impressive set of malaria drug candidates, including some novel compounds at early stages of development, through effective management of its R&D portfolio. The unexpected speed of MMV's work has created an urgent need to expand and accelerate work on downstream issues relating to access and delivery of MMV products, and to strengthen and utilize other partnerships to this end. 2.8 The recommendations for action of the evaluation were many and detailed. Highlights include: Stakeholders should support the expansion of MMV's mandate to "discover, develop, deliver." MMV should strengthen ESAC's technical competence in the design and execution of clinical trials, and introduce honoraria for ESAC members in recognition of their contributions. MMV should engage TDR and RBM in ESAC. MMV should strengthen its portfolio management with new expertise, new tools and additional staff. MMV should undertake special efforts to establish effective collaborative mechanisms between MMV and WHO and its other partners, and should organize a high-level independent review of MMV's interaction with TDR and RBM.13 Donors should sustain and increase their financial commitment over the next five years to ensure MMV success, and be mindful of the risks of failure in contemplating their future funding strategy. 2.9 MMV's Board and management have responded positively to the evaluation. Annex G contains a summary of the key findings and recommendations of the evaluation team, as set out in the evaluation study, along with a current status report on each of them from the 12. Independent Evaluation Group­World Bank and DAC Network on Development Evaluation, 2007, Sourcebook for Evaluating Global and Regional Partnership Programs -- Indicative Principles and Standards, pages 34­35. Evaluations of more mature programs are also expected to give greater attention to impacts and potential changes in strategic direction, including devolution and possible exit. 13. The evaluation text was worded in such a fashion as to address this recommendation, indirectly, to both MMV and WHO. 12 MMV staff. MMV management took exception to one point in the draft evaluation, namely, the description of the MMV management and staff as a "secretariat." MMV personnel thought that this incorrectly characterized MMV in administrative terms as being comparable to a public sector bureaucracy such as TDR, in contrast to the entrepreneurial private sector world from which they came and which they were endeavoring to replicate in their MMV portfolio management approach. Some tension remains between the largely public sector orientation of the evaluation team and of WHO and TDR, and the largely private sector background and organizational culture of MMV.14 Nonetheless, working relations between MMV and RBM are excellent, and the RBM Executive Director chairs the MMV Access and Delivery Advisory Committee. The recommendation of the evaluation team for an independent review of cooperation between MMV and WHO, and the establishment of special collaborative mechanisms between them, has had no specific MMV follow-up. 2.10 Mindful of the importance of long-term funding of MMV for its success in mobilizing its drug R&D partners and appropriately managing its R&D portfolio, the MMV evaluation team recommended that the donors utilize a replenishment model for product development public-private partnerships (PD-PPPs) similar to that of GFATM, which involves high level annual pledging conferences bringing together its major official donors. According to MMV, no such replenishment model has since been discussed or is being considered by donors. MMV is fully aware that it needs to continue and expand its efforts to increase the number of bilateral donors and the level of their support. Impact of the Evaluation 2.11 The evaluation has enhanced MMV's legitimacy by making it better known and giving it a stamp of endorsement from well-regarded, independent outsiders. The evaluation has validated the shift in MMV's agenda, which was already underway, to include access and delivery of new drugs in its mission. But the biggest tangible impact has been the increase in donor financing subsequent to the evaluation. The evaluation has contributed, in the spirit of an advocacy document, to an environment encouraging donor assistance to MMV. MMV has skillfully drawn upon the evaluation in its annual reports and dialogue with donors. Donor financing, particularly but not exclusively through the Gates Foundation, has grown remarkably since the evaluation. More specifically, the positive results in the evaluation have facilitated additional financial support from DFID and the Wellcome Trust. 14. This is not to suggest that TDR has not collaborated with private industry. While TDR has tended to concentrate on basic research, it has a history of such collaboration, and served as the incubator for ideas that led to the establishment of MMV (W. E. Gutteridge, 2006, "TDR Collaboration with the pharmaceutical industry"). 13 3. The Effectiveness of MMV15 Relevance 3.1 MMV's mission of reducing the burden of malaria in disease endemic countries by discovering, developing and delivering new affordable antimalarial drugs, and its objective of registering at least one new drug with stringent drug authorities by 2010 and at least one additional drug every five years thereafter, are fully consistent with current global challenges and concerns in the health sector, and with the needs and priorities of beneficiary countries and groups. 3.2 There is a strong international consensus on the desirability of reducing the prevalence of malaria in disease endemic countries (which are mostly low-income countries) and of the need for global collective action in order to do so. This consensus is reflected, among other things, in the establishment of the Roll Back Malaria Partnership (RBM) in 1998, in the inclusion of malaria in the Millennium Declaration by the UN General Assembly and the MDGs in 2000,16 and in the creation of the Global Fund to Fight AIDS, TB and Malaria (GFATM) in 2002.17 The Bank's involvement in MMV is also consistent with the Bank's strategic framework for global programs and partnerships, with the Bank's 1997 and 2007 health sector strategies, and with the establishment of the Bank's Malaria Booster Program in 2005 (Box 4). Malaria is a significant contributor to low incomes and poor aggregate growth. At the microeconomic level, estimates of the direct and indirect costs of malaria vary, from 0.75 percent of GNP in Pakistan to as much as 7­13 percent in Nigeria and 9­18 percent for small farmers in Kenya.18 3.3 This international consensus is shared by beneficiary countries as reflected, among other things, in the targets established by the Abuja Summit in April 2000 (attended by over 50 malaria endemic countries) that (a) by the year 2005 at least 60 percent of those suffering from malaria should have prompt access to affordable and appropriate treatment within 24 hours of the onset of symptoms, and (b) by the year 2010 malaria mortality should be reduced by 50 percent. In spite of the Abuja Summit and other actions at the sectoral level, the extent of ownership of the malaria issue by the Bank's client countries in Africa is less clear at the level of macroeconomic strategy. An analysis of PRSPs and I-PRSPs for 27 African countries showed that the inclusion of strategies and actions to achieve malaria control targets was generally low.19 15. This section of the GPR takes the evaluation as its point of departure but goes beyond it with information from outside sources, including especially interviews carried out for the GPR. See Annex F for persons consulted. 16. MDG 6 to "combat HIV/AIDS, malaria, and other diseases" establishes a target to "have halted by 2015 and begun to reverse the incidence of malaria and other major diseases." 17. For additional evidence, see also Annex C on key internal and external events in relation to the establishment and operation of MMV. 18. As summarized from various sources in World Bank, 2005, Rolling Back Malaria ­ The World Bank Global Strategy and Booster Program. 19. World Bank, 2005, Rolling Back Malaria ­ The World Bank Global Strategy and Booster Program. 14 Box 4. MMV's Objectives and Strategies Are Consistent with the Bank's Global and Sector Strategies The Bank's participation in MMV is consistent with the Bank's strategic framework for global programs and partnerships (of May 2005) which identifies the control of communicable diseases including malaria as one of the five priority global public goods issues for Bank engagement. This includes "vaccines and drug development for major communicable diseases in developing countries." About 10 percent of the Bank's DGF resources are allocated to global health programs for controlling communicable diseases. The Bank's involvement in the founding of MMV in 1999 was a concrete expression of its 1997 HNP Sector Strategy. In accordance with its three main objectives -- (a) improving health outcomes for the poor, (b) enhancing the performance of health systems, and (c) securing sustainable financing -- the Strategy stated that the Bank would strengthen its collaboration with other agencies on international health R&D, including through the Global Forum for Health Research. Viewing the Global Forum as a mechanism to focus R&D resources more tightly on priority subjects, the Strategy stated that the Bank would collaborate, in partnership with the Global Forum, with the pipeline for health products needed by poor people in low income countries, including new malaria drugs. The Strategy also observed that Bank clients are often left out of partnerships. The new HNP Sector Strategy, endorsed by the Bank's Board in April 2007, sets its central goal as strengthening health systems for results on the ground for poor people. The Strategy foresees improved strategic engagement and agreement with global partners on a collaborative division of labor for the benefit of client countries, and sees the Bank's own comparative advantage as being greater in the access and delivery of products and services than in the creation of medical technologies such as malaria drugs. But, without new malaria drugs, the economic and social benefits of Bank projects and activities aimed at health system strengthening in poor endemic countries would be reduced. The new Strategy observes that the extensive engagement of the HNP Anchor with 55 organizations or initiatives at the global level poses internal challenges for the Bank. The action plan associated with the new HNP Strategy calls for the review and reorientation of the DGF's health sector grants towards areas of Bank comparative advantage. The HNP Anchor expects to have clear decisions about the strategic roadmap for these grants early in FY08. 3.4 Since the establishment of MMV in 1999, the relevance of its central objective of bringing new affordable malaria drugs onto the market in poor countries has increased, as donors have increased their pledges of development assistance, especially for health and especially in Africa. All the major global and regional initiatives with respect to malaria (Annex C) presume, and some explicitly support, the development of new, affordable malaria drugs to take the place of existing drugs whose effectiveness is rapidly declining. 3.5 The discovery and development of new, affordable drugs for the treatment of neglected diseases such as malaria that are endemic in many countries is clearly a global public good that is undersupplied by the commercial private sector. Once a new drug is registered with stringent national drug authorities, the expenditures involved in its discovery and development (including three phases of clinical trials) are sunk costs, and the marginal social cost of the knowledge contained in the new drug is zero. Although research-based pharmaceutical companies that could undertake the discovery and development of new drugs could patent this intellectual property, they have little interest in markets for drugs to treat neglected diseases in low-income countries.20 Private sector R&D for neglected diseases such as malaria declined from 1975 until the end of the 1990s, and only 13 new drugs for neglected diseases were 20. Mary Moran et al., 2005, The New Landscape of Neglected Disease Development. 15 developed during this period. Without some public intervention, the private sector alone cannot be expected to discover, develop and market new antimalarial drugs. 3.6 At the time of MMV's establishment in 1999, relying on the "pull" of effective market demand to drive the development of new drugs through private pharmaceutical R&D would not have been effective because the private sector had effectively withdrawn from drug R&D on its own account. There was no practical alternative to intervening at the global level with "push" resources for the production of this evident public good. It also made sense for donors to pool their resources since the development of a stream of new antimalarial drugs might be considered a "best shot" global public good.21 For donors to contribute additional resources to TDR to discover and develop antimalarial drugs might have been a possibility, but TDR is a research-driven organization, whereas MMV is product-driven. Furthermore, TDR's almost exclusively public sector focus, and its broad mandate across eight diseases, risked losing a focus on specific products to combat malaria. 3.7 Since 1999, there has been a growing consensus that public-private partnerships for product development (PD-PPPs) represent an innovative and sound model for the development of new health and medical products of vital public concern to developing countries, including new malaria drugs. Indeed, there has been a remarkable surge in new projects for neglected diseases particularly with the Drugs for Neglected Diseases Initiative (DNDi) of Médecins sans Frontières (MSF). DNDi launched its first new antimalarial drug in March 2007. This initial launch and MMV's larger pipeline argue for cooperation between DNDi and MMV. However, an element of competition may also be healthy. 3.8 In contrast with MMV's R&D work, it is less evident that MMV's new access and delivery work -- on the downstream end of the drug R&D and supply chain (Figure 4) -- represents a global public good. Individual units of malaria drugs are rival and excludable, and therefore private goods at the point of consumption, although the degree of excludability will depend on the institutional arrangements for distribution -- whether free of charge or for a small price. In fact, as one interviewee observed, malaria is largely a "private disease" managed at the household level, with between 60 and 90 percent of first line therapy obtained in the private sector in many countries. There is, however, a minimal public good element associated with reduction in transmission from increasing cure rates, and some spillover of this benefit from one country to another in the case of small countries. National-level policies 21. This refers to the manner in which the individual efforts of the program partners contribute or add up to the collective outcome for the program as a whole -- whether "best shot," "summation," or "weakest link." For "best shot" aggregation technologies (such an AIDS or malaria vaccine ), the partners should pool their efforts because the collective outcome equals that of the best individualized effort. For "summation" technologies (such as mitigating climate change), the collective outcome equals the sum of the individual efforts. Therefore, one partner's contribution (or lack thereof) can substitute for (or nullify) another partner's contribution. For "weakest link" technologies (such as the eradication of an infectious disease), the smallest provision (or lack thereof) determines the collective outcome. If one necessary partner does not do anything, the disease will not be eradicated. For a current treatment of these different aggregation technologies, see Scott Barrett, 2006, "Making International Cooperation Pay: Financing as a Strategic Inventive," in Inge Kaul and Pedro Conceição, eds., 2006, The New Public Finance: Responding to Global Challenges. 16 Figure 4. The Drug R&D and Supply Chain Upstream Drug Discovery & Downstream Groundwork Development Access · Consensus on · Sustainable Accessibility INTE INTE global priorities financing · Long-term funding · Supportive drug commitments RFA Availability RFA policies · Knowledge of · Developing country disease burden C C regulatory approval Acceptability E · Knowledge of E current drug MMV's Core · Manufacturing · Trained providers/ supply and new R&D Activities Affordability IN INTE informed customers drug demand TER · Efficient public and · Research capacity private sector Rational Use RF · Support for basic distribution FAC science ACE · Pharmaco Quality · Intellectual property vigilance E protection Assurance Source: MMV. and institutions in relation to the access and delivery of malaria drugs22 are national public goods, and policy and analytical work to improve these could be carried out, and even financed, at the national level. Such work will have global public good characteristics to the extent that lessons learned in one country are readily transferable to other countries, and to the extent that some policy and institutional issues in relation to access and delivery are international in scope.23 3.9 The importance of access and delivery issues and the potential for learning lessons of broad, multi-country application suggest that work on access and delivery should be sponsored and at least in part financed on a global basis, rather than solely at the national level. While MMV's R&D activities clearly belong in the category of financing global investments to produce global public goods -- in IEG's classification of common activities undertaken by global and regional partnership programs (Annex Table 4) -- the access and delivery work belongs largely in the categories of generating and disseminating knowledge, advocacy, and supporting national-level policy, institutional and technical reforms.24 3.10 This finding does not answer the critical question facing MMV and its partners of their respective roles in access and delivery issues. The MMV Board has yet fully to come to grips with MMV's approach to access and delivery of its new drugs, a consensus has yet to be reached among those concerned on the appropriateness and nature of MMV's activities in this area, and the contours of MMV's activities in this area will undoubtedly evolve.25 In the 22. For example, deciding when to change a national malaria drug policy from one drug to another, or whether to accept multiple products; and how to ensure adequate distribution and private markets for the new drugs. 23. For examples, determining the roles of various actors in WHO. 24. Categories 2, 4, and 6 in Annex Table 4 on common GRPP activities. 25. The range of issues likely to arise is indicated by the areas of expertise sought by MMV for its Access and Delivery Advisory Committee: Epidemiology, drug regulation and quality assurance, malaria treatment and 17 interviews undertaken for this GPR, the policy and institutional issues associated with the access and delivery of new malaria drugs were widely considered to be as difficult, if not more difficult, than the R&D problems to be surmounted before registration of a new malaria drug (Box 5). 3.11 MMV has so far decided to move forward with learning from the rollout of the recent introduction of artemisinin combination therapy (ACTs) in some countries, with case studies on such practical issues as product pricing and follow-up, initially in Uganda. With financial support from Ireland and the Netherlands, MMV is also engaged in a wide set of consultations in malaria-endemic countries on its access and delivery work. Generally, MMV expects to act as a facilitator and to follow the partnership model of its R&D work, including working with national partners and RBM, without directly acting on the issues itself. Possibilities for collaboration with the Bank are also evident. MMV may wish to consider developing a logframe for its access and delivery activities, ideally in consultation with its principal stakeholders, as a vehicle to clarify the objectives and roles of its many different partners who are likely to engage in this new area. Box 5. What Is Involved in Access and Delivery of New Drugs? Ensuring access and delivery of new malaria drugs requires that many commodity supply and health system development issues be addressed: · Sustainable drug financing · Supportive national drug policies and practices · Fast track regulatory approval · Low cost manufacturing · Trained public and private sector health care providers · Well-oriented public and private pharmacists · Informed consumers · Efficient public sector distribution · Efficient private markets · Monitoring feedback from tracking patient reactions to new drugs. Large numbers of actors need to be involved, including Ministries of Health, WHO, private industry, national drug regulatory authorities, NGOs, and donors. Source: Annex Tables 7 and 8. Efficacy 3.12 MMV has clearly achieved one of its initial objectives, of establishing and managing a portfolio of candidates for new malaria drugs passing in appropriate fashion through the various phases of discovery and clinical development that precede registration by national drug regulatory authorities. It is also very likely to meet and probably exceed its specific initial goal of registering at least one new drug by 2010. As of March 2007, MMV expected coverage, national drug policy formulation, drug financing, drug pricing, supply chain and logistics management, local delivery systems, demand creation and marketing, communications, economics of malaria, and drug product manufacture. Many of these issues typically arise in connection with Bank-financed health operations in malaria endemic countries. 18 that its first 2 dossiers would be submitted for registration by the end of 2007. The public sector target price for a full course of treatment was expected to be one dollar or less, which is within the performance metrics of MMV.26 MMV was also poised to deliver 2 or 3 new combination antimalarial drugs by 2010, beginning as early as 2008.27 However, no assessment of MMV's impact with respect to the prevalence of malaria is feasible at this time, nor is it likely to be feasible for an extended period into the future because MMV products are not yet on the market and their health impact is even further away. In any case it will continue to be important to have on-going replenishment of the pipeline of new candidate drugs, in the face of (a) the risks, indeed the expectation, that a number of candidates for new products will be dropped before or at various transitions in the R&D process, (b) the probability that the malaria parasite will develop resistance to new drugs placed on the market, and (c) the need for multiple drugs for different malaria indications (such as children, adults, pregnant women) and for back-up drugs for use if one drug fails to cure the disease in specific patients. 3.13 Inputs. As resource mobilization has permitted, MMV has expanded its spending with additional projects and personnel. MMV's staffing has generally expanded more rapidly than planned since the beginning, and its total expenditures have expanded more rapidly than planned since 2004 (Table 3). Although the numbers of personnel have been consistently higher than planned, MMV is widely perceived to be a lean organization. In any event, both the external evaluation and its principal financial supporter (the Gates Foundation) have found that the MMV team needs to be expanded and strengthened to maintain the momentum generated in its first five years. In addition, the original staffing plans did not anticipate the expansion of MMV into access and delivery issues. Table 3. MMV Expenditures and Personnel -- Plans and Actuals, 2000­2006 2000 2001 2002 2003 2004 2005 2006 Personnel (Full Time Equivalent, FTE) /1 Initial Business Plan 3.0 5.25 6.0 6.75 6.75 7.5 7.0 2003­2007 Business Plan n/a n/a n/a 11 11­12 12­13 12­14 Actual 8 10 7 11 13 14 21 Total Expenditures (US$ millions) Initial Business Plan 7.8 18.3 19.3 25.9 24.2 28.2 26.6 2003­2007 Business Plan n/a n/a n/a 19.3 27.8 32.1 34.1 Actual 3.2 8.3 12.6 19.2 28.6 30.7 51.5 /1 Includes support staff and excludes outsourced activities; definitions by MMV may not be entirely consistent, particularly in 2001. In-kind contributions by MMV's private sector partners, sometimes estimated to equal MMV's direct financial contributions, are not in the table. Source: MMV business plans and financial statements. 26. However, it might be noted that the $1 treatment cost figure is well above the roughly $0.10 cost for the chloroquine treatment used for many years in Africa but now, due to drug resistance, being replaced increasingly by ACTs costing $1­$2 per treatment. 27. MMV letter of inquiry to Bill and Melinda Gates Foundation, March 5, 2007. 19 3.14 Outputs. MMV's principal "output" is malaria drug projects at various stages of discovery and clinical development in its R&D portfolio. The number of such projects has generally exceeded plans (Table 4). This has been due to MMV's success in fund-raising, making it possible to accept more proposals than had been anticipated under MMV's prudent business planning. MMV could state as early as 2004 in its Annual Report that science was no longer the barrier to antimalarial drug innovation. By this year its portfolio had become the largest malaria drug research effort ever mounted. 3.15 Table 4 assumes a traditional and somewhat static public infrastructure business model. It does not capture the dynamic of moving projects through the various phases of discovery and clinical development, nor the process of brokering collaboration among researchers in industry and academic, nor how MMV maintains a balanced portfolio by selecting projects at appropriate points in discovery and development for financing and inclusion in the MMV portfolio. Modeling portfolio size, composition, and performance under alternative sets of assumptions about successful rates of transition of a potential drug product from one phase of discovery or development to the next and under alternative financial scenarios is a critical input to MMV's business planning process.28 3.16 MMV's achievements have had the unintended consequence that its funders, reinforced by the external evaluation, have successfully pressed it to expand its mission beyond drug registration further downstream to access and delivery of new malaria drugs. Thus MMV has developed a capacity to learn from its own experience. This is especially Table 4. MMV's Drug Project Pipeline -- Plans and Actuals, 2000­2006 /1 2000 2001 2002 2003 2004 2005 2006 Drug Discovery Projects Initial Business Plan 3 6 7 7 7 7 7 2003­2007 Business Plan n/a n/a n/a /2 /2 /2 /2 Actual 3 6 7 5 12 8 22 Drug Development Projects Initial Business Plan 0 1 1 2 2 3 2 2003­2007 Business Plan n/a n/a n/a /2 /2 /2 /2 Actual 0 0 2 5 12 8 6 Projects Terminated by MMV 1 0 1 3 2 3 7 /1 This table includes only projects under full contract, for ease of comparison. The MMV initial business plan was established prior to the acceptance of combination therapies as a norm, yet many of the projects reflect this new position. The business plans also cannot take into account MMV's mini-portfolios, now with three private firms. MMV's demanding approach to selection of project proposals for its pipeline has led only around ten percent to be selected. /2 The 2003­2007 business plan did not quantify the number of projects expected to be in the MMV portfolio because of the uncertainties in project management in the complex environment faced by MMV. The plan foresaw addition of five new projects every other year to the MMV R&D portfolio; at that time this was thought to be challenging for MMV. Source: MMV business plans and MMV Chief Financial Officer, March 2007. 28. Details are contained in MMV's business plans. See Annex Figure 2 for the average successful transition rates in the various phases of drug discovery and development that are used in MMV's business planning. 20 reflected in the differences between its initial business plan, which was largely a hypothetical exercise for a somewhat risky and untried entity, and its 2003­2007 business plan, which shows a much greater sophistication in its modeling and interaction with MMV's management and Board of Directors. 3.17 As MMV moves its access and delivery agenda to the country level, and endeavors to indirectly reach patients and consumers, its ability to establish and manage operational linkages with new and different actors will become increasingly important. Because malaria is largely managed at the household level, MMV rightly sees private sector distribution as key to success in reaching the vast majority of the poor population in endemic countries with new drugs.29 MMV will also need to come to grips with a large number of difficult policy and institutional issues, at both the global and national level, before its access and delivery work can be fully effective. Efficiency 3.18 MMV's spending is significantly driven by its available financial resources and the dynamic management of its portfolio. Management and administrative costs, including the cost of MMV's Board and stakeholders meetings, have tended to decline as a share of total expenditures since start-up in the year 2000 (Table 5). These have represented less than 12 percent of total costs over 7 years of operation, and represented less than 8 percent in the most recent year (2006). This share seems reasonable given that MMV's critical value added lies in its portfolio management activity. 3.19 MMV's expected spending on bringing malaria drugs to market is modest, relatively speaking. Academic estimates of the costs incurred by "big pharma" in bringing a new chemical entity through the entirety of the R&D pipeline to registration, and then onto the market in industrial countries were around $800 million in the 1990s. More recent estimates would likely be higher.30 In contrast, MMV expects that its cost will be about $150 million Table 5. MMV Expenditures, 2000­2006 (US$ millions) 2000 2001 2002 2003 2004 2005 2006 Total Drug R&D 2.3 6.7 10.5 17.0 23.8 27.2 46.9 134.4 Access and 0 0 0 0 0 0 0.7 0.7 Delivery Management and 1.0 1.6 2.5 2.3 2.9 3.5 3.8 17.6 Administration /1 Total 3.3 8.3 13.0 19.3 26.7 30.5 51.4 152.7 Percent share of 30.3 19.3 19.2 11.9 10.9 12.9 7.4 11.6 Mgt. and Admin. /1 Includes Board and stakeholder expenses Source: MMV annual reports, 2000­2005; MMV financial statements, 2006. 29. Annex Figure 4 summarizes the timeline of the various activities from regulatory approval through private and public sector rollout of a new drug. 30. Joseph A. DiMasi et al., 2003, "The Price of Innovation: New Estimates of Drug Development Costs." 21 for a new malaria drug, including the costs of failure. This substantial difference in costs is due to many factors, including the high cost of capital for the "big pharma" enterprise, the extensive in-kind and pro bono support for MMV from its private industry partners, the lower cost of clinical trials for antimalarial drugs compared to drugs for chronic diseases, the inclusion of some projects in the MMV pipeline that were already relatively advanced by in- sourcing extant intellectual property, and skilled management of the R&D pipeline by MMV staff with the support of its ESAC. Finally, it must also be acknowledged that only two of the four MMV research projects that are most likely to lead to early registration of a new drug represent new chemical entities as distinct from reformulations of existing compounds. 3.20 Another perspective on MMV's efficiency, that of allocative efficiency, comes from comparing R&D funding for malaria with R&D funding on other diseases in relation to the burden of the disease as measured by Disability-Adjusted Life Years (DALYs). According to a study sponsored by the Malaria R&D Alliance, R&D funding for malaria in 2001­2002 was approximately $6.20 for each DALY lost to the disease (Table 6). This was much lower than for other major diseases prevalent in developing countries, such as HIV/AIDS ($24.26) and TB ($10.88) per DALY lost. By way of contrast, the same study estimated that R&D on diabetes, a chronic disease particularly widespread in the industrial countries, was $102.07 per DALY. If malaria R&D had been funded at the average of all medical conditions addressed in the Malaria R&D Alliance study, it would have received more than $3 billion in annual funding, compared to the total estimate of $288 million for all malaria-related research in 2001­2002. 3.21 Looking at the efficiency of MMV from the standpoint of the immediate recipients of its funding -- whether "big pharma," small biotech or academic enterprises, or specialized contract research organizations -- the issue is less the relatively low cost of MMV's drug R&D work than MMV's "fit" in the new dynamics of the drug R&D industry. The new landscape of neglected disease R&D is dominated by PD-PPPs such as MMV, and the Table 6. Disease Burden and Funding Comparison, 2001­2002 Condition Global Disease Burden R&D Funding R&D Funding per (million DALYs) /1 (US$ millions) DALY (US$) Cardiovascular 148.19 9,402 63.45 HIV/AIDS 84.46 2,049 24.26 Malaria 46.49 288 6.20 Tuberculosis 34.74 378 10.88 Diabetes 16.19 1653 102.07 Dengue 0.62 58 94.16 /1 Disability-Adjusted Life Year -- a measure of healthy life lost Note that these figures, while comparable among diseases, cover more than investment in drug R&D, including much intramural research (such as U.S. Department of Defense and NIH) and basic research, drug discovery and development (MMV's raison d'être), vaccine development and vaccine trials, vector control research, development of malaria diagnostics, and implementation research. Antimalarial drug discovery and development was estimated at $120 million in 2004, represent 37 percent of the total of $323 million spent of malaria R&D. MMV, with $27 million in receipts in 2004, represented a relatively small share of this. Source: Malaria R&D Alliance, "Malaria Research and Development -- An Assessment of Global Investment," November 2005. 22 PD-PPP model is ideal for each type of partner, each for its own reasons. Large pharmaceutical enterprises, which are increasingly following a "no profit ­ no loss" business approach to diseases of the developing world, avoid incurring the full cost, risk, and liability of shouldering the entire drug R&D process. Biotechnology companies in developing countries gain access to knowledge and skills from enterprises with which they partner on specific research projects -- MMV's partnerships with the Indian generic drug manufacturer Ranbaxy being a case in point. Contract research organizations (CROs) provide specialized services to PPPs such as MMV and its collaborators.31 The existence of MMV permits donors to reduce their transactions costs by internalizing choices that they might otherwise have to make about likely "winners" in the sweepstakes for new antimalarial drugs. MMV's knowledge, and access to even more specialized expertise through its ESAC, reduces the costs for donors of supporting appropriate malaria drug research. Governance 3.22 The legitimacy of MMV has grown in recent years, due to a number of factors: (1) the positive external evaluation, (2) MMV's success in fund-raising and portfolio management, (3) its increasing engagement of researchers and research institutions in endemic countries, (4) its holding key meetings in countries where malaria is widespread and MMV-sponsored research is under way, (5) the voice of WHO on the MMV Board, 32 and (6) the conscious and successful efforts to ensure beneficiary country members constitute more than the majority of its Access and Delivery Advisory Committee. If well executed, MMV's access and delivery activities should contribute to MMV's legitimacy. Nonetheless, in the interest of further enhancing its legitimacy as the Board's role shifts more and more to wide strategic issues, MMV may wish to consider increasing the developing country membership on its Board. MMV may also wish to envisage broader engagement with NGOs and others in the design and execution of its access and delivery work program -- particularly with advocacy NGOs that have played and continue to play an important role in pharmaceutical policies on the global scene. The present members of the MMV Access and Delivery Advisory Committee tend to have a more technical background and orientation to its work. 3.23 The initial MMV Board Chair was Dame Bridget Ogilvie, a well-known and highly regarded British research scientist and former head of the Wellcome Trust. The current Board Chair is Baroness Lynda Chalker, a former Minister responsible for the British Department for International Development. Senior personnel from the pharmaceutical industry are members. There are no NGO representatives. Having leaders of the stature of Bridget Ogilvie and Lynda Chalker has greatly helped MMV to gain attention, political and other contacts, and financial resources. MMV has also been fortunate to have a respected President and CEO with a background in the UK Medical Research Council and in a well-regarded private pharmaceutical enterprise engaged in R&D. Now, as MMV matures, the Board is reportedly 31. This discussion draws heavily on Moran et al, 2005, "The New Landscape of Neglected Disease Drug Development." 32. It is worth noting in this connection that the external evaluation recommended that both the Board and ESAC should continue to include the best qualified individuals, but should also be mindful of the need for "appropriate gender and geographical representation." 23 giving greater attention to issues of strategy and finance. As MMV's financial resource gap remains large, Board members will likely need to give increasing attention to fund-raising -- an important governance responsibility. As MMV's leadership needs evolve, the role of the Nominating Committee is also becoming more important. 3.24 In light of the commercial and IPR risks involved in MMV's work, confidence and confidentiality are an important feature of its governance. All Board members, all MMV personnel, and all members of MMV advisory committees are required to sign a confidentiality and conflict of interest agreement before being allowed access to relevant scientific data. Annual "statements of interest" documents are completed by all, to indicate direct or indirect involvement with any project under discussion. MMV has been fortunate to attract the principal players in the malaria community to its various governance structures. It was natural that these individuals would also be involved in certain MMV projects, and that conflicts of interest would arise. Several cases have arisen. They have been reviewed and accepted as necessary to MMV's work by its Board of Directors, Without accepting these narrow conflicts of interest and managing the participation of the people concerned in other aspects of MMV's work, the organization would probably have had difficulty in inducing some top quality scientists to participate. On the whole, MMV seems to have handled the conflict of interest issue reasonably well. 3.25 Subject to the requirements of confidentiality in its R&D work, MMV operates as a very transparent organization. Documents of interest, including complete annual reports, are routinely available on its Web site. MMV personnel and its current and former Board chairpersons were highly responsive to requests for information and interviews for this review. Current and former members of the Board, along with ESAC and ADAC members, and MMV donors, are listed separately on the Web site. MMV staff members are also shown on the Web site. The MMV R&D strategy is available, and individual projects -- including project leaders and partners and MMV staff contacts -- are shown in the MMV annual report. The upbeat tone of the annual reports is that of a publicly held company presenting itself to its shareholders and the general public. Sustainability of the Program and Prospects for the Future 3.26 MMV's future spending can be expected to increase, particularly as it has been very successful in raising funds. However, its financial requirements, for an expanding R&D portfolio, and for growing activity on its access and delivery agenda, are continuing to expand rapidly. These substantially exceed available income (Figure 5), and pose a formidable fund-raising challenge to MMV and its financial partners.33 As of March 2007, MMV estimated that its funding gap stood between $300 and $400 million, according to the budget scenario chosen.34 At the same time it must be acknowledged that MMV has a truly remarkable fund-raising record. 33. It may be noted in this connection that the OED 2004b study of the World Bank's involvement in six global health programs had already found that even to convert the recent growth in PD-PPPs into usable products for the poor needed substantially larger investments. This meant, the study continued, that program partners needed to mobilize more resources for health research that benefited the poor on a long-term predictable basis. 34. MMV Letter of Inquiry to Bill and Melinda Gates Foundation, March 5, 2007. 24 Figure 5. MMV: Available Income and Annual Expenditures on R&D Projects, Administration and Access and Delivery Activity -- Actual, 2000­2007, and Projected, 2008­2010 US$ Millions Cumulative 80 Expenditures 70 Access Available Income & Delivery 60 Including Carryover Administration 50 Annual Income Received/Pledged R&D Projects 40 30 20 10 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Source: Chief Financial Officer, MMV, March 2007. 3.27 In the initial business plan prepared by MMV at the time of its establishment the possibility was raised of creating an endowment, for long-term sustainability of MMV R&D spending, by devoting a defined share of annual expenditures to an endowment. This idea has not been pursued. 3.28 MMV's R&D contracts for products in the later stages of development foresee the possibility of cost recovery as MMV earns royalties on its new IPR, but whether this could effectively become, or should become an important element in MMV revenue, is not clear. Market segmentation might permit some recovery in private sector premium markets, such as international travelers, but these are not likely to represent a significant portion of total sales value. 3.29 While the relevance, efficacy and efficiency of MMV's R&D activities are evident, it is too early to reach conclusions on the relevance, efficacy and efficiency of MMV's new and highly demanding downstream access and delivery agenda. At some stage, the work being sponsored by MMV on MMV products will need to be generalized in a broader effort involving additional partners and stakeholders and in a different institutional framework covering all new malaria drugs. MMV's present lack of institutional comparative advantage in this area stands in sharp contrast to its effectively established comparative advantage on 25 malaria drug R&D. MMV's access and delivery work demands individual and organizational skills, and involves interfaces that are not traditional for MMV. MMV has recruited two highly skilled people for this work, and MMV has access to the RBM partnership and network in WHO -- the Executive Secretary of RBM being the chair of MMV's Access and Delivery Advisory Committee. Recent changes in WHO management are also thought likely to facilitate cooperation between MMV and WHO, although there appear to exist some tensions in this respect among the WHO units concerned. It remains to be seen to what extent and how MMV will be able to reconcile its private sector entrepreneurial style with the public sector requirements for resolution of policy and institutional issues in access and delivery. 3.30 MMV will also need to consider how it positions itself in relation to GFATM and the United States -- two principal financiers of malaria control and treatment programs in developing countries. As its role evolves, MMV will have to determine how to respond to pressures from developing country partners to assume a conscious capacity building role, beyond the natural capacity building that takes place within the framework of its partnerships with developing country research partners. MMV has resisted moving into research capacity building -- an area where TDR is already active and would appear to have a comparative advantage relative to MMV. 4. World Bank Performance in the MMV Partnership 4.1 The Bank's roles in the MMV partnership have been limited -- as a co-founder and a small financier. The Bank's HNP leadership played a critical role in the establishment of MMV, and the Bank has subsequently made annual DGF grants to MMV of $500,000 or $750,000 from fiscal years 2000 to 2006 inclusive. Since the notion of a PD-PPP was risky and untried at the time of MMV's founding, the confidence and trust which other donors had in the Bank were important in the formative stages and start-up of MMV, to both the private and public sector partners, including WHO and official donors. The Bank's willingness to contribute DGF resources has been particularly important to other official donors, even though the Bank's contribution has represented a declining share of MMV resources since 2000 (Figure 6).35 Otherwise, the Bank has been a relatively silent partner: it has not served on any of the governing bodies or advisory committees of MMV, and has only inconsistently attended its annual stakeholders' meetings. MMV would welcome greater involvement of the Bank in its work at both the global and country levels, but specific terms of reference and clear definitions of roles would be needed. 4.2 Table 7 summarizes the Bank's performance in the MMV partnership, both at the founding and currently, as well as potential roles in the future. At the global level, the Bank contributed its convening power to the establishment of MMV, although the Bank did not carry out any sector work directly connected to the establishment of MMV. The discussions surrounding the establishment of MMV generally took place in connection with the Global Forum for Health Research (GFHR). The Bank's annual DGF grant to MMV has been an 35. No Bank-administered trust funds have been involved. 26 Figure 6. World Bank: A Small and Declining Share in MMV Financial Resources 7% 6% 5% 4% 3% 2% 1% 0% 2000 2001 2002 2003 2004 2005 2006 Source: Chief Financial Officer, MMV, March 2007. earmarked amount within its larger DGF grant to GFHR and with an individual grant agreement signed between the Bank and MMV. This arrangement has created some ambiguities in oversight and inhibited transparency, since the annual DGF reports do not indicate how much of the Bank's grant to GFHR goes to MMV. 4.3 At the country level, the Bank had no role in MMV at the outset. However, HNP submissions concerning MMV to the DGF Council have observed that the Bank's involvement in supporting health programs in developing countries gives it unique perspectives on research needs.36 The Bank appears to have brought these perspectives to discussions in the Global Forum for Health Research but not in the determination of the MMV portfolio. Greater possibilities for building country operational linkages are rapidly emerging in connection with MMV's access and delivery agenda, though these are poorly defined at present. 4.4 The Bank has been largely disengaged from performing any oversight of MMV, there being no terms of reference for the staff concerned and minimal administrative budget support ($9,500 in FY06). The HNP Department has made no formal assessment of MMV's achievements.37 The HNP Sector Board has tended to focus on the Global Forum rather than on MMV.38 This is hardly surprising, given the number and size of HNP partnerships, and the almost non-existent budget allocation for HNP oversight of these partnerships, including 36. An undated paper on the HNP DGF module for MMV observes also that the Bank can help define treatment research needs that are common across settings and to which solutions are greatly facilitated by concerted action. This would appear to be more applicable to the GFHR than to MMV. 37. In January 2004 the HNP Sector Board expressed interest in a more strategic framework for DGF programs, which might include consideration of an overarching "Communicable Disease" program. 38. Meeting on the DGF proposals for FY04, the Sector Board minutes of the January 30, 2003, report an agreement that the Sector Board would take time to review in detail each of the DGF-funded programs with the respective task manager during the course of the year. It is not clear whether such a review of MMV took place. 27 Table 7. The Bank's Performance as a Partner in MMV Dimension MMV Founding (1999) MMV Today (2007) Potential Future Roles Using its comparative Through advocacy and Bank financial Convening power could, if advantage at the global early financial participation remains requested, become level: (1) global mandate commitment, the Bank important to some official important to reaching and reach, (2) convening contributed credibility to financiers and other common understanding(s) power, and (3) catalyzing build confidence of stakeholders in MMV. on WHO and other part- other resources and partners; Bank role ner roles in access and partners for the program. consistent with 1997 HNP delivery activities relating Sector Strategy to new malaria drugs. /1 anticipating strengthening The Bank could also of collaboration with other make a policy decision to agencies on international support coordinated health R&D, including definition and funding of a new malaria drugs. global health research agenda along the lines of its role in the CGIAR. Contributing its No linkages at the country Initial discussions under As MMV expands country comparative advantage at level. way between MMV and level activity on access the country level: (1) the Bank's Malaria and delivery, potential for multi-sectoral capacity, (2) Booster Program staff; country linkages will grow analytical expertise, and specifics undetermined rapidly in HNP operations. (3) country-level and unresourced. Possibilities include Bank knowledge. support for drug policy, regulation, and purchase under HNP operations. /2 Exercising effective and Loose oversight; no TOR Very distant contact; Bank With appropriate independent oversight of for the Bank's global widely perceived to be assignment, Bank could its involvement in the program team leader. unengaged, aside from enter into mutually program. minimal funding; no TOR. beneficial strategic dialogue with MMV and provide continuing assurance on appropriate public-private balance in MMV's work. Identification and Apparently not discussed. Not considered, but risks Reputational and country management of risks. to Bank seem to be low, operations risks may arise aside from possible as MMV increases reputational risk in the country-level engage- eyes of some observers ment, if the Bank does not from early withdrawal. substantially engage at this level. Facilitating an effective, Not considered. Current DGF funding Issue may be less flexible and transparent under Window 1; disengagement than disengagement strategy, continuing financial redefinition of role, as appropriate. engagement foreseen gradually away from DGF until 2020; reconsider- funding towards ation under HNP DGF orchestration of country strategy review foreseen operations linkages. in FY08. /1 This would necessarily include the roles of various units within WHO. /2 Participation in financing Phase III and Phase IV clinical trials under HNP SWAP operations could also be considered. Source: The five dimensions of Bank performance as a partner in the program correspond to the evaluation questions in Annex Table 3 in Annex A. 28 MMV. Thus this GPR finds that the Bank's internal accountability systems have been weak in regard to MMV. Interviews undertaken for this report make clear that enhancing the Bank's oversight of MMV could help to assure MMV's official donors, including the Bank, of an appropriate public-private balance in MMV.39 4.5 Reputational risks for the Bank were high at the outset of MMV, since PD-PPPs were untried, but these risks were apparently not considered in approving the original DGF grant for MMV for FY2000. While current risks to be Bank would appear to be low, reputational and country operations risks could increase in the future if the Bank withdraws precipitously or does not substantially engage with MMV at the country level. A disengagement strategy was not considered at the outset and receives little attention now, since the DGF provides its grants to MMV under Window 1 (its long-term grant window). In the future, the issue may be less about disengaging than about redefining the Bank's role in the partnership. 4.6 For the future, the benefits to the Bank and its clients from the Bank's engagement with MMV are real and substantial. For the Bank, the interviews carried out for this GPR make clear that the Bank's engagement with MMV brings reputational benefits, certainly in the minds of MMV's other donors and among its private partners and possibly more widely. For the Bank's clients, the Bank's continuing financial involvement would accelerate the day when new, affordable malaria drugs will be available to reduce the burden of malaria, especially among the Bank's poorest client populations in Africa. 4.7 The discussions by the HNP Sector Board planned for FY08 on its future partnership and grant strategy will provide a suitable opportunity for the Bank, as a relatively small financier, to explore a range of options for engaging in the future funding and operational strategies of MMV, and for global health research more generally (Box 6). Neither external nor internal interviewees for this review were clear on how the Bank wishes to define its future role in relation to MMV. While malaria drug R&D would appear to be a candidate for phasing out support under the new (2007) HNP Sector Strategy, it could also be argued that Bank support for a larger health research agenda, with priorities set by outside specialists, could be a critical dimension of support to health systems. The debate on these issues could also represent an occasion for the Bank to focus in practical, operational terms on where its support for a global public good, such as R&D on new malaria drugs, should end, and to what extent it should support R&D applications at the country level, as a national public good, through Bank operations at that level. 4.8 Regardless of the outcome of these discussions, the Bank will need to pay attention to the development and marketing of new drugs, especially drugs for malaria, because of the demand coming from its clients. According to data from the Bank's Business Warehouse database that were assembled for an HNP discussion paper, the Bank issued non-objection communications concerning pharmaceutical and medical product contracts under Bank operations with an estimated value of $400 million over only four years, from FY99 through FY02.40 39. These challenges for the Bank are not unique to MMV, but the focus of the present report remains on MMV. 40. Rosa Rodriguez-Mongulo and Juan Rovira, 2004, "An Analysis of Pharmaceutical Lending by the World Bank." More recent data are not available. 29 Box 6. Options for Future Bank Engagement with MMV In consultation with MMV and other stakeholders, the Bank's HNP Sector Board and DGF Council may wish to consider a range of options for future support for MMV and health research, including: · Alerting MMV and its financial partners soon that -- despite the internal planning horizon for exit in 2020 -- the Bank may have to reallocate its DGF resources now devoted to MMV to other PD-PPPs when MMV succeeds in registering a new drug; alternatively, a conscious exit strategy could be envisaged with attendant costs and benefits. · Informing MMV that, regardless of the level of its financial support, the Bank would like to engage with MMV strategically as well as at the country level on the issues of access and delivery of new malaria drugs and is prepared to make the administrative budget and human resource allocations and assignments required. · Facilitating a careful and comprehensive definition of the roles of WHO (including its multiple internal actors), the Bank, other partners in MMV's access and delivery work, but only if they request this support; such an action would be justified by the importance of client country capacity to benefit from, and Bank operations to support, new, affordable MMV products; it would be in a Bank convening or convening support role. · Offering support including financial resources, probably through the Global Forum and/or TDR, for the creation of a common health research fund in which donors would pool funds, or at least for establishment of a network or mechanism for coordination of donor health research funding decisions.41 4.9 MMV is also developing and changing in ways that create grounds for the Bank to rethink its somewhat distanced posture towards MMV that has prevailed in recent years. The MMV access and delivery agenda brings MMV directly into contact with the Bank's HNP clients and operational issues in ways that the MMV R&D activity has not. The issue is likely to be posed less as an issue of national health research on medical issues -- an area where the Bank has been relatively inactive -- and more as an issue of health systems and service delivery, since MMV's access and delivery work involves issues in health systems policies and development (from pharmaceutical regulation through public and private sector distribution channels), where the Bank sees a comparative advantage for itself in its new HNP strategy.42 4.10 As MMV work on access and delivery grows, the Bank's Malaria Booster Program is becoming a logical point of contact and cooperation with MMV. However, the Bank staff concerned seem to be overwhelmed meeting daily operational requirements and left with little time to devote to what could become an entirely new collaborative activity with MMV. As MMV drug products gradually move out of the R&D pipeline towards the market, Bank task managers can be expected to receive client requests to support financing of drug purchases and related investments. IFC investment in manufacturing and distribution projects for MMV products could also become an important part of the Bank's agenda with MMV. 41. Because bilateral donors are generally reluctant to support health research, the Bank might have to play an advocacy and convening role in this area, at least initially. 42. In the 2007 HNP Strategy, the limited research focus is on public health surveillance and health systems development, including development of statistical capacity, rather than on local medical research. This contrasts somewhat with the support for Medical Research Councils and Pasteur Institute collaborations that has been provided by the UK and France. 30 More active engagement would inevitably require further clarification on the timing and schedules for WHO prequalification of new MMV drugs. Yet, the present review finds that the Bank's staffing, budget resources and incentives have been insufficient to link its country activities with MMV. Fortunately, MMV is at an early enough stage in its work in this area for the Bank to rethink its posture and to ensure the required allocation of human and financial resources as well as the design of internal accountability structures that could create synergies between the Bank's HNP operations and MMV's access and delivery work. 4.11 Health research and agricultural research have much to learn from each other. IEG's meta-evaluation of Consultative Group on Agricultural Research (CGIAR) in 2003 offers many relevant comparisons and lessons for international health research, including MMV (Box 7). A subsequent IEG review of six global health programs in 200443 observed that for over 30 years the Bank has exercised stronger and more consistent leadership, both globally and at the country level, in mobilizing resources for research on agriculture compared to health. In contrast to agriculture, the Bank's financial commitments to health research of a public goods nature both internationally and nationally in developing countries have been relatively small and sporadic. The World Bank and Health Research 4.12 The Bank supports health research financially through TDR (generally, more technical) and the Global Forum (generally, more analytic and policy-oriented). Other entities are also involved, such as the Malaria Vaccine Initiative (MVI) and the large research agenda of UNAIDS.44 The operation of these multiple health research-sponsoring entities -- sometimes in cooperation with each other, sometimes in competition -- inevitably raises the question whether the international community should agree to establish a common fund that would allocate financial resources for health research on a more rational basis that gets beyond the political and policy priorities of individual funding agencies. While the CGIAR does not engage in pooled funding, it has endeavored to ensure coordinated funding of agricultural research since the early 1970s. It seems that health research funders have generally preferred to play a more direct role in choosing research ventures than those funding agricultural research.45 The growing range of research initiatives and programs on the health problems and diseases of developing countries (Table 8) cries out for greater coherence. Programs tend to be largely donor-driven without much research coordination and prioritization. While the GFHR has undertaken work on methodologies for health research prioritization, it has not endeavored to sponsor research prioritization as such. It reports a broad consensus regarding research priorities, but it has not yet turned this conclusion into specific, practical operational proposals for research resource allocation. 43. OED, 2004b, "Global Health Programs, Millennium Development Goals, and the World Bank's Role." 44. UNAIDS-sponsored research relates less to product R&D and more to health systems and guidelines issues, similar to MMV's access and delivery agenda. 45. However, IEG's meta-evaluation of the CGIAR found that the influence of individual donors (and their domestic constituencies) on the research expenditures of CGIAR centers has increased since the mid-1990s. 31 Box 7. Health Research and Agricultural Research: Comparison Between MMV and the CGIAR The experience of the Consultative Group on International Agricultural Research (CGIAR) is rich in comparisons and lessons for health research and MMV. · The arrangements for funding the 15 CGIAR international agricultural research centers have shifted over the years, with important incentive and practical effects on the research funded by the CGIAR system. It will be in the interest of MMV to maintain as stable a financing system as possible. · The traditional and most successful dimension of CGIAR research, the area of its greatest comparative advantage, and the activity most unambiguously of a GPG character, has been its scientific work on improving plant germplasm. The same considerations apply to MMV drug R&D. · CGIAR has also supported natural resource management research, but so far less successfully, because such research has a higher degree of country specificity. This is more comparable with MMV's access and delivery activity, which also has a high level of country specificity, and therefore presents an important note of caution for MMV. · The CGIAR system is being pulled in two opposite directions. On the one hand, the CGIAR System is insufficiently centralized to deal adequately with advances in biology and intellectual property rights; on the other, its research centers are not conducting sufficiently coordinated research of a highly decentralized nature on natural resource management, which calls for active partnerships with national agricultural extension and research programs. In contrast, MMV is highly centralized and able to follow and respond to changes in science and IPR. But in the future, MMV may need to carefully consider the nature of its relationships at the country level, including the possibilities for collaborative funding at the national level, in connection with its growing access and delivery work program. Source (on CGIAR): Operations Evaluation Department, 2004a, The CGIAR at 31: An Independent Meta- Evaluation of the Consultative Group on International Agricultural Research. 4.13 Once PD-PPPs reach late stage Phase III clinical trials and product registration, and launch access & delivery activities, dependable and sustainable funding will be essential if the previous investment in R&D is not to be wasted. Although the issue is well known, tangible, concrete action needs to be taken to bridge funding gaps if effective, affordable products are to become available to poor endemic country populations.46 It is notable that the Donor Coordination Group for PD-PPPs in which the Bank participates is discussing the possibility of moving beyond the simple exchange of information about MMV and other biomedical research and their financial requirements, and establishing a small, one-person secretariat. An initial step could be to agree on common performance metrics,47 and subsequent steps could include coordination of work programs, priorities, and funding decisions. Collaborative but separate fund-raising could also be considered but is probably not feasible at this stage because of competition among agencies for the same donor resources. 4.14 Since the Bank has a substantial institutional interest in promoting a sound health research agenda, it would be appropriate for the Bank not only to strengthen its engagement 46. Very rough estimates of a PD-PPP funding gap of $1 billion were made by the Donor Coordination Group in 2006, but substantial analytic work would be required to prepare and document reliable estimates. 47. Roy Widdus and Katherine White, "Financing Strategies for Product Development and the Potential Role of Public-Private Partnerships," Initiative on Public-Private Partnerships for Health, November 2004. 32 Table 8. Global Partnership Programs on Malaria and Other Communicable Disease Name Disease(s) Malaria Research Access and Delivery (Y/N) (Y/N) PD-PPPs DNDi 5 diseases Y Y GATB TB N n/a IOWH 4 diseases Y Y MMV Malaria Y Y Other health research programs and research coordination GFHR n/a N N IAVI AIDS N n/a MVI Malaria Y N (vaccine only) TDR 10 diseases Y Y Providing technical assistance and advocacy RBM Malaria Y N Stop TB TB UNAIDS HIV/AIDS N n/a Financing country-level programs GAVI Diseases other than N n/a AIDS and malaria GFATM AIDS, TB, malaria Operational research N only, in country projects Source: Author and Web sites of the research programs and initiatives. in the existing Donor Coordination Group but also to consider the possibility of gradually merging some or all of its DGF funding of health research, perhaps through the development of a new mechanism for coordinated resource allocation across health research priorities by the Global Forum and TDR. In this way the Bank could begin to operate in more of a "wholesale" than a "retail" mode in supporting health research. DGF funding would be used to stimulate the establishment of a network, just as it was used to facilitate the establishment of MMV. In the interviews for this report it was also suggested that one possibility to consider for sustainable funding of MMV and other programs would be the creation of a new International Financial Facility for Neglected Diseases, (IFFnd), comparable to the International Financial Facility for Immunization (IFFim) recently established for accelerated development assistance funding of immunization-related research and immunization delivery activities. The perspectives coming from Bank operational engagement at the country level -- repeatedly cited as a reason for the Bank to engage with MMV -- might be brought more effectively to bear on the definition and monitoring of overall health research programs in such an IFFnd, or a health research network,48 than through separate participation in MMV 48. Creation of a health research network was proposed in 2003 by the Director of the Fogarty International Center at NIH (Gerald T. Keusch and Carol A. Medlin, 2003, "Tapping the Power of Small Institutions," Nature, Vol. 422, April 10). 33 and other single disease programs. Such leadership would require conscious decisions and the allocation of substantial human and financial resources.49 5. Lessons 5.1 The following lessons have emerged from the experience of the Bank's partnership with MMV: · Product development public-private partnerships represent an innovative and sound model for the discovery and development of new health and medical products of vital public concern to developing countries, including new malaria drugs. As exemplified by the case of MMV, however, such PD-PPPs raise particular issues including (a) increasing legitimacy through appropriate membership in their governing bodies and other means to facilitate respect for beneficiary perspectives and an appropriate balance between public and private interests in the partnership; (b) establishing an appropriate planning and monitoring framework in an environment of rapid change and managed innovation by entrepreneurial teams;50 and (c) ensuring financial sustainability for long-term survival and success. · MMV's entry into access and delivery issues raises entirely new policy and institutional challenges for MMV. These are areas in which it has no institutional comparative advantage and little institutional experience but also new opportunities for exploiting synergies with Bank operations at the country level. Effective coordination and consultation with other players will be essential, including with the various units in WHO, GFATM, the Bank and other partners active at the level of individual countries, as well as with the key players in developing country governments concerned with health services, health policy, and drug regulation. · As the interviews undertaken for this study make clear, highly focused global programs such as MMV may look upon the Bank too narrowly, as primarily a source of funds, and could pay greater attention to other possible roles and opportunities to engage the Bank and benefit from multiple Bank roles. Possible new roles for the Bank at the level of MMV's access and delivery agenda are evident, although policy support and resources for them are currently inadequate within the Bank. There are also unexploited and unresourced opportunities for MMV and the Bank to engage at the global level on advocacy, on strategy, and on the appropriate balance in public- private partnerships. 49. IEG has previously proposed action by the Bank in this area, in its Phase 2 report on the Bank's global programs. The report observed that global health research for the poor is grossly underfunded. It also foresaw the Bank identifying under-funded long-term global public goods programs that benefit the poor, such as a global health research and product development network for diseases that disproportionately affect the poor, and using the Bank's convening power to mobilize additional resources for this (Operations Evaluation Department of the World Bank, 2004c). 50. A traditional logframe, of the kind used by the Bank for public infrastructure investments, may be unsuitable for entrepreneurial product-oriented R&D of the kind conducted by MMV. 34 · The Bank needs a more proactive and conscious approach to its participation in public-private partnerships such as MMV. Bank oversight of MMV has been minimal. The relevant Bank Departments and task managers need to redefine and communicate the roles which the Bank is willing to play in PD-PPPs as their needs for Bank engagement evolve, as their work programs change, and as the Bank's priorities shift.51 The Bank's budget systems and incentives, focused as they are on immediate analytic and advisory tasks and lending products at the country level, tend to discourage providing strategic support to programs such as MMV and funding the staff time required to establish the linkages with the Bank's country work. While the Bank's financial support for MMV's R&D activity has become less important over time as MMV has matured and demonstrated capacity to achieve its goals, a different kind of support centered around MMV's work on access and delivery now merits consideration. The Bank's strategic engagement in such partnerships also calls for consultation with its donor partners, as in the Donor Coordination Group for PD- PPPs, which may have expectations of the Bank in relation to oversight and other matters.52 51. The risks of fickleness in priority setting by the Bank were, however, a theme in several observers' comments for this GPR. 52. These activities would also require terms of reference for the staff concerned and they should take place within the context of guidelines for Bank participation in global programs, as IEG recommended in its Phase 2 Report on global programs, and as accepted by Bank Management in its formal response to this report (Operations Evaluation Department, 2004c). 35 References Alilio, Martin S., Ib C. Bygbjerg, and Joel G. Breman. "Are Multilateral Malaria Research and Control Programs the Most Successful? Lessons from the past 100 years in Africa." American Journal of Tropical Medicine and Hygiene, 71, 2004. Arrow, Kenneth, Claire Panosian, and Hellen Gelband, eds. Saving Lives, Buying Time: Economics of Malaria Drugs in an Age of Resistance, Institute of Medicine, 1994. Attridge, C. James, and Alexander S. Preker. "Improving Access to Medicines in Developing Countries -- Application of New Institutional Economics to the Analysis of Manufacturing and Distribution Issues," HNP Discussion Paper, World Bank, March 2005. Bathurst, Ian, and Chris Hentschel. "Medicines for Malaria Venture: Sustaining Antimalarial Drug Development," Trends in Parasitology, July 2006. Cohen, Jon. "Public-Private Partnerships Proliferate," Science, January 13, 2006. Daar, Abdallah S. et al. Towards Evolution and Growth, Fourth External Review of the Special Program for Research and Training in Tropical Diseases (TDR), Document TDR/JCB (29)/06.7, May 30, 2006. DiMasi, Joseph A., Ronald W. Hansen, and Henry G. Grabowski. "The Price of Innovation: New Estimates of Drug Development Costs," Journal of Health Economics, v. 22, March 2003. Enemark, Ulrika Anita Alban, and Enrique C.S. Vazques. "Purchasing Pharmaceuticals," HNP Discussion Paper, World Bank, September 2004. Fairlamb, Alan, Keith Bragman, Hassan Mshinda, and Adetokunbo Lucas (Team Leader). Independent Review of Medicines for Malaria Venture, DFID Health Resource Center, May 2005. Guttridge, W.E. "TDR Collaboration with the pharmaceutical industry," Transactions of the Royal Society of Tropical Medicine and Hygiene, Vol. 100, Supplement 1, December 2006. IEG (Independent Evaluation Group). The CGIAR at 31: An Independent Meta-Evaluation of the Consultative Group on International Agricultural Research, 2004a. ------. "Global Health Programs, Millennium Development Goals, and the World Bank's Role," Case Study, 2004b. ------. Addressing the Challenges of Globalization: An Independent Evaluation of the World Bank's Approach to Global Programs, 2004c. Independent Evaluation Group­World Bank and DAC Network on Development Evaluation. Sourcebook for Evaluating Global and Regional Partnership Programs -- Indicative Principles and Standards, 2007. Jamison, Dean, et al., eds., Disease Control Priorities in Developing Countries, 2nd Edition, WHO, World Bank, Bill and Melinda Gates Foundation, 2006. Kaplan, Warren, and Richard Laing. "Local Production of Pharmaceuticals: Industrial Policy and Access to Medicines," HNP Discussion Paper, World Bank, January 2005. Malaria R&D Alliance. "Malaria Research and Development: An Assessment of Global Investment," November 2005. Medicines for Malaria Venture. Business Plan, (draft), March 3, 2000. ------. Business Plan, 2003­2007, November 2003. ------. "Planning for Success -- Study Overview and Key Results," Bill and Melinda Gates Foundation and The Boston Consulting Group, Fall 2005. ------. Annual Report(s), 2000­2005; Annual Report 2006 (draft). 36 ------. Statuts de la Fondation "MMV Medicines for Malaria Venture," November 2002. Milstien, Julie B. Amie Batson, and Albert I. Wertheimer. "Vaccines and Drugs: Characteristics of their Use to Meet Public Health Goals," HNP Discussion Paper, World Bank, March 2005. Moran, Mary, et al. "The New Landscape of Neglected Disease Drug Development," London School of Economic and Political Science and Wellcome Trust, 2005. Murray, Christopher J. L., and Alan D. Lopez, eds. The Global Burden of Disease, WHO, Harvard School of Public Health and World Bank, 1996. Nikolic, Irina A., and Harald Maikisch. "Public-Private Partnerships and Collaboration in the Health Sector -- An Overview with Case Studies from recent European Experience," HNP Discussion Paper, World Bank, October 2006. Nwaka, Solomon, and Robert G. Ridley. "Virtual Drug Discovery and Development for Neglected Diseases through Public-Private Partnerships," Nature Reviews, Vol. 2, Issues 11, November 2003. Rodriguez-Monguio, Rosa, and Juan Rovira. "An Analysis of Pharmaceutical Lending by the World Bank," HNP Discussion Paper, World Bank, March 2005. Wagstaff, Adam, and Mariam Claeson. "Rising to the Challenges -- The Millennium Development Goals for Health," World Bank, 2004. Wheeler, Craig, and Seth Berkley. "Initial Lessons from Public-Private Partnerships in Drug and Vaccine Development," Bulletin of the World Health Organization, 2001, 79. Widdus, Roy, and Katherine White. "Combating Diseases associated with Poverty -- Financing Strategies for Product Development and the Potential Role of Public-Private Partnerships," Initiative on Public- Private Partnerships for Health, Global Forum for Health Research, November 2004. World Bank. Health, Nutrition and Population Sector Strategy, 1997. ------. Healthy Development -- The World Bank Strategy for Health, Nutrition and Population Results, February 2007. ------. Rolling Back Malaria: The World Bank Global Strategy and Booster Program, 2005. 37 Annex A Annex A. Evaluation Framework for Global Program Reviews Note: This evaluation framework is a general framework that has been designed to cover the wide range of such programs in which the World Bank is involved, encompassing policy and knowledge networks, technical assistance programs, and investment programs. It is not expected that every global program review will cover every question in this table in detail. Annex Table 1. Assessing the Independence and Quality of the Evaluation Evaluation Questions 1. Evaluation process To what extent was the GRPP evaluation independent of the management of the program, according to the following criteria: · Organizational independence? · Behavioral independence and protection from interference? · Avoidance of conflicts of interest? Factors to take into account in answering these questions include: · Who commissioned and managed the evaluation? · Who approved the terms of reference and selected the evaluation team? · To whom the evaluation team reported, and how the evaluation was reviewed? · Any other factors that hindered the independence of the evaluation such as an inadequate budget, or restrictions on access to information, travel, sampling, etc.? 2. Monitoring and evaluation framework of the program To what extent was the evaluation based on an effective M&E framework of the program with: · Clear and coherent objectives and strategies that give focus and direction to the program? · An expected results chain or logical framework? · Measurable indicators that meet the monitoring and reporting needs of the governing body and management of the program? · Systematic and regular processes for collecting and managing data? 3. Evaluation approach and scope To what extent was the evaluation objectives-based and evidence-based? To what extent did the evaluation use a results-based framework -- constructed either by the program or by the evaluators? To what extent did the evaluation address: · Relevance · Governance and management · Efficacy · Resource mobilization and financial management · Efficiency or cost-effectiveness · Sustainability, risk, and strategy for devolution or exit 4. Evaluation instruments To what extent did the evaluation utilize the following instruments: · Desk and document review · Consultations/interviews and with whom · Literature review · Structured surveys and of whom · Site visits and for what purpose: for interviewing implementers/beneficiaries, or for observing activities being implemented or completed Annex A 38 Evaluation Questions · Case studies · Other 5. Evaluation feedback To what extent have the findings of the evaluation been reflected in: · The objectives, strategies, design, or scale of the program? · The governance, management, and financing of the program? · The monitoring and evaluation framework of the program? Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program Every review is expected to cover the first four criteria in the following table: (a) relevance, (b) efficacy, (c) efficiency, and (d) governance and management. A review may also cover (e) resource mobilization and financial management and (f) sustainability, risk, and strategies for devolution or exit if the latter are important issues for the program at the time of GPR, and if there is sufficient information available on which to base an independent opinion. Evaluation Criteria and Questions Relevance: The extent to which the objectives and design of the program are consistent with (a) current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. 1. Supply-side relevance -- the existence of an international consensus that global/regional collective action is required. To what extent does the program reflect an international consensus on the need for action, on the definition of the problem being addressed, on priorities, and on strategies for action? Is the original consensus that led to the creation of the program still present? Is the program still needed to address specific global/regional public concerns? Take into account the origin of the program in answering these questions: · Is the program formally responsible for implementing an international convention? · Did the program arise out of an international conference? · Is the program facilitating the implementation of formal standards and approaches? · Is the program primarily donor-driven? Did donors establish the program with little consultation with developing countries? · Is the program primarily Bank-driven? Did the World Bank found the program and then seek other partners? 2. Demand-side relevance -- alignment with beneficiary needs, priorities, and strategies. To what extent are the objectives consistent with the needs, priorities, and strategies of beneficiary countries as articulated in the countries' own PRSPs, and in donors' strategies such as the World Bank CASs, and the UN Development Assistance Frameworks? To what extent has the voice of developing and transition countries been expressed in the international consensus underlying the program? 3. Vertical relevance -- consistency with the subsidiarity principle. To what extent are the activities of the program being carried out at the most appropriate level -- global, regional, national, or local -- in terms of efficiency and responsiveness to the needs of beneficiaries? To what extent are the activities of the program competing with or substituting for activities that individual donors or countries could do more efficiently by themselves? Pay particular attention to those programs that, on the face of it, are primarily supporting the provision of national or local public goods. 39 Annex A Evaluation Criteria and Questions 4. Horizontal relevance -- the absence of alternative sources of supply. What is the comparative advantage, value added, or core competency of the program relative to other GRPPs with similar or complementary objectives? To what extent is the program providing additional funding, advocacy, or technical capacity that is otherwise unavailable to meet the program's objectives? To what extent are the good and services being provided by the program in the nature of public goods? Are there alternative ways of providing these goods and services, such as by the private sector under regular market conditions? 5. Relevance of the design of the program To what extent are the strategies and priority activities of the program appropriate for achieving its objectives? What are the major activities of the program: · Policy and knowledge networking? · Financing country and local-level technical assistance? · Financing investments to deliver national, regional, or global public goods? (See Annex Table 4.) Has the program articulated an expected results chain or logical framework, along with assumptions that relate the progress of activities with the achievement of the objectives? Does the results chain identify the extent to which the achievement of the objectives depends on the effective functioning of bureaucracies, markets, or collectivities? If so, to what extent are these assumptions valid? For programs providing global or regional public goods, is the design of the program consistent with the way in which the individual efforts of the partners contribute to the collective outcome for the program as a whole -- whether "best shot", "summation", or "weakest link?" Efficacy: The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. 6. Achievement of objectives To what extent have the stated objectives of the program been achieved, or has satisfactory progress been made towards achieving these objectives? To what extent are there implicit objectives that are well understood and agreed upon by the partners and to which the program should also be held accountable? To what extent are there any positive, unintended outcomes of the program that have been convincingly document? To what extent have these assessments by the program or the evaluation been evidence-based? 7. Progress of activities, outputs, and outcomes. To what extent has the program or the evaluation measured the progress of activities, outputs, and outcomes? How did the program or the evaluation aggregate its outputs and outcomes at all levels -- global, regional, national, and local -- to provide an overall summary of its results? To what extent have factors such as changes in the location of the program, its legal structure, or governance processes affected the outputs and outcomes of the program? To what extent have there been outcomes that can be uniquely attributed to the partnership itself -- such as the scale of or joint activities made possible by its organizational setup as a GRPP, or its institutional linkages to a host organization? 8. Linkages to country or local-level activities. To what extent has the program established effective operational linkages with country-level activities, taking into account that: · The desired nature of these linkages will vary according to the objectives, design, and implementation of each program? · Positive outcomes at the country or local level are generally a joint product of both global/regional and county- level activities? Annex A 40 Evaluation Criteria and Questions Efficiency or cost-effectiveness: Efficiency -- the extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results. Cost-effectiveness -- the extent to which the program has achieved or is expected to achieve its results at a lower cost compared with alternatives. 9. Efficiency To what extent is it possible to place a monetary value on the benefits arising from the activities of the program? To what extent has the program or the evaluation conducted impact evaluations of representative program activities? To what extent has the program or the evaluation analyzed the program's costs in broad categories (such as overhead vs. activity costs), and categorized the program's activities and associated benefits, even if these cannot be valued in monetary terms? 10. Cost-effectiveness To what extent is the program measuring up against its own business plans: · Has the program cost more or less than planned? How did it measure up against its own costing schedule? · Have there been any obvious cases of inefficiency or wasted resources? To what extent is the program delivering its activities cost-effectively in comparison with alternatives: · How do actual costs compare with benchmarks from similar programs or activities? · Are the overhead costs of governing and managing the program reasonable and appropriate in relation to the objectives and activities of the program? How does the program compare with traditional development assistance programs: · For beneficiary countries, has receiving the development assistance through the GRPP increased the transactions costs compared with traditional development assistance programs? · For donors, has delivering the development assistance through the GRPP reduced donor costs by harmonizing efforts among donors or by reducing overlapping work (such as through joint supervision, monitoring and evaluation)? Governance and management: Governance -- the structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. Management -- the day-to-day operation of the program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Whereas governance is concerned with "doing the right thing," management is concerned with "doing things right." 11. Compliance with generally accepted principles of good governance. To what extent are the governance and management structures and processes well articulated and working well to bring about legitimate and effective governance and management? To what extent do governance and management practices comply with the following seven principles: · Legitimacy -- the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large? · Accountability -- the extent to which accountability is defined, accepted, and exercised along the chain of command and control within a program, starting with the annual general meeting of the members or parties at the top and going down to the executive board, the chief executive officer, task team leaders, implementers, and in some cases, to the beneficiaries of the program? · Responsibility -- the extent to which the program accepts and exercises responsibility to stakeholders who are not directly involved in the governance of the program and who are not part of the direct chain of accountability in the implementation of the program? 41 Annex A Evaluation Criteria and Questions · Fairness -- the extent to which partners and participants, similarly situated, have equal opportunity to influence the program and to receive benefits from the program? · Transparency -- the extent to which a program's decision making, reporting, and evaluation processes are open and freely available to the general public? · Efficiency -- the extent to which the governance and management structures enhance efficiency or cost- effectiveness in the allocation and use of the program's resources? · Probity -- the adherence by all persons in leadership positions to high standards of ethics and professional conduct over and above compliance with the rules and regulations governing the operation of the program? 12. Partnerships and participation To what extent has the program identified a complete list of stakeholders, or "stakeholder map", including the agreed- upon or perceived roles and responsibilities of the categories of stakeholders identified? To what extent is this a routine programmatic function, updated regularly, and transparently available? Has the program adopted primarily a shareholder model of governance (in which membership on the governing body is limited to financial and other contributors), or a stakeholder model (in which membership also includes non- contributors)? To what extent, if any, is the program's legitimacy being sacrificed in order to achieve greater efficiency, or vice-versa? 13. Programs located in host organizations To what extent is the location of the program in the Bank or other partner organization adversely affecting the governance, management, or other aspects of the program, such as compliance with the principles of transparency and fairness? For which functions is the program manager accountable to the host organization and the governing body of the program, respectively? Are conflicts of interest being managed appropriately? To what extent does the host organization play such a dominant role in the program, thereby reducing the incentives of other partners to participate effectively, or reducing the ability of the host organization to look at the weaknesses of the program objectively? Resource mobilization and financial management: Resource mobilization -- the processes by which resources are solicited by a program and provided by donors and partners. Financial management -- the processes that govern the recording and use of funds, including allocation processes, crediting and debiting of accounts, controls that restrict use, accounting, and periodic financial reporting systems. In cases where funds accumulate over time, this would also include the management of the cash and investment portfolio. 14. Resource mobilization To what extent has the program succeeded in raising financial resources commensurate with its objectives? And from what sources -- the Bank, bilateral donors, foundations, etc.? To what extent has the program succeeded in diversifying its funding beyond a small number of donors? To what extent are the sources of funding for the program (including donor restrictions on the use of resources) affecting, positively or negatively: · The strategic focus of the program? · The outputs and outcomes of the program? · The governance and management of the program? · The sustainability of the program? Annex A 42 Evaluation Criteria and Questions 15. Financial management Are there any issues that have emerged during the course of the review in relation to: · The quality of financial management and accounting? · The methods, criteria, and processes for allocating funds among different activities of the program? · Financial management during the early stages of the program? Sustainability, risk, and strategy for devolution or exit: Sustainability -- When applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When applied to a program itself, the extent to which the organization or program is likely to continue its operational activities over time. Devolution or exit strategy -- a proactive strategy to change the design of a program, to devolve some of its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. 16. Sustainability of the benefits of the program's activities What is the risk, at the time of evaluation, that the development outcomes (or expected outcomes) of the program will not be maintained (or realized)? This depends on (a) the likelihood that some changes may occur that are detrimental to maintaining or realizing the expected outcomes, and (b) the affect on the expected outcomes if some or all of these changes actually materialize? 17. Sustainability of the program This will depend on a number of factors, such as the continued legitimacy of the program, its financial stability, its continuity of effective management, and its ability to withstand changing market or other conditions. To what extent is there still a sufficient convergence or accommodation of interests among the major partners to sustain the program financially? To what extent has the program developed institutional capacity such as performance- based management, personnel policies, learning programs, and knowledge management that help to sustain a program? In what areas could the program improve in order to enhance its sustainability, such as better marketing of the program's achievements in order to sustain its reputation? 18. Prospects for continuation and strategies for devolution or exit To what extent should the program be sustained? Is the continuation of the program the best way of sustaining the results achieved? Should the design of the program be modified as a result of changed circumstances, either positive or negative? What other alternatives should be considered to sustain the program's results more cost-effectively, in the light of the previous evaluation findings with respect to relevance, efficacy, efficiency, and sustainability: · Reinventing the program with the same governance? · Phasing out the program? · Continuing country or local-level activities with or without devolution of implementation? · Seeking alternative financing arrangements, such as revenue-generation, or self-financing to reduce dependency on external sources? · "Spinning off" from the host organization? 43 Annex A Annex Table 3. Assessing the Bank's Performance as a Partner in the Program Evaluation Questions 1. Comparative advantage at the global/regional level. To what extent is the Bank playing up to its comparative advantages at the global/regional level -- its global mandate and reach and convening power? To what extent is the Bank's presence as a partner in the program catalyzing other resources and partners for the program? 2. Comparative advantage at the country level. To what extent is the Bank contributing multi-sector capacity, analytical expertise, and country-level knowledge to the program? To what extent has the Bank's country operations established linkages to the GRPP, where appropriate, to enhance the effectiveness of both? 3. Oversight. To what extent is the Bank exercising effective and independent oversight of its involvement in the program, as appropriate, whether the program is housed in the Bank or externally managed? To what extent is the Bank's oversight independent of the management of the program? To what extent does the Bank's representative on the governing body have a clear terms of reference? 4. Risks and risk management. To what extent have the risks associated with the program been identified and are being effectively managed? For example, IEG identified the following risks in its global review: · Bank bears a disproportionate share of responsibility for governing and managing in-house programs? · Confusion at the country level between global program activities, Bank activities, and Borrower activities? · Representation of NGOs and the commercial private sector on program governing bodies? · Unclear role and application of Bank's safeguards? · Trust-funded consultants and seconded staff representing the Bank on some program governing bodies? 5. Disengagement strategy. To what extent is the Bank engaged at the appropriate level in relation to the Bank's new strategic framework: · Watching brief? · Research and knowledge exchange? · Policy or advocacy network? · Operational platform? To what extent is the Bank facilitating an effective, flexible, and transparent disengagement strategy for the program, in relation to the Bank's objectives for its involvement in the program: · The program declares "mission accomplished" and closes? · The program continues and the Bank withdraws from all aspects of its participation? · The program continues and the Bank remains engaged, but the degree of the Bank's engagement in some or all aspects (such as financing) declines over time? Annex A 44 Annex Table 4. Common GRPP Activities Policy and knowledge networking 1. Facilitating communica- This includes providing a central point of contact and communication among practitioners tion among practitioners who are working the sector or area of development to facilitate the sharing of analytical in the sector results. It might also include the financing of case studies and comparative studies. 2. Generating and This comprises two related activities. The first is gathering, analyzing and disseminating disseminating information, for example, on the evolving HIV/AIDS epidemic and responses to it, information and including epidemiological data collection and analysis, needs assessment, resource knowledge flows, and country readiness. The second is the systematic assembling and dissemination of knowledge (not merely information) with respect to best practices in a sector on a global/regional basis. 3. Improving donor This should be an active process, not just the side effect of other program activities. This coordination may involve resolving difficult interagency issues in order to improve alignment and efficiency in delivering development assistance. 4. Advocacy This comprises proactive interaction with policymakers and decision makers concerning approaches to development in a sector, commonly in the context of global, regional, or country-level forums. This is intended to create reform conditions in developing countries, as distinct from physical and institutional investments in public goods, and is more proactive than generating and disseminating information and knowledge. 5. Implementing Rules are generally formal. Standards can be formal or informal, and binding or conventions, rules, or nonbinding, but implementing standards involves more than simply advocating an formal and informal approach to development in a sector. In general, there should be some costs associated standards and norms with noncompliance. Costs can come in many forms, including exposure to financial contagion, bad financial ratings by the IMF and other rating agencies, with consequent impacts on access to private finance; lack of access to OECD markets for failing to meet food safety standards, or even the consequences of failing to be seen as progressive in international circles. Financing technical assistance 6. Supporting national- This is more directed to specific tasks than advocacy. This represents concrete level policy, institutional, involvement in specific and ongoing policy, institutional, and technical reform processes and technical reforms in a sector, from deciding on a reform strategy to implementation of new policies and regulations in a sector. It is more than just conducting studies unless the studies are strategic in nature and specific to the reform issue in question. 7. Capacity strengthening This refers to strengthening the capacity of human resources through proactive training and training (in courses or on-the-job), as well as collaborative work with the active involvement of developing country partners. 8. Catalyzing public or This includes improving regulatory frameworks for private investment and implementing private investments in pilot investments projects. the sector Financing investments 9. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver loans and credits (more than the financing of studies), the benefits of which accrue national public goods primarily at the national level. 10. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver loans and credits (more than the financing of studies) to deliver public goods such as global/regional public conserving biodiversity of global significance and reducing emissions of ozone-depleting goods substances and carbon dioxide, the benefits of which accrue globally. 11. Financing global/ This refers to financing research and development for new products and technologies. regional investments to These are generally physical products or processes -- the hardware as opposed to the deliver global/regional software of development. public goods 45 Annex B Annex B. MMV Objectives, Collaboration Principles, and Program Activities Malaria Drug Research in Context The efficiency and cost-effectiveness of MMV's drug R&D activity can best be understood within the context of changes in the environment for drug R&D on diseases of poor countries.53 Following significant success on malaria control in connection with the construction of the Panama Canal in the first decade of the 20th Century, military research and concerns were central to malaria research in the US. In the years after World War II, a major effort was made through WHO, ultimately unsuccessfully, to eradicate malaria, using DDT to kill the mosquito vector as the campaign's centerpiece. For many years after the War chloroquine was an inexpensive drug of choice for malaria treatment, but resistance has become widespread. Malaria drug research was sponsored by major pharmaceutical firms and the defense establishment, especially in the United States. Broader malariology was centered in the United Kingdom. MMV's work has been at the center of recent changes in the pharmaceutical industry, and has both encouraged and been stimulated by them. Despite the absence of significant new government R&D incentives, there has been a remarkable upsurge in R&D for neglected diseases by 2005. Large pharmaceutical enterprises had been through a period of mergers starting in 1995. Many of the multinationals down-sized, with loss of skills and knowledge relevant to malaria R&D. This led to a shift from in-house R&D towards licensing-in IPR from small biotech companies and academia. Meanwhile, major markets such as China and India were already developing high-skill and low-cost R&D capacity, which encourages the R&D contracting model of MMV. Under pressure from the public and key actors, large pharmaceutical companies began increasingly to operate on neglected diseases in a "no profit-no loss" model. For these companies research on neglected diseases is an expression of corporate social responsibility rather than a commercial concern. The benefits accrue largely to corporate reputation and public image, but there are also valuable introductions to developing country markets and researchers, and benefits in employee morale. The smaller biotech companies in the industrial world tend to be heavy on capacity in early stages of drug R&D but low on capacity for clinical development activity through large human trials. This environment suited MMV's approach of "virtual" R&D, with nearly all activity outsourced and some projects licensed-in to provide appropriate balance in the research portfolio. Overview of MMV's Objectives, Guiding Principles, and Mode of Operation According to MMV's agreed statement of collaboration principles adopted in 2004, MMV's central objective is to ensure the sustainable and continuous generation of appropriate new 53. Moran et al., 2005, and Widdus and White, 2004. Annex B 46 malaria medicines that are accessible to all of those in need in developing countries at the lowest prices practicable. The medicines should be available for distribution utilizing either or both public and private sector channels, to ensure the central objective is met. Such distribution should be consistent with ethical use, public health impact and national drug policies. It is intended that products sold in developing countries would be preferentially priced, at profit margins more comparable to those traditionally associated with generic products than those associated with new products. Prices in non-malaria endemic countries may be determined by the commercial collaborator through normal commercial considerations. While MMV must seek to achieve its objectives in collaboration with both commercial and non-commercial organizations, it has the responsibility to ensure that viable projects are pursued to completion. It requires sufficient overall influence in the collaboration to achieve this. Moreover, MMV may terminate a given collaboration agreement for any reason at any time upon 90 days written notice if it decides not to proceed with the research program. MMV requires intellectual property rights on a royalty free basis to the relevant intellectual property in the field of malaria and developed through the collaboration necessary to meet its objectives. MMV seeks rights in relevant background intellectual property necessary to achieve its objectives. While publication of research results is encouraged, it is intended that the collaborators will agree to necessary controls on publication so as to preserve intellectual property rights -- in particular patent rights and confidential information. MMV would not normally have a desire to retain any interest in relevant intellectual property rights for use outside the field of malaria or to constrain such use by its collaborators. MMV expects that income from sales of products for the treatment of malaria in non-malaria endemic countries that incorporate intellectual property developed through the collaboration will further the objective of ensuring that malaria medicines are readily accessible to all of those in need in developing countries at the lowest prices practicable. Moreover, MMV expects that income from sales of products outside the field of malaria that incorporate intellectual property developed through the collaboration will also serve to further that objective. MMV works to achieve its objectives by inviting, screening, financing, and supervising the execution of research proposals by contract research organizations in the public and private sectors throughout the world. Its Expert Scientific Advisory Committee (ESAC) plays a key role in the screening, selection and annual supervision of research proposals and projects, and helping them to pass through the various stages of drug development, from identification of potential leads stemming from basic research, through pre-clinical and clinical development, including large scale trials in human subjects. The entire process culminates in the registration of new drugs with competent national drug regulatory authorities. Drug Discovery and Development MMV's work on drug discovery and development is captured in several figures and graphs. Annex Figure 1 shows MMV's R&D portfolio in July 2006. It reveals that MMV has products at all stages of discovery and development. It also shows the transitions through 47 Annex B Annex Figure 1. MMV Portfolio, July 2006: Products at All Stages of Development which candidate drugs pass on the way to registration with competent national drug authorities. Annex Figure 2 presents the average successful project transition rates through drug development phases, as used by MMV for portfolio planning and management. The figure illustrates dramatically the low probability that any given candidate drug will pass successfully through all phases of discovery and development to approval for marketing by the appropriate public authority. Annex Figure 3 gives the number of projects in the MMV R&D portfolio in each year of its operation, and reveals that terminations are an important part of MMV's portfolio management task. Annex B 48 Annex Figure 2. MMV: Successful Project Transition Rates Used for Portfolio Planning and Management Utilization Registration/ Approval 95 % Project success rates at each phase Phase 3 65 % Phase 2 50 % Phase 1 70 % Pre-clinical 55 % Lead Optimization 55 % Lead Identification 65 % Early Discovery 30 % 0 100 200 300 400 500 600 700 800 900 1000 Source: MMV Science Team Member Assessments; Industry Figures; Paraxel's 2002/2003 Sourcebook. Annex Figure 3. The Evolving MMV R&D Portfolio: An Increasing and Dynamic Number of Projects for Drug Discovery and Clinical Development, 2000­2006, with Terminations 30 Number of Projects Per Stage 25 20 15 10 5 0 2000 2001 2002 2003 2004 2005 2006 Discovery projects Clinical development Total discovery & development Terminated projects Source: MMV CFO, March 2007. 49 Annex B Access and Delivery of New Drugs The market failure on R&D that led to the creation of MMV extends to downstream issues severely limiting patients' access to effective treatment. This failure of the market to address the health needs of the most vulnerable is clearly evident in the continued use, through drug outlets, of ineffective treatment such as chloroquine as the drug of choice by the rural poor -- a choice driven largely by cost and availability. In 2005, MMV and its stakeholders were concerned that the antimalarials soon to emerge from its R&D pipeline might not reach its target population segments, and might simply result in limited uptake and health impact without an appropriate delivery strategy. A multi- country assessment confirmed these concerns and led MMV to expand its mandate from Discover and Develop to the third D -- Deliver. MMV's principles concerning access and delivery of new malaria drugs, and the many stages of activity from registration through distribution of a new product are summarized in: Downstream access prerequisites (Annex Table 5) Key partners and key partner activities relating to downstream access prerequisites (Annex Table 6) A schematic schedule towards access and delivery of new drugs (Annex Figure 4). MMV's work program on access and delivery is only beginning to get under way. However, a 2005 "Planning for Success" consulting study for MMV by the Boston Consulting Group (BCG) with the Bill and Melinda Gates Foundation sheds light on MMV thinking. The study started from the fact that currently available malaria drugs are cheap, widely available but increasingly ineffective due to the development of resistance to them. Multiple stakeholders are recognized to be involved in policy-making, funding, manufacturing and distribution, as confirmed in Annex Table 6. Against this background the BCG study asked what interventions are needed, how can an adequate and cost-effective supply of antimalarial drugs be secured, what are the key determinants of policy adoption globally and in malaria endemic countries, and what are the key threats and opportunities? The consultants interviewed multiple stakeholders and undertook research in six countries, including four in Africa. Among policy issues, the BCG team underscored the importance of engaging global and local players in WHO and Ministries of Health, sharing clinical trial data, launching post- distribution studies, and ensuring inclusion of the new drug(s) in essential drugs lists and treatment guidelines, with help from WHO. On financing and procurement issues, the study addressed GFATM applications, but did not consider the Bank. In the area of manufacturing, it underscored the importance of WHO pre-qualification and development of forecasts. It emphasized the determination of channels and competition strategies for new drug distribution and design of private and public sector rollouts. The study created a model to assess and update demand in the public and private sectors in different areas; MMV can share the model with interested stakeholders in the malaria community. It considered the experience of endemic countries that have adopted ACTs into their malaria drug policies. Annex B 50 Annex Table 5. MMV: Downstream Access Prerequisites Prerequisites Challenges to Prerequisites Potential Global Responses · Cost of the antimalarials will most likely · Donor funding to subsidize costs of be higher than existing drugs antimalarials in both the public and Sustainable Financing private sectors · Competing health needs and stretched health budgets · Tiered pricing in the public and private sectors · Vested interests to continue use of · Adequate time to sensitize recipient existing antimalarials countries about new drugs and their benefits, training required · Vast number of stakeholders involved Supportive Drug in drug policy and time required to Policies change national drug policies · Vertical disease interests can distort more comprehensive drug policy planning · Insufficient staff, poor training · Sensitization of regulatory bodies and key stakeholders before dossiers are Fast Track Regulatory · Bureaucratic regulations leading to completed Approval delays · Provision of necessary data · Inadequate financial resources · Initial years of low volume sales do not · Donor subsidization of drugs to allow allow for economies of scale for initial higher sales volumes Low Cost Manufacturing · Complex compounds may require · Subsidize production or increase low- expensive, multiple step production cost production capacity in developing processes countries · Insufficient medical provider staff, poor · Training of public and private medical Trained Providers/ training and lack of financial incentives providers Informed Consumers (in the public sector) · Provision of consumer information · Limited health literacy by consumers materials · Public Sector. Poor quality of storage and · Training on proper storage for both inventory control; Limited number of public and private medical providers government clinics, particularly in rural Efficient Public and · Pilot introductions to test efficacy of areas; Diversion of public sector private distribution channels Private Sector commodities to the private sector · Differentiated packaging in public and Distribution · Private Sector. Poor quality of drugs due private sectors to inadequate regulation or enforcement; Incentives to over · Stimulate demand in private sector prescribe in private sector through promotional materials Feedback from · Lack of systems to capture drug · Design of surveillance systems, data Pharmacovigilance utilization information requirements to inform future R&D Source: MMV. 51 Annex B Annex Table 6. Access Prerequisites: Key Partners and Partner Activities Downstream Access Prerequisites Key Partners Key Partner Activities · Industry · Pharmaceutical and other · FDA/EMEA contracted partners Fast-Track Regulatory conduct drug registration Follow On Drug Approval process in endemic Development countries Components · Industry · High-quality manufacturing Mission-Driven · Contract manufacturers that attains drug Manufacturing affordability · WHO/RBM · Articulate need for donor · Global Fund purchase or subsidization of drugs in endemic Purchase or · Donors countries Subsidization of Drugs Sustainable Financing · Key OECD governments by Donors · Provide funding to endemic countries for purchase of developed drugs · WHO/RBM · Sponsor consultations · TDR among health officials in Supportive Drug endemic countries to foster Policies in Endemic · Ministries of Health supportive drug policies, Countries new guidelines, etc. for the introduction of new drugs · WHO/RBM · Create training materials, · NGOs conduct trainings and follow-ups with public providers medical sector Conducive Setting for Trained Providers/ private and (if appropriate) Drug Access in Informed Consumers to ensure that they Endemic Countries understand benefits and risks of new drugs and are able to communicate to patients · WHO/RBM · Financial support for · Donors training of public sector Efficient Public and logistical personnel to · Ministries of Health Private Sector ensure efficient distribution Distribution · Industry and possible subsidization · NGOs of drugs in the private sector · TDR · Creation of monitoring · Industry protocols and systems to Tracking of Patient capture patient reaction to Feedback from Reactions to Drugs in drugs Pharmacovigilance Endemic Countries · Transmittal of information to MMV or partners Source: MMV. Annex B 52 Annex Figure 4. MMV: Towards Access and Delivery of New Malaria Drugs First country Public sector approval adoption Product development 3 yrs 2 yrs 1 yr 1 yr 2 yrs 3 yrs 4 yrs 5 yrs 6 yrs 7 yrs pathway Pre- launch Post- launch Phase III trials Regulatory SRA(1) NRA(2) Share trial data Global Policy Treatment guidelines (WHO) PQ(4) EDL(3) Phase IV research Share trial data WHO/country information sharing Local Policy Treatment guidelines EDL First forecast Ongoing forecasting Manufacturing Secure raw materials CMC / network strategy Ensure proper scale up Share trial data WHO information sharing GFATM App. Funds dispersed Financing and Procurement procurement Drug delivery Set country strategy Distribution Private sector roll-out Ongoing dist. Public sector roll-out Ongoing dist. Post IEC distribution Pharmacovigilance (1) Stringent Regulatory Authority (e.g., EMEA, FDA, other) (2) National Regulatory Authority endemic country; Ongoing activity may require additional small scale local studies (3) Essential Drug List; (4) Pre-qualification Direct uptake impact Source: WHO website, GFATM research, interviews 53 Annex B The BCG study stressed the importance of MMV engagement with global agencies like GFATM and WHO, especially in Africa, and of dialogue and early engagement with countries being expected to change their drugs of choice. It pointed out that the majority of malaria drugs are distributed through the private sector and that low price is the primary purchase driver but also that patient awareness and attitudes present significant challenges to uptake. The study found that research indicated price and sustainability concerns have slowed ACT programs in the public market. It thus raised the issue of a global subsidy, as proposed by the US IOM.54 The consultant study concluded that operational experience and effectiveness studies are required for local policy adoption. It found counterfeit drugs and poor quality as significant problems in the private sector drug market, and saw a need for various approaches to reduce monotherapy and increase combination drugs. On the basis of the experience of Vietnam in introducing artemisinin-based drugs, the study found that multi- prong rollout of new malaria drugs can dramatically reduce the burden of the disease. From nearly 24 cases per 1,000 persons in 1986, Vietnam's burden fell to 2 cases per 1,000 people at the turn of the millennium. The BCG study concluded that MMV will increasingly get involved in access and delivery through partnerships and advocacy roles. It stressed the importance of information sharing between endemic countries and drug development agencies, and for helping the countries to be aware of the drug development pipeline which may affect their policy-making. Helping countries to deal with the difficult problem of multiple products, rather than a single, simple malaria drug regimen, is part of the agenda. MMV's approach to access and delivery is patient-oriented and geared towards facilitation of delivery of MMV's new products. It aims to ensure that MMV drugs meet the needs of its target population with regard to price, distribution channels, and information -- in other words, the right drug, at the right place, at the right price, with the right information, at the right time. The mission of MMV's work on access and delivery is to accelerate the speed of product adoption, expand its reach and help shape future product development. The timely availability of data and dossiers for review by various regulatory authorities, both national and international, should accelerate the registration of MMV products and their inclusion on treatment guidelines and procurement lists. This is a precondition for public sector use, an important source of antimalarials for MMV's target segment of women, children, and the poor. MMV is undertaking a critical path analysis to map out the product adoption pathway. MMV will focus on how to responsibly improve access to artemisinin combination therapy (ACT) through the private sector, which remains a vital source of antimalarials in view of its wider coverage and reach. This sector also serves as a back-up to public sector provision of ACTs; public sector product stock-outs are frequently recorded in most countries. ACT penetration of this sector is less than 5 percent partly due to its relatively higher cost and its prescription-only status; they preclude sale through the informal private sector. 54. Arrow et al., 1994. Annex B 54 MMV's direct engagement in country programs is expected to facilitate a better understanding of the antimalarial market as well as "product readiness" issues critical to correct product dispensing and use. The readiness of health systems, though critical, is beyond MMV's remit. MMV expects to continue the successful partnership model at the core of its operations and to collaborate closely with organizations at both global and country level. With direct engagement at the country level with authorities, health care providers, and patients, MMV's access and delivery activities are expected subsequently to reflect the voice of the customer in the product development process. At the operational level MMV has launched access and delivery studies in Uganda, in collaboration with the Ministry of Health, on implementation of an ACT subsidy and prices of antimalarial drugs. It is also discussing implementation issues in the proposal for a global ACT subsidy put forward by IOM. Supply chain issues -- especially procurement -- are on the agenda. On a wider plane, MMV is cooperating with the Gates Foundation on a 10 country study of price and supply chain issues. Five years of country-specific data are to be collected, and the approach is to see malaria drugs as a public good or at least as a merit good. WHO Headquarters staff and WHO AFRO are reported to be involved. As of March 2007 the study was said to be still at the stage of discussions with potential executing agencies. 55 Annex C Annex C. Program Timeline: Key Internal and External Events Year and Month Events Internal to MMV Events External to MMV 1997 WHO, TDR, World Bank, NIH, GFHR, Rockefeller Foundation, the Swiss Agency for Development and Cooperation, the Wellcome Trust, and other officials begin discussions, also with the International Federation of Pharmaceutical Manufacturers Associations and the British drug industry, on mechanisms to support discovery and development of new malaria drugs 1998 July Launch of Roll Back Malaria partnership (RBM) by WHO, UNICEF, UNDP, and the World Bank 1999 November Formal launch of MMV as a non-profit MMV is the first of four PD-PPPs that Swiss Foundation by WHO Director- are established early in the new General and others; MMV is initially millennium housed at TDR; Dame Bridget Ogilvie, former head of the Wellcome Trust, becomes first Board Chairperson 2000 March Completion of MMV Business Plan, prepared with assistance of the Boston Consulting Group and financial support of the Rockefeller Foundation March Bill and Melinda Gates Foundation commits $25 million for MMV; release of initial funds, even before MMV initial capital of $4 million, creates confidence to permit staff recruitment and start-up of activity April MMV has 4 projects in its portfolio 54 of 55 endemic countries attend the Abuja Summit sponsored by RBM, and resolve to ensure that by the year 2005 at least 60 percent of those suffering from malaria have prompt access to, and are able to correctly use, affordable and appropriate treatment within 24 hours of the onset of symptoms; they also commit to a 50 percent reduction on malaria mortality by 2010. Annex C 56 Year and Month Events Internal to MMV Events External to MMV July UN Secretary-General and G­8 propose creation of Global Fund to Fight AIDS, TB, and Malaria (GFATM) September Adoption of Millennium Declaration by UN General Assembly endorsing the Millennium Development Goals (MDGs) contributes to a public policy environment in industrial counties supportive of attention to malaria in the developing countries 2001 April African leaders commit to devoting at least 15 percent of national budgets to health. December Publication by WHO of report of Commission on Macroeconomics and Health, which calls for increased support for R&D targeting the poor and lamenting low funding of MMV 2002 January GFATM starts operation; its Board declines to fund research October Publication of malaria parasite (plasmodium falciparum) genome opens new possibilities for malaria drug discovery November Third Multilateral Initiative Pan-African Malaria Conference, Tanzania, brings together 1,000 participants 2003 September MMV has more than 20 projects in its drug discovery and development portfolio November Release of MMV Business Plan 2003­ 2007, prepared with assistance from the Foundation Strategy Group 2004 May Donor Coordination Group founded for PD-PPPs; it is subsequently renamed PD-PPP Funders Group. May­June MMV initiates holding key meetings in beneficiary countries with meetings of the MMV Board of Directors and other stakeholders in Maputo, Mozambique, under the patronage of the President of Mozambique 57 Annex C Year and Month Events Internal to MMV Events External to MMV July IOM publishes report of panel led by Nobel Laureate economist Kenneth Arrow, "Saving Lives, Buying Time: the Economics of Malaria Drugs in an Age of Resistance," calling for increased funding for malaria drug research, with 50 percent to MMV October Publication of "Copenhagen Consensus" report of panel of independent economists, who place control of malaria among only four "very good" projects for use of development cooperation funds 2005 January UK assumes Presidency of G­8, with emphasis on better health and Africa; this is helpful to the external environment for MMV and in donor support April Launch of World Bank Malaria Booster Program envisaging $500­$1,000 million in new commitments for malaria control, over five years May Completion of external evaluation of Release of Report of "Commission for MMV commissioned by donors Africa" by UK Prime Minister; Commission calls for incentives for pharmaceutical companies to invest in research in new medicines, and for donors to fund health research and advance market commitments for new medicines. May Meetings of MMV Board of Directors "World Malaria Report" released by and donor stakeholders in Bangkok, RBM estimates that the continual Thailand, in cooperation with the development of new antimalarials for Faculty of Tropical Medicine populations at endemic risk, including special groups such as children and pregnant women, at the rate dictated by the development of drug resistance will cost at least US$ 30 million per year, possibly more after 2006 when more projects move into the expensive phases of clinical development. June Launch of President's Malaria Initiative in United States, including a pledge to increase US malaria funding by more than $1.2 billion over five years to reduce deaths due to malaria by 50 percent in 15 African countries Annex C 58 Year and Month Events Internal to MMV Events External to MMV 2006 March MMV completes transition to International Financial Reporting Standards with issuance of its Annual Report for 2005 May Former British MP and Minister of State for Overseas Development Baroness Lynda Chalker of Wallassey succeeds Dame Bridget Ogilvie as Chairperson of the MMV Board of Directors July Preparatory Meeting in Geneva of MMV Access and Delivery Advisory Committee (ADAC), including extensive list of observers from Bill and Melinda Gates Foundation, WHO, MMV- supported research entities, industry, GFATM, and TDR, but not World Bank July Meeting(s) of MMV Board of Directors and stakeholders in Zambia, in association with MMV-sponsored clinical trials there December White House Malaria Summit in Washington 2007 January Preparation is initiated of new MMV Business Plan 2007­2012, with assistance of Boston Consulting Group; draft is expected to be discussed with MMV Board of Directors in May, and completion expected in November March Launch of all-party report on financing malaria in London and, with World Bank President Wolfowitz present, in Johannesburg April MMV has 35 projects in its portfolio RBM presents $250-$300 million global subsidy plan for ACTs to AU ministers, aiming to see 300 percent improvement in access to treatment; while recognizing that ACTs are expensive, RBM hopes for formal launch in 2008 June OECD High Level Meeting on Medicines for Neglected and Emerging Diseases in the Netherlands focuses on TB and malaria. Note: All data as of June 2007 59 Annex D Annex D. Members of Key MMV Governing Bodies Board of Directors Name MMV Position Professional background and affiliation Baroness Chalker of Chairperson Member of Parliament, Minister of State for Foreign Wallasey, Lynda and Commonwealth Affairs; consultant on public- private interface Dr. Anarfi Asamoa- Board Member Assistant Director-General, WHO;1 former Director of Baah Medical Services, Ghana; medical doctor James M. T. Board Member Former Director, Glaxo Wellcome; Chair-elect of Cochrane British Red Cross Prof. Winston Observer Chair, Expert Scientific Advisory Committee (ESAC) of Gutteridge MMV; Visiting Professor, London School of Hygiene and Tropical Medicine; former Chief, Product R&D, WHO Tropical Disease Research Program Dr. Chris Hentschel Board Member President and CEO, MMV; biochemist; former CEO, UK Medical Research Council's Collaborative Center Prof. Trevor Jones Board Member Chair, ReNeuron, Ltd (a stem cell biotech company); former Director-General of the Association of the British Pharmaceutical Industry (ABPI) Dr. R. A. Mashelkar Board Member Director General, Council of Scientific and Industrial Research, India; former Director, National Chemical Laboratory, Pune; chemical engineering scientist. Dr. Pascoal Mocumbi Board Member Former Prime Minister and Minister of Health, Mozambique; medical doctor Dr. Carlos Morel Board Member Former Director of TDR, WHO; former WHO Executive Board member; medical doctor and molecular biologist Dr. Regina Board Member Director, Infectious Disease Program, Bill and Melinda Rabinovich Gates Foundation; former Chief of Clinical and Regulatory Affairs Branch, National Institute of Allergy and Infectious Diseases (NIAID); vaccine research scientist Dr. Leon Rosenberg Board Member Professor, Molecular Biology, Princeton University; former Chief Scientific Officer of Bristol- Myers Squibb; member, Institute of Medicine and National Academy of Sciences /1 In March 2007, Dr. Asamoah-Baah was appointed Deputy-Director General of WHO. Note: The Board of Directors has three Committees: Remuneration, Audit, and Nominations Annex D 60 Expert Scientific Advisory Committee (ESAC) Name Position Scientific background Win Gutteridge Chair Consultant and visiting Professor, London School of Hygiene and Tropical Medicine Richard Auty Member Chair, MNLpharma, Ltd, Director, Salient Consulting Ltd and Visiting Professor of Medicine, University of Malawi George Aynilian Member Director of drug development with expertise in clinical research and international regulatory affairs William Charman Member Professor of Pharmaceutics, Monash University, Australia Virander Chaugan Member Director, Malaria Research Group, International Center for Genetic Engineering and Biotechnology, India Awa Coll Seck Observer Chair, MMV ADAV; Executive Secretary, Roll Back Malaria Partnership, WHO; former Minister of Health, Senegal, and former Director of Policy, UNAIDS David Floyd Member Chief Scientific Officer and Executive Vice- President, Pharmacopeia Drug Discovery, USA Brian Greenwood Member Professor of Clinical Tropical Medicine, London School of Hygiene and Tropical Medicine, Director of Malaria Center and Gates Malaria Partnership David Matthews Member Expertise in methods of drug discovery; founder of and now retired from Pfizer, La Jolla Margaret Phillips Member Professor, Department of Pharmacology, University of Texas Southwestern Medical School, Dallas Zulfiqarali Member Professor of Clinical Parasitology Gulamhussien Premji Maria Paris Member Senior Medical Director, ENANTA Pharmaceuticals, USA David Roos Member Professor of Biology and Director, University of Pennsylvania Genomics Institute Jurg Seiler Member Consultant in pharmacology, toxicology and regulatory affairs; former group leader, Swiss Agency for Therapeutic Products Dennis Schmatz Member Vice-President, Merck Research Laboratories, USA and Head of Tsukuba Research Institute, Japan Bob Snow Member Head, Malaria Epidemiology, Public Health Group, KEMRI/Wellcome Trust Program, Nairobi and Oxford Henrietta Ukwu Member Vice President, World Wide Regulatory Affairs, Wyeth Research Inc., USA Thomas E. Wellems Member Chief, Laboratory of Malaria Vector Research, NIAID, NIH, USA Kitima Yuthavong Member Vice President, Thailand Center of Excellence for Life Sciences 61 Annex D Access and Delivery Advisory Committee (ADAC) Name Position Professional background Awa Coll Seck Chair Executive Secretary, Roll Back Malaria Partnership, WHO; former Minister of Health, Senegal, and former Director of Policy, UNAIDS Dora Akunyili Member Director General, Nigerian National Agency for Drug Administration and Control; Professor of Pharmacology at University of Nsukka, Nigeria Joseph Amoussou Member President, Union of Private Pharmacists of Benin; private pharmacist in Porto Novo, Benin Issa Diop Member Pharmaceutical advisor, Ministry of Health, Senegal; former President, African Association of Central Pharmaceutical Procurement Agencies Win Gutteridge Observer Chair, MMV ESAC; distinguished academic and industry experience in pharmaceutical matters; former Chief of Product T&D, WHO-UNDP-IBRD Tropical Disease Research Program Paul Lalvani Member Expertise in procurement and supply chain management; international consultant; formerly with Global Fund to Fight AIDS, TB and Malaria P A Narayan Member Vice President, Emerging Markets, Strides Arcolab Ltd, India Daniel Ngamije Member Director, National Malaria Control Program, Rwanda; medical doctor with experience and publications on clinical trials Naawa Sipilanyambe Member Acting Director, Malaria Control Program, Zambia; medical doctor and honorary lecturer Bob Snow Member Head of Malaria Public Health and Epidemiology Group of Kenya Medical Research Institute/Wellcome Trust Program, Nairobi Francisco Songane Member Director, Partnership for Maternal, Newborn and Child Health; former Minister of Health, Mozambique Ambrose Talisuna Member Head of Division of Epidemiology and Surveillance, Ministry of Health, Uganda; coordinator of MMV- funded malaria clinical trial Marcel Tanner Member Director, Swiss Tropical Institute, Professor of Epidemiology and Medical Parasitology, Basel Geoff Targett Member Professor Emeritus, London School of Hygiene and Tropical Medicine; former Head, Department of Medical Parasitology, LSHTM; Deputy Director, Gates Malaria Partnership Invited observers to the Access and Delivery Advisory Committee: Global Fund to Fight AIDS, TB, and Malaria; Roll Back Malaria Partnership; UNICEF Procurement and Supply Division; WHO GMP Department; World Bank. Annex D 62 Authorization for Phase III Advancement Committee (APAC) /1 Institutional Origin Role in APAC /2 MMV Expert Scientific Advisory Committee ­ Chair Member MMV - Chief Executive Officer Member MMV - Chief Scientific Officer Member MMV- Chief Financial Officer Member MMV ­ Vice President, Global Access /3 Member Stakeholders/partners -- Bill and Melinda Gates Foundation Member Stakeholders/partners -- WHO Global Malaria Program Member /1 Unlike the ESAC and ACAC, members of the APAC participate in an institutional rather than individual capacity. /2 The Committee selects its own chair for each meeting. /3 The position of Vice-President, Global Access, has not been filled in MMV; in these circumstances, the MMV Director, Global Access, participates in APAC. 63 Annex D Executive Management of MMV Name Position Professional background Christopher President and Chief Bio-pharmaceutical executive, Senior Research Hentschel Executive Officer Fellow, Wharton Business School Emerging Technology Program J. Carl Craft Chief Scientific Former Head of Drug Discovery and Development, Officer Anti-Infective Department, Abbott Laboratories Diana Cotran Vice President, Knowledge of multinational corporate sector and Operations experience of cross-cultural relations and management; MBA with emphasis on human resources Peter Potter-Lesage Chief Financial International finance and banking executive, with Officer and Donor Swiss banking and non-profit experience Relations Penny Grewal Director, Global Extensive experience applying private sector skills Access and approaches to public health programs and partnerships Marion Hutt Business Expertise in developing and maintaining Development relationships with partners and organizing events Manager P.V. Venugopal International Former Executive Director, Dr. Reddy's Operations Director Laboratories, India; experience in drug discovery and development, including IPR and drug counterfeiting Anna Wang Vice President, Experience as a communications, political and Public Affairs business executive Source: MMV, March 2007. Annex E 64 Annex E. MMV Finances Annex Table 7. MMV Statement of Income and Expenditures for Years Ending December 31 2004 2005 2006 INCOME Donation Revenues Private Foundations & Individual Donors 21,664,632 37,375,004 14,889,579 UN Agencies 750,000 750,000 750,000 Government Agencies 4,679,781 4,879,963 12,797,857 Corporate & Corporate Foundations 750,000 750,000 750,000 Subtotal 27,844,413 43,754,967 29,187,435 Other Income Financial Income, Net 158,097 635,425 1,289,113 Project Balance Reimbursements 647,466 65,866 97,556 Other 147,583 314,096 44,598 Subtotal 953,146 1,015,387 1431267 Total Income 28,797,559 44,770,355 30,618,703 EXPENDITURES Research & Development Expenditure Project-Related Variable Expenditure 23,604,547 26,844,576 46,646,940 Expert Scientific Advisory Committee Expenses 200,865 321,758 296,312 Subtotal 23,805,412 27,166,334 46,943,252 Access Expenditure Access-Related Variable Expenditure 0 0 697,556 Access & Delivery Advisory Committee 0 0 34,278 Subtotal 0 0 731,834 Foundation Board & Stakeholder Expenses 128,641 136,952 291,300 General and Administrative Expenses Staff-Related Benefits / Compensation 1,643,297 1,735,094 1,915,567 Office And Occupancy 378,970 391,881 565,200 Travel Expenses 151,804 194,638 196,638 Fundraising 221,782 264,924 168,571 Professional & Legal Fees 49,943 402,967 121,747 Training, Education & Journals 80,734 153,927 81,312 IT Expenses 52,590 87,366 153,704 Communications 176,858 127,267 187,728 Depreciation 42,659 56,193 103,638 Other 4,415 5,456 13,284 Subtotal 2,803,052 3,419,713 3,507,390 Total Expenditures 26,737,105 30,722,999 51,473,776 Income ­ Expenditures 2,060,454 14,047,356 (20,855,073) 65 Annex E Annex Table 8. MMV Balance Sheet as of December 31 2004 2005 2006 ASSETS Current Assets Cash And Cash Equivalents 16,213,654 37,672,107 19,826,097 Donations Receivable 0 4,577 0 Project Balance Reimbursements 647,466 0 0 Accounts Receivable 20,852 75,101 36,354 Recoverable Withholding Tax 45,947 223,201 466,685 Project Related Prepaid Expenses 1,431,118 5,621,519 25,645 Total Current Assets 18,359,037 43,596,505 20,354,780 Long Term Assets Guarantees 50,130 43,381 74,213 Fixed Assets, Net 57,527 52,883 220,578 Total Long Term Assets 107,657 96,264 294,790 Total Assets 18,466,694 43,692,769 20,649,570 LIABILITIES AND CAPITAL & RESERVES Current Liabilities Accrued R&D Commitments 1,175,991 2,047,249 5,794,953 Deferred Income 0 10,000,000 3,919,607 Other Creditors 74,125 314,722 328,990 Accrued Expenses 475,326 561,226 778,486 Short-Term Provisions 266,000 246,965 160,000 Total Current Liabilities 1,991,442 13,170,162 10,982,036 Capital & Reserves Foundation Capital 4,000,000 4,000,000 4,000,000 Operations Reserve 12,025,109 23,772,498 5,207,866 Foreign Exchange Reserve 450,143 398,206 459,667 Donor Restricted Reserve 0 2,351,903 0 Total Capital & Reserves 16,475,252 30,522,607 9,667,534 Total Liabilities And Capital & Reserves 18,466,694 43,692,769 20,692,769 Annex Table 9. The World Bank's Financial Participation in MMV, 2000­2006 (US$ millions) 2000 2001 2002 2003 2004 2005 2006 Total World Bank .500 .750 .500 .750 .750 .750 .750 4.750 Other Donors 8.210 14.738 10.178 20.171 27.094 43.004 28.437 151.833 Total 8.709 15.488 10,678 20.921 27.844 43.755 29.187 156.583 World Bank Share 5.7 4.8 4.7 3.6 2.7 1.7 2.6 3.0 (percent) Source: MMV. Annex F 66 Annex F. Persons Consulted55 Person Position Date of Interview Medicines for Malaria Venture (MMV) Dr. Christopher President and CEO March 15, London Hentschel Mr. Peter Potter- Chief Financial Officer and Donor March 12, Geneva Lesage Relations Dr. Carl Craft Chief Scientific Officer March 12, Geneva Dr. Win Gutteridge Chairperson, Expert Scientific Advisory March 16, London Committee (ASAC) Dame Bridget Ogilvie Former Chairperson, MMV Board of March 16, London Directors Baroness Lynda Chairperson, MMV Board of Directors March 16, London Chalker Ms. Marion Hutt Business Development Manager March 13, Geneva Ms. Penny Grewal Director, Global Access March 13, Geneva Ms. Renia Coghlan Associate Director, Global Access March 12, Geneva MMV Consultants Mr. Colin Boyle MMV Consultant, Boston Consulting March 27, by phone Group Mr. Michael Dybbs MMV Consultant, Boston Consulting March 27, by phone Group MMV External Evaluation Team Prof. Adetokunbo Team Leader March 23, by phone Lucas World Health Organization Dr. Awa Marie Coll- Executive Director, Roll Back Malaria March 29, by phone Seck WHO, and Chairperson, Access and Delivery Advisory Committee, MMV 55. Special thanks are given to Mr. Peter Potter-Lesage of MMV, for providing information, responding to questions, and facilitating contacts with others capable of providing useful insights. For the London interviews, particular appreciation is expressed for the assistance of Ms. Susan Dykes, MMV Advisor, based in London. 67 Annex F Person Position Date of Interview Dr. Robert Ridley Director, TDR March 12, Geneva Global Forum for Health Research Prof. Stephen Matlin Executive Director March 13, Geneva Dr. Louis Currat Former Executive Director April 17, by phone Department for International Development (DFID) Ms. Sue Kinn Head, Research Department Unit for March 22, by phone Health Netherlands Development Cooperation Dr. Harry van Senior Health Advisor, Social and April 24, by phone Schooten Institutional Development Department Bill and Melinda Gates Foundation Dr. Regina Chief Infectious Diseases Advisor; April 3, by phone Rabinovich member of the MMV Board of Directors Wellcome Trust Dr. Val Snewin International Activities Manager March 15, London Dr. David Carr Policy Officer March 15, London GlaxoSmithKline (GSK) Mr. Ian Boulton Director, Global Commercial Strategy, March 16, London Diseases of the Developing World (DDW) Mr. Martin Bates Project Leader, Infectious Disease, DDW March 16, London World Bank Dr. Olusoji Adeyi Coordinator, Public Health Programs, April 4, Washington HNP Hub Dr. Ok Pannenborg Sr. Advisor, HNP, Africa Region March 20, Washington Dr. Anne Maryse Coordinator, Malaria Booster Program, April 5, Washington Pierre-Louis Africa Region Ms. Sophia Sr. Partnerships Officer, DGF Secretariat March 28, Washington Drewnowski Annex G 68 Annex G. Recommendations of the Independent Evaluation and Program Response Annex Table 10. Status of Recommendations and Program Response, as of March 2007 Findings Recommendations Program Response Scientific Strengths Strong technical guidance Provide honoraria Honoraria provided for ESAC from Expert Scientific chair, also for members' Advisory Committee (ESAC) duties additional to annual meetings (mentoring and review of new calls for proposals) Weaknesses Gaps in expertise -- Strengthen ESAC & Science ESAC increased to 18 statistics, clinical science -- team members with relevant in ESAC & the Science team; expertise. limited expertise in clinical trials & field work Science team: 3 additional staff Medical Director in recruitment Opportunities Innovative ideas and leads Maintain MMV's capacity to Fully maintained & increased: from academia follow up on useful leads R&D Portfolio now 35 (21 in 2005) Risks/Threats Failure to take up promising Ensure that MMV has Internal scientific resources leads may undermine morale enough resources to substantially increased as in academia maintain its momentum; seen above. MMV now works with 80 partners worldwide including the pharmaceutical industry, academia and contract research organizations (CROs) Managerial/ Operational Strengths Strong, effective Governing Expand and strengthen Management team now Board & small dedicated Management team constituted as an Executive Management Team Committee with 7 members. Total headcount 22 (12 in 2005), with expected 25 or 26 by end 2007 69 Annex G Findings Recommendations Program Response Weaknesses Science team heavily Expand Science team to fill Additional MMV science staff: dependent on identified gaps in expertise complementary expertise in 3 Associate Scientific ESAC directors 3 mini-portfolios = Novartis, GSK, Broad/Genzyme involve more management from partners Opportunities Potential new partners Develop & strengthen Sustained effort in this area identified for downstream partnerships with WHO & with multiple segments of functions other partners WHO and other partners. Risks/Threats Difficulties in developing Careful task analysis of Boston Consulting Group effective partnerships for the downstream issues, (BCG) consultancy with the downstream work. identification of potential Gates Foundation "Planning partners and matching for Success" evaluated this capacity to needs. area for facilitation by MMV. New "Access & Delivery" area created within MMV and 2 staff members hired. Financial Strengths Generous support especially Donors continue to give high MMV has put effort and from private sector donors. priority to MMV energy into fundraising with existing and new donors. Received/pledged funding now totals $273 million to 2010 ($113 million to 2007) Weaknesses Present financial Donors should ensure steady Funding flows much arrangements do not ensure flow of resources to maintain improved, with a steady flow of funds steady progress of MMV disbursements earlier in the projects year and a more predictable, robust cash flow situation. This was due to direct discussion and negotiation between MMV financial management and individual donors. Opportunities MMV's rapid progress on Donor group should respond Donor Coordination Group as discovery and early positively to MMV's needs such has not responded to development provides needs of PD-PPPs. opportunities for fast progress towards its goals. Risks/Threats Stalling of progress in Donors should consider No such replenishment managing a very promising using a replenishment model model for PD-PPPs portfolio similar to that adopted by the discussed, implemented or Global Fund to Fight AIDS, considered by donors. One Tuberculosis and Malaria innovative solution would be an IFFnd. Annex H 70 Annex H. Response of the Program to IEG's Global Program Review Succinct and clear, the Global Program Review (GPR) written by the Independent Evaluation Group displays an admirable understanding of Medicines for Malaria Venture (MMV), its origins and evolution, as well as the challenges it has overcome and those it is currently facing. We are indeed honored that the evaluation considers MMV to be "a successful product development public private partnership (PD-PPP) in the field of malaria." From Preface to Appendices, the IEG has successfully and constructively analyzed MMV's achievements over the past two years, using the 2005 Donor Coordination Group (DCG)/DFID evaluation of MMV as its baseline. It is very encouraging to see that the MMV business model is deemed worth emulating -- MMV was chosen for a Global Program Review "because it provides lessons for the design and operation of other global programs -- in particular, for public-private partnerships for health research, and for international support of health research more generally." And it is also heartening to know that the GPR considers MMV a transparent organization. Transparency, accountability, and flexibility are indeed major guiding principles underpinning MMV operations. To all intents and purposes, the GPR sees MMV as "an effective PD-PPP." What began in 1999 as an inspired but untested pioneering idea has now grown into a fully-fledged efficient virtual drug development organization that is now "an accepted part of the landscape of drug R&D on diseases of developing countries." Our goals have evolved flexibly, too. Shaped by stakeholder feedback we have expanded our mandate from "discover and develop to registration" by adding a third dimension -- deliver. Today, MMV is poised to over-deliver on its original promise by registering not just one but several new antimalarial drugs by 2010. As the report states, "In this respect, MMV must be considered a success." This success did not come effortlessly. From its very inception, MMV has had to overcome many and constant obstacles not least to find sufficient funding and source sufficient high quality projects promising enough to develop its now robust portfolio. From a small organization of two in 1999, it has grown to a still lean 22 people seven years later to "maintain the momentum generated in its first five years." As its financial situation became more secure, MMV has confidently expanded its drug development pipeline from four antimalarial projects in 2000 to over 35 projects today. This balanced portfolio includes five projects in clinical development and 19 new classes of drugs with many novel modes of action in the early discovery phase. Unlike some sister organizations, MMV prides itself as "product- driven" not "research-driven.". Moreover the products are clearly destined as "global public goods." This strategy has resulted in and required us to assemble the largest- ever jointly-managed portfolio of antimalarial R&D projects in the history of drug development. It might seem at first glance that MMV has overreached itself. For instance, how is it going to continue to fund these 35 projects as they progress through the pipeline? But a full early pipeline is a strategic choice informed by pharmaceutical R&D practice and experience -- 71 Annex H the discovery portfolio has to remain populated and balanced to counter the certainty of project attrition, parasite resistance, and the need for new drugs for different malaria indications. MMV's success can thus be attributed to the successful integration of several imperatives: tireless fund-raising; the ability to mobilize resources from government agencies, philanthropic or corporate foundations and international organizations; the well-regulated, transparent reporting of expenditures of donor money; and critically professional portfolio management. As the GPR states, "MMV has a truly remarkable fundraising record" and goes on to rightly say that the World Bank's annual contribution of $500,000 or $750,000 to MMV for a total of $4,750,000 from 2000 to 2006, although merely 3% of overall commitments, was "critical to attracting other official donors." The converse is also true, namely that the loss of the Bank in MMV's donor list could lead to further donor attrition -- its funding commitment can thus be seen as catalytic in both directions. By the same token an increase from 3 percent to say 6 percent would be really welcomed as significant by the broader malaria community. The report identifies MMV as an "efficient allocator of resources to finance potential new malaria drugs," and recognizes the "efficiencies of a large portfolio with regard to resource allocation across candidate drugs that could not be realized with a small portfolio." We agree and would add that MMV has witnessed a declining share of management and administration expenses and spends most of its funds (over 90 percent) on its core activity of research and development (2006 figures). None of this should however allow complacency about the longer term funding challenges. In spite of this rigorously managed spending, MMV's current financial gap through 2012 stands at $300­400 million. The GPR pulls no punches about its concerns in some areas. It expresses concern about MMV's entry into the Access and Delivery arena in 2006. This move was of course strongly encouraged and supported by our primary funder, the Bill and Melinda Gates foundation. Its purpose -- a program to ensure that MMV's newly registered products do not languish on warehouse shelves but rather get used as public goods -- is not in itself controversial. Rather the question amounts to "why MMV?" The simple answer is that a that a major analysis by an independent external group (the Boston Consulting Group) concluded that MMV was best placed and had the most incentive to do the job well for products from its own pipeline. Board and stakeholders by and large agree. The first of MMV's four new ICH quality ACTs is expected to gain market authorization in 2008 and the others by 2010. The imminent availability of these new products has propelled the MMV's access work from theory to practice in short order. Clearly, efficient access and delivery is a daunting task and much too big for MMV alone, yet we must be highly involved in this process if our new drugs are to have public health impact. Access is rightly recognized as the next big challenge facing MMV, as it will have to engage with a number of policy and institutional issues beyond its current competence. In doing so, it fully intends to "establish strong links with new and different actors." Some consider access issues as difficult; even as inherently more difficult than the myriad issues around registration of new ICH standard malaria drugs. We disagree, remembering the many similar doubts about R&D that were expressed earlier on -- doubts that have been Annex H 72 reversed by evidence. Nevertheless we do recognize the relative novelty and complexity of this endeavor and have gone a long way in honing and refining the Access and Delivery strategy. Expert advice and constructive criticism from numerous distinguished stakeholders and mentors has already informed the process and will continue to do so. We also plan to "learn by doing" -- something that small flexible organizations can often do well. The MMV Board & Stakeholder's meeting in Uganda in May 2007 marked a turning point when a revised and fully focused access and delivery facilitation strategy was unanimously endorsed. The GPR mentions that apart from increasing its funding commitments to health research, the World Bank should get more closely involved via its country operations with MMV activities, especially Access and Delivery, and offers several options for future engagement. The Bank's comparative advantage includes "advice to governments on the regulatory framework for public-private collaboration in the health sector -- a crucial element of MMV's new access and delivery agenda." We would of course be delighted to work with, and benefit from, multiple Bank roles as advocate, strategist, and arbitrator at both global and country levels. We have already had meetings to investigate possibilities of cooperating with the Bank's Malaria Booster Program and look forward to strengthening ties. We wholeheartedly welcome the GPR's recommendation that the Bank "adopt a more proactive and conscious approach to its participation" in PD-PPPs such as MMV. The GPR evaluation is based on 27 interviews of MMV personnel, donors, research partners and other stakeholders. It is gratifying for MMV to emerge from this meticulous, wide- ranging and indisputably fair assessment of MMV's efficacy, efficiency, governance, sustainability, and future prospects with strong overall endorsement. We have worked hard to achieve this and appreciate it ­ but most of all are motivated by the excitement of our mission and the feeling that it can succeed. The Global Program Review Series The following reviews are available from IEG. Volume #1, Issue #1: ProVention Consortium Issue #2: Medicines for Malaria Venture Issue #3: Development Gateway Foundation Issue #4: Cities Alliance Volume #2, Issue #1: Critical Ecosystem Partnership Fund Issue #2: Association for the Development of Education in Africa Issue #3: Population and Reproductive Health Capacity Building Program Issue #4: International Land Coalition Volume #3, Issue #1: Consultative Group to Assist the Poor GPR The Medicines for Malaria Venture (MMV) is a public-private partnership that was estab- lished in 1999 to fund and manage the discovery and development of new antimalarial drugs in response to the increasing incidence of and mortality from malaria, the declining efficiency of first- and second-line treatments, and the withdrawal of major pharmaceutical firms from developing new antimalarial drugs. This review found MMV to be an effective program that is efficiently managing a portfolio of candidates for new malaria drugs through the various phases of drug discovery and clinical development that precede formal registration with pub- lic authorities and marketing in the public and private sectors. MMV's success in fund raising has permitted it to establish a strong pipeline of new malaria drugs that are expected to be affordable in developing countries. MMV's decision to expand its activities to encompass facilitation of the access and delivery of its malaria drugs raises new policy and institutional challenges for MMV, as well as new opportunities for exploiting synergies with the World Bank's country operations. Effective coordination and consultation with other key players at the global and country levels will also be essential.